Commit | Line | Data |
---|---|---|
684ffa2d MCC |
1 | Media Controller devices |
2 | ------------------------ | |
3 | ||
4 | Media Controller | |
5 | ~~~~~~~~~~~~~~~~ | |
6 | ||
74604b73 | 7 | The media controller userspace API is documented in |
da83c888 | 8 | :ref:`the Media Controller uAPI book <media_controller>`. This document focus |
684ffa2d MCC |
9 | on the kernel-side implementation of the media framework. |
10 | ||
11 | Abstract media device model | |
12 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
13 | ||
14 | Discovering a device internal topology, and configuring it at runtime, is one | |
15 | of the goals of the media framework. To achieve this, hardware devices are | |
16 | modelled as an oriented graph of building blocks called entities connected | |
17 | through pads. | |
18 | ||
19 | An entity is a basic media hardware building block. It can correspond to | |
20 | a large variety of logical blocks such as physical hardware devices | |
21 | (CMOS sensor for instance), logical hardware devices (a building block | |
22 | in a System-on-Chip image processing pipeline), DMA channels or physical | |
23 | connectors. | |
24 | ||
25 | A pad is a connection endpoint through which an entity can interact with | |
26 | other entities. Data (not restricted to video) produced by an entity | |
27 | flows from the entity's output to one or more entity inputs. Pads should | |
28 | not be confused with physical pins at chip boundaries. | |
29 | ||
30 | A link is a point-to-point oriented connection between two pads, either | |
31 | on the same entity or on different entities. Data flows from a source | |
32 | pad to a sink pad. | |
33 | ||
34 | Media device | |
35 | ^^^^^^^^^^^^ | |
36 | ||
fc78c7c7 | 37 | A media device is represented by a struct :c:type:`media_device` |
74604b73 MCC |
38 | instance, defined in ``include/media/media-device.h``. |
39 | Allocation of the structure is handled by the media device driver, usually by | |
40 | embedding the :c:type:`media_device` instance in a larger driver-specific | |
41 | structure. | |
684ffa2d MCC |
42 | |
43 | Drivers register media device instances by calling | |
7b998bae MCC |
44 | :c:func:`__media_device_register()` via the macro ``media_device_register()`` |
45 | and unregistered by calling :c:func:`media_device_unregister()`. | |
684ffa2d MCC |
46 | |
47 | Entities | |
48 | ^^^^^^^^ | |
49 | ||
fc78c7c7 | 50 | Entities are represented by a struct :c:type:`media_entity` |
74604b73 MCC |
51 | instance, defined in ``include/media/media-entity.h``. The structure is usually |
52 | embedded into a higher-level structure, such as | |
041d8211 | 53 | :c:type:`v4l2_subdev` or :c:type:`video_device` |
74604b73 | 54 | instances, although drivers can allocate entities directly. |
684ffa2d MCC |
55 | |
56 | Drivers initialize entity pads by calling | |
7b998bae | 57 | :c:func:`media_entity_pads_init()`. |
684ffa2d MCC |
58 | |
59 | Drivers register entities with a media device by calling | |
7b998bae | 60 | :c:func:`media_device_register_entity()` |
684ffa2d | 61 | and unregistred by calling |
7b998bae | 62 | :c:func:`media_device_unregister_entity()`. |
684ffa2d MCC |
63 | |
64 | Interfaces | |
65 | ^^^^^^^^^^ | |
66 | ||
74604b73 | 67 | Interfaces are represented by a |
fc78c7c7 | 68 | struct :c:type:`media_interface` instance, defined in |
74604b73 MCC |
69 | ``include/media/media-entity.h``. Currently, only one type of interface is |
70 | defined: a device node. Such interfaces are represented by a | |
fc78c7c7 | 71 | struct :c:type:`media_intf_devnode`. |
684ffa2d MCC |
72 | |
73 | Drivers initialize and create device node interfaces by calling | |
7b998bae | 74 | :c:func:`media_devnode_create()` |
684ffa2d | 75 | and remove them by calling: |
7b998bae | 76 | :c:func:`media_devnode_remove()`. |
684ffa2d MCC |
77 | |
78 | Pads | |
79 | ^^^^ | |
fc78c7c7 | 80 | Pads are represented by a struct :c:type:`media_pad` instance, |
74604b73 MCC |
81 | defined in ``include/media/media-entity.h``. Each entity stores its pads in |
82 | a pads array managed by the entity driver. Drivers usually embed the array in | |
83 | a driver-specific structure. | |
684ffa2d MCC |
84 | |
85 | Pads are identified by their entity and their 0-based index in the pads | |
86 | array. | |
74604b73 | 87 | |
fc78c7c7 MCC |
88 | Both information are stored in the struct :c:type:`media_pad`, |
89 | making the struct :c:type:`media_pad` pointer the canonical way | |
3c7d91ef | 90 | to store and pass link references. |
684ffa2d MCC |
91 | |
92 | Pads have flags that describe the pad capabilities and state. | |
93 | ||
74604b73 MCC |
94 | ``MEDIA_PAD_FL_SINK`` indicates that the pad supports sinking data. |
95 | ``MEDIA_PAD_FL_SOURCE`` indicates that the pad supports sourcing data. | |
96 | ||
97 | .. note:: | |
684ffa2d | 98 | |
74604b73 MCC |
99 | One and only one of ``MEDIA_PAD_FL_SINK`` or ``MEDIA_PAD_FL_SOURCE`` must |
100 | be set for each pad. | |
684ffa2d MCC |
101 | |
102 | Links | |
103 | ^^^^^ | |
104 | ||
fc78c7c7 | 105 | Links are represented by a struct :c:type:`media_link` instance, |
74604b73 | 106 | defined in ``include/media/media-entity.h``. There are two types of links: |
684ffa2d | 107 | |
74604b73 | 108 | **1. pad to pad links**: |
684ffa2d MCC |
109 | |
110 | Associate two entities via their PADs. Each entity has a list that points | |
111 | to all links originating at or targeting any of its pads. | |
112 | A given link is thus stored twice, once in the source entity and once in | |
113 | the target entity. | |
114 | ||
115 | Drivers create pad to pad links by calling: | |
7b998bae MCC |
116 | :c:func:`media_create_pad_link()` and remove with |
117 | :c:func:`media_entity_remove_links()`. | |
684ffa2d | 118 | |
74604b73 | 119 | **2. interface to entity links**: |
684ffa2d MCC |
120 | |
121 | Associate one interface to a Link. | |
122 | ||
123 | Drivers create interface to entity links by calling: | |
7b998bae MCC |
124 | :c:func:`media_create_intf_link()` and remove with |
125 | :c:func:`media_remove_intf_links()`. | |
684ffa2d MCC |
126 | |
127 | .. note:: | |
128 | ||
129 | Links can only be created after having both ends already created. | |
130 | ||
131 | Links have flags that describe the link capabilities and state. The | |
7b998bae MCC |
132 | valid values are described at :c:func:`media_create_pad_link()` and |
133 | :c:func:`media_create_intf_link()`. | |
684ffa2d MCC |
134 | |
135 | Graph traversal | |
136 | ^^^^^^^^^^^^^^^ | |
137 | ||
138 | The media framework provides APIs to iterate over entities in a graph. | |
139 | ||
140 | To iterate over all entities belonging to a media device, drivers can use | |
141 | the media_device_for_each_entity macro, defined in | |
74604b73 MCC |
142 | ``include/media/media-device.h``. |
143 | ||
144 | .. code-block:: c | |
684ffa2d | 145 | |
74604b73 | 146 | struct media_entity *entity; |
684ffa2d | 147 | |
74604b73 MCC |
148 | media_device_for_each_entity(entity, mdev) { |
149 | // entity will point to each entity in turn | |
150 | ... | |
151 | } | |
684ffa2d MCC |
152 | |
153 | Drivers might also need to iterate over all entities in a graph that can be | |
154 | reached only through enabled links starting at a given entity. The media | |
155 | framework provides a depth-first graph traversal API for that purpose. | |
156 | ||
74604b73 MCC |
157 | .. note:: |
158 | ||
159 | Graphs with cycles (whether directed or undirected) are **NOT** | |
160 | supported by the graph traversal API. To prevent infinite loops, the graph | |
161 | traversal code limits the maximum depth to ``MEDIA_ENTITY_ENUM_MAX_DEPTH``, | |
162 | currently defined as 16. | |
684ffa2d MCC |
163 | |
164 | Drivers initiate a graph traversal by calling | |
7b998bae | 165 | :c:func:`media_entity_graph_walk_start()` |
684ffa2d MCC |
166 | |
167 | The graph structure, provided by the caller, is initialized to start graph | |
168 | traversal at the given entity. | |
169 | ||
170 | Drivers can then retrieve the next entity by calling | |
7b998bae | 171 | :c:func:`media_entity_graph_walk_next()` |
684ffa2d | 172 | |
74604b73 | 173 | When the graph traversal is complete the function will return ``NULL``. |
684ffa2d MCC |
174 | |
175 | Graph traversal can be interrupted at any moment. No cleanup function call | |
176 | is required and the graph structure can be freed normally. | |
177 | ||
178 | Helper functions can be used to find a link between two given pads, or a pad | |
179 | connected to another pad through an enabled link | |
7b998bae MCC |
180 | :c:func:`media_entity_find_link()` and |
181 | :c:func:`media_entity_remote_pad()`. | |
684ffa2d MCC |
182 | |
183 | Use count and power handling | |
184 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
185 | ||
186 | Due to the wide differences between drivers regarding power management | |
187 | needs, the media controller does not implement power management. However, | |
fc78c7c7 | 188 | the struct :c:type:`media_entity` includes a ``use_count`` |
74604b73 | 189 | field that media drivers |
684ffa2d MCC |
190 | can use to track the number of users of every entity for power management |
191 | needs. | |
192 | ||
74604b73 MCC |
193 | The :c:type:`media_entity<media_entity>`.\ ``use_count`` field is owned by |
194 | media drivers and must not be | |
684ffa2d | 195 | touched by entity drivers. Access to the field must be protected by the |
74604b73 | 196 | :c:type:`media_device`.\ ``graph_mutex`` lock. |
684ffa2d MCC |
197 | |
198 | Links setup | |
199 | ^^^^^^^^^^^ | |
200 | ||
201 | Link properties can be modified at runtime by calling | |
7b998bae | 202 | :c:func:`media_entity_setup_link()`. |
684ffa2d MCC |
203 | |
204 | Pipelines and media streams | |
205 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
206 | ||
207 | When starting streaming, drivers must notify all entities in the pipeline to | |
208 | prevent link states from being modified during streaming by calling | |
7b998bae | 209 | :c:func:`media_entity_pipeline_start()`. |
684ffa2d MCC |
210 | |
211 | The function will mark all entities connected to the given entity through | |
212 | enabled links, either directly or indirectly, as streaming. | |
213 | ||
fc78c7c7 | 214 | The struct :c:type:`media_pipeline` instance pointed to by |
74604b73 | 215 | the pipe argument will be stored in every entity in the pipeline. |
fc78c7c7 | 216 | Drivers should embed the struct :c:type:`media_pipeline` |
74604b73 | 217 | in higher-level pipeline structures and can then access the |
fc78c7c7 | 218 | pipeline through the struct :c:type:`media_entity` |
74604b73 | 219 | pipe field. |
684ffa2d | 220 | |
7b998bae | 221 | Calls to :c:func:`media_entity_pipeline_start()` can be nested. |
74604b73 | 222 | The pipeline pointer must be identical for all nested calls to the function. |
684ffa2d | 223 | |
7b998bae | 224 | :c:func:`media_entity_pipeline_start()` may return an error. In that case, |
74604b73 | 225 | it will clean up any of the changes it did by itself. |
684ffa2d MCC |
226 | |
227 | When stopping the stream, drivers must notify the entities with | |
7b998bae | 228 | :c:func:`media_entity_pipeline_stop()`. |
684ffa2d | 229 | |
7b998bae MCC |
230 | If multiple calls to :c:func:`media_entity_pipeline_start()` have been |
231 | made the same number of :c:func:`media_entity_pipeline_stop()` calls | |
74604b73 MCC |
232 | are required to stop streaming. |
233 | The :c:type:`media_entity`.\ ``pipe`` field is reset to ``NULL`` on the last | |
234 | nested stop call. | |
684ffa2d | 235 | |
74604b73 | 236 | Link configuration will fail with ``-EBUSY`` by default if either end of the |
684ffa2d | 237 | link is a streaming entity. Links that can be modified while streaming must |
74604b73 | 238 | be marked with the ``MEDIA_LNK_FL_DYNAMIC`` flag. |
684ffa2d MCC |
239 | |
240 | If other operations need to be disallowed on streaming entities (such as | |
241 | changing entities configuration parameters) drivers can explicitly check the | |
242 | media_entity stream_count field to find out if an entity is streaming. This | |
243 | operation must be done with the media_device graph_mutex held. | |
244 | ||
245 | Link validation | |
246 | ^^^^^^^^^^^^^^^ | |
247 | ||
7b998bae | 248 | Link validation is performed by :c:func:`media_entity_pipeline_start()` |
74604b73 MCC |
249 | for any entity which has sink pads in the pipeline. The |
250 | :c:type:`media_entity`.\ ``link_validate()`` callback is used for that | |
251 | purpose. In ``link_validate()`` callback, entity driver should check | |
252 | that the properties of the source pad of the connected entity and its own | |
253 | sink pad match. It is up to the type of the entity (and in the end, the | |
254 | properties of the hardware) what matching actually means. | |
684ffa2d MCC |
255 | |
256 | Subsystems should facilitate link validation by providing subsystem specific | |
257 | helper functions to provide easy access for commonly needed information, and | |
258 | in the end provide a way to use driver-specific callbacks. | |
259 | ||
260 | .. kernel-doc:: include/media/media-device.h | |
261 | ||
262 | .. kernel-doc:: include/media/media-devnode.h | |
263 | ||
264 | .. kernel-doc:: include/media/media-entity.h |