Commit | Line | Data |
---|---|---|
400e64df OBC |
1 | Remote Processor Framework |
2 | ||
3 | 1. Introduction | |
4 | ||
5 | Modern SoCs typically have heterogeneous remote processor devices in asymmetric | |
6 | multiprocessing (AMP) configurations, which may be running different instances | |
7 | of operating system, whether it's Linux or any other flavor of real-time OS. | |
8 | ||
9 | OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. | |
10 | In a typical configuration, the dual cortex-A9 is running Linux in a SMP | |
11 | configuration, and each of the other three cores (two M3 cores and a DSP) | |
12 | is running its own instance of RTOS in an AMP configuration. | |
13 | ||
14 | The remoteproc framework allows different platforms/architectures to | |
15 | control (power on, load firmware, power off) those remote processors while | |
16 | abstracting the hardware differences, so the entire driver doesn't need to be | |
17 | duplicated. In addition, this framework also adds rpmsg virtio devices | |
18 | for remote processors that supports this kind of communication. This way, | |
19 | platform-specific remoteproc drivers only need to provide a few low-level | |
20 | handlers, and then all rpmsg drivers will then just work | |
21 | (for more information about the virtio-based rpmsg bus and its drivers, | |
22 | please read Documentation/rpmsg.txt). | |
23 | ||
24 | 2. User API | |
25 | ||
26 | int rproc_boot(struct rproc *rproc) | |
27 | - Boot a remote processor (i.e. load its firmware, power it on, ...). | |
28 | If the remote processor is already powered on, this function immediately | |
29 | returns (successfully). | |
30 | Returns 0 on success, and an appropriate error value otherwise. | |
31 | Note: to use this function you should already have a valid rproc | |
32 | handle. There are several ways to achieve that cleanly (devres, pdata, | |
33 | the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we | |
34 | might also consider using dev_archdata for this). See also | |
35 | rproc_get_by_name() below. | |
36 | ||
37 | void rproc_shutdown(struct rproc *rproc) | |
38 | - Power off a remote processor (previously booted with rproc_boot()). | |
39 | In case @rproc is still being used by an additional user(s), then | |
40 | this function will just decrement the power refcount and exit, | |
41 | without really powering off the device. | |
42 | Every call to rproc_boot() must (eventually) be accompanied by a call | |
43 | to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. | |
44 | Notes: | |
45 | - we're not decrementing the rproc's refcount, only the power refcount. | |
46 | which means that the @rproc handle stays valid even after | |
47 | rproc_shutdown() returns, and users can still use it with a subsequent | |
48 | rproc_boot(), if needed. | |
49 | - don't call rproc_shutdown() to unroll rproc_get_by_name(), exactly | |
50 | because rproc_shutdown() _does not_ decrement the refcount of @rproc. | |
51 | To decrement the refcount of @rproc, use rproc_put() (but _only_ if | |
52 | you acquired @rproc using rproc_get_by_name()). | |
53 | ||
54 | struct rproc *rproc_get_by_name(const char *name) | |
55 | - Find an rproc handle using the remote processor's name, and then | |
56 | boot it. If it's already powered on, then just immediately return | |
57 | (successfully). Returns the rproc handle on success, and NULL on failure. | |
58 | This function increments the remote processor's refcount, so always | |
59 | use rproc_put() to decrement it back once rproc isn't needed anymore. | |
60 | Note: currently rproc_get_by_name() and rproc_put() are not used anymore | |
61 | by the rpmsg bus and its drivers. We need to scrutinize the use cases | |
62 | that still need them, and see if we can migrate them to use the non | |
63 | name-based boot/shutdown interface. | |
64 | ||
65 | void rproc_put(struct rproc *rproc) | |
66 | - Decrement @rproc's power refcount and shut it down if it reaches zero | |
67 | (essentially by just calling rproc_shutdown), and then decrement @rproc's | |
68 | validity refcount too. | |
69 | After this function returns, @rproc may _not_ be used anymore, and its | |
70 | handle should be considered invalid. | |
71 | This function should be called _iff_ the @rproc handle was grabbed by | |
72 | calling rproc_get_by_name(). | |
73 | ||
74 | 3. Typical usage | |
75 | ||
76 | #include <linux/remoteproc.h> | |
77 | ||
78 | /* in case we were given a valid 'rproc' handle */ | |
79 | int dummy_rproc_example(struct rproc *my_rproc) | |
80 | { | |
81 | int ret; | |
82 | ||
83 | /* let's power on and boot our remote processor */ | |
84 | ret = rproc_boot(my_rproc); | |
85 | if (ret) { | |
86 | /* | |
87 | * something went wrong. handle it and leave. | |
88 | */ | |
89 | } | |
90 | ||
91 | /* | |
92 | * our remote processor is now powered on... give it some work | |
93 | */ | |
94 | ||
95 | /* let's shut it down now */ | |
96 | rproc_shutdown(my_rproc); | |
97 | } | |
98 | ||
99 | 4. API for implementors | |
100 | ||
101 | struct rproc *rproc_alloc(struct device *dev, const char *name, | |
102 | const struct rproc_ops *ops, | |
103 | const char *firmware, int len) | |
104 | - Allocate a new remote processor handle, but don't register | |
105 | it yet. Required parameters are the underlying device, the | |
106 | name of this remote processor, platform-specific ops handlers, | |
107 | the name of the firmware to boot this rproc with, and the | |
108 | length of private data needed by the allocating rproc driver (in bytes). | |
109 | ||
110 | This function should be used by rproc implementations during | |
111 | initialization of the remote processor. | |
112 | After creating an rproc handle using this function, and when ready, | |
113 | implementations should then call rproc_register() to complete | |
114 | the registration of the remote processor. | |
115 | On success, the new rproc is returned, and on failure, NULL. | |
116 | ||
117 | Note: _never_ directly deallocate @rproc, even if it was not registered | |
118 | yet. Instead, if you just need to unroll rproc_alloc(), use rproc_free(). | |
119 | ||
120 | void rproc_free(struct rproc *rproc) | |
121 | - Free an rproc handle that was allocated by rproc_alloc. | |
122 | This function should _only_ be used if @rproc was only allocated, | |
123 | but not registered yet. | |
124 | If @rproc was already successfully registered (by calling | |
125 | rproc_register()), then use rproc_unregister() instead. | |
126 | ||
127 | int rproc_register(struct rproc *rproc) | |
128 | - Register @rproc with the remoteproc framework, after it has been | |
129 | allocated with rproc_alloc(). | |
130 | This is called by the platform-specific rproc implementation, whenever | |
131 | a new remote processor device is probed. | |
132 | Returns 0 on success and an appropriate error code otherwise. | |
133 | Note: this function initiates an asynchronous firmware loading | |
134 | context, which will look for virtio devices supported by the rproc's | |
135 | firmware. | |
136 | If found, those virtio devices will be created and added, so as a result | |
137 | of registering this remote processor, additional virtio drivers might get | |
138 | probed. | |
139 | Currently, though, we only support a single RPMSG virtio vdev per remote | |
140 | processor. | |
141 | ||
142 | int rproc_unregister(struct rproc *rproc) | |
143 | - Unregister a remote processor, and decrement its refcount. | |
144 | If its refcount drops to zero, then @rproc will be freed. If not, | |
145 | it will be freed later once the last reference is dropped. | |
146 | ||
147 | This function should be called when the platform specific rproc | |
148 | implementation decides to remove the rproc device. it should | |
149 | _only_ be called if a previous invocation of rproc_register() | |
150 | has completed successfully. | |
151 | ||
152 | After rproc_unregister() returns, @rproc is _not_ valid anymore and | |
153 | it shouldn't be used. More specifically, don't call rproc_free() | |
154 | or try to directly free @rproc after rproc_unregister() returns; | |
155 | none of these are needed, and calling them is a bug. | |
156 | ||
157 | Returns 0 on success and -EINVAL if @rproc isn't valid. | |
158 | ||
159 | 5. Implementation callbacks | |
160 | ||
161 | These callbacks should be provided by platform-specific remoteproc | |
162 | drivers: | |
163 | ||
164 | /** | |
165 | * struct rproc_ops - platform-specific device handlers | |
166 | * @start: power on the device and boot it | |
167 | * @stop: power off the device | |
168 | * @kick: kick a virtqueue (virtqueue id given as a parameter) | |
169 | */ | |
170 | struct rproc_ops { | |
171 | int (*start)(struct rproc *rproc); | |
172 | int (*stop)(struct rproc *rproc); | |
173 | void (*kick)(struct rproc *rproc, int vqid); | |
174 | }; | |
175 | ||
176 | Every remoteproc implementation should at least provide the ->start and ->stop | |
177 | handlers. If rpmsg functionality is also desired, then the ->kick handler | |
178 | should be provided as well. | |
179 | ||
180 | The ->start() handler takes an rproc handle and should then power on the | |
181 | device and boot it (use rproc->priv to access platform-specific private data). | |
182 | The boot address, in case needed, can be found in rproc->bootaddr (remoteproc | |
183 | core puts there the ELF entry point). | |
184 | On success, 0 should be returned, and on failure, an appropriate error code. | |
185 | ||
186 | The ->stop() handler takes an rproc handle and powers the device down. | |
187 | On success, 0 is returned, and on failure, an appropriate error code. | |
188 | ||
189 | The ->kick() handler takes an rproc handle, and an index of a virtqueue | |
190 | where new message was placed in. Implementations should interrupt the remote | |
191 | processor and let it know it has pending messages. Notifying remote processors | |
192 | the exact virtqueue index to look in is optional: it is easy (and not | |
193 | too expensive) to go through the existing virtqueues and look for new buffers | |
194 | in the used rings. | |
195 | ||
196 | 6. Binary Firmware Structure | |
197 | ||
198 | At this point remoteproc only supports ELF32 firmware binaries. However, | |
199 | it is quite expected that other platforms/devices which we'd want to | |
200 | support with this framework will be based on different binary formats. | |
201 | ||
202 | When those use cases show up, we will have to decouple the binary format | |
203 | from the framework core, so we can support several binary formats without | |
204 | duplicating common code. | |
205 | ||
206 | When the firmware is parsed, its various segments are loaded to memory | |
207 | according to the specified device address (might be a physical address | |
208 | if the remote processor is accessing memory directly). | |
209 | ||
210 | In addition to the standard ELF segments, most remote processors would | |
211 | also include a special section which we call "the resource table". | |
212 | ||
213 | The resource table contains system resources that the remote processor | |
214 | requires before it should be powered on, such as allocation of physically | |
215 | contiguous memory, or iommu mapping of certain on-chip peripherals. | |
216 | Remotecore will only power up the device after all the resource table's | |
217 | requirement are met. | |
218 | ||
219 | In addition to system resources, the resource table may also contain | |
220 | resource entries that publish the existence of supported features | |
221 | or configurations by the remote processor, such as trace buffers and | |
222 | supported virtio devices (and their configurations). | |
223 | ||
224 | Currently the resource table is just an array of: | |
225 | ||
226 | /** | |
227 | * struct fw_resource - describes an entry from the resource section | |
228 | * @type: resource type | |
229 | * @id: index number of the resource | |
230 | * @da: device address of the resource | |
231 | * @pa: physical address of the resource | |
232 | * @len: size, in bytes, of the resource | |
233 | * @flags: properties of the resource, e.g. iommu protection required | |
234 | * @reserved: must be 0 atm | |
235 | * @name: name of resource | |
236 | */ | |
237 | struct fw_resource { | |
238 | u32 type; | |
239 | u32 id; | |
240 | u64 da; | |
241 | u64 pa; | |
242 | u32 len; | |
243 | u32 flags; | |
244 | u8 reserved[16]; | |
245 | u8 name[48]; | |
246 | } __packed; | |
247 | ||
248 | Some resources entries are mere announcements, where the host is informed | |
249 | of specific remoteproc configuration. Other entries require the host to | |
250 | do something (e.g. reserve a requested resource) and possibly also reply | |
251 | by overwriting a member inside 'struct fw_resource' with info about the | |
252 | allocated resource. | |
253 | ||
254 | Different resource entries use different members of this struct, | |
255 | with different meanings. This is pretty limiting and error-prone, | |
256 | so the plan is to move to variable-length TLV-based resource entries, | |
257 | where each resource will begin with a type and length fields, followed by | |
258 | its own specific structure. | |
259 | ||
260 | Here are the resource types that are currently being used: | |
261 | ||
262 | /** | |
263 | * enum fw_resource_type - types of resource entries | |
264 | * | |
265 | * @RSC_CARVEOUT: request for allocation of a physically contiguous | |
266 | * memory region. | |
267 | * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. | |
268 | * @RSC_TRACE: announces the availability of a trace buffer into which | |
269 | * the remote processor will be writing logs. In this case, | |
270 | * 'da' indicates the device address where logs are written to, | |
271 | * and 'len' is the size of the trace buffer. | |
272 | * @RSC_VRING: request for allocation of a virtio vring (address should | |
273 | * be indicated in 'da', and 'len' should contain the number | |
274 | * of buffers supported by the vring). | |
275 | * @RSC_VIRTIO_DEV: announces support for a virtio device, and serves as | |
276 | * the virtio header. 'da' contains the virtio device | |
277 | * features, 'pa' holds the virtio guest features (host | |
278 | * will write them here after they're negotiated), 'len' | |
279 | * holds the virtio status, and 'flags' holds the virtio | |
280 | * device id (currently only VIRTIO_ID_RPMSG is supported). | |
281 | */ | |
282 | enum fw_resource_type { | |
283 | RSC_CARVEOUT = 0, | |
284 | RSC_DEVMEM = 1, | |
285 | RSC_TRACE = 2, | |
286 | RSC_VRING = 3, | |
287 | RSC_VIRTIO_DEV = 4, | |
288 | RSC_VIRTIO_CFG = 5, | |
289 | }; | |
290 | ||
291 | Most of the resource entries share the basic idea of address/length | |
292 | negotiation with the host: the firmware usually asks for memory | |
293 | of size 'len' bytes, and the host needs to allocate it and provide | |
294 | the device/physical address (when relevant) in 'da'/'pa' respectively. | |
295 | ||
296 | If the firmware is compiled with hard coded device addresses, and | |
297 | can't handle dynamically allocated 'da' values, then the 'da' field | |
298 | will contain the expected device addresses (today we actually only support | |
299 | this scheme, as there aren't yet any use cases for dynamically allocated | |
300 | device addresses). | |
301 | ||
302 | We also expect that platform-specific resource entries will show up | |
303 | at some point. When that happens, we could easily add a new RSC_PLAFORM | |
304 | type, and hand those resources to the platform-specific rproc driver to handle. | |
305 | ||
306 | 7. Virtio and remoteproc | |
307 | ||
308 | The firmware should provide remoteproc information about virtio devices | |
309 | that it supports, and their configurations: a RSC_VIRTIO_DEV resource entry | |
310 | should specify the virtio device id, and subsequent RSC_VRING resource entries | |
311 | should indicate the vring size (i.e. how many buffers do they support) and | |
312 | where should they be mapped (i.e. which device address). Note: the alignment | |
313 | between the consumer and producer parts of the vring is assumed to be 4096. | |
314 | ||
315 | At this point we only support a single virtio rpmsg device per remote | |
316 | processor, but the plan is to remove this limitation. In addition, once we | |
317 | move to TLV-based resource table, the plan is to have a single RSC_VIRTIO | |
318 | entry per supported virtio device, which will include the virtio header, | |
319 | the vrings information and the virtio config space. | |
320 | ||
321 | Of course, RSC_VIRTIO resource entries are only good enough for static | |
322 | allocation of virtio devices. Dynamic allocations will also be made possible | |
323 | using the rpmsg bus (similar to how we already do dynamic allocations of | |
324 | rpmsg channels; read more about it in rpmsg.txt). |