Commit | Line | Data |
---|---|---|
1da177e4 | 1 | |
74da15eb GG |
2 | How To Write Linux PCI Drivers |
3 | ||
4 | by Martin Mares <mj@ucw.cz> on 07-Feb-2000 | |
5 | updated by Grant Grundler <grundler@parisc-linux.org> on 23-Dec-2006 | |
1da177e4 LT |
6 | |
7 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
74da15eb GG |
8 | The world of PCI is vast and full of (mostly unpleasant) surprises. |
9 | Since each CPU architecture implements different chip-sets and PCI devices | |
10 | have different requirements (erm, "features"), the result is the PCI support | |
11 | in the Linux kernel is not as trivial as one would wish. This short paper | |
12 | tries to introduce all potential driver authors to Linux APIs for | |
13 | PCI device drivers. | |
14 | ||
15 | A more complete resource is the third edition of "Linux Device Drivers" | |
16 | by Jonathan Corbet, Alessandro Rubini, and Greg Kroah-Hartman. | |
17 | LDD3 is available for free (under Creative Commons License) from: | |
18 | ||
19 | http://lwn.net/Kernel/LDD3/ | |
20 | ||
21 | However, keep in mind that all documents are subject to "bit rot". | |
22 | Refer to the source code if things are not working as described here. | |
23 | ||
24 | Please send questions/comments/patches about Linux PCI API to the | |
25 | "Linux PCI" <linux-pci@atrey.karlin.mff.cuni.cz> mailing list. | |
26 | ||
1da177e4 LT |
27 | |
28 | ||
29 | 0. Structure of PCI drivers | |
30 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
74da15eb GG |
31 | PCI drivers "discover" PCI devices in a system via pci_register_driver(). |
32 | Actually, it's the other way around. When the PCI generic code discovers | |
33 | a new device, the driver with a matching "description" will be notified. | |
34 | Details on this below. | |
35 | ||
36 | pci_register_driver() leaves most of the probing for devices to | |
37 | the PCI layer and supports online insertion/removal of devices [thus | |
38 | supporting hot-pluggable PCI, CardBus, and Express-Card in a single driver]. | |
39 | pci_register_driver() call requires passing in a table of function | |
40 | pointers and thus dictates the high level structure of a driver. | |
41 | ||
42 | Once the driver knows about a PCI device and takes ownership, the | |
43 | driver generally needs to perform the following initialization: | |
1da177e4 LT |
44 | |
45 | Enable the device | |
74da15eb GG |
46 | Request MMIO/IOP resources |
47 | Set the DMA mask size (for both coherent and streaming DMA) | |
48 | Allocate and initialize shared control data (pci_allocate_coherent()) | |
49 | Access device configuration space (if needed) | |
50 | Register IRQ handler (request_irq()) | |
51 | Initialize non-PCI (i.e. LAN/SCSI/etc parts of the chip) | |
52 | Enable DMA/processing engines | |
53 | ||
54 | When done using the device, and perhaps the module needs to be unloaded, | |
55 | the driver needs to take the follow steps: | |
56 | Disable the device from generating IRQs | |
57 | Release the IRQ (free_irq()) | |
58 | Stop all DMA activity | |
59 | Release DMA buffers (both streaming and coherent) | |
60 | Unregister from other subsystems (e.g. scsi or netdev) | |
61 | Release MMIO/IOP resources | |
1da177e4 LT |
62 | Disable the device |
63 | ||
74da15eb GG |
64 | Most of these topics are covered in the following sections. |
65 | For the rest look at LDD3 or <linux/pci.h> . | |
1da177e4 LT |
66 | |
67 | If the PCI subsystem is not configured (CONFIG_PCI is not set), most of | |
74da15eb GG |
68 | the PCI functions described below are defined as inline functions either |
69 | completely empty or just returning an appropriate error codes to avoid | |
70 | lots of ifdefs in the drivers. | |
71 | ||
1da177e4 LT |
72 | |
73 | ||
74da15eb GG |
74 | 1. pci_register_driver() call |
75 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
1da177e4 | 76 | |
74da15eb GG |
77 | PCI device drivers call pci_register_driver() during their |
78 | initialization with a pointer to a structure describing the driver | |
79 | (struct pci_driver): | |
80 | ||
81 | field name Description | |
82 | ---------- ------------------------------------------------------ | |
1da177e4 LT |
83 | id_table Pointer to table of device ID's the driver is |
84 | interested in. Most drivers should export this | |
85 | table using MODULE_DEVICE_TABLE(pci,...). | |
74da15eb GG |
86 | |
87 | probe This probing function gets called (during execution | |
88 | of pci_register_driver() for already existing | |
89 | devices or later if a new device gets inserted) for | |
90 | all PCI devices which match the ID table and are not | |
91 | "owned" by the other drivers yet. This function gets | |
92 | passed a "struct pci_dev *" for each device whose | |
93 | entry in the ID table matches the device. The probe | |
94 | function returns zero when the driver chooses to | |
95 | take "ownership" of the device or an error code | |
96 | (negative number) otherwise. | |
97 | The probe function always gets called from process | |
98 | context, so it can sleep. | |
99 | ||
100 | remove The remove() function gets called whenever a device | |
101 | being handled by this driver is removed (either during | |
102 | deregistration of the driver or when it's manually | |
103 | pulled out of a hot-pluggable slot). | |
104 | The remove function always gets called from process | |
105 | context, so it can sleep. | |
106 | ||
1da177e4 | 107 | suspend Put device into low power state. |
74da15eb GG |
108 | suspend_late Put device into low power state. |
109 | ||
110 | resume_early Wake device from low power state. | |
1da177e4 | 111 | resume Wake device from low power state. |
74da15eb GG |
112 | |
113 | (Please see Documentation/power/pci.txt for descriptions | |
114 | of PCI Power Management and the related functions.) | |
115 | ||
74da15eb GG |
116 | shutdown Hook into reboot_notifier_list (kernel/sys.c). |
117 | Intended to stop any idling DMA operations. | |
118 | Useful for enabling wake-on-lan (NIC) or changing | |
119 | the power state of a device before reboot. | |
120 | e.g. drivers/net/e100.c. | |
121 | ||
4b5ff469 | 122 | err_handler See Documentation/PCI/pci-error-recovery.txt |
74da15eb | 123 | |
1da177e4 | 124 | |
74da15eb | 125 | The ID table is an array of struct pci_device_id entries ending with an |
9f9351bb | 126 | all-zero entry; use of the macro DEFINE_PCI_DEVICE_TABLE is the preferred |
90a1ba0c | 127 | method of declaring the table. Each entry consists of: |
74da15eb GG |
128 | |
129 | vendor,device Vendor and device ID to match (or PCI_ANY_ID) | |
1da177e4 | 130 | |
1da177e4 | 131 | subvendor, Subsystem vendor and device ID to match (or PCI_ANY_ID) |
74da15eb GG |
132 | subdevice, |
133 | ||
134 | class Device class, subclass, and "interface" to match. | |
135 | See Appendix D of the PCI Local Bus Spec or | |
136 | include/linux/pci_ids.h for a full list of classes. | |
137 | Most drivers do not need to specify class/class_mask | |
138 | as vendor/device is normally sufficient. | |
139 | ||
140 | class_mask limit which sub-fields of the class field are compared. | |
141 | See drivers/scsi/sym53c8xx_2/ for example of usage. | |
142 | ||
1da177e4 | 143 | driver_data Data private to the driver. |
74da15eb GG |
144 | Most drivers don't need to use driver_data field. |
145 | Best practice is to use driver_data as an index | |
146 | into a static list of equivalent device types, | |
147 | instead of using it as a pointer. | |
1da177e4 | 148 | |
1da177e4 | 149 | |
74da15eb GG |
150 | Most drivers only need PCI_DEVICE() or PCI_DEVICE_CLASS() to set up |
151 | a pci_device_id table. | |
1da177e4 | 152 | |
74da15eb GG |
153 | New PCI IDs may be added to a device driver pci_ids table at runtime |
154 | as shown below: | |
1da177e4 LT |
155 | |
156 | echo "vendor device subvendor subdevice class class_mask driver_data" > \ | |
74da15eb GG |
157 | /sys/bus/pci/drivers/{driver}/new_id |
158 | ||
159 | All fields are passed in as hexadecimal values (no leading 0x). | |
6ba18636 JD |
160 | The vendor and device fields are mandatory, the others are optional. Users |
161 | need pass only as many optional fields as necessary: | |
162 | o subvendor and subdevice fields default to PCI_ANY_ID (FFFFFFFF) | |
74da15eb GG |
163 | o class and classmask fields default to 0 |
164 | o driver_data defaults to 0UL. | |
165 | ||
b41d6cf3 JD |
166 | Note that driver_data must match the value used by any of the pci_device_id |
167 | entries defined in the driver. This makes the driver_data field mandatory | |
168 | if all the pci_device_id entries have a non-zero driver_data value. | |
169 | ||
74da15eb GG |
170 | Once added, the driver probe routine will be invoked for any unclaimed |
171 | PCI devices listed in its (newly updated) pci_ids list. | |
1da177e4 LT |
172 | |
173 | When the driver exits, it just calls pci_unregister_driver() and the PCI layer | |
174 | automatically calls the remove hook for all devices handled by the driver. | |
175 | ||
74da15eb GG |
176 | |
177 | 1.1 "Attributes" for driver functions/data | |
178 | ||
1da177e4 LT |
179 | Please mark the initialization and cleanup functions where appropriate |
180 | (the corresponding macros are defined in <linux/init.h>): | |
181 | ||
182 | __init Initialization code. Thrown away after the driver | |
183 | initializes. | |
184 | __exit Exit code. Ignored for non-modular drivers. | |
74da15eb GG |
185 | |
186 | ||
187 | __devinit Device initialization code. | |
188 | Identical to __init if the kernel is not compiled | |
189 | with CONFIG_HOTPLUG, normal function otherwise. | |
1da177e4 LT |
190 | __devexit The same for __exit. |
191 | ||
74da15eb GG |
192 | Tips on when/where to use the above attributes: |
193 | o The module_init()/module_exit() functions (and all | |
194 | initialization functions called _only_ from these) | |
195 | should be marked __init/__exit. | |
1da177e4 | 196 | |
74da15eb | 197 | o Do not mark the struct pci_driver. |
1da177e4 | 198 | |
90a1ba0c | 199 | o The ID table array should be marked __devinitconst; this is done |
9f9351bb | 200 | automatically if the table is declared with DEFINE_PCI_DEVICE_TABLE(). |
1da177e4 | 201 | |
74da15eb GG |
202 | o The probe() and remove() functions should be marked __devinit |
203 | and __devexit respectively. All initialization functions | |
204 | exclusively called by the probe() routine, can be marked __devinit. | |
205 | Ditto for remove() and __devexit. | |
206 | ||
26ba05e4 GG |
207 | o If mydriver_remove() is marked with __devexit(), then all address |
208 | references to mydriver_remove must use __devexit_p(mydriver_remove) | |
74da15eb GG |
209 | (in the struct pci_driver declaration for example). |
210 | __devexit_p() will generate the function name _or_ NULL if the | |
211 | function will be discarded. For an example, see drivers/net/tg3.c. | |
212 | ||
213 | o Do NOT mark a function if you are not sure which mark to use. | |
214 | Better to not mark the function than mark the function wrong. | |
215 | ||
216 | ||
217 | ||
218 | 2. How to find PCI devices manually | |
219 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
220 | ||
221 | PCI drivers should have a really good reason for not using the | |
222 | pci_register_driver() interface to search for PCI devices. | |
223 | The main reason PCI devices are controlled by multiple drivers | |
224 | is because one PCI device implements several different HW services. | |
225 | E.g. combined serial/parallel port/floppy controller. | |
226 | ||
227 | A manual search may be performed using the following constructs: | |
1da177e4 LT |
228 | |
229 | Searching by vendor and device ID: | |
230 | ||
231 | struct pci_dev *dev = NULL; | |
232 | while (dev = pci_get_device(VENDOR_ID, DEVICE_ID, dev)) | |
233 | configure_device(dev); | |
234 | ||
235 | Searching by class ID (iterate in a similar way): | |
236 | ||
237 | pci_get_class(CLASS_ID, dev) | |
238 | ||
239 | Searching by both vendor/device and subsystem vendor/device ID: | |
240 | ||
74da15eb | 241 | pci_get_subsys(VENDOR_ID,DEVICE_ID, SUBSYS_VENDOR_ID, SUBSYS_DEVICE_ID, dev). |
1da177e4 | 242 | |
74da15eb | 243 | You can use the constant PCI_ANY_ID as a wildcard replacement for |
1da177e4 LT |
244 | VENDOR_ID or DEVICE_ID. This allows searching for any device from a |
245 | specific vendor, for example. | |
246 | ||
74da15eb | 247 | These functions are hotplug-safe. They increment the reference count on |
1da177e4 LT |
248 | the pci_dev that they return. You must eventually (possibly at module unload) |
249 | decrement the reference count on these devices by calling pci_dev_put(). | |
250 | ||
251 | ||
1da177e4 | 252 | |
74da15eb GG |
253 | 3. Device Initialization Steps |
254 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
255 | ||
256 | As noted in the introduction, most PCI drivers need the following steps | |
257 | for device initialization: | |
1da177e4 | 258 | |
74da15eb GG |
259 | Enable the device |
260 | Request MMIO/IOP resources | |
261 | Set the DMA mask size (for both coherent and streaming DMA) | |
262 | Allocate and initialize shared control data (pci_allocate_coherent()) | |
263 | Access device configuration space (if needed) | |
264 | Register IRQ handler (request_irq()) | |
265 | Initialize non-PCI (i.e. LAN/SCSI/etc parts of the chip) | |
266 | Enable DMA/processing engines. | |
267 | ||
268 | The driver can access PCI config space registers at any time. | |
269 | (Well, almost. When running BIST, config space can go away...but | |
270 | that will just result in a PCI Bus Master Abort and config reads | |
271 | will return garbage). | |
272 | ||
273 | ||
274 | 3.1 Enable the PCI device | |
275 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | |
276 | Before touching any device registers, the driver needs to enable | |
277 | the PCI device by calling pci_enable_device(). This will: | |
278 | o wake up the device if it was in suspended state, | |
279 | o allocate I/O and memory regions of the device (if BIOS did not), | |
280 | o allocate an IRQ (if BIOS did not). | |
281 | ||
282 | NOTE: pci_enable_device() can fail! Check the return value. | |
74da15eb GG |
283 | |
284 | [ OS BUG: we don't check resource allocations before enabling those | |
285 | resources. The sequence would make more sense if we called | |
286 | pci_request_resources() before calling pci_enable_device(). | |
287 | Currently, the device drivers can't detect the bug when when two | |
288 | devices have been allocated the same range. This is not a common | |
289 | problem and unlikely to get fixed soon. | |
290 | ||
291 | This has been discussed before but not changed as of 2.6.19: | |
292 | http://lkml.org/lkml/2006/3/2/194 | |
293 | ] | |
294 | ||
295 | pci_set_master() will enable DMA by setting the bus master bit | |
296 | in the PCI_COMMAND register. It also fixes the latency timer value if | |
6a479079 BH |
297 | it's set to something bogus by the BIOS. pci_clear_master() will |
298 | disable DMA by clearing the bus master bit. | |
74da15eb GG |
299 | |
300 | If the PCI device can use the PCI Memory-Write-Invalidate transaction, | |
1da177e4 LT |
301 | call pci_set_mwi(). This enables the PCI_COMMAND bit for Mem-Wr-Inval |
302 | and also ensures that the cache line size register is set correctly. | |
74da15eb | 303 | Check the return value of pci_set_mwi() as not all architectures |
694625c0 RD |
304 | or chip-sets may support Memory-Write-Invalidate. Alternatively, |
305 | if Mem-Wr-Inval would be nice to have but is not required, call | |
306 | pci_try_set_mwi() to have the system do its best effort at enabling | |
307 | Mem-Wr-Inval. | |
74da15eb GG |
308 | |
309 | ||
310 | 3.2 Request MMIO/IOP resources | |
311 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
312 | Memory (MMIO), and I/O port addresses should NOT be read directly | |
313 | from the PCI device config space. Use the values in the pci_dev structure | |
314 | as the PCI "bus address" might have been remapped to a "host physical" | |
315 | address by the arch/chip-set specific kernel support. | |
1da177e4 | 316 | |
395cf969 | 317 | See Documentation/io-mapping.txt for how to access device registers |
74da15eb GG |
318 | or device memory. |
319 | ||
320 | The device driver needs to call pci_request_region() to verify | |
321 | no other device is already using the same address resource. | |
322 | Conversely, drivers should call pci_release_region() AFTER | |
1da177e4 | 323 | calling pci_disable_device(). |
74da15eb GG |
324 | The idea is to prevent two devices colliding on the same address range. |
325 | ||
326 | [ See OS BUG comment above. Currently (2.6.19), The driver can only | |
327 | determine MMIO and IO Port resource availability _after_ calling | |
328 | pci_enable_device(). ] | |
329 | ||
330 | Generic flavors of pci_request_region() are request_mem_region() | |
331 | (for MMIO ranges) and request_region() (for IO Port ranges). | |
332 | Use these for address resources that are not described by "normal" PCI | |
333 | BARs. | |
334 | ||
335 | Also see pci_request_selected_regions() below. | |
336 | ||
337 | ||
338 | 3.3 Set the DMA mask size | |
339 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | |
340 | [ If anything below doesn't make sense, please refer to | |
341 | Documentation/DMA-API.txt. This section is just a reminder that | |
342 | drivers need to indicate DMA capabilities of the device and is not | |
343 | an authoritative source for DMA interfaces. ] | |
344 | ||
345 | While all drivers should explicitly indicate the DMA capability | |
346 | (e.g. 32 or 64 bit) of the PCI bus master, devices with more than | |
347 | 32-bit bus master capability for streaming data need the driver | |
348 | to "register" this capability by calling pci_set_dma_mask() with | |
349 | appropriate parameters. In general this allows more efficient DMA | |
350 | on systems where System RAM exists above 4G _physical_ address. | |
351 | ||
352 | Drivers for all PCI-X and PCIe compliant devices must call | |
353 | pci_set_dma_mask() as they are 64-bit DMA devices. | |
354 | ||
355 | Similarly, drivers must also "register" this capability if the device | |
356 | can directly address "consistent memory" in System RAM above 4G physical | |
357 | address by calling pci_set_consistent_dma_mask(). | |
358 | Again, this includes drivers for all PCI-X and PCIe compliant devices. | |
359 | Many 64-bit "PCI" devices (before PCI-X) and some PCI-X devices are | |
360 | 64-bit DMA capable for payload ("streaming") data but not control | |
361 | ("consistent") data. | |
362 | ||
363 | ||
364 | 3.4 Setup shared control data | |
365 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
366 | Once the DMA masks are set, the driver can allocate "consistent" (a.k.a. shared) | |
367 | memory. See Documentation/DMA-API.txt for a full description of | |
368 | the DMA APIs. This section is just a reminder that it needs to be done | |
369 | before enabling DMA on the device. | |
370 | ||
371 | ||
372 | 3.5 Initialize device registers | |
373 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
374 | Some drivers will need specific "capability" fields programmed | |
375 | or other "vendor specific" register initialized or reset. | |
376 | E.g. clearing pending interrupts. | |
377 | ||
378 | ||
379 | 3.6 Register IRQ handler | |
380 | ~~~~~~~~~~~~~~~~~~~~~~~~ | |
59c51591 | 381 | While calling request_irq() is the last step described here, |
74da15eb GG |
382 | this is often just another intermediate step to initialize a device. |
383 | This step can often be deferred until the device is opened for use. | |
384 | ||
385 | All interrupt handlers for IRQ lines should be registered with IRQF_SHARED | |
386 | and use the devid to map IRQs to devices (remember that all PCI IRQ lines | |
387 | can be shared). | |
388 | ||
389 | request_irq() will associate an interrupt handler and device handle | |
390 | with an interrupt number. Historically interrupt numbers represent | |
391 | IRQ lines which run from the PCI device to the Interrupt controller. | |
392 | With MSI and MSI-X (more below) the interrupt number is a CPU "vector". | |
393 | ||
394 | request_irq() also enables the interrupt. Make sure the device is | |
395 | quiesced and does not have any interrupts pending before registering | |
396 | the interrupt handler. | |
397 | ||
398 | MSI and MSI-X are PCI capabilities. Both are "Message Signaled Interrupts" | |
399 | which deliver interrupts to the CPU via a DMA write to a Local APIC. | |
400 | The fundamental difference between MSI and MSI-X is how multiple | |
401 | "vectors" get allocated. MSI requires contiguous blocks of vectors | |
402 | while MSI-X can allocate several individual ones. | |
403 | ||
404 | MSI capability can be enabled by calling pci_enable_msi() or | |
405 | pci_enable_msix() before calling request_irq(). This causes | |
406 | the PCI support to program CPU vector data into the PCI device | |
407 | capability registers. | |
408 | ||
409 | If your PCI device supports both, try to enable MSI-X first. | |
410 | Only one can be enabled at a time. Many architectures, chip-sets, | |
411 | or BIOSes do NOT support MSI or MSI-X and the call to pci_enable_msi/msix | |
412 | will fail. This is important to note since many drivers have | |
413 | two (or more) interrupt handlers: one for MSI/MSI-X and another for IRQs. | |
414 | They choose which handler to register with request_irq() based on the | |
415 | return value from pci_enable_msi/msix(). | |
416 | ||
417 | There are (at least) two really good reasons for using MSI: | |
418 | 1) MSI is an exclusive interrupt vector by definition. | |
419 | This means the interrupt handler doesn't have to verify | |
420 | its device caused the interrupt. | |
421 | ||
422 | 2) MSI avoids DMA/IRQ race conditions. DMA to host memory is guaranteed | |
423 | to be visible to the host CPU(s) when the MSI is delivered. This | |
424 | is important for both data coherency and avoiding stale control data. | |
425 | This guarantee allows the driver to omit MMIO reads to flush | |
426 | the DMA stream. | |
427 | ||
428 | See drivers/infiniband/hw/mthca/ or drivers/net/tg3.c for examples | |
429 | of MSI/MSI-X usage. | |
430 | ||
431 | ||
432 | ||
433 | 4. PCI device shutdown | |
434 | ~~~~~~~~~~~~~~~~~~~~~~~ | |
435 | ||
436 | When a PCI device driver is being unloaded, most of the following | |
437 | steps need to be performed: | |
438 | ||
439 | Disable the device from generating IRQs | |
440 | Release the IRQ (free_irq()) | |
441 | Stop all DMA activity | |
442 | Release DMA buffers (both streaming and consistent) | |
443 | Unregister from other subsystems (e.g. scsi or netdev) | |
444 | Disable device from responding to MMIO/IO Port addresses | |
445 | Release MMIO/IO Port resource(s) | |
446 | ||
447 | ||
448 | 4.1 Stop IRQs on the device | |
449 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
450 | How to do this is chip/device specific. If it's not done, it opens | |
451 | the possibility of a "screaming interrupt" if (and only if) | |
452 | the IRQ is shared with another device. | |
453 | ||
454 | When the shared IRQ handler is "unhooked", the remaining devices | |
455 | using the same IRQ line will still need the IRQ enabled. Thus if the | |
456 | "unhooked" device asserts IRQ line, the system will respond assuming | |
457 | it was one of the remaining devices asserted the IRQ line. Since none | |
458 | of the other devices will handle the IRQ, the system will "hang" until | |
459 | it decides the IRQ isn't going to get handled and masks the IRQ (100,000 | |
460 | iterations later). Once the shared IRQ is masked, the remaining devices | |
461 | will stop functioning properly. Not a nice situation. | |
462 | ||
463 | This is another reason to use MSI or MSI-X if it's available. | |
464 | MSI and MSI-X are defined to be exclusive interrupts and thus | |
465 | are not susceptible to the "screaming interrupt" problem. | |
466 | ||
467 | ||
468 | 4.2 Release the IRQ | |
469 | ~~~~~~~~~~~~~~~~~~~ | |
470 | Once the device is quiesced (no more IRQs), one can call free_irq(). | |
471 | This function will return control once any pending IRQs are handled, | |
472 | "unhook" the drivers IRQ handler from that IRQ, and finally release | |
473 | the IRQ if no one else is using it. | |
474 | ||
475 | ||
476 | 4.3 Stop all DMA activity | |
477 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | |
478 | It's extremely important to stop all DMA operations BEFORE attempting | |
479 | to deallocate DMA control data. Failure to do so can result in memory | |
480 | corruption, hangs, and on some chip-sets a hard crash. | |
1da177e4 | 481 | |
74da15eb GG |
482 | Stopping DMA after stopping the IRQs can avoid races where the |
483 | IRQ handler might restart DMA engines. | |
484 | ||
485 | While this step sounds obvious and trivial, several "mature" drivers | |
486 | didn't get this step right in the past. | |
487 | ||
488 | ||
489 | 4.4 Release DMA buffers | |
490 | ~~~~~~~~~~~~~~~~~~~~~~~ | |
491 | Once DMA is stopped, clean up streaming DMA first. | |
492 | I.e. unmap data buffers and return buffers to "upstream" | |
493 | owners if there is one. | |
494 | ||
495 | Then clean up "consistent" buffers which contain the control data. | |
496 | ||
497 | See Documentation/DMA-API.txt for details on unmapping interfaces. | |
498 | ||
499 | ||
500 | 4.5 Unregister from other subsystems | |
501 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
502 | Most low level PCI device drivers support some other subsystem | |
503 | like USB, ALSA, SCSI, NetDev, Infiniband, etc. Make sure your | |
504 | driver isn't losing resources from that other subsystem. | |
505 | If this happens, typically the symptom is an Oops (panic) when | |
506 | the subsystem attempts to call into a driver that has been unloaded. | |
507 | ||
508 | ||
509 | 4.6 Disable Device from responding to MMIO/IO Port addresses | |
510 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
511 | io_unmap() MMIO or IO Port resources and then call pci_disable_device(). | |
512 | This is the symmetric opposite of pci_enable_device(). | |
513 | Do not access device registers after calling pci_disable_device(). | |
514 | ||
515 | ||
516 | 4.7 Release MMIO/IO Port Resource(s) | |
517 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
518 | Call pci_release_region() to mark the MMIO or IO Port range as available. | |
519 | Failure to do so usually results in the inability to reload the driver. | |
520 | ||
521 | ||
522 | ||
523 | 5. How to access PCI config space | |
1da177e4 | 524 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
74da15eb GG |
525 | |
526 | You can use pci_(read|write)_config_(byte|word|dword) to access the config | |
1da177e4 LT |
527 | space of a device represented by struct pci_dev *. All these functions return 0 |
528 | when successful or an error code (PCIBIOS_...) which can be translated to a text | |
529 | string by pcibios_strerror. Most drivers expect that accesses to valid PCI | |
530 | devices don't fail. | |
531 | ||
74da15eb | 532 | If you don't have a struct pci_dev available, you can call |
1da177e4 LT |
533 | pci_bus_(read|write)_config_(byte|word|dword) to access a given device |
534 | and function on that bus. | |
535 | ||
74da15eb | 536 | If you access fields in the standard portion of the config header, please |
1da177e4 LT |
537 | use symbolic names of locations and bits declared in <linux/pci.h>. |
538 | ||
74da15eb | 539 | If you need to access Extended PCI Capability registers, just call |
1da177e4 LT |
540 | pci_find_capability() for the particular capability and it will find the |
541 | corresponding register block for you. | |
542 | ||
543 | ||
1da177e4 LT |
544 | |
545 | 6. Other interesting functions | |
546 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
74da15eb | 547 | |
1da177e4 LT |
548 | pci_find_slot() Find pci_dev corresponding to given bus and |
549 | slot numbers. | |
550 | pci_set_power_state() Set PCI Power Management state (0=D0 ... 3=D3) | |
551 | pci_find_capability() Find specified capability in device's capability | |
552 | list. | |
1da177e4 LT |
553 | pci_resource_start() Returns bus start address for a given PCI region |
554 | pci_resource_end() Returns bus end address for a given PCI region | |
555 | pci_resource_len() Returns the byte length of a PCI region | |
556 | pci_set_drvdata() Set private driver data pointer for a pci_dev | |
557 | pci_get_drvdata() Return private driver data pointer for a pci_dev | |
558 | pci_set_mwi() Enable Memory-Write-Invalidate transactions. | |
559 | pci_clear_mwi() Disable Memory-Write-Invalidate transactions. | |
560 | ||
561 | ||
74da15eb | 562 | |
1da177e4 LT |
563 | 7. Miscellaneous hints |
564 | ~~~~~~~~~~~~~~~~~~~~~~ | |
74da15eb GG |
565 | |
566 | When displaying PCI device names to the user (for example when a driver wants | |
567 | to tell the user what card has it found), please use pci_name(pci_dev). | |
1da177e4 LT |
568 | |
569 | Always refer to the PCI devices by a pointer to the pci_dev structure. | |
570 | All PCI layer functions use this identification and it's the only | |
571 | reasonable one. Don't use bus/slot/function numbers except for very | |
572 | special purposes -- on systems with multiple primary buses their semantics | |
573 | can be pretty complex. | |
574 | ||
1da177e4 LT |
575 | Don't try to turn on Fast Back to Back writes in your driver. All devices |
576 | on the bus need to be capable of doing it, so this is something which needs | |
577 | to be handled by platform and generic code, not individual drivers. | |
578 | ||
579 | ||
74da15eb | 580 | |
9b860b8c IO |
581 | 8. Vendor and device identifications |
582 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
9b860b8c | 583 | |
fa964e1a | 584 | One is not required to add new device ids to include/linux/pci_ids.h. |
74da15eb GG |
585 | Please add PCI_VENDOR_ID_xxx for vendors and a hex constant for device ids. |
586 | ||
587 | PCI_VENDOR_ID_xxx constants are re-used. The device ids are arbitrary | |
588 | hex numbers (vendor controlled) and normally used only in a single | |
589 | location, the pci_device_id table. | |
590 | ||
591 | Please DO submit new vendor/device ids to pciids.sourceforge.net project. | |
592 | ||
9b860b8c | 593 | |
9b860b8c IO |
594 | |
595 | 9. Obsolete functions | |
1da177e4 | 596 | ~~~~~~~~~~~~~~~~~~~~~ |
74da15eb | 597 | |
1da177e4 LT |
598 | There are several functions which you might come across when trying to |
599 | port an old driver to the new PCI interface. They are no longer present | |
600 | in the kernel as they aren't compatible with hotplug or PCI domains or | |
601 | having sane locking. | |
602 | ||
74da15eb GG |
603 | pci_find_device() Superseded by pci_get_device() |
604 | pci_find_subsys() Superseded by pci_get_subsys() | |
605 | pci_find_slot() Superseded by pci_get_slot() | |
606 | ||
607 | ||
608 | The alternative is the traditional PCI device driver that walks PCI | |
609 | device lists. This is still possible but discouraged. | |
610 | ||
611 | ||
612 | ||
d48b5d3a | 613 | 10. MMIO Space and "Write Posting" |
74da15eb GG |
614 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
615 | ||
616 | Converting a driver from using I/O Port space to using MMIO space | |
617 | often requires some additional changes. Specifically, "write posting" | |
618 | needs to be handled. Many drivers (e.g. tg3, acenic, sym53c8xx_2) | |
619 | already do this. I/O Port space guarantees write transactions reach the PCI | |
620 | device before the CPU can continue. Writes to MMIO space allow the CPU | |
621 | to continue before the transaction reaches the PCI device. HW weenies | |
622 | call this "Write Posting" because the write completion is "posted" to | |
623 | the CPU before the transaction has reached its destination. | |
624 | ||
625 | Thus, timing sensitive code should add readl() where the CPU is | |
626 | expected to wait before doing other work. The classic "bit banging" | |
627 | sequence works fine for I/O Port space: | |
628 | ||
629 | for (i = 8; --i; val >>= 1) { | |
630 | outb(val & 1, ioport_reg); /* write bit */ | |
631 | udelay(10); | |
632 | } | |
633 | ||
634 | The same sequence for MMIO space should be: | |
635 | ||
636 | for (i = 8; --i; val >>= 1) { | |
637 | writeb(val & 1, mmio_reg); /* write bit */ | |
638 | readb(safe_mmio_reg); /* flush posted write */ | |
639 | udelay(10); | |
640 | } | |
641 | ||
642 | It is important that "safe_mmio_reg" not have any side effects that | |
643 | interferes with the correct operation of the device. | |
644 | ||
645 | Another case to watch out for is when resetting a PCI device. Use PCI | |
646 | Configuration space reads to flush the writel(). This will gracefully | |
647 | handle the PCI master abort on all platforms if the PCI device is | |
648 | expected to not respond to a readl(). Most x86 platforms will allow | |
649 | MMIO reads to master abort (a.k.a. "Soft Fail") and return garbage | |
650 | (e.g. ~0). But many RISC platforms will crash (a.k.a."Hard Fail"). | |
651 |