Commit | Line | Data |
---|---|---|
15b49bee YZ |
1 | PCI Express I/O Virtualization Howto |
2 | Copyright (C) 2009 Intel Corporation | |
3 | Yu Zhao <yu.zhao@intel.com> | |
4 | ||
2597ba76 DD |
5 | Update: November 2012 |
6 | -- sysfs-based SRIOV enable-/disable-ment | |
7 | Donald Dutile <ddutile@redhat.com> | |
15b49bee YZ |
8 | |
9 | 1. Overview | |
10 | ||
11 | 1.1 What is SR-IOV | |
12 | ||
13 | Single Root I/O Virtualization (SR-IOV) is a PCI Express Extended | |
14 | capability which makes one physical device appear as multiple virtual | |
15 | devices. The physical device is referred to as Physical Function (PF) | |
16 | while the virtual devices are referred to as Virtual Functions (VF). | |
17 | Allocation of the VF can be dynamically controlled by the PF via | |
18 | registers encapsulated in the capability. By default, this feature is | |
19 | not enabled and the PF behaves as traditional PCIe device. Once it's | |
20 | turned on, each VF's PCI configuration space can be accessed by its own | |
21 | Bus, Device and Function Number (Routing ID). And each VF also has PCI | |
22 | Memory Space, which is used to map its register set. VF device driver | |
23 | operates on the register set so it can be functional and appear as a | |
24 | real existing PCI device. | |
25 | ||
26 | 2. User Guide | |
27 | ||
28 | 2.1 How can I enable SR-IOV capability | |
29 | ||
2597ba76 DD |
30 | Multiple methods are available for SR-IOV enablement. |
31 | In the first method, the device driver (PF driver) will control the | |
32 | enabling and disabling of the capability via API provided by SR-IOV core. | |
33 | If the hardware has SR-IOV capability, loading its PF driver would | |
34 | enable it and all VFs associated with the PF. Some PF drivers require | |
35 | a module parameter to be set to determine the number of VFs to enable. | |
36 | In the second method, a write to the sysfs file sriov_numvfs will | |
37 | enable and disable the VFs associated with a PCIe PF. This method | |
38 | enables per-PF, VF enable/disable values versus the first method, | |
39 | which applies to all PFs of the same device. Additionally, the | |
40 | PCI SRIOV core support ensures that enable/disable operations are | |
41 | valid to reduce duplication in multiple drivers for the same | |
42 | checks, e.g., check numvfs == 0 if enabling VFs, ensure | |
43 | numvfs <= totalvfs. | |
44 | The second method is the recommended method for new/future VF devices. | |
15b49bee YZ |
45 | |
46 | 2.2 How can I use the Virtual Functions | |
47 | ||
48 | The VF is treated as hot-plugged PCI devices in the kernel, so they | |
49 | should be able to work in the same way as real PCI devices. The VF | |
50 | requires device driver that is same as a normal PCI device's. | |
51 | ||
52 | 3. Developer Guide | |
53 | ||
54 | 3.1 SR-IOV API | |
55 | ||
56 | To enable SR-IOV capability: | |
2597ba76 | 57 | (a) For the first method, in the driver: |
15b49bee YZ |
58 | int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn); |
59 | 'nr_virtfn' is number of VFs to be enabled. | |
2597ba76 DD |
60 | (b) For the second method, from sysfs: |
61 | echo 'nr_virtfn' > \ | |
62 | /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs | |
15b49bee YZ |
63 | |
64 | To disable SR-IOV capability: | |
2597ba76 | 65 | (a) For the first method, in the driver: |
15b49bee | 66 | void pci_disable_sriov(struct pci_dev *dev); |
2597ba76 DD |
67 | (b) For the second method, from sysfs: |
68 | echo 0 > \ | |
69 | /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_numvfs | |
15b49bee YZ |
70 | |
71 | To notify SR-IOV core of Virtual Function Migration: | |
2597ba76 | 72 | (a) In the driver: |
15b49bee YZ |
73 | irqreturn_t pci_sriov_migration(struct pci_dev *dev); |
74 | ||
75 | 3.2 Usage example | |
76 | ||
77 | Following piece of code illustrates the usage of the SR-IOV API. | |
78 | ||
63a29f74 | 79 | static int dev_probe(struct pci_dev *dev, const struct pci_device_id *id) |
15b49bee YZ |
80 | { |
81 | pci_enable_sriov(dev, NR_VIRTFN); | |
82 | ||
83 | ... | |
84 | ||
85 | return 0; | |
86 | } | |
87 | ||
63a29f74 | 88 | static void dev_remove(struct pci_dev *dev) |
15b49bee YZ |
89 | { |
90 | pci_disable_sriov(dev); | |
91 | ||
92 | ... | |
93 | } | |
94 | ||
95 | static int dev_suspend(struct pci_dev *dev, pm_message_t state) | |
96 | { | |
97 | ... | |
98 | ||
99 | return 0; | |
100 | } | |
101 | ||
102 | static int dev_resume(struct pci_dev *dev) | |
103 | { | |
104 | ... | |
105 | ||
106 | return 0; | |
107 | } | |
108 | ||
109 | static void dev_shutdown(struct pci_dev *dev) | |
110 | { | |
111 | ... | |
112 | } | |
113 | ||
2597ba76 DD |
114 | static int dev_sriov_configure(struct pci_dev *dev, int numvfs) |
115 | { | |
116 | if (numvfs > 0) { | |
117 | ... | |
118 | pci_enable_sriov(dev, numvfs); | |
119 | ... | |
120 | return numvfs; | |
121 | } | |
122 | if (numvfs == 0) { | |
123 | .... | |
124 | pci_disable_sriov(dev); | |
125 | ... | |
126 | return 0; | |
127 | } | |
128 | } | |
129 | ||
15b49bee YZ |
130 | static struct pci_driver dev_driver = { |
131 | .name = "SR-IOV Physical Function driver", | |
132 | .id_table = dev_id_table, | |
133 | .probe = dev_probe, | |
63a29f74 | 134 | .remove = dev_remove, |
15b49bee YZ |
135 | .suspend = dev_suspend, |
136 | .resume = dev_resume, | |
137 | .shutdown = dev_shutdown, | |
2597ba76 | 138 | .sriov_configure = dev_sriov_configure, |
15b49bee | 139 | }; |