Commit | Line | Data |
---|---|---|
32e62c63 BH |
1 | MEMORY ATTRIBUTE ALIASING ON IA-64 |
2 | ||
3 | Bjorn Helgaas | |
4 | <bjorn.helgaas@hp.com> | |
5 | May 4, 2006 | |
6 | ||
7 | ||
8 | MEMORY ATTRIBUTES | |
9 | ||
10 | Itanium supports several attributes for virtual memory references. | |
11 | The attribute is part of the virtual translation, i.e., it is | |
12 | contained in the TLB entry. The ones of most interest to the Linux | |
13 | kernel are: | |
14 | ||
15 | WB Write-back (cacheable) | |
16 | UC Uncacheable | |
17 | WC Write-coalescing | |
18 | ||
19 | System memory typically uses the WB attribute. The UC attribute is | |
20 | used for memory-mapped I/O devices. The WC attribute is uncacheable | |
21 | like UC is, but writes may be delayed and combined to increase | |
22 | performance for things like frame buffers. | |
23 | ||
24 | The Itanium architecture requires that we avoid accessing the same | |
25 | page with both a cacheable mapping and an uncacheable mapping[1]. | |
26 | ||
27 | The design of the chipset determines which attributes are supported | |
28 | on which regions of the address space. For example, some chipsets | |
29 | support either WB or UC access to main memory, while others support | |
30 | only WB access. | |
31 | ||
32 | MEMORY MAP | |
33 | ||
34 | Platform firmware describes the physical memory map and the | |
35 | supported attributes for each region. At boot-time, the kernel uses | |
36 | the EFI GetMemoryMap() interface. ACPI can also describe memory | |
37 | devices and the attributes they support, but Linux/ia64 currently | |
38 | doesn't use this information. | |
39 | ||
40 | The kernel uses the efi_memmap table returned from GetMemoryMap() to | |
41 | learn the attributes supported by each region of physical address | |
42 | space. Unfortunately, this table does not completely describe the | |
43 | address space because some machines omit some or all of the MMIO | |
44 | regions from the map. | |
45 | ||
46 | The kernel maintains another table, kern_memmap, which describes the | |
47 | memory Linux is actually using and the attribute for each region. | |
48 | This contains only system memory; it does not contain MMIO space. | |
49 | ||
50 | The kern_memmap table typically contains only a subset of the system | |
51 | memory described by the efi_memmap. Linux/ia64 can't use all memory | |
52 | in the system because of constraints imposed by the identity mapping | |
53 | scheme. | |
54 | ||
55 | The efi_memmap table is preserved unmodified because the original | |
56 | boot-time information is required for kexec. | |
57 | ||
58 | KERNEL IDENTITY MAPPINGS | |
59 | ||
60 | Linux/ia64 identity mappings are done with large pages, currently | |
61 | either 16MB or 64MB, referred to as "granules." Cacheable mappings | |
62 | are speculative[2], so the processor can read any location in the | |
63 | page at any time, independent of the programmer's intentions. This | |
64 | means that to avoid attribute aliasing, Linux can create a cacheable | |
65 | identity mapping only when the entire granule supports cacheable | |
66 | access. | |
67 | ||
68 | Therefore, kern_memmap contains only full granule-sized regions that | |
69 | can referenced safely by an identity mapping. | |
70 | ||
71 | Uncacheable mappings are not speculative, so the processor will | |
72 | generate UC accesses only to locations explicitly referenced by | |
73 | software. This allows UC identity mappings to cover granules that | |
74 | are only partially populated, or populated with a combination of UC | |
75 | and WB regions. | |
76 | ||
77 | USER MAPPINGS | |
78 | ||
79 | User mappings are typically done with 16K or 64K pages. The smaller | |
80 | page size allows more flexibility because only 16K or 64K has to be | |
81 | homogeneous with respect to memory attributes. | |
82 | ||
83 | POTENTIAL ATTRIBUTE ALIASING CASES | |
84 | ||
85 | There are several ways the kernel creates new mappings: | |
86 | ||
87 | mmap of /dev/mem | |
88 | ||
89 | This uses remap_pfn_range(), which creates user mappings. These | |
90 | mappings may be either WB or UC. If the region being mapped | |
91 | happens to be in kern_memmap, meaning that it may also be mapped | |
92 | by a kernel identity mapping, the user mapping must use the same | |
93 | attribute as the kernel mapping. | |
94 | ||
95 | If the region is not in kern_memmap, the user mapping should use | |
96 | an attribute reported as being supported in the EFI memory map. | |
97 | ||
98 | Since the EFI memory map does not describe MMIO on some | |
99 | machines, this should use an uncacheable mapping as a fallback. | |
100 | ||
101 | mmap of /sys/class/pci_bus/.../legacy_mem | |
102 | ||
103 | This is very similar to mmap of /dev/mem, except that legacy_mem | |
104 | only allows mmap of the one megabyte "legacy MMIO" area for a | |
105 | specific PCI bus. Typically this is the first megabyte of | |
106 | physical address space, but it may be different on machines with | |
107 | several VGA devices. | |
108 | ||
109 | "X" uses this to access VGA frame buffers. Using legacy_mem | |
110 | rather than /dev/mem allows multiple instances of X to talk to | |
111 | different VGA cards. | |
112 | ||
113 | The /dev/mem mmap constraints apply. | |
114 | ||
115 | However, since this is for mapping legacy MMIO space, WB access | |
116 | does not make sense. This matters on machines without legacy | |
117 | VGA support: these machines may have WB memory for the entire | |
118 | first megabyte (or even the entire first granule). | |
119 | ||
120 | On these machines, we could mmap legacy_mem as WB, which would | |
121 | be safe in terms of attribute aliasing, but X has no way of | |
122 | knowing that it is accessing regular memory, not a frame buffer, | |
123 | so the kernel should fail the mmap rather than doing it with WB. | |
124 | ||
125 | read/write of /dev/mem | |
126 | ||
127 | This uses copy_from_user(), which implicitly uses a kernel | |
128 | identity mapping. This is obviously safe for things in | |
129 | kern_memmap. | |
130 | ||
131 | There may be corner cases of things that are not in kern_memmap, | |
132 | but could be accessed this way. For example, registers in MMIO | |
133 | space are not in kern_memmap, but could be accessed with a UC | |
134 | mapping. This would not cause attribute aliasing. But | |
135 | registers typically can be accessed only with four-byte or | |
136 | eight-byte accesses, and the copy_from_user() path doesn't allow | |
137 | any control over the access size, so this would be dangerous. | |
138 | ||
139 | ioremap() | |
140 | ||
141 | This returns a kernel identity mapping for use inside the | |
142 | kernel. | |
143 | ||
144 | If the region is in kern_memmap, we should use the attribute | |
145 | specified there. Otherwise, if the EFI memory map reports that | |
146 | the entire granule supports WB, we should use that (granules | |
147 | that are partially reserved or occupied by firmware do not appear | |
148 | in kern_memmap). Otherwise, we should use a UC mapping. | |
149 | ||
150 | PAST PROBLEM CASES | |
151 | ||
152 | mmap of various MMIO regions from /dev/mem by "X" on Intel platforms | |
153 | ||
154 | The EFI memory map may not report these MMIO regions. | |
155 | ||
156 | These must be allowed so that X will work. This means that | |
157 | when the EFI memory map is incomplete, every /dev/mem mmap must | |
158 | succeed. It may create either WB or UC user mappings, depending | |
159 | on whether the region is in kern_memmap or the EFI memory map. | |
160 | ||
161 | mmap of 0x0-0xA0000 /dev/mem by "hwinfo" on HP sx1000 with VGA enabled | |
162 | ||
163 | See https://bugzilla.novell.com/show_bug.cgi?id=140858. | |
164 | ||
165 | The EFI memory map reports the following attributes: | |
166 | 0x00000-0x9FFFF WB only | |
167 | 0xA0000-0xBFFFF UC only (VGA frame buffer) | |
168 | 0xC0000-0xFFFFF WB only | |
169 | ||
170 | This mmap is done with user pages, not kernel identity mappings, | |
171 | so it is safe to use WB mappings. | |
172 | ||
173 | The kernel VGA driver may ioremap the VGA frame buffer at 0xA0000, | |
174 | which will use a granule-sized UC mapping covering 0-0xFFFFF. This | |
175 | granule covers some WB-only memory, but since UC is non-speculative, | |
176 | the processor will never generate an uncacheable reference to the | |
177 | WB-only areas unless the driver explicitly touches them. | |
178 | ||
179 | mmap of 0x0-0xFFFFF legacy_mem by "X" | |
180 | ||
181 | If the EFI memory map reports this entire range as WB, there | |
182 | is no VGA MMIO hole, and the mmap should fail or be done with | |
183 | a WB mapping. | |
184 | ||
185 | There's no easy way for X to determine whether the 0xA0000-0xBFFFF | |
186 | region is a frame buffer or just memory, so I think it's best to | |
187 | just fail this mmap request rather than using a WB mapping. As | |
188 | far as I know, there's no need to map legacy_mem with WB | |
189 | mappings. | |
190 | ||
191 | Otherwise, a UC mapping of the entire region is probably safe. | |
192 | The VGA hole means the region will not be in kern_memmap. The | |
193 | HP sx1000 chipset doesn't support UC access to the memory surrounding | |
194 | the VGA hole, but X doesn't need that area anyway and should not | |
195 | reference it. | |
196 | ||
197 | mmap of 0xA0000-0xBFFFF legacy_mem by "X" on HP sx1000 with VGA disabled | |
198 | ||
199 | The EFI memory map reports the following attributes: | |
200 | 0x00000-0xFFFFF WB only (no VGA MMIO hole) | |
201 | ||
202 | This is a special case of the previous case, and the mmap should | |
203 | fail for the same reason as above. | |
204 | ||
205 | NOTES | |
206 | ||
207 | [1] SDM rev 2.2, vol 2, sec 4.4.1. | |
208 | [2] SDM rev 2.2, vol 2, sec 4.4.6. |