Commit | Line | Data |
---|---|---|
5ba9f198 | 1 | |
4767a9e7 | 2 | RFC: Common Trace Format (CTF) Proposal (pre-v1.7) |
5ba9f198 MD |
3 | |
4 | Mathieu Desnoyers, EfficiOS Inc. | |
5 | ||
6 | The goal of the present document is to propose a trace format that suits the | |
cc089c3a | 7 | needs of the embedded, telecom, high-performance and kernel communities. It is |
5ba9f198 | 8 | based on the Common Trace Format Requirements (v1.4) document. It is designed to |
cc089c3a | 9 | allow traces to be natively generated by the Linux kernel, Linux user-space |
6c7226e9 MD |
10 | applications written in C/C++, and hardware components. One major element of |
11 | CTF is the Trace Stream Description Language (TSDL) which flexibility | |
12 | enables description of various binary trace stream layouts. | |
cc089c3a MD |
13 | |
14 | The latest version of this document can be found at: | |
15 | ||
16 | git tree: git://git.efficios.com/ctf.git | |
17 | gitweb: http://git.efficios.com/?p=ctf.git | |
5ba9f198 MD |
18 | |
19 | A reference implementation of a library to read and write this trace format is | |
20 | being implemented within the BabelTrace project, a converter between trace | |
21 | formats. The development tree is available at: | |
22 | ||
23 | git tree: git://git.efficios.com/babeltrace.git | |
24 | gitweb: http://git.efficios.com/?p=babeltrace.git | |
25 | ||
26 | ||
27 | 1. Preliminary definitions | |
28 | ||
3bf79539 MD |
29 | - Event Trace: An ordered sequence of events. |
30 | - Event Stream: An ordered sequence of events, containing a subset of the | |
31 | trace event types. | |
32 | - Event Packet: A sequence of physically contiguous events within an event | |
33 | stream. | |
5ba9f198 MD |
34 | - Event: This is the basic entry in a trace. (aka: a trace record). |
35 | - An event identifier (ID) relates to the class (a type) of event within | |
3bf79539 MD |
36 | an event stream. |
37 | e.g. event: irq_entry. | |
5ba9f198 MD |
38 | - An event (or event record) relates to a specific instance of an event |
39 | class. | |
3bf79539 MD |
40 | e.g. event: irq_entry, at time X, on CPU Y |
41 | - Source Architecture: Architecture writing the trace. | |
42 | - Reader Architecture: Architecture reading the trace. | |
5ba9f198 MD |
43 | |
44 | ||
45 | 2. High-level representation of a trace | |
46 | ||
3bf79539 MD |
47 | A trace is divided into multiple event streams. Each event stream contains a |
48 | subset of the trace event types. | |
5ba9f198 | 49 | |
3bf79539 MD |
50 | The final output of the trace, after its generation and optional transport over |
51 | the network, is expected to be either on permanent or temporary storage in a | |
52 | virtual file system. Because each event stream is appended to while a trace is | |
53 | being recorded, each is associated with a separate file for output. Therefore, | |
54 | a stored trace can be represented as a directory containing one file per stream. | |
5ba9f198 | 55 | |
6672e9e1 MD |
56 | Meta-data description associated with the trace contains information on |
57 | trace event types expressed in the Trace Stream Description Language | |
58 | (TSDL). This language describes: | |
5ba9f198 MD |
59 | |
60 | - Trace version. | |
61 | - Types available. | |
6672e9e1 | 62 | - Per-trace event header description. |
3bf79539 | 63 | - Per-stream event header description. |
6672e9e1 | 64 | - Per-stream event context description. |
5ba9f198 | 65 | - Per-event |
3bf79539 | 66 | - Event type to stream mapping. |
5ba9f198 MD |
67 | - Event type to name mapping. |
68 | - Event type to ID mapping. | |
6672e9e1 | 69 | - Event context description. |
5ba9f198 MD |
70 | - Event fields description. |
71 | ||
72 | ||
3bf79539 | 73 | 3. Event stream |
5ba9f198 | 74 | |
6672e9e1 MD |
75 | An event stream can be divided into contiguous event packets of variable |
76 | size. These subdivisions have a variable size. An event packet can | |
77 | contain a certain amount of padding at the end. The stream header is | |
78 | repeated at the beginning of each event packet. The rationale for the | |
79 | event stream design choices is explained in Appendix B. Stream Header | |
80 | Rationale. | |
5ba9f198 | 81 | |
3bf79539 MD |
82 | The event stream header will therefore be referred to as the "event packet |
83 | header" throughout the rest of this document. | |
5ba9f198 MD |
84 | |
85 | ||
86 | 4. Types | |
87 | ||
1fad7a85 MD |
88 | Types are organized as type classes. Each type class belong to either of two |
89 | kind of types: basic types or compound types. | |
90 | ||
5ba9f198 MD |
91 | 4.1 Basic types |
92 | ||
1fad7a85 MD |
93 | A basic type is a scalar type, as described in this section. It includes |
94 | integers, GNU/C bitfields, enumerations, and floating point values. | |
5ba9f198 MD |
95 | |
96 | 4.1.1 Type inheritance | |
97 | ||
80fd2569 MD |
98 | Type specifications can be inherited to allow deriving types from a |
99 | type class. For example, see the uint32_t named type derived from the "integer" | |
100 | type class below ("Integers" section). Types have a precise binary | |
101 | representation in the trace. A type class has methods to read and write these | |
102 | types, but must be derived into a type to be usable in an event field. | |
5ba9f198 MD |
103 | |
104 | 4.1.2 Alignment | |
105 | ||
106 | We define "byte-packed" types as aligned on the byte size, namely 8-bit. | |
107 | We define "bit-packed" types as following on the next bit, as defined by the | |
370eae99 | 108 | "Integers" section. |
5ba9f198 | 109 | |
6672e9e1 | 110 | Each basic type must specify its alignment, in bits. Examples of |
ec4404a7 MD |
111 | possible alignments are: bit-packed (align = 1), byte-packed (align = |
112 | 8), or word-aligned (e.g. align = 32 or align = 64). The choice depends | |
113 | on the architecture preference and compactness vs performance trade-offs | |
114 | of the implementation. Architectures providing fast unaligned write | |
115 | byte-packed basic types to save space, aligning each type on byte | |
116 | boundaries (8-bit). Architectures with slow unaligned writes align types | |
117 | on specific alignment values. If no specific alignment is declared for a | |
118 | type, it is assumed to be bit-packed for integers with size not multiple | |
119 | of 8 bits and for gcc bitfields. All other basic types are byte-packed | |
120 | by default. It is however recommended to always specify the alignment | |
121 | explicitly. Alignment values must be power of two. Compound types are | |
122 | aligned as specified in their individual specification. | |
5ba9f198 | 123 | |
6672e9e1 | 124 | TSDL meta-data attribute representation of a specific alignment: |
5ba9f198 MD |
125 | |
126 | align = value; /* value in bits */ | |
127 | ||
128 | 4.1.3 Byte order | |
129 | ||
3bf79539 MD |
130 | By default, the native endianness of the source architecture the trace is used. |
131 | Byte order can be overridden for a basic type by specifying a "byte_order" | |
132 | attribute. Typical use-case is to specify the network byte order (big endian: | |
133 | "be") to save data captured from the network into the trace without conversion. | |
134 | If not specified, the byte order is native. | |
5ba9f198 | 135 | |
6672e9e1 | 136 | TSDL meta-data representation: |
5ba9f198 MD |
137 | |
138 | byte_order = native OR network OR be OR le; /* network and be are aliases */ | |
139 | ||
140 | 4.1.4 Size | |
141 | ||
142 | Type size, in bits, for integers and floats is that returned by "sizeof()" in C | |
143 | multiplied by CHAR_BIT. | |
144 | We require the size of "char" and "unsigned char" types (CHAR_BIT) to be fixed | |
145 | to 8 bits for cross-endianness compatibility. | |
146 | ||
6672e9e1 | 147 | TSDL meta-data representation: |
5ba9f198 MD |
148 | |
149 | size = value; (value is in bits) | |
150 | ||
151 | 4.1.5 Integers | |
152 | ||
6672e9e1 MD |
153 | Signed integers are represented in two-complement. Integer alignment, |
154 | size, signedness and byte ordering are defined in the TSDL meta-data. | |
155 | Integers aligned on byte size (8-bit) and with length multiple of byte | |
156 | size (8-bit) correspond to the C99 standard integers. In addition, | |
157 | integers with alignment and/or size that are _not_ a multiple of the | |
158 | byte size are permitted; these correspond to the C99 standard bitfields, | |
159 | with the added specification that the CTF integer bitfields have a fixed | |
160 | binary representation. A MIT-licensed reference implementation of the | |
161 | CTF portable bitfields is available at: | |
5ba9f198 MD |
162 | |
163 | http://git.efficios.com/?p=babeltrace.git;a=blob;f=include/babeltrace/bitfield.h | |
164 | ||
165 | Binary representation of integers: | |
166 | ||
167 | - On little and big endian: | |
168 | - Within a byte, high bits correspond to an integer high bits, and low bits | |
169 | correspond to low bits. | |
170 | - On little endian: | |
171 | - Integer across multiple bytes are placed from the less significant to the | |
172 | most significant. | |
173 | - Consecutive integers are placed from lower bits to higher bits (even within | |
174 | a byte). | |
175 | - On big endian: | |
176 | - Integer across multiple bytes are placed from the most significant to the | |
177 | less significant. | |
178 | - Consecutive integers are placed from higher bits to lower bits (even within | |
179 | a byte). | |
180 | ||
181 | This binary representation is derived from the bitfield implementation in GCC | |
182 | for little and big endian. However, contrary to what GCC does, integers can | |
6672e9e1 | 183 | cross units boundaries (no padding is required). Padding can be explicitly |
5ba9f198 MD |
184 | added (see 4.1.6 GNU/C bitfields) to follow the GCC layout if needed. |
185 | ||
6672e9e1 | 186 | TSDL meta-data representation: |
5ba9f198 | 187 | |
80fd2569 | 188 | integer { |
5ba9f198 MD |
189 | signed = true OR false; /* default false */ |
190 | byte_order = native OR network OR be OR le; /* default native */ | |
191 | size = value; /* value in bits, no default */ | |
192 | align = value; /* value in bits */ | |
2152348f | 193 | } |
5ba9f198 | 194 | |
80fd2569 | 195 | Example of type inheritance (creation of a uint32_t named type): |
5ba9f198 | 196 | |
359894ac | 197 | typealias integer { |
9e4e34e9 | 198 | size = 32; |
5ba9f198 MD |
199 | signed = false; |
200 | align = 32; | |
38b8da21 | 201 | } := uint32_t; |
5ba9f198 | 202 | |
80fd2569 | 203 | Definition of a named 5-bit signed bitfield: |
5ba9f198 | 204 | |
359894ac | 205 | typealias integer { |
5ba9f198 MD |
206 | size = 5; |
207 | signed = true; | |
208 | align = 1; | |
38b8da21 | 209 | } := int5_t; |
5ba9f198 MD |
210 | |
211 | 4.1.6 GNU/C bitfields | |
212 | ||
213 | The GNU/C bitfields follow closely the integer representation, with a | |
214 | particularity on alignment: if a bitfield cannot fit in the current unit, the | |
80fd2569 MD |
215 | unit is padded and the bitfield starts at the following unit. The unit size is |
216 | defined by the size of the type "unit_type". | |
5ba9f198 | 217 | |
6672e9e1 | 218 | TSDL meta-data representation: |
80fd2569 MD |
219 | |
220 | unit_type name:size: | |
221 | ||
5ba9f198 MD |
222 | As an example, the following structure declared in C compiled by GCC: |
223 | ||
224 | struct example { | |
225 | short a:12; | |
226 | short b:5; | |
227 | }; | |
228 | ||
2152348f MD |
229 | The example structure is aligned on the largest element (short). The second |
230 | bitfield would be aligned on the next unit boundary, because it would not fit in | |
231 | the current unit. | |
5ba9f198 MD |
232 | |
233 | 4.1.7 Floating point | |
234 | ||
6672e9e1 | 235 | The floating point values byte ordering is defined in the TSDL meta-data. |
5ba9f198 MD |
236 | |
237 | Floating point values follow the IEEE 754-2008 standard interchange formats. | |
238 | Description of the floating point values include the exponent and mantissa size | |
239 | in bits. Some requirements are imposed on the floating point values: | |
240 | ||
241 | - FLT_RADIX must be 2. | |
242 | - mant_dig is the number of digits represented in the mantissa. It is specified | |
243 | by the ISO C99 standard, section 5.2.4, as FLT_MANT_DIG, DBL_MANT_DIG and | |
244 | LDBL_MANT_DIG as defined by <float.h>. | |
245 | - exp_dig is the number of digits represented in the exponent. Given that | |
246 | mant_dig is one bit more than its actual size in bits (leading 1 is not | |
247 | needed) and also given that the sign bit always takes one bit, exp_dig can be | |
248 | specified as: | |
249 | ||
250 | - sizeof(float) * CHAR_BIT - FLT_MANT_DIG | |
251 | - sizeof(double) * CHAR_BIT - DBL_MANT_DIG | |
252 | - sizeof(long double) * CHAR_BIT - LDBL_MANT_DIG | |
253 | ||
6672e9e1 | 254 | TSDL meta-data representation: |
5ba9f198 | 255 | |
80fd2569 | 256 | floating_point { |
ec4404a7 MD |
257 | exp_dig = value; |
258 | mant_dig = value; | |
259 | byte_order = native OR network OR be OR le; | |
260 | align = value; | |
2152348f | 261 | } |
5ba9f198 MD |
262 | |
263 | Example of type inheritance: | |
264 | ||
359894ac | 265 | typealias floating_point { |
5ba9f198 MD |
266 | exp_dig = 8; /* sizeof(float) * CHAR_BIT - FLT_MANT_DIG */ |
267 | mant_dig = 24; /* FLT_MANT_DIG */ | |
268 | byte_order = native; | |
ec4404a7 | 269 | align = 32; |
38b8da21 | 270 | } := float; |
5ba9f198 MD |
271 | |
272 | TODO: define NaN, +inf, -inf behavior. | |
273 | ||
ec4404a7 MD |
274 | Bit-packed, byte-packed or larger alignments can be used for floating |
275 | point values, similarly to integers. | |
276 | ||
5ba9f198 MD |
277 | 4.1.8 Enumerations |
278 | ||
279 | Enumerations are a mapping between an integer type and a table of strings. The | |
280 | numerical representation of the enumeration follows the integer type specified | |
6672e9e1 MD |
281 | by the meta-data. The enumeration mapping table is detailed in the enumeration |
282 | description within the meta-data. The mapping table maps inclusive value | |
283 | ranges (or single values) to strings. Instead of being limited to simple | |
3bf79539 | 284 | "value -> string" mappings, these enumerations map |
80fd2569 | 285 | "[ start_value ... end_value ] -> string", which map inclusive ranges of |
3bf79539 MD |
286 | values to strings. An enumeration from the C language can be represented in |
287 | this format by having the same start_value and end_value for each element, which | |
288 | is in fact a range of size 1. This single-value range is supported without | |
4767a9e7 | 289 | repeating the start and end values with the value = string declaration. |
80fd2569 | 290 | |
a9b83695 | 291 | enum name : integer_type { |
359894ac | 292 | somestring = start_value1 ... end_value1, |
80fd2569 MD |
293 | "other string" = start_value2 ... end_value2, |
294 | yet_another_string, /* will be assigned to end_value2 + 1 */ | |
295 | "some other string" = value, | |
296 | ... | |
297 | }; | |
298 | ||
299 | If the values are omitted, the enumeration starts at 0 and increment of 1 for | |
300 | each entry: | |
301 | ||
a9b83695 | 302 | enum name : unsigned int { |
80fd2569 MD |
303 | ZERO, |
304 | ONE, | |
305 | TWO, | |
306 | TEN = 10, | |
307 | ELEVEN, | |
3bf79539 | 308 | }; |
5ba9f198 | 309 | |
80fd2569 | 310 | Overlapping ranges within a single enumeration are implementation defined. |
5ba9f198 | 311 | |
2152348f MD |
312 | A nameless enumeration can be declared as a field type or as part of a typedef: |
313 | ||
a9b83695 | 314 | enum : integer_type { |
2152348f MD |
315 | ... |
316 | } | |
317 | ||
c2742c56 MD |
318 | Enumerations omitting the container type ": integer_type" use the "int" |
319 | type (for compatibility with C99). The "int" type must be previously | |
320 | declared. E.g.: | |
321 | ||
322 | typealias integer { size = 32; align = 32; signed = true } := int; | |
323 | ||
324 | enum { | |
325 | ... | |
326 | } | |
327 | ||
1fad7a85 | 328 | |
5ba9f198 MD |
329 | 4.2 Compound types |
330 | ||
1fad7a85 MD |
331 | Compound are aggregation of type declarations. Compound types include |
332 | structures, variant, arrays, sequences, and strings. | |
333 | ||
5ba9f198 MD |
334 | 4.2.1 Structures |
335 | ||
336 | Structures are aligned on the largest alignment required by basic types | |
337 | contained within the structure. (This follows the ISO/C standard for structures) | |
338 | ||
6672e9e1 | 339 | TSDL meta-data representation of a named structure: |
5ba9f198 | 340 | |
80fd2569 MD |
341 | struct name { |
342 | field_type field_name; | |
343 | field_type field_name; | |
344 | ... | |
345 | }; | |
5ba9f198 MD |
346 | |
347 | Example: | |
348 | ||
80fd2569 MD |
349 | struct example { |
350 | integer { /* Nameless type */ | |
351 | size = 16; | |
352 | signed = true; | |
353 | align = 16; | |
354 | } first_field_name; | |
6672e9e1 | 355 | uint64_t second_field_name; /* Named type declared in the meta-data */ |
3bf79539 | 356 | }; |
5ba9f198 MD |
357 | |
358 | The fields are placed in a sequence next to each other. They each possess a | |
359 | field name, which is a unique identifier within the structure. | |
360 | ||
2152348f | 361 | A nameless structure can be declared as a field type or as part of a typedef: |
80fd2569 MD |
362 | |
363 | struct { | |
364 | ... | |
2152348f | 365 | } |
80fd2569 | 366 | |
ec4404a7 MD |
367 | Alignment for a structure compound type can be forced to a minimum value |
368 | by adding an "align" specifier after the declaration of a structure | |
369 | body. This attribute is read as: align(value). The value is specified in | |
370 | bits. The structure will be aligned on the maximum value between this | |
371 | attribute and the alignment required by the basic types contained within | |
372 | the structure. e.g. | |
373 | ||
374 | struct { | |
375 | ... | |
376 | } align(32) | |
377 | ||
77a98c82 | 378 | 4.2.2 Variants (Discriminated/Tagged Unions) |
fcba70d4 | 379 | |
fdf2bb05 MD |
380 | A CTF variant is a selection between different types. A CTF variant must |
381 | always be defined within the scope of a structure or within fields | |
382 | contained within a structure (defined recursively). A "tag" enumeration | |
383 | field must appear in either the same lexical scope, prior to the variant | |
384 | field (in field declaration order), in an uppermost lexical scope (see | |
6c7226e9 | 385 | Section 7.3.1), or in an uppermost dynamic scope (see Section 7.3.2). |
fdf2bb05 MD |
386 | The type selection is indicated by the mapping from the enumeration |
387 | value to the string used as variant type selector. The field to use as | |
388 | tag is specified by the "tag_field", specified between "< >" after the | |
389 | "variant" keyword for unnamed variants, and after "variant name" for | |
390 | named variants. | |
fcba70d4 MD |
391 | |
392 | The alignment of the variant is the alignment of the type as selected by the tag | |
393 | value for the specific instance of the variant. The alignment of the type | |
394 | containing the variant is independent of the variant alignment. The size of the | |
395 | variant is the size as selected by the tag value for the specific instance of | |
396 | the variant. | |
397 | ||
398 | A named variant declaration followed by its definition within a structure | |
399 | declaration: | |
400 | ||
401 | variant name { | |
402 | field_type sel1; | |
403 | field_type sel2; | |
404 | field_type sel3; | |
405 | ... | |
406 | }; | |
407 | ||
408 | struct { | |
a9b83695 | 409 | enum : integer_type { sel1, sel2, sel3, ... } tag_field; |
fcba70d4 MD |
410 | ... |
411 | variant name <tag_field> v; | |
412 | } | |
413 | ||
414 | An unnamed variant definition within a structure is expressed by the following | |
6672e9e1 | 415 | TSDL meta-data: |
fcba70d4 MD |
416 | |
417 | struct { | |
a9b83695 | 418 | enum : integer_type { sel1, sel2, sel3, ... } tag_field; |
fcba70d4 MD |
419 | ... |
420 | variant <tag_field> { | |
421 | field_type sel1; | |
422 | field_type sel2; | |
423 | field_type sel3; | |
424 | ... | |
425 | } v; | |
426 | } | |
427 | ||
428 | Example of a named variant within a sequence that refers to a single tag field: | |
429 | ||
430 | variant example { | |
431 | uint32_t a; | |
432 | uint64_t b; | |
433 | short c; | |
434 | }; | |
435 | ||
436 | struct { | |
a9b83695 | 437 | enum : uint2_t { a, b, c } choice; |
15850440 | 438 | variant example <choice> v[unsigned int]; |
fcba70d4 MD |
439 | } |
440 | ||
441 | Example of an unnamed variant: | |
442 | ||
443 | struct { | |
a9b83695 | 444 | enum : uint2_t { a, b, c, d } choice; |
fcba70d4 MD |
445 | /* Unrelated fields can be added between the variant and its tag */ |
446 | int32_t somevalue; | |
447 | variant <choice> { | |
448 | uint32_t a; | |
449 | uint64_t b; | |
450 | short c; | |
451 | struct { | |
452 | unsigned int field1; | |
453 | uint64_t field2; | |
454 | } d; | |
455 | } s; | |
456 | } | |
457 | ||
458 | Example of an unnamed variant within an array: | |
459 | ||
460 | struct { | |
a9b83695 | 461 | enum : uint2_t { a, b, c } choice; |
fcba70d4 MD |
462 | variant <choice> { |
463 | uint32_t a; | |
464 | uint64_t b; | |
465 | short c; | |
15850440 | 466 | } v[10]; |
fcba70d4 MD |
467 | } |
468 | ||
469 | Example of a variant type definition within a structure, where the defined type | |
470 | is then declared within an array of structures. This variant refers to a tag | |
471 | located in an upper lexical scope. This example clearly shows that a variant | |
472 | type definition referring to the tag "x" uses the closest preceding field from | |
473 | the lexical scope of the type definition. | |
474 | ||
475 | struct { | |
a9b83695 | 476 | enum : uint2_t { a, b, c, d } x; |
fcba70d4 MD |
477 | |
478 | typedef variant <x> { /* | |
479 | * "x" refers to the preceding "x" enumeration in the | |
480 | * lexical scope of the type definition. | |
481 | */ | |
482 | uint32_t a; | |
483 | uint64_t b; | |
484 | short c; | |
485 | } example_variant; | |
486 | ||
487 | struct { | |
a9b83695 | 488 | enum : int { x, y, z } x; /* This enumeration is not used by "v". */ |
fcba70d4 | 489 | example_variant v; /* |
a9b83695 | 490 | * "v" uses the "enum : uint2_t { a, b, c, d }" |
fcba70d4 MD |
491 | * tag. |
492 | */ | |
493 | } a[10]; | |
494 | } | |
495 | ||
496 | 4.2.3 Arrays | |
5ba9f198 | 497 | |
6672e9e1 MD |
498 | Arrays are fixed-length. Their length is declared in the type |
499 | declaration within the meta-data. They contain an array of "inner type" | |
500 | elements, which can refer to any type not containing the type of the | |
501 | array being declared (no circular dependency). The length is the number | |
502 | of elements in an array. | |
5ba9f198 | 503 | |
6672e9e1 | 504 | TSDL meta-data representation of a named array: |
80fd2569 MD |
505 | |
506 | typedef elem_type name[length]; | |
5ba9f198 | 507 | |
2152348f | 508 | A nameless array can be declared as a field type within a structure, e.g.: |
5ba9f198 | 509 | |
2152348f | 510 | uint8_t field_name[10]; |
80fd2569 | 511 | |
ec4404a7 | 512 | Arrays are always aligned on their element alignment requirement. |
5ba9f198 | 513 | |
fcba70d4 | 514 | 4.2.4 Sequences |
5ba9f198 MD |
515 | |
516 | Sequences are dynamically-sized arrays. They start with an integer that specify | |
517 | the length of the sequence, followed by an array of "inner type" elements. | |
3bf79539 | 518 | The length is the number of elements in the sequence. |
5ba9f198 | 519 | |
6672e9e1 | 520 | TSDL meta-data representation for a named sequence: |
80fd2569 MD |
521 | |
522 | typedef elem_type name[length_type]; | |
523 | ||
524 | A nameless sequence can be declared as a field type, e.g.: | |
525 | ||
80fd2569 MD |
526 | long field_name[int]; |
527 | ||
528 | The length type follows the integer types specifications, and the sequence | |
5ba9f198 MD |
529 | elements follow the "array" specifications. |
530 | ||
fcba70d4 | 531 | 4.2.5 Strings |
5ba9f198 MD |
532 | |
533 | Strings are an array of bytes of variable size and are terminated by a '\0' | |
6672e9e1 MD |
534 | "NULL" character. Their encoding is described in the TSDL meta-data. In |
535 | absence of encoding attribute information, the default encoding is | |
536 | UTF-8. | |
5ba9f198 | 537 | |
6672e9e1 | 538 | TSDL meta-data representation of a named string type: |
80fd2569 | 539 | |
359894ac | 540 | typealias string { |
5ba9f198 | 541 | encoding = UTF8 OR ASCII; |
38b8da21 | 542 | } := name; |
5ba9f198 | 543 | |
80fd2569 MD |
544 | A nameless string type can be declared as a field type: |
545 | ||
546 | string field_name; /* Use default UTF8 encoding */ | |
5ba9f198 | 547 | |
ec4404a7 MD |
548 | Strings are always aligned on byte size. |
549 | ||
3bf79539 MD |
550 | 5. Event Packet Header |
551 | ||
6672e9e1 MD |
552 | The event packet header consists of two parts: the "event packet header" |
553 | is the same for all streams of a trace. The second part, the "event | |
554 | packet context", is described on a per-stream basis. Both are described | |
555 | in the TSDL meta-data. The packets are aligned on architecture-page-sized | |
556 | addresses. | |
3bf79539 | 557 | |
6672e9e1 | 558 | Event packet header (all fields are optional, specified by TSDL meta-data): |
3bf79539 | 559 | |
6672e9e1 MD |
560 | - Magic number (CTF magic number: 0xC1FC1FC1) specifies that this is a |
561 | CTF packet. This magic number is optional, but when present, it should | |
562 | come at the very beginning of the packet. | |
563 | - Trace UUID, used to ensure the event packet match the meta-data used. | |
564 | (note: we cannot use a meta-data checksum in every cases instead of a | |
565 | UUID because meta-data can be appended to while tracing is active) | |
566 | This field is optional. | |
567 | - Stream ID, used as reference to stream description in meta-data. | |
568 | This field is optional if there is only one stream description in the | |
569 | meta-data, but becomes required if there are more than one stream in | |
570 | the TSDL meta-data description. | |
3bf79539 | 571 | |
6672e9e1 | 572 | Event packet context (all fields are optional, specified by TSDL meta-data): |
3bf79539 MD |
573 | |
574 | - Event packet content size (in bytes). | |
575 | - Event packet size (in bytes, includes padding). | |
576 | - Event packet content checksum (optional). Checksum excludes the event packet | |
577 | header. | |
578 | - Per-stream event packet sequence count (to deal with UDP packet loss). The | |
579 | number of significant sequence counter bits should also be present, so | |
b11853af | 580 | wrap-arounds are dealt with correctly. |
6672e9e1 | 581 | - Time-stamp at the beginning and time-stamp at the end of the event packet. |
3bf79539 MD |
582 | Both timestamps are written in the packet header, but sampled respectively |
583 | while (or before) writing the first event and while (or after) writing the | |
584 | last event in the packet. The inclusive range between these timestamps should | |
585 | include all event timestamps assigned to events contained within the packet. | |
5ba9f198 | 586 | - Events discarded count |
3bf79539 MD |
587 | - Snapshot of a per-stream free-running counter, counting the number of |
588 | events discarded that were supposed to be written in the stream prior to | |
589 | the first event in the event packet. | |
5ba9f198 | 590 | * Note: producer-consumer buffer full condition should fill the current |
3bf79539 | 591 | event packet with padding so we know exactly where events have been |
5ba9f198 | 592 | discarded. |
3bf79539 MD |
593 | - Lossless compression scheme used for the event packet content. Applied |
594 | directly to raw data. New types of compression can be added in following | |
595 | versions of the format. | |
5ba9f198 MD |
596 | 0: no compression scheme |
597 | 1: bzip2 | |
598 | 2: gzip | |
3bf79539 MD |
599 | 3: xz |
600 | - Cypher used for the event packet content. Applied after compression. | |
5ba9f198 MD |
601 | 0: no encryption |
602 | 1: AES | |
3bf79539 | 603 | - Checksum scheme used for the event packet content. Applied after encryption. |
5ba9f198 MD |
604 | 0: no checksum |
605 | 1: md5 | |
606 | 2: sha1 | |
607 | 3: crc32 | |
608 | ||
6672e9e1 | 609 | 5.1 Event Packet Header Description |
3bf79539 | 610 | |
fc5425db | 611 | The event packet header layout is indicated by the trace packet.header |
6672e9e1 MD |
612 | field. Here is a recommended structure type for the packet header with |
613 | the fields typically expected (although these fields are each optional): | |
fc5425db | 614 | |
80fd2569 MD |
615 | struct event_packet_header { |
616 | uint32_t magic; | |
617 | uint8_t trace_uuid[16]; | |
3bf79539 | 618 | uint32_t stream_id; |
80fd2569 | 619 | }; |
5ba9f198 | 620 | |
fc5425db MD |
621 | trace { |
622 | ... | |
623 | packet.header := struct event_packet_header; | |
624 | }; | |
625 | ||
6672e9e1 MD |
626 | If the magic number is not present, tools such as "file" will have no |
627 | mean to discover the file type. | |
628 | ||
629 | If the trace_uuid is not present, no validation that the meta-data | |
37b39e47 MD |
630 | actually corresponds to the stream is performed. |
631 | ||
632 | If the stream_id packet header field is missing, the trace can only | |
633 | contain a single stream. Its "id" field can be left out, and its events | |
634 | don't need to declare a "stream_id" field. | |
635 | ||
636 | ||
3bf79539 MD |
637 | 5.2 Event Packet Context Description |
638 | ||
639 | Event packet context example. These are declared within the stream declaration | |
6672e9e1 | 640 | in the meta-data. All these fields are optional. If the packet size field is |
6a7c61df | 641 | missing, the whole stream only contains a single packet. If the content |
f6b81726 MD |
642 | size field is missing, the packet is filled (no padding). The content |
643 | and packet sizes include all headers. | |
3bf79539 MD |
644 | |
645 | An example event packet context type: | |
646 | ||
80fd2569 | 647 | struct event_packet_context { |
3bf79539 MD |
648 | uint64_t timestamp_begin; |
649 | uint64_t timestamp_end; | |
650 | uint32_t checksum; | |
651 | uint32_t stream_packet_count; | |
652 | uint32_t events_discarded; | |
653 | uint32_t cpu_id; | |
654 | uint32_t/uint16_t content_size; | |
655 | uint32_t/uint16_t packet_size; | |
656 | uint8_t stream_packet_count_bits; /* Significant counter bits */ | |
657 | uint8_t compression_scheme; | |
658 | uint8_t encryption_scheme; | |
3b0f8e4d | 659 | uint8_t checksum_scheme; |
3bf79539 | 660 | }; |
5ba9f198 | 661 | |
fcba70d4 | 662 | |
5ba9f198 MD |
663 | 6. Event Structure |
664 | ||
665 | The overall structure of an event is: | |
666 | ||
6672e9e1 MD |
667 | 1 - Stream Packet Context (as specified by the stream meta-data) |
668 | 2 - Event Header (as specified by the stream meta-data) | |
669 | 3 - Stream Event Context (as specified by the stream meta-data) | |
670 | 4 - Event Context (as specified by the event meta-data) | |
671 | 5 - Event Payload (as specified by the event meta-data) | |
5ba9f198 | 672 | |
fdf2bb05 | 673 | This structure defines an implicit dynamic scoping, where variants |
7d9d7e92 MD |
674 | located in inner structures (those with a higher number in the listing |
675 | above) can refer to the fields of outer structures (with lower number in | |
6c7226e9 | 676 | the listing above). See Section 7.3 TSDL Scopes for more detail. |
5ba9f198 | 677 | |
fdf2bb05 | 678 | 6.1 Event Header |
fcba70d4 | 679 | |
6672e9e1 | 680 | Event headers can be described within the meta-data. We hereby propose, as an |
fcba70d4 MD |
681 | example, two types of events headers. Type 1 accommodates streams with less than |
682 | 31 event IDs. Type 2 accommodates streams with 31 or more event IDs. | |
5ba9f198 | 683 | |
3bf79539 MD |
684 | One major factor can vary between streams: the number of event IDs assigned to |
685 | a stream. Luckily, this information tends to stay relatively constant (modulo | |
5ba9f198 | 686 | event registration while trace is being recorded), so we can specify different |
3bf79539 | 687 | representations for streams containing few event IDs and streams containing |
6672e9e1 MD |
688 | many event IDs, so we end up representing the event ID and time-stamp as |
689 | densely as possible in each case. | |
5ba9f198 | 690 | |
fcba70d4 MD |
691 | The header is extended in the rare occasions where the information cannot be |
692 | represented in the ranges available in the standard event header. They are also | |
3bf79539 MD |
693 | used in the rare occasions where the data required for a field could not be |
694 | collected: the flag corresponding to the missing field within the missing_fields | |
695 | array is then set to 1. | |
5ba9f198 | 696 | |
6672e9e1 MD |
697 | Types uintX_t represent an X-bit unsigned integer, as declared with |
698 | either: | |
5ba9f198 | 699 | |
6672e9e1 MD |
700 | typealias integer { size = X; align = X; signed = false } := uintX_t; |
701 | ||
702 | or | |
703 | ||
704 | typealias integer { size = X; align = 1; signed = false } := uintX_t; | |
5ba9f198 | 705 | |
fdf2bb05 | 706 | 6.1.1 Type 1 - Few event IDs |
5ba9f198 MD |
707 | |
708 | - Aligned on 32-bit (or 8-bit if byte-packed, depending on the architecture | |
709 | preference). | |
5ba9f198 | 710 | - Native architecture byte ordering. |
fcba70d4 MD |
711 | - For "compact" selection |
712 | - Fixed size: 32 bits. | |
713 | - For "extended" selection | |
714 | - Size depends on the architecture and variant alignment. | |
5ba9f198 | 715 | |
80fd2569 | 716 | struct event_header_1 { |
fcba70d4 MD |
717 | /* |
718 | * id: range: 0 - 30. | |
719 | * id 31 is reserved to indicate an extended header. | |
720 | */ | |
a9b83695 | 721 | enum : uint5_t { compact = 0 ... 30, extended = 31 } id; |
fcba70d4 MD |
722 | variant <id> { |
723 | struct { | |
724 | uint27_t timestamp; | |
725 | } compact; | |
726 | struct { | |
727 | uint32_t id; /* 32-bit event IDs */ | |
728 | uint64_t timestamp; /* 64-bit timestamps */ | |
729 | } extended; | |
730 | } v; | |
5ba9f198 MD |
731 | }; |
732 | ||
5ba9f198 | 733 | |
fdf2bb05 | 734 | 6.1.2 Type 2 - Many event IDs |
5ba9f198 | 735 | |
fcba70d4 | 736 | - Aligned on 16-bit (or 8-bit if byte-packed, depending on the architecture |
5ba9f198 | 737 | preference). |
5ba9f198 | 738 | - Native architecture byte ordering. |
fcba70d4 MD |
739 | - For "compact" selection |
740 | - Size depends on the architecture and variant alignment. | |
741 | - For "extended" selection | |
742 | - Size depends on the architecture and variant alignment. | |
5ba9f198 | 743 | |
80fd2569 | 744 | struct event_header_2 { |
fcba70d4 MD |
745 | /* |
746 | * id: range: 0 - 65534. | |
747 | * id 65535 is reserved to indicate an extended header. | |
748 | */ | |
a9b83695 | 749 | enum : uint16_t { compact = 0 ... 65534, extended = 65535 } id; |
fcba70d4 MD |
750 | variant <id> { |
751 | struct { | |
752 | uint32_t timestamp; | |
753 | } compact; | |
754 | struct { | |
755 | uint32_t id; /* 32-bit event IDs */ | |
756 | uint64_t timestamp; /* 64-bit timestamps */ | |
757 | } extended; | |
758 | } v; | |
5ba9f198 MD |
759 | }; |
760 | ||
5ba9f198 MD |
761 | |
762 | 6.2 Event Context | |
763 | ||
6672e9e1 MD |
764 | The event context contains information relative to the current event. |
765 | The choice and meaning of this information is specified by the TSDL | |
766 | stream and event meta-data descriptions. The stream context is applied | |
767 | to all events within the stream. The stream context structure follows | |
768 | the event header. The event context is applied to specific events. Its | |
769 | structure follows the stream context structure. | |
5ba9f198 | 770 | |
fcba70d4 MD |
771 | An example of stream-level event context is to save the event payload size with |
772 | each event, or to save the current PID with each event. These are declared | |
6672e9e1 | 773 | within the stream declaration within the meta-data: |
5ba9f198 | 774 | |
fcba70d4 MD |
775 | stream { |
776 | ... | |
6672e9e1 | 777 | event.context := struct { |
80fd2569 MD |
778 | uint pid; |
779 | uint16_t payload_size; | |
6672e9e1 | 780 | }; |
fcba70d4 MD |
781 | }; |
782 | ||
783 | An example of event-specific event context is to declare a bitmap of missing | |
784 | fields, only appended after the stream event context if the extended event | |
785 | header is selected. NR_FIELDS is the number of fields within the event (a | |
786 | numeric value). | |
5ba9f198 | 787 | |
fcba70d4 MD |
788 | event { |
789 | context = struct { | |
790 | variant <id> { | |
791 | struct { } compact; | |
792 | struct { | |
793 | uint1_t missing_fields[NR_FIELDS]; /* missing event fields bitmap */ | |
794 | } extended; | |
795 | } v; | |
796 | }; | |
797 | ... | |
798 | } | |
5ba9f198 MD |
799 | |
800 | 6.3 Event Payload | |
801 | ||
802 | An event payload contains fields specific to a given event type. The fields | |
6672e9e1 | 803 | belonging to an event type are described in the event-specific meta-data |
5ba9f198 MD |
804 | within a structure type. |
805 | ||
806 | 6.3.1 Padding | |
807 | ||
808 | No padding at the end of the event payload. This differs from the ISO/C standard | |
809 | for structures, but follows the CTF standard for structures. In a trace, even | |
810 | though it makes sense to align the beginning of a structure, it really makes no | |
811 | sense to add padding at the end of the structure, because structures are usually | |
812 | not followed by a structure of the same type. | |
813 | ||
814 | This trick can be done by adding a zero-length "end" field at the end of the C | |
815 | structures, and by using the offset of this field rather than using sizeof() | |
3bf79539 | 816 | when calculating the size of a structure (see Appendix "A. Helper macros"). |
5ba9f198 MD |
817 | |
818 | 6.3.2 Alignment | |
819 | ||
820 | The event payload is aligned on the largest alignment required by types | |
821 | contained within the payload. (This follows the ISO/C standard for structures) | |
822 | ||
823 | ||
6c7226e9 MD |
824 | 7. Trace Stream Description Language (TSDL) |
825 | ||
826 | The Trace Stream Description Language (TSDL) allows expression of the | |
827 | binary trace streams layout in a C99-like Domain Specific Language | |
828 | (DSL). | |
829 | ||
830 | ||
6672e9e1 | 831 | 7.1 Meta-data |
6c7226e9 MD |
832 | |
833 | The trace stream layout description is located in the trace meta-data. | |
834 | The meta-data is itself located in a stream identified by its name: | |
835 | "metadata". | |
5ba9f198 | 836 | |
6672e9e1 MD |
837 | The meta-data description can be expressed in two different formats: |
838 | text-only and packet-based. The text-only description facilitates | |
839 | generation of meta-data and provides a convenient way to enter the | |
840 | meta-data information by hand. The packet-based meta-data provides the | |
841 | CTF stream packet facilities (checksumming, compression, encryption, | |
842 | network-readiness) for meta-data stream generated and transported by a | |
843 | tracer. | |
844 | ||
845 | The text-only meta-data file is a plain text TSDL description. | |
846 | ||
847 | The packet-based meta-data is made of "meta-data packets", which each | |
848 | start with a meta-data packet header. The packet-based meta-data | |
849 | description is detected by reading the magic number "0x75D11D57" at the | |
850 | beginning of the file. This magic number is also used to detect the | |
851 | endianness of the architecture by trying to read the CTF magic number | |
852 | and its counterpart in reversed endianness. The events within the | |
853 | meta-data stream have no event header nor event context. Each event only | |
854 | contains a "string" payload. Each meta-data packet start with a special | |
855 | packet header, specific to the meta-data stream, which contains, | |
856 | exactly: | |
857 | ||
858 | struct metadata_packet_header { | |
2daeaa3a | 859 | uint32_t magic; /* 0x75D11D57 */ |
6672e9e1 MD |
860 | uint8_t trace_uuid[16]; /* Unique Universal Identifier */ |
861 | uint32_t checksum; /* 0 if unused */ | |
862 | uint32_t content_size; /* in bits */ | |
863 | uint32_t packet_size; /* in bits */ | |
864 | uint8_t compression_scheme; /* 0 if unused */ | |
865 | uint8_t encryption_scheme; /* 0 if unused */ | |
866 | uint8_t checksum_scheme; /* 0 if unused */ | |
867 | }; | |
868 | ||
869 | The packet-based meta-data can be converted to a text-only meta-data by | |
870 | concatenating all the strings in contains. | |
4fafe1ad | 871 | |
6672e9e1 MD |
872 | In the textual representation of the meta-data, the text contained |
873 | within "/*" and "*/", as well as within "//" and end of line, are | |
874 | treated as comments. Boolean values can be represented as true, TRUE, | |
875 | or 1 for true, and false, FALSE, or 0 for false. Within the string-based | |
876 | meta-data description, the trace UUID is represented as a string of | |
877 | hexadecimal digits and dashes "-". In the event packet header, the trace | |
878 | UUID is represented as an array of bytes. | |
fcba70d4 | 879 | |
fdf2bb05 | 880 | |
6c7226e9 | 881 | 7.2 Declaration vs Definition |
fdf2bb05 MD |
882 | |
883 | A declaration associates a layout to a type, without specifying where | |
884 | this type is located in the event structure hierarchy (see Section 6). | |
885 | This therefore includes typedef, typealias, as well as all type | |
886 | specifiers. In certain circumstances (typedef, structure field and | |
887 | variant field), a declaration is followed by a declarator, which specify | |
888 | the newly defined type name (for typedef), or the field name (for | |
889 | declarations located within structure and variants). Array and sequence, | |
890 | declared with square brackets ("[" "]"), are part of the declarator, | |
a9b83695 | 891 | similarly to C99. The enumeration base type is specified by |
6c7226e9 | 892 | ": enum_base", which is part of the type specifier. The variant tag |
a9b83695 | 893 | name, specified between "<" ">", is also part of the type specifier. |
fdf2bb05 MD |
894 | |
895 | A definition associates a type to a location in the event structure | |
b9606a77 MD |
896 | hierarchy (see Section 6). This association is denoted by ":=", as shown |
897 | in Section 7.3. | |
fdf2bb05 MD |
898 | |
899 | ||
6c7226e9 | 900 | 7.3 TSDL Scopes |
fdf2bb05 | 901 | |
6c7226e9 MD |
902 | TSDL uses two different types of scoping: a lexical scope is used for |
903 | declarations and type definitions, and a dynamic scope is used for | |
904 | variants references to tag fields. | |
fdf2bb05 | 905 | |
6c7226e9 | 906 | 7.3.1 Lexical Scope |
fdf2bb05 | 907 | |
d285084f MD |
908 | Each of "trace", "stream", "event", "struct" and "variant" have their own |
909 | nestable declaration scope, within which types can be declared using "typedef" | |
fdf2bb05 | 910 | and "typealias". A root declaration scope also contains all declarations |
7d9d7e92 | 911 | located outside of any of the aforementioned declarations. An inner |
fdf2bb05 | 912 | declaration scope can refer to type declared within its container |
7d9d7e92 MD |
913 | lexical scope prior to the inner declaration scope. Redefinition of a |
914 | typedef or typealias is not valid, although hiding an upper scope | |
fdf2bb05 MD |
915 | typedef or typealias is allowed within a sub-scope. |
916 | ||
6c7226e9 | 917 | 7.3.2 Dynamic Scope |
fdf2bb05 | 918 | |
7d9d7e92 MD |
919 | A dynamic scope consists in the lexical scope augmented with the |
920 | implicit event structure definition hierarchy presented at Section 6. | |
921 | The dynamic scope is only used for variant tag definitions. It is used | |
922 | at definition time to look up the location of the tag field associated | |
923 | with a variant. | |
924 | ||
925 | Therefore, variants in lower levels in the dynamic scope (e.g. event | |
926 | context) can refer to a tag field located in upper levels (e.g. in the | |
927 | event header) by specifying, in this case, the associated tag with | |
928 | <header.field_name>. This allows, for instance, the event context to | |
929 | define a variant referring to the "id" field of the event header as | |
930 | selector. | |
fdf2bb05 MD |
931 | |
932 | The target dynamic scope must be specified explicitly when referring to | |
933 | a field outside of the local static scope. The dynamic scope prefixes | |
934 | are thus: | |
935 | ||
e0d9e2c7 | 936 | - Trace Packet Header: <trace.packet.header. >, |
7d9d7e92 MD |
937 | - Stream Packet Context: <stream.packet.context. >, |
938 | - Event Header: <stream.event.header. >, | |
939 | - Stream Event Context: <stream.event.context. >, | |
940 | - Event Context: <event.context. >, | |
941 | - Event Payload: <event.fields. >. | |
fdf2bb05 MD |
942 | |
943 | Multiple declarations of the same field name within a single scope is | |
944 | not valid. It is however valid to re-use the same field name in | |
945 | different scopes. There is no possible conflict, because the dynamic | |
946 | scope must be specified when a variant refers to a tag field located in | |
947 | a different dynamic scope. | |
948 | ||
457d8b0a MD |
949 | The information available in the dynamic scopes can be thought of as the |
950 | current tracing context. At trace production, information about the | |
951 | current context is saved into the specified scope field levels. At trace | |
952 | consumption, for each event, the current trace context is therefore | |
953 | readable by accessing the upper dynamic scopes. | |
954 | ||
fdf2bb05 | 955 | |
6c7226e9 | 956 | 7.4 TSDL Examples |
d285084f | 957 | |
6672e9e1 | 958 | The grammar representing the TSDL meta-data is presented in Appendix C. |
7df6b93a | 959 | TSDL Grammar. This section presents a rather lighter reading that |
6672e9e1 | 960 | consists in examples of TSDL meta-data, with template values. |
969f30c0 | 961 | |
350bb633 MD |
962 | The stream "id" can be left out if there is only one stream in the |
963 | trace. The event "id" field can be left out if there is only one event | |
964 | in a stream. | |
965 | ||
5ba9f198 | 966 | trace { |
fdf2bb05 | 967 | major = value; /* Trace format version */ |
5ba9f198 | 968 | minor = value; |
fdf2bb05 | 969 | uuid = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; /* Trace UUID */ |
58997e9e | 970 | byte_order = be OR le; /* Endianness (required) */ |
fc5425db MD |
971 | packet.header := struct { |
972 | uint32_t magic; | |
973 | uint8_t trace_uuid[16]; | |
974 | uint32_t stream_id; | |
975 | }; | |
3bf79539 | 976 | }; |
5ba9f198 | 977 | |
3bf79539 MD |
978 | stream { |
979 | id = stream_id; | |
fdf2bb05 | 980 | /* Type 1 - Few event IDs; Type 2 - Many event IDs. See section 6.1. */ |
4fa992a5 MD |
981 | event.header := event_header_1 OR event_header_2; |
982 | event.context := struct { | |
77a98c82 | 983 | ... |
3bf79539 | 984 | }; |
4fa992a5 | 985 | packet.context := struct { |
77a98c82 | 986 | ... |
3bf79539 MD |
987 | }; |
988 | }; | |
5ba9f198 MD |
989 | |
990 | event { | |
3d13ef1a | 991 | name = event_name; |
3bf79539 | 992 | id = value; /* Numeric identifier within the stream */ |
67f02e24 | 993 | stream_id = stream_id; |
4fa992a5 | 994 | context := struct { |
fcba70d4 MD |
995 | ... |
996 | }; | |
4fa992a5 | 997 | fields := struct { |
80fd2569 MD |
998 | ... |
999 | }; | |
3bf79539 | 1000 | }; |
5ba9f198 MD |
1001 | |
1002 | /* More detail on types in section 4. Types */ | |
1003 | ||
3d13ef1a MD |
1004 | /* |
1005 | * Named types: | |
1006 | * | |
4fa992a5 | 1007 | * Type declarations behave similarly to the C standard. |
3d13ef1a MD |
1008 | */ |
1009 | ||
80af8ac6 | 1010 | typedef aliased_type_specifiers new_type_declarators; |
2152348f | 1011 | |
3d13ef1a | 1012 | /* e.g.: typedef struct example new_type_name[10]; */ |
80fd2569 | 1013 | |
4fa992a5 MD |
1014 | /* |
1015 | * typealias | |
1016 | * | |
1017 | * The "typealias" declaration can be used to give a name (including | |
80af8ac6 MD |
1018 | * pointer declarator specifier) to a type. It should also be used to |
1019 | * map basic C types (float, int, unsigned long, ...) to a CTF type. | |
1020 | * Typealias is a superset of "typedef": it also allows assignment of a | |
38b8da21 | 1021 | * simple variable identifier to a type. |
4fa992a5 MD |
1022 | */ |
1023 | ||
1024 | typealias type_class { | |
80fd2569 | 1025 | ... |
38b8da21 | 1026 | } := type_specifiers type_declarator; |
2152348f | 1027 | |
3d13ef1a MD |
1028 | /* |
1029 | * e.g.: | |
4fa992a5 | 1030 | * typealias integer { |
3d13ef1a MD |
1031 | * size = 32; |
1032 | * align = 32; | |
1033 | * signed = false; | |
38b8da21 | 1034 | * } := struct page *; |
359894ac MD |
1035 | * |
1036 | * typealias integer { | |
1037 | * size = 32; | |
1038 | * align = 32; | |
1039 | * signed = true; | |
38b8da21 | 1040 | * } := int; |
3d13ef1a | 1041 | */ |
80fd2569 MD |
1042 | |
1043 | struct name { | |
3bf79539 MD |
1044 | ... |
1045 | }; | |
5ba9f198 | 1046 | |
fcba70d4 MD |
1047 | variant name { |
1048 | ... | |
1049 | }; | |
1050 | ||
a9b83695 | 1051 | enum name : integer_type { |
3bf79539 MD |
1052 | ... |
1053 | }; | |
1054 | ||
2152348f | 1055 | |
4fa992a5 MD |
1056 | /* |
1057 | * Unnamed types, contained within compound type fields, typedef or typealias. | |
1058 | */ | |
2152348f | 1059 | |
80fd2569 MD |
1060 | struct { |
1061 | ... | |
2152348f | 1062 | } |
5ba9f198 | 1063 | |
ec4404a7 MD |
1064 | struct { |
1065 | ... | |
1066 | } align(value) | |
1067 | ||
fcba70d4 MD |
1068 | variant { |
1069 | ... | |
1070 | } | |
1071 | ||
a9b83695 | 1072 | enum : integer_type { |
80fd2569 | 1073 | ... |
2152348f MD |
1074 | } |
1075 | ||
1076 | typedef type new_type[length]; | |
3bf79539 | 1077 | |
2152348f MD |
1078 | struct { |
1079 | type field_name[length]; | |
1080 | } | |
1081 | ||
1082 | typedef type new_type[length_type]; | |
1083 | ||
1084 | struct { | |
1085 | type field_name[length_type]; | |
1086 | } | |
1087 | ||
1088 | integer { | |
80fd2569 | 1089 | ... |
2152348f | 1090 | } |
3bf79539 | 1091 | |
2152348f | 1092 | floating_point { |
80fd2569 | 1093 | ... |
2152348f MD |
1094 | } |
1095 | ||
1096 | struct { | |
1097 | integer_type field_name:size; /* GNU/C bitfield */ | |
1098 | } | |
1099 | ||
1100 | struct { | |
1101 | string field_name; | |
1102 | } | |
3bf79539 | 1103 | |
fcba70d4 | 1104 | |
3bf79539 | 1105 | A. Helper macros |
5ba9f198 MD |
1106 | |
1107 | The two following macros keep track of the size of a GNU/C structure without | |
1108 | padding at the end by placing HEADER_END as the last field. A one byte end field | |
1109 | is used for C90 compatibility (C99 flexible arrays could be used here). Note | |
1110 | that this does not affect the effective structure size, which should always be | |
1111 | calculated with the header_sizeof() helper. | |
1112 | ||
1113 | #define HEADER_END char end_field | |
1114 | #define header_sizeof(type) offsetof(typeof(type), end_field) | |
3bf79539 MD |
1115 | |
1116 | ||
1117 | B. Stream Header Rationale | |
1118 | ||
1119 | An event stream is divided in contiguous event packets of variable size. These | |
1120 | subdivisions allow the trace analyzer to perform a fast binary search by time | |
1121 | within the stream (typically requiring to index only the event packet headers) | |
1122 | without reading the whole stream. These subdivisions have a variable size to | |
1123 | eliminate the need to transfer the event packet padding when partially filled | |
1124 | event packets must be sent when streaming a trace for live viewing/analysis. | |
1125 | An event packet can contain a certain amount of padding at the end. Dividing | |
1126 | streams into event packets is also useful for network streaming over UDP and | |
1127 | flight recorder mode tracing (a whole event packet can be swapped out of the | |
1128 | buffer atomically for reading). | |
1129 | ||
1130 | The stream header is repeated at the beginning of each event packet to allow | |
1131 | flexibility in terms of: | |
1132 | ||
1133 | - streaming support, | |
1134 | - allowing arbitrary buffers to be discarded without making the trace | |
1135 | unreadable, | |
1136 | - allow UDP packet loss handling by either dealing with missing event packet | |
1137 | or asking for re-transmission. | |
1138 | - transparently support flight recorder mode, | |
1139 | - transparently support crash dump. | |
1140 | ||
6c7226e9 MD |
1141 | |
1142 | C. TSDL Grammar | |
fcba70d4 | 1143 | |
4fa992a5 | 1144 | /* |
6c7226e9 | 1145 | * Common Trace Format (CTF) Trace Stream Description Language (TSDL) Grammar. |
4fa992a5 MD |
1146 | * |
1147 | * Inspired from the C99 grammar: | |
1148 | * http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf (Annex A) | |
6c7226e9 MD |
1149 | * and c++1x grammar (draft) |
1150 | * http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3291.pdf (Annex A) | |
4fa992a5 MD |
1151 | * |
1152 | * Specialized for CTF needs by including only constant and declarations from | |
1153 | * C99 (excluding function declarations), and by adding support for variants, | |
6c7226e9 MD |
1154 | * sequences and CTF-specific specifiers. Enumeration container types |
1155 | * semantic is inspired from c++1x enum-base. | |
4fa992a5 MD |
1156 | */ |
1157 | ||
1158 | 1) Lexical grammar | |
1159 | ||
1160 | 1.1) Lexical elements | |
1161 | ||
1162 | token: | |
1163 | keyword | |
1164 | identifier | |
1165 | constant | |
1166 | string-literal | |
1167 | punctuator | |
1168 | ||
1169 | 1.2) Keywords | |
1170 | ||
1171 | keyword: is one of | |
1172 | ||
ec4404a7 | 1173 | align |
4fa992a5 MD |
1174 | const |
1175 | char | |
1176 | double | |
1177 | enum | |
1178 | event | |
1179 | floating_point | |
1180 | float | |
1181 | integer | |
1182 | int | |
1183 | long | |
1184 | short | |
1185 | signed | |
1186 | stream | |
1187 | string | |
1188 | struct | |
1189 | trace | |
3e1e1a78 | 1190 | typealias |
4fa992a5 MD |
1191 | typedef |
1192 | unsigned | |
1193 | variant | |
1194 | void | |
1195 | _Bool | |
1196 | _Complex | |
1197 | _Imaginary | |
1198 | ||
1199 | ||
1200 | 1.3) Identifiers | |
1201 | ||
1202 | identifier: | |
1203 | identifier-nondigit | |
1204 | identifier identifier-nondigit | |
1205 | identifier digit | |
1206 | ||
1207 | identifier-nondigit: | |
1208 | nondigit | |
1209 | universal-character-name | |
1210 | any other implementation-defined characters | |
1211 | ||
1212 | nondigit: | |
1213 | _ | |
1214 | [a-zA-Z] /* regular expression */ | |
1215 | ||
1216 | digit: | |
1217 | [0-9] /* regular expression */ | |
1218 | ||
1219 | 1.4) Universal character names | |
1220 | ||
1221 | universal-character-name: | |
1222 | \u hex-quad | |
1223 | \U hex-quad hex-quad | |
1224 | ||
1225 | hex-quad: | |
1226 | hexadecimal-digit hexadecimal-digit hexadecimal-digit hexadecimal-digit | |
1227 | ||
1228 | 1.5) Constants | |
1229 | ||
1230 | constant: | |
1231 | integer-constant | |
1232 | enumeration-constant | |
1233 | character-constant | |
1234 | ||
1235 | integer-constant: | |
1236 | decimal-constant integer-suffix-opt | |
1237 | octal-constant integer-suffix-opt | |
1238 | hexadecimal-constant integer-suffix-opt | |
1239 | ||
1240 | decimal-constant: | |
1241 | nonzero-digit | |
1242 | decimal-constant digit | |
1243 | ||
1244 | octal-constant: | |
1245 | 0 | |
1246 | octal-constant octal-digit | |
1247 | ||
1248 | hexadecimal-constant: | |
1249 | hexadecimal-prefix hexadecimal-digit | |
1250 | hexadecimal-constant hexadecimal-digit | |
1251 | ||
1252 | hexadecimal-prefix: | |
1253 | 0x | |
1254 | 0X | |
1255 | ||
1256 | nonzero-digit: | |
1257 | [1-9] | |
1258 | ||
1259 | integer-suffix: | |
1260 | unsigned-suffix long-suffix-opt | |
1261 | unsigned-suffix long-long-suffix | |
1262 | long-suffix unsigned-suffix-opt | |
1263 | long-long-suffix unsigned-suffix-opt | |
1264 | ||
1265 | unsigned-suffix: | |
1266 | u | |
1267 | U | |
1268 | ||
1269 | long-suffix: | |
1270 | l | |
1271 | L | |
1272 | ||
1273 | long-long-suffix: | |
1274 | ll | |
1275 | LL | |
1276 | ||
4fa992a5 MD |
1277 | enumeration-constant: |
1278 | identifier | |
1279 | string-literal | |
1280 | ||
1281 | character-constant: | |
1282 | ' c-char-sequence ' | |
1283 | L' c-char-sequence ' | |
1284 | ||
1285 | c-char-sequence: | |
1286 | c-char | |
1287 | c-char-sequence c-char | |
1288 | ||
1289 | c-char: | |
1290 | any member of source charset except single-quote ('), backslash | |
1291 | (\), or new-line character. | |
1292 | escape-sequence | |
1293 | ||
1294 | escape-sequence: | |
1295 | simple-escape-sequence | |
1296 | octal-escape-sequence | |
1297 | hexadecimal-escape-sequence | |
1298 | universal-character-name | |
1299 | ||
1300 | simple-escape-sequence: one of | |
1301 | \' \" \? \\ \a \b \f \n \r \t \v | |
1302 | ||
1303 | octal-escape-sequence: | |
1304 | \ octal-digit | |
1305 | \ octal-digit octal-digit | |
1306 | \ octal-digit octal-digit octal-digit | |
1307 | ||
1308 | hexadecimal-escape-sequence: | |
1309 | \x hexadecimal-digit | |
1310 | hexadecimal-escape-sequence hexadecimal-digit | |
1311 | ||
1312 | 1.6) String literals | |
1313 | ||
1314 | string-literal: | |
1315 | " s-char-sequence-opt " | |
1316 | L" s-char-sequence-opt " | |
1317 | ||
1318 | s-char-sequence: | |
1319 | s-char | |
1320 | s-char-sequence s-char | |
1321 | ||
1322 | s-char: | |
1323 | any member of source charset except double-quote ("), backslash | |
1324 | (\), or new-line character. | |
1325 | escape-sequence | |
1326 | ||
1327 | 1.7) Punctuators | |
1328 | ||
1329 | punctuator: one of | |
1330 | [ ] ( ) { } . -> * + - < > : ; ... = , | |
1331 | ||
1332 | ||
1333 | 2) Phrase structure grammar | |
1334 | ||
1335 | primary-expression: | |
1336 | identifier | |
1337 | constant | |
1338 | string-literal | |
1339 | ( unary-expression ) | |
1340 | ||
1341 | postfix-expression: | |
1342 | primary-expression | |
1343 | postfix-expression [ unary-expression ] | |
1344 | postfix-expression . identifier | |
1345 | postfix-expressoin -> identifier | |
1346 | ||
1347 | unary-expression: | |
1348 | postfix-expression | |
1349 | unary-operator postfix-expression | |
1350 | ||
1351 | unary-operator: one of | |
1352 | + - | |
1353 | ||
4fa992a5 MD |
1354 | assignment-operator: |
1355 | = | |
1356 | ||
b9606a77 MD |
1357 | type-assignment-operator: |
1358 | := | |
1359 | ||
4fa992a5 MD |
1360 | constant-expression: |
1361 | unary-expression | |
1362 | ||
1363 | constant-expression-range: | |
1364 | constant-expression ... constant-expression | |
1365 | ||
1366 | 2.2) Declarations: | |
1367 | ||
1368 | declaration: | |
689e04b4 | 1369 | declaration-specifiers declarator-list-opt ; |
4fa992a5 MD |
1370 | ctf-specifier ; |
1371 | ||
1372 | declaration-specifiers: | |
689e04b4 | 1373 | storage-class-specifier declaration-specifiers-opt |
4fa992a5 MD |
1374 | type-specifier declaration-specifiers-opt |
1375 | type-qualifier declaration-specifiers-opt | |
1376 | ||
1377 | declarator-list: | |
1378 | declarator | |
1379 | declarator-list , declarator | |
1380 | ||
d285084f MD |
1381 | abstract-declarator-list: |
1382 | abstract-declarator | |
1383 | abstract-declarator-list , abstract-declarator | |
1384 | ||
4fa992a5 MD |
1385 | storage-class-specifier: |
1386 | typedef | |
1387 | ||
1388 | type-specifier: | |
1389 | void | |
1390 | char | |
1391 | short | |
1392 | int | |
1393 | long | |
1394 | float | |
1395 | double | |
1396 | signed | |
1397 | unsigned | |
1398 | _Bool | |
1399 | _Complex | |
cfdd51ec | 1400 | _Imaginary |
9dfcfc0f MD |
1401 | struct-specifier |
1402 | variant-specifier | |
4fa992a5 MD |
1403 | enum-specifier |
1404 | typedef-name | |
1405 | ctf-type-specifier | |
1406 | ||
ec4404a7 MD |
1407 | align-attribute: |
1408 | align ( constant-expression ) | |
1409 | ||
4fa992a5 | 1410 | struct-specifier: |
ec4404a7 MD |
1411 | struct identifier-opt { struct-or-variant-declaration-list-opt } align-attribute-opt |
1412 | struct identifier align-attribute-opt | |
4fa992a5 MD |
1413 | |
1414 | struct-or-variant-declaration-list: | |
1415 | struct-or-variant-declaration | |
1416 | struct-or-variant-declaration-list struct-or-variant-declaration | |
1417 | ||
1418 | struct-or-variant-declaration: | |
1419 | specifier-qualifier-list struct-or-variant-declarator-list ; | |
550aca33 | 1420 | declaration-specifiers storage-class-specifier declaration-specifiers declarator-list ; |
38b8da21 MD |
1421 | typealias declaration-specifiers abstract-declarator-list := declaration-specifiers abstract-declarator-list ; |
1422 | typealias declaration-specifiers abstract-declarator-list := declarator-list ; | |
4fa992a5 MD |
1423 | |
1424 | specifier-qualifier-list: | |
1425 | type-specifier specifier-qualifier-list-opt | |
1426 | type-qualifier specifier-qualifier-list-opt | |
1427 | ||
1428 | struct-or-variant-declarator-list: | |
1429 | struct-or-variant-declarator | |
1430 | struct-or-variant-declarator-list , struct-or-variant-declarator | |
1431 | ||
1432 | struct-or-variant-declarator: | |
1433 | declarator | |
1434 | declarator-opt : constant-expression | |
1435 | ||
1436 | variant-specifier: | |
1437 | variant identifier-opt variant-tag-opt { struct-or-variant-declaration-list } | |
1438 | variant identifier variant-tag | |
1439 | ||
1440 | variant-tag: | |
1441 | < identifier > | |
1442 | ||
1443 | enum-specifier: | |
1444 | enum identifier-opt { enumerator-list } | |
1445 | enum identifier-opt { enumerator-list , } | |
1446 | enum identifier | |
a9b83695 MD |
1447 | enum identifier-opt : declaration-specifiers { enumerator-list } |
1448 | enum identifier-opt : declaration-specifiers { enumerator-list , } | |
4fa992a5 MD |
1449 | |
1450 | enumerator-list: | |
1451 | enumerator | |
1452 | enumerator-list , enumerator | |
1453 | ||
1454 | enumerator: | |
1455 | enumeration-constant | |
1456 | enumeration-constant = constant-expression | |
1457 | enumeration-constant = constant-expression-range | |
1458 | ||
1459 | type-qualifier: | |
1460 | const | |
1461 | ||
1462 | declarator: | |
1463 | pointer-opt direct-declarator | |
1464 | ||
1465 | direct-declarator: | |
1466 | identifier | |
1467 | ( declarator ) | |
1468 | direct-declarator [ type-specifier ] | |
1469 | direct-declarator [ constant-expression ] | |
1470 | ||
d285084f MD |
1471 | abstract-declarator: |
1472 | pointer-opt direct-abstract-declarator | |
1473 | ||
1474 | direct-abstract-declarator: | |
1475 | identifier-opt | |
1476 | ( abstract-declarator ) | |
1477 | direct-abstract-declarator [ type-specifier ] | |
1478 | direct-abstract-declarator [ constant-expression ] | |
1479 | direct-abstract-declarator [ ] | |
1480 | ||
4fa992a5 | 1481 | pointer: |
3b0f8e4d MD |
1482 | * type-qualifier-list-opt |
1483 | * type-qualifier-list-opt pointer | |
4fa992a5 MD |
1484 | |
1485 | type-qualifier-list: | |
1486 | type-qualifier | |
1487 | type-qualifier-list type-qualifier | |
1488 | ||
4fa992a5 MD |
1489 | typedef-name: |
1490 | identifier | |
1491 | ||
1492 | 2.3) CTF-specific declarations | |
1493 | ||
1494 | ctf-specifier: | |
1495 | event { ctf-assignment-expression-list-opt } | |
1496 | stream { ctf-assignment-expression-list-opt } | |
1497 | trace { ctf-assignment-expression-list-opt } | |
38b8da21 MD |
1498 | typealias declaration-specifiers abstract-declarator-list := declaration-specifiers abstract-declarator-list ; |
1499 | typealias declaration-specifiers abstract-declarator-list := declarator-list ; | |
4fa992a5 MD |
1500 | |
1501 | ctf-type-specifier: | |
1502 | floating_point { ctf-assignment-expression-list-opt } | |
1503 | integer { ctf-assignment-expression-list-opt } | |
1504 | string { ctf-assignment-expression-list-opt } | |
1505 | ||
1506 | ctf-assignment-expression-list: | |
1507 | ctf-assignment-expression | |
1508 | ctf-assignment-expression-list ; ctf-assignment-expression | |
1509 | ||
1510 | ctf-assignment-expression: | |
1511 | unary-expression assignment-operator unary-expression | |
1512 | unary-expression type-assignment-operator type-specifier | |
550aca33 | 1513 | declaration-specifiers storage-class-specifier declaration-specifiers declarator-list |
38b8da21 MD |
1514 | typealias declaration-specifiers abstract-declarator-list := declaration-specifiers abstract-declarator-list |
1515 | typealias declaration-specifiers abstract-declarator-list := declarator-list |