Clarify that timestamp begin/end need to be complete clock values
[ctf.git] / common-trace-format-specification.md
CommitLineData
941a19cf
PP
1# Common Trace Format (CTF) Specification (v1.8.2)
2
3**Author**: Mathieu Desnoyers, [EfficiOS Inc.](http://www.efficios.com/)
4
5The goal of the present document is to specify a trace format that suits
6the needs of the embedded, telecom, high-performance and kernel
7communities. It is based on the
8[Common Trace Format Requirements (v1.4)](http://git.efficios.com/?p=ctf.git;a=blob_plain;f=common-trace-format-reqs.txt;hb=master)
9document. It is designed to allow traces to be natively generated by the
10Linux kernel, Linux user space applications written in C/C++, and
11hardware components. One major element of CTF is the Trace Stream
12Description Language (TSDL) which flexibility enables description of
13various binary trace stream layouts.
14
15The latest version of this document can be found at:
16
17 * Git: `git clone git://git.efficios.com/ctf.git`
18 * [Gitweb](http://git.efficios.com/?p=ctf.git)
19
20A reference implementation of a library to read and write this trace
21format is being implemented within the
22[Babeltrace](http://www.efficios.com/babeltrace) project, a converter
23between trace formats. The development tree is available at:
24
25 * Git: `git clone git://git.efficios.com/babeltrace.git`
26 * [Gitweb](http://git.efficios.com/?p=babeltrace.git)
27
28The [CE Workgroup](http://www.linuxfoundation.org/collaborate/workgroups/celf)
29of the Linux Foundation, [Ericsson](http://www.ericsson.com/), and
30[EfficiOS](http://www.efficios.com/) have sponsored this work.
31
32**Contents**:
33
34 1. Preliminary definitions
35 2. High-level representation of a trace
36 3. Event stream
37 4. Types
38 4.1 Basic types
39 4.1.1 Type inheritance
40 4.1.2 Alignment
41 4.1.3 Byte order
42 4.1.4 Size
43 4.1.5 Integers
44 4.1.6 GNU/C bitfields
45 4.1.7 Floating point
46 4.1.8 Enumerations
47 4.2 Compound types
48 4.2.1 Structures
49 4.2.2 Variants (discriminated/tagged unions)
50 4.2.3 Arrays
51 4.2.4 Sequences
52 4.2.5 Strings
53 5. Event packet header
54 5.1 Event packet header description
55 5.2 Event packet context description
56 6. Event structure
57 6.1 Event header
58 6.1.1 Type 1: few event IDs
59 6.1.2 Type 2: many event IDs
60 6.2 Stream event context and event context
61 6.3 Event payload
62 6.3.1 Padding
63 6.3.2 Alignment
64 7. Trace Stream Description Language (TSDL)
65 7.1 Metadata
66 7.2 Declaration vs definition
67 7.3 TSDL scopes
68 7.3.1 Lexical scope
69 7.3.2 Static and dynamic scopes
70 7.4 TSDL examples
71 8. Clocks
72 A. Helper macros
73 B. Stream header rationale
74 C. TSDL Grammar
75 C.1 Lexical grammar
76 C.1.1 Lexical elements
77 C.1.2 Keywords
78 C.1.3 Identifiers
79 C.1.4 Universal character names
80 C.1.5 Constants
81 C.1.6 String literals
82 C.1.7 Punctuators
83 C.2 Phrase structure grammar
84 C.2.2 Declarations:
85 C.2.3 CTF-specific declarations
86
87
88## 1. Preliminary definitions
89
90 * **Event trace**: an ordered sequence of events.
91 * **Event stream**: an ordered sequence of events, containing a
92 subset of the trace event types.
93 * **Event packet**: a sequence of physically contiguous events within
94 an event stream.
95 * **Event**: this is the basic entry in a trace. Also known as
96 a _trace record_.
97 * An **event identifier** (ID) relates to the class (a type) of
98 event within an event stream, e.g. event `irq_entry`.
99 * An **event** (or event record) relates to a specific instance of
100 an event class, e.g. event `irq_entry`, at time _X_, on CPU _Y_.
101 * Source architecture: architecture writing the trace.
102 * Reader architecture: architecture reading the trace.
103
104
105## 2. High-level representation of a trace
106
107A _trace_ is divided into multiple event streams. Each event stream
108contains a subset of the trace event types.
109
110The final output of the trace, after its generation and optional
111transport over the network, is expected to be either on permanent or
112temporary storage in a virtual file system. Because each event stream
113is appended to while a trace is being recorded, each is associated with
114a distinct set of files for output. Therefore, a stored trace can be
115represented as a directory containing zero, one or more files
116per stream.
117
118Metadata description associated with the trace contains information on
119trace event types expressed in the _Trace Stream Description Language_
120(TSDL). This language describes:
121
122 * Trace version
123 * Types available
124 * Per-trace event header description
125 * Per-stream event header description
126 * Per-stream event context description
127 * Per-event
128 * Event type to stream mapping
129 * Event type to name mapping
130 * Event type to ID mapping
131 * Event context description
132 * Event fields description
133
134
135## 3. Event stream
136
137An _event stream_ can be divided into contiguous event packets of
138variable size. An event packet can contain a certain amount of padding
139at the end. The stream header is repeated at the beginning of each
140event packet. The rationale for the event stream design choices is
141explained in [Stream header rationale](#specB).
142
143The event stream header will therefore be referred to as the
144_event packet header_ throughout the rest of this document.
145
146
147## 4. Types
148
149Types are organized as type classes. Each type class belong to either
150of two kind of types: _basic types_ or _compound types_.
151
152
153### 4.1 Basic types
154
155A basic type is a scalar type, as described in this section. It
156includes integers, GNU/C bitfields, enumerations, and floating
157point values.
158
159
160#### 4.1.1 Type inheritance
161
162Type specifications can be inherited to allow deriving types from a
163type class. For example, see the uint32_t named type derived from the
164[_integer_ type](#spec4.1.5) class. Types have a precise binary
165representation in the trace. A type class has methods to read and write
166these types, but must be derived into a type to be usable in an event
167field.
168
169
170#### 4.1.2 Alignment
171
172We define _byte-packed_ types as aligned on the byte size, namely 8-bit.
173We define _bit-packed_ types as following on the next bit, as defined
174by the [Integers](#spec4.1.5) section.
175
176Each basic type must specify its alignment, in bits. Examples of
177possible alignments are: bit-packed (`align = 1`), byte-packed
178(`align = 8`), or word-aligned (e.g. `align = 32` or `align = 64`).
179The choice depends on the architecture preference and compactness vs
180performance trade-offs of the implementation. Architectures providing
181fast unaligned write byte-packed basic types to save space, aligning
182each type on byte boundaries (8-bit). Architectures with slow unaligned
183writes align types on specific alignment values. If no specific
184alignment is declared for a type, it is assumed to be bit-packed for
185integers with size not multiple of 8 bits and for gcc bitfields. All
186other basic types are byte-packed by default. It is however recommended
187to always specify the alignment explicitly. Alignment values must be
188power of two. Compound types are aligned as specified in their
189individual specification.
190
191The base offset used for field alignment is the start of the packet
192containing the field. For instance, a field aligned on 32-bit needs to
193be at an offset multiple of 32-bit from the start of the packet that
194contains it.
195
196TSDL metadata attribute representation of a specific alignment:
197
198~~~ tsdl
199align = /* value in bits */;
200~~~
201
202#### 4.1.3 Byte order
203
204By default, byte order of a basic type is the byte order described in
205the trace description. It can be overridden by specifying a
206`byte_order` attribute for a basic type. Typical use-case is to specify
207the network byte order (big endian: `be`) to save data captured from
208the network into the trace without conversion.
209
210TSDL metadata representation:
211
212~~~ tsdl
213/* network and be are aliases */
214byte_order = /* native OR network OR be OR le */;
215~~~
216
217The `native` keyword selects the byte order described in the trace
218description. The `network` byte order is an alias for big endian.
219
220Even though the trace description section is not per se a type, for
221sake of clarity, it should be noted that `native` and `network` byte
222orders are only allowed within type declaration. The `byte_order`
223specified in the trace description section only accepts `be` or `le`
224values.
225
226
227#### 4.1.4 Size
228
229Type size, in bits, for integers and floats is that returned by
230`sizeof()` in C multiplied by `CHAR_BIT`. We require the size of `char`
231and `unsigned char` types (`CHAR_BIT`) to be fixed to 8 bits for
232cross-endianness compatibility.
233
234TSDL metadata representation:
235
236~~~ tsdl
237size = /* value is in bits */;
238~~~
239
240
241#### 4.1.5 Integers
242
243Signed integers are represented in two-complement. Integer alignment,
244size, signedness and byte ordering are defined in the TSDL metadata.
245Integers aligned on byte size (8-bit) and with length multiple of byte
246size (8-bit) correspond to the C99 standard integers. In addition,
247integers with alignment and/or size that are _not_ a multiple of the
248byte size are permitted; these correspond to the C99 standard bitfields,
249with the added specification that the CTF integer bitfields have a fixed
250binary representation. Integer size needs to be a positive integer.
251Integers of size 0 are **forbidden**. An MIT-licensed reference
252implementation of the CTF portable bitfields is available
253[here](http://git.efficios.com/?p=babeltrace.git;a=blob;f=include/babeltrace/bitfield.h).
254
255Binary representation of integers:
256
257 * On little and big endian:
258 * Within a byte, high bits correspond to an integer high bits, and
259 low bits correspond to low bits
260 * On little endian:
261 * Integer across multiple bytes are placed from the less significant
262 to the most significant
263 * Consecutive integers are placed from lower bits to higher bits
264 (even within a byte)
265 * On big endian:
266 * Integer across multiple bytes are placed from the most significant
267 to the less significant
268 * Consecutive integers are placed from higher bits to lower bits
269 (even within a byte)
270
271This binary representation is derived from the bitfield implementation
272in GCC for little and big endian. However, contrary to what GCC does,
273integers can cross units boundaries (no padding is required). Padding
274can be [explicitly added](#spec4.1.6) to follow the GCC layout if needed.
275
276TSDL metadata representation:
277
278~~~ tsdl
279integer {
280 signed = /* true OR false */; /* default: false */
281 byte_order = /* native OR network OR be OR le */; /* default: native */
282 size = /* value in bits */; /* no default */
283 align = /* value in bits */;
284
285 /* base used for pretty-printing output; default: decimal */
286 base = /* decimal OR dec OR d OR i OR u OR 10 OR hexadecimal OR hex
287 OR x OR X OR p OR 16 OR octal OR oct OR o OR 8 OR binary
288 OR b OR 2 */;
289
290 /* character encoding */
291 encoding = /* none or UTF8 or ASCII */; /* default: none */
292}
293~~~
294
295Example of type inheritance (creation of a `uint32_t` named type):
296
297~~~ tsdl
298typealias integer {
299 size = 32;
300 signed = false;
301 align = 32;
302} := uint32_t;
303~~~
304
305Definition of a named 5-bit signed bitfield:
306
307~~~ tsdl
308typealias integer {
309 size = 5;
310 signed = true;
311 align = 1;
312} := int5_t;
313~~~
314
315The character encoding field can be used to specify that the integer
316must be printed as a text character when read. e.g.:
317
318~~~ tsdl
319typealias integer {
320 size = 8;
321 align = 8;
322 signed = false;
323 encoding = UTF8;
324} := utf_char;
325~~~
326
327#### 4.1.6 GNU/C bitfields
328
329The GNU/C bitfields follow closely the integer representation, with a
330particularity on alignment: if a bitfield cannot fit in the current
331unit, the unit is padded and the bitfield starts at the following unit.
332The unit size is defined by the size of the type `unit_type`.
333
334TSDL metadata representation:
335
336~~~ tsdl
337unit_type name:size;
338~~~
339
340As an example, the following structure declared in C compiled by GCC:
341
342~~~ tsdl
343struct example {
344 short a:12;
345 short b:5;
346};
347~~~
348
349The example structure is aligned on the largest element (short). The
350second bitfield would be aligned on the next unit boundary, because it
351would not fit in the current unit.
352
353
354#### 4.1.7 Floating point
355
356The floating point values byte ordering is defined in the TSDL metadata.
357
358Floating point values follow the IEEE 754-2008 standard interchange
359formats. Description of the floating point values include the exponent
360and mantissa size in bits. Some requirements are imposed on the
361floating point values:
362
363* `FLT_RADIX` must be 2.
364* `mant_dig` is the number of digits represented in the mantissa. It is
365 specified by the ISO C99 standard, section 5.2.4, as `FLT_MANT_DIG`,
366 `DBL_MANT_DIG` and `LDBL_MANT_DIG` as defined by `<float.h>`.
367* `exp_dig` is the number of digits represented in the exponent. Given
368 that `mant_dig` is one bit more than its actual size in bits (leading
369 1 is not needed) and also given that the sign bit always takes one
370 bit, `exp_dig` can be specified as:
371 * `sizeof(float) * CHAR_BIT - FLT_MANT_DIG`
372 * `sizeof(double) * CHAR_BIT - DBL_MANT_DIG`
373 * `sizeof(long double) * CHAR_BIT - LDBL_MANT_DIG`
374
375TSDL metadata representation:
376
377~~~ tsdl
378floating_point {
379 exp_dig = /* value */;
380 mant_dig = /* value */;
381 byte_order = /* native OR network OR be OR le */;
382 align = /* value */;
383}
384~~~
385
386Example of type inheritance:
387
388~~~ tsdl
389typealias floating_point {
390 exp_dig = 8; /* sizeof(float) * CHAR_BIT - FLT_MANT_DIG */
391 mant_dig = 24; /* FLT_MANT_DIG */
392 byte_order = native;
393 align = 32;
394} := float;
395~~~
396
397TODO: define NaN, +inf, -inf behavior.
398
399Bit-packed, byte-packed or larger alignments can be used for floating
400point values, similarly to integers.
401
402
403#### 4.1.8 Enumerations
404
405Enumerations are a mapping between an integer type and a table of
406strings. The numerical representation of the enumeration follows the
407integer type specified by the metadata. The enumeration mapping table
408is detailed in the enumeration description within the metadata. The
409mapping table maps inclusive value ranges (or single values) to strings.
410Instead of being limited to simple `value -> string` mappings, these
411enumerations map `[ start_value ... end_value ] -> string`, which map
412inclusive ranges of values to strings. An enumeration from the C
413language can be represented in this format by having the same
414`start_value` and `end_value` for each mapping, which is in fact a
415range of size 1. This single-value range is supported without repeating
416the start and end values with the `value = string` declaration.
417Enumerations need to contain at least one entry.
418
419~~~ tsdl
420enum name : integer_type {
421 somestring = /* start_value1 */ ... /* end_value1 */,
422 "other string" = /* start_value2 */ ... /* end_value2 */,
423 yet_another_string, /* will be assigned to end_value2 + 1 */
424 "some other string" = /* value */,
425 /* ... */
426}
427~~~
428
429If the values are omitted, the enumeration starts at 0 and increment
430of 1 for each entry. An entry with omitted value that follows a range
431entry takes as value the `end_value` of the previous range + 1:
432
433~~~ tsdl
434enum name : unsigned int {
435 ZERO,
436 ONE,
437 TWO,
438 TEN = 10,
439 ELEVEN,
440}
441~~~
442
443Overlapping ranges within a single enumeration are implementation
444defined.
445
446A nameless enumeration can be declared as a field type or as part of
447a `typedef`:
448
449~~~ tsdl
450enum : integer_type {
451 /* ... */
452}
453~~~
454
455Enumerations omitting the container type `: integer_type` use the `int`
456type (for compatibility with C99). The `int` type _must be_ previously
457declared, e.g.:
458
459~~~ tsdl
460typealias integer { size = 32; align = 32; signed = true; } := int;
461
462enum {
463 /* ... */
464}
465~~~
466
bc21db73
MD
467An enumeration field can have an integral value for which the associated
468enumeration type does not map to a string.
469
941a19cf
PP
470### 4.2 Compound types
471
472Compound are aggregation of type declarations. Compound types include
473structures, variant, arrays, sequences, and strings.
474
475
476#### 4.2.1 Structures
477
478Structures are aligned on the largest alignment required by basic types
479contained within the structure. (This follows the ISO/C standard for
480structures)
481
482TSDL metadata representation of a named structure:
483
484~~~ tsdl
485struct name {
486 field_type field_name;
487 field_type field_name;
488 /* ... */
489};
490~~~
491
492Example:
493
494~~~ tsdl
495struct example {
496 integer { /* nameless type */
497 size = 16;
498 signed = true;
499 align = 16;
500 } first_field_name;
501 uint64_t second_field_name; /* named type declared in the metadata */
502};
503~~~
504
505The fields are placed in a sequence next to each other. They each
506possess a field name, which is a unique identifier within the structure.
507The identifier is not allowed to use any [reserved keyword](#specC.1.2).
508Replacing reserved keywords with underscore-prefixed field names is
509**recommended**. Fields starting with an underscore should have their
510leading underscore removed by the CTF trace readers.
511
512A nameless structure can be declared as a field type or as part of
513a `typedef`:
514
515~~~ tsdl
516struct {
517 /* ... */
518}
519~~~
520
521Alignment for a structure compound type can be forced to a minimum
522value by adding an `align` specifier after the declaration of a
523structure body. This attribute is read as: `align(value)`. The value is
524specified in bits. The structure will be aligned on the maximum value
525between this attribute and the alignment required by the basic types
526contained within the structure. e.g.
527
528~~~ tsdl
529struct {
530 /* ... */
531} align(32)
532~~~
533
534#### 4.2.2 Variants (discriminated/tagged unions)
535
536A CTF variant is a selection between different types. A CTF variant must
537always be defined within the scope of a structure or within fields
538contained within a structure (defined recursively). A _tag_ enumeration
539field must appear in either the same static scope, prior to the variant
540field (in field declaration order), in an upper static scope, or in an
541upper dynamic scope (see [Static and dynamic scopes](#spec7.3.2)).
542The type selection is indicated by the mapping from the enumeration
543value to the string used as variant type selector. The field to use as
544tag is specified by the `tag_field`, specified between `< >` after the
545`variant` keyword for unnamed variants, and after _variant name_ for
546named variants. It is not required that each enumeration mapping appears
547as variant type tag field. It is also not required that each variant
548type tag appears as enumeration mapping. However, it is required that
549any enumeration mapping encountered within a stream has a matching
550variant type tag field.
551
552The alignment of the variant is the alignment of the type as selected
553by the tag value for the specific instance of the variant. The size of
554the variant is the size as selected by the tag value for the specific
555instance of the variant.
556
557The alignment of the type containing the variant is independent of the
558variant alignment. For instance, if a structure contains two fields, a
55932-bit integer, aligned on 32 bits, and a variant, which contains two
560choices: either a 32-bit field, aligned on 32 bits, or a 64-bit field,
561aligned on 64 bits, the alignment of the outmost structure will be
56232-bit (the alignment of its largest field, disregarding the alignment
563of the variant). The alignment of the variant will depend on the
564selector: if the variant's 32-bit field is selected, its alignment will
565be 32-bit, or 64-bit otherwise. It is important to note that variants
566are specifically tailored for compactness in a stream. Therefore, the
567relative offsets of compound type fields can vary depending on the
568offset at which the compound type starts if it contains a variant
569that itself contains a type with alignment larger than the largest field
570contained within the compound type. This is caused by the fact that the
571compound type may contain the enumeration that select the variant's
572choice, and therefore the alignment to be applied to the compound type
573cannot be determined before encountering the enumeration.
574
575Each variant type selector possess a field name, which is a unique
576identifier within the variant. The identifier is not allowed to use any
577[reserved keyword](#C.1.2). Replacing reserved keywords with
578underscore-prefixed field names is recommended. Fields starting with an
579underscore should have their leading underscore removed by the CTF trace
580readers.
581
582A named variant declaration followed by its definition within a
583structure declaration:
584
585~~~ tsdl
586variant name {
587 field_type sel1;
588 field_type sel2;
589 field_type sel3;
590 /* ... */
591};
592
593struct {
594 enum : integer_type { sel1, sel2, sel3, /* ... */ } tag_field;
595 /* ... */
596 variant name <tag_field> v;
597}
598~~~
599
600An unnamed variant definition within a structure is expressed by the
601following TSDL metadata:
602
603~~~ tsdl
604struct {
605 enum : integer_type { sel1, sel2, sel3, /* ... */ } tag_field;
606 /* ... */
607 variant <tag_field> {
608 field_type sel1;
609 field_type sel2;
610 field_type sel3;
611 /* ... */
612 } v;
613}
614~~~
615
616Example of a named variant within a sequence that refers to a single
617tag field:
618
619~~~ tsdl
620variant example {
621 uint32_t a;
622 uint64_t b;
623 short c;
624};
625
626struct {
627 enum : uint2_t { a, b, c } choice;
628 unsigned int seqlen;
629 variant example <choice> v[seqlen];
630}
631~~~
632
633Example of an unnamed variant:
634
635~~~ tsdl
636struct {
637 enum : uint2_t { a, b, c, d } choice;
638
639 /* Unrelated fields can be added between the variant and its tag */
640 int32_t somevalue;
641 variant <choice> {
642 uint32_t a;
643 uint64_t b;
644 short c;
645 struct {
646 unsigned int field1;
647 uint64_t field2;
648 } d;
649 } s;
650}
651~~~
652
653Example of an unnamed variant within an array:
654
655~~~ tsdl
656struct {
657 enum : uint2_t { a, b, c } choice;
658 variant <choice> {
659 uint32_t a;
660 uint64_t b;
661 short c;
662 } v[10];
663}
664~~~
665
666Example of a variant type definition within a structure, where the
667defined type is then declared within an array of structures. This
668variant refers to a tag located in an upper static scope. This example
669clearly shows that a variant type definition referring to the tag `x`
670uses the closest preceding field from the static scope of the type
671definition.
672
673~~~ tsdl
674struct {
675 enum : uint2_t { a, b, c, d } x;
676
677 /*
678 * "x" refers to the preceding "x" enumeration in the
679 * static scope of the type definition.
680 */
681 typedef variant <x> {
682 uint32_t a;
683 uint64_t b;
684 short c;
685 } example_variant;
686
687 struct {
688 enum : int { x, y, z } x; /* This enumeration is not used by "v". */
689
690 /* "v" uses the "enum : uint2_t { a, b, c, d }" tag. */
691 example_variant v;
692 } a[10];
693}
694~~~
695
696
697#### 4.2.3 Arrays
698
699Arrays are fixed-length. Their length is declared in the type
700declaration within the metadata. They contain an array of _inner type_
701elements, which can refer to any type not containing the type of the
702array being declared (no circular dependency). The length is the number
703of elements in an array.
704
705TSDL metadata representation of a named array:
706
707~~~ tsdl
708typedef elem_type name[/* length */];
709~~~
710
711A nameless array can be declared as a field type within a
712structure, e.g.:
713
714~~~ tsdl
715uint8_t field_name[10];
716~~~
717
718Arrays are always aligned on their element alignment requirement.
719
720
721#### 4.2.4 Sequences
722
723Sequences are dynamically-sized arrays. They refer to a _length_
724unsigned integer field, which must appear in either the same static
725scope, prior to the sequence field (in field declaration order),
726in an upper static scope, or in an upper dynamic scope
727(see [Static and dynamic scopes](#spec7.3.2)). This length field represents
728the number of elements in the sequence. The sequence per se is an
729array of _inner type_ elements.
730
731TSDL metadata representation for a sequence type definition:
732
733~~~ tsdl
734struct {
735 unsigned int length_field;
736 typedef elem_type typename[length_field];
737 typename seq_field_name;
738}
739~~~
740
741A sequence can also be declared as a field type, e.g.:
742
743~~~ tsdl
744struct {
745 unsigned int length_field;
746 long seq_field_name[length_field];
747}
748~~~
749
750Multiple sequences can refer to the same length field, and these length
751fields can be in a different upper dynamic scope, e.g., assuming the
752`stream.event.header` defines:
753
754~~~ tsdl
755stream {
756 /* ... */
757 id = 1;
758 event.header := struct {
759 uint16_t seq_len;
760 };
761};
762
763event {
764 /* ... */
765 stream_id = 1;
766 fields := struct {
767 long seq_a[stream.event.header.seq_len];
768 char seq_b[stream.event.header.seq_len];
769 };
770};
771~~~
772
773The sequence elements follow the [array](#spec4.2.3) specifications.
774
775
776#### 4.2.5 Strings
777
778Strings are an array of _bytes_ of variable size and are terminated by
779a `'\0'` "NULL" character. Their encoding is described in the TSDL
780metadata. In absence of encoding attribute information, the default
781encoding is UTF-8.
782
783TSDL metadata representation of a named string type:
784
785~~~ tsdl
786typealias string {
787 encoding = /* UTF8 OR ASCII */;
788} := name;
789~~~
790
791A nameless string type can be declared as a field type:
792
793~~~ tsdl
794string field_name; /* use default UTF8 encoding */
795~~~
796
797Strings are always aligned on byte size.
798
799
800## 5. Event packet header
801
802The event packet header consists of two parts: the
803_event packet header_ is the same for all streams of a trace. The
804second part, the _event packet context_, is described on a per-stream
805basis. Both are described in the TSDL metadata.
806
807Event packet header (all fields are optional, specified by
808TSDL metadata):
809
810 * **Magic number** (CTF magic number: 0xC1FC1FC1) specifies that this is
811 a CTF packet. This magic number is optional, but when present, it
812 should come at the very beginning of the packet.
813 * **Trace UUID**, used to ensure the event packet match the metadata used.
814 Note: we cannot use a metadata checksum in every cases instead of a
815 UUID because metadata can be appended to while tracing is active.
816 This field is optional.
817 * **Stream ID**, used as reference to stream description in metadata.
818 This field is optional if there is only one stream description in
819 the metadata, but becomes required if there are more than one
820 stream in the TSDL metadata description.
821
822Event packet context (all fields are optional, specified by
823TSDL metadata):
824
825 * Event packet **content size** (in bits).
826 * Event packet **size** (in bits, includes padding).
827 * Event packet content checksum. Checksum excludes the event packet
828 header.
829 * Per-stream event **packet sequence count** (to deal with UDP packet
830 loss). The number of significant sequence counter bits should also
831 be present, so wrap-arounds are dealt with correctly.
832 * Time-stamp at the beginning and timestamp at the end of the event
833 packet. Both timestamps are written in the packet header, but
834 sampled respectively while (or before) writing the first event and
835 while (or after) writing the last event in the packet. The inclusive
836 range between these timestamps should include all event timestamps
837 assigned to events contained within the packet. The timestamp at the
838 beginning of an event packet is guaranteed to be below or equal the
6c466647 839 timestamp at the end of that event packet. The timestamp at the
ed8509fc
MD
840 beginning of an event packet is guaranteed to be grater than or
841 equal to timestamps at the beginning of any prior packet within the
842 same stream. The timestamp at the end of an event packet is
843 guaranteed to be less than or equal to the timestamps at the end of
844 any following packet within the same stream. See [Clocks](#spec8)
845 for more detail.
941a19cf
PP
846 * **Events discarded count**. Snapshot of a per-stream
847 free-running counter, counting the number of events discarded that
848 were supposed to be written in the stream after the last event in
849 the event packet. Note: producer-consumer buffer full condition can
850 fill the current event packet with padding so we know exactly where
851 events have been discarded. However, if the buffer full condition
852 chooses not to fill the current event packet with padding, all we
853 know about the timestamp range in which the events have been
854 discarded is that it is somewhere between the beginning and the end
855 of the packet.
856 * Lossless **compression scheme** used for the event packet content.
857 Applied directly to raw data. New types of compression can be added
858 in following versions of the format.
859 * 0: no compression scheme
860 * 1: bzip2
861 * 2: gzip
862 * 3: xz
863 * **Cypher** used for the event packet content. Applied after
864 compression.
865 * 0: no encryption
866 * 1: AES
867 * **Checksum scheme** used for the event packet content. Applied after
868 encryption.
869 * 0: no checksum
870 * 1: md5
871 * 2: sha1
872 * 3: crc32
873
874
875### 5.1 Event packet header description
876
877The event packet header layout is indicated by the
878`trace.packet.header` field. Here is a recommended structure type for
879the packet header with the fields typically expected (although these
880fields are each optional):
881
882~~~ tsdl
883struct event_packet_header {
884 uint32_t magic;
885 uint8_t uuid[16];
886 uint32_t stream_id;
887};
888
889trace {
890 /* ... */
891 packet.header := struct event_packet_header;
892};
893~~~
894
9ab09151
PP
895If the magic number (`magic` field) is not present,
896tools such as `file` will have no mean to discover the file type.
941a19cf 897
9ab09151
PP
898If the `uuid` field is not present, no validation that the metadata
899actually corresponds to the stream is performed.
941a19cf 900
9ab09151 901If the `stream_id` packet header field is missing, the trace can only
941a19cf
PP
902contain a single stream. Its `id` field can be left out, and its events
903don't need to declare a `stream_id` field.
904
905
906### 5.2 Event packet context description
907
908Event packet context example. These are declared within the stream
909declaration in the metadata. All these fields are optional. If the
910packet size field is missing, the whole stream only contains a single
911packet. If the content size field is missing, the packet is filled
912(no padding). The content and packet sizes include all headers.
913
914An example event packet context type:
915
916~~~ tsdl
917struct event_packet_context {
918 uint64_t timestamp_begin;
919 uint64_t timestamp_end;
920 uint32_t checksum;
921 uint32_t stream_packet_count;
922 uint32_t events_discarded;
923 uint32_t cpu_id;
924 uint64_t content_size;
925 uint64_t packet_size;
926 uint8_t compression_scheme;
927 uint8_t encryption_scheme;
928 uint8_t checksum_scheme;
929};
930~~~
931
932
933## 6. Event Structure
934
935The overall structure of an event is:
936
937 1. Event header (as specified by the stream metadata)
938 2. Stream event context (as specified by the stream metadata)
939 3. Event context (as specified by the event metadata)
940 4. Event payload (as specified by the event metadata)
941
942This structure defines an implicit dynamic scoping, where variants
943located in inner structures (those with a higher number in the listing
944above) can refer to the fields of outer structures (with lower number
945in the listing above). See [TSDL scopes](#spec7.3) for more detail.
946
947The total length of an event is defined as the difference between the
948end of its event payload and the end of the previous event's event
949payload. Therefore, it includes the event header alignment padding, and
950all its fields and their respective alignment padding. Events of length
9510 are forbidden.
952
953
954### 6.1 Event header
955
956Event headers can be described within the metadata. We hereby propose,
957as an example, two types of events headers. Type 1 accommodates streams
958with less than 31 event IDs. Type 2 accommodates streams with 31 or
959more event IDs.
960
961One major factor can vary between streams: the number of event IDs
962assigned to a stream. Luckily, this information tends to stay
963relatively constant (modulo event registration while trace is being
964recorded), so we can specify different representations for streams
965containing few event IDs and streams containing many event IDs, so we
966end up representing the event ID and timestamp as densely as possible
967in each case.
968
969The header is extended in the rare occasions where the information
970cannot be represented in the ranges available in the standard event
971header. They are also used in the rare occasions where the data
972required for a field could not be collected: the flag corresponding to
973the missing field within the `missing_fields` array is then set to 1.
974
975Types `uintX_t` represent an `X`-bit unsigned integer, as declared with
976either:
977
978~~~ tsdl
979typealias integer {
980 size = /* X */;
981 align = /* X */;
982 signed = false;
983} := uintX_t;
984~~~
985
986or
987
988~~~ tsdl
989typealias integer {
990 size = /* X */;
991 align = 1;
992 signed = false;
993} := uintX_t;
994~~~
995
996For more information about timestamp fields, see [Clocks](#spec8).
997
998
999#### 6.1.1 Type 1: few event IDs
1000
1001 * Aligned on 32-bit (or 8-bit if byte-packed, depending on the
1002 architecture preference)
1003 * Native architecture byte ordering
1004 * For `compact` selection, fixed size of 32 bits
1005 * For "extended" selection, size depends on the architecture and
1006 variant alignment
1007
1008~~~ tsdl
1009struct event_header_1 {
1010 /*
1011 * id: range: 0 - 30.
1012 * id 31 is reserved to indicate an extended header.
1013 */
1014 enum : uint5_t { compact = 0 ... 30, extended = 31 } id;
1015 variant <id> {
1016 struct {
1017 uint27_t timestamp;
1018 } compact;
1019 struct {
1020 uint32_t id; /* 32-bit event IDs */
1021 uint64_t timestamp; /* 64-bit timestamps */
1022 } extended;
1023 } v;
1024} align(32); /* or align(8) */
1025~~~
1026
1027
1028#### 6.1.2 Type 2: many event IDs
1029
1030 * Aligned on 16-bit (or 8-bit if byte-packed, depending on the
1031 architecture preference)
1032 * Native architecture byte ordering
1033 * For `compact` selection, size depends on the architecture and
1034 variant alignment
1035 * For `extended` selection, size depends on the architecture and
1036 variant alignment
1037
1038~~~ tsdl
1039struct event_header_2 {
1040 /*
1041 * id: range: 0 - 65534.
1042 * id 65535 is reserved to indicate an extended header.
1043 */
1044 enum : uint16_t { compact = 0 ... 65534, extended = 65535 } id;
1045 variant <id> {
1046 struct {
1047 uint32_t timestamp;
1048 } compact;
1049 struct {
1050 uint32_t id; /* 32-bit event IDs */
1051 uint64_t timestamp; /* 64-bit timestamps */
1052 } extended;
1053 } v;
1054} align(16); /* or align(8) */
1055~~~
1056
1057
1058### 6.2 Stream event context and event context
1059
1060The event context contains information relative to the current event.
1061The choice and meaning of this information is specified by the TSDL
1062stream and event metadata descriptions. The stream context is applied
1063to all events within the stream. The stream context structure follows
1064the event header. The event context is applied to specific events. Its
1065structure follows the stream context structure.
1066
1067An example of stream-level event context is to save the event payload
1068size with each event, or to save the current PID with each event.
1069These are declared within the stream declaration within the metadata:
1070
1071~~~ tsdl
1072stream {
1073 /* ... */
1074 event.context := struct {
1075 uint pid;
1076 uint16_t payload_size;
1077 };
1078};
1079~~~
1080
1081An example of event-specific event context is to declare a bitmap of
1082missing fields, only appended after the stream event context if the
1083extended event header is selected. `NR_FIELDS` is the number of fields
1084within the event (a numeric value).
1085
1086~~~ tsdl
1087event {
1088 context := struct {
1089 variant <id> {
1090 struct { } compact;
1091 struct {
1092 /* missing event fields bitmap */
1093 uint1_t missing_fields[NR_FIELDS];
1094 } extended;
1095 } v;
1096 };
1097 /* ... */
1098}
1099~~~
1100
1101
1102### 6.3 Event payload
1103
1104An event payload contains fields specific to a given event type. The
1105fields belonging to an event type are described in the event-specific
1106metadata within a structure type.
1107
1108
1109#### 6.3.1 Padding
1110
1111No padding at the end of the event payload. This differs from the ISO/C
1112standard for structures, but follows the CTF standard for structures.
1113In a trace, even though it makes sense to align the beginning of a
1114structure, it really makes no sense to add padding at the end of the
1115structure, because structures are usually not followed by a structure
1116of the same type.
1117
1118This trick can be done by adding a zero-length `end` field at the end
1119of the C structures, and by using the offset of this field rather than
1120using `sizeof()` when calculating the size of a structure
1121(see [Helper macros](#specA)).
1122
1123
1124#### 6.3.2 Alignment
1125
1126The event payload is aligned on the largest alignment required by types
1127contained within the payload. This follows the ISO/C standard for
1128structures.
1129
1130
1131## 7. Trace Stream Description Language (TSDL)
1132
1133The Trace Stream Description Language (TSDL) allows expression of the
1134binary trace streams layout in a C99-like Domain Specific Language
1135(DSL).
1136
1137
1138### 7.1 Meta-data
1139
1140The trace stream layout description is located in the trace metadata.
1141The metadata is itself located in a stream identified by its name:
1142`metadata`.
1143
1144The metadata description can be expressed in two different formats:
1145text-only and packet-based. The text-only description facilitates
1146generation of metadata and provides a convenient way to enter the
1147metadata information by hand. The packet-based metadata provides the
1148CTF stream packet facilities (checksumming, compression, encryption,
1149network-readiness) for metadata stream generated and transported by a
1150tracer.
1151
1152The text-only metadata file is a plain-text TSDL description. This file
1153must begin with the following characters to identify the file as a CTF
1154TSDL text-based metadata file (without the double-quotes):
1155
1156~~~ text
1157"/* CTF"
1158~~~
1159
1160It must be followed by a space, and the version of the specification
1161followed by the CTF trace, e.g.:
1162
1163~~~ text
1164" 1.8"
1165~~~
1166
1167These characters allow automated discovery of file type and CTF
1168specification version. They are interpreted as a the beginning of a
1169comment by the TSDL metadata parser. The comment can be continued to
1170contain extra commented characters before it is closed.
1171
1172The packet-based metadata is made of _metadata packets_, which each
1173start with a metadata packet header. The packet-based metadata
1174description is detected by reading the magic number 0x75D11D57 at the
1175beginning of the file. This magic number is also used to detect the
1176endianness of the architecture by trying to read the CTF magic number
1177and its counterpart in reversed endianness. The events within the
1178metadata stream have no event header nor event context. Each event only
1179contains a special _sequence_ payload, which is a sequence of bits which
1180length is implicitly calculated by using the
1181`trace.packet.header.content_size` field, minus the packet header size.
1182The formatting of this sequence of bits is a plain-text representation
1183of the TSDL description. Each metadata packet start with a special
1184packet header, specific to the metadata stream, which contains,
1185exactly:
1186
1187~~~ tsdl
1188struct metadata_packet_header {
1189 uint32_t magic; /* 0x75D11D57 */
1190 uint8_t uuid[16]; /* Unique Universal Identifier */
1191 uint32_t checksum; /* 0 if unused */
1192 uint32_t content_size; /* in bits */
1193 uint32_t packet_size; /* in bits */
1194 uint8_t compression_scheme; /* 0 if unused */
1195 uint8_t encryption_scheme; /* 0 if unused */
1196 uint8_t checksum_scheme; /* 0 if unused */
1197 uint8_t major; /* CTF spec version major number */
1198 uint8_t minor; /* CTF spec version minor number */
1199};
1200~~~
1201
1202The packet-based metadata can be converted to a text-only metadata by
1203concatenating all the strings it contains.
1204
1205In the textual representation of the metadata, the text contained
1206within `/*` and `*/`, as well as within `//` and end of line, are
1207treated as comments. Boolean values can be represented as `true`,
1208`TRUE`, or `1` for true, and `false`, `FALSE`, or `0` for false. Within
1209the string-based metadata description, the trace UUID is represented as
1210a string of hexadecimal digits and dashes `-`. In the event packet
1211header, the trace UUID is represented as an array of bytes.
1212
1213
1214### 7.2 Declaration vs definition
1215
1216A declaration associates a layout to a type, without specifying where
1217this type is located in the event [structure hierarchy](#spec6).
1218This therefore includes `typedef`, `typealias`, as well as all type
1219specifiers. In certain circumstances (`typedef`, structure field and
1220variant field), a declaration is followed by a declarator, which specify
1221the newly defined type name (for `typedef`), or the field name (for
1222declarations located within structure and variants). Array and sequence,
1223declared with square brackets (`[` `]`), are part of the declarator,
1224similarly to C99. The enumeration base type is specified by
1225`: enum_base`, which is part of the type specifier. The variant tag
1226name, specified between `<` `>`, is also part of the type specifier.
1227
1228A definition associates a type to a location in the event
1229[structure hierarchy](#spec6). This association is denoted by `:=`,
1230as shown in [TSDL scopes](#spec7.3).
1231
1232
1233### 7.3 TSDL scopes
1234
1235TSDL uses three different types of scoping: a lexical scope is used for
1236declarations and type definitions, and static and dynamic scopes are
1237used for variants references to tag fields (with relative and absolute
1238path lookups) and for sequence references to length fields.
1239
1240
1241#### 7.3.1 Lexical Scope
1242
1243Each of `trace`, `env`, `stream`, `event`, `struct` and `variant` have
1244their own nestable declaration scope, within which types can be declared
1245using `typedef` and `typealias`. A root declaration scope also contains
1246all declarations located outside of any of the aforementioned
1247declarations. An inner declaration scope can refer to type declared
1248within its container lexical scope prior to the inner declaration scope.
1249Redefinition of a typedef or typealias is not valid, although hiding an
1250upper scope typedef or typealias is allowed within a sub-scope.
1251
1252
1253#### 7.3.2 Static and dynamic scopes
1254
1255A local static scope consists in the scope generated by the declaration
1256of fields within a compound type. A static scope is a local static scope
1257augmented with the nested sub-static-scopes it contains.
1258
1259A dynamic scope consists in the static scope augmented with the
1260implicit [event structure](#spec6) definition hierarchy.
1261
1262Multiple declarations of the same field name within a local static scope
1263is not valid. It is however valid to re-use the same field name in
1264different local scopes.
1265
1266Nested static and dynamic scopes form lookup paths. These are used for
1267variant tag and sequence length references. They are used at the variant
1268and sequence definition site to look up the location of the tag field
1269associated with a variant, and to lookup up the location of the length
1270field associated with a sequence.
1271
1272Variants and sequences can refer to a tag field either using a relative
1273path or an absolute path. The relative path is relative to the scope in
1274which the variant or sequence performing the lookup is located.
1275Relative paths are only allowed to lookup within the same static scope,
1276which includes its nested static scopes. Lookups targeting parent static
1277scopes need to be performed with an absolute path.
1278
1279Absolute path lookups use the full path including the dynamic scope
1280followed by a `.` and then the static scope. Therefore, variants (or
1281sequences) in lower levels in the dynamic scope (e.g., event context)
1282can refer to a tag (or length) field located in upper levels
1283(e.g., in the event header) by specifying, in this case, the associated
1284tag with `<stream.event.header.field_name>`. This allows, for instance,
1285the event context to define a variant referring to the `id` field of
1286the event header as selector.
1287
1288The dynamic scope prefixes are thus:
1289
1290 * Trace environment: `<env. >`
1291 * Trace packet header: `<trace.packet.header. >`
1292 * Stream packet context: `<stream.packet.context. >`
1293 * Event header: `<stream.event.header. >`
1294 * Stream event context: `<stream.event.context. >`
1295 * Event context: `<event.context. >`
1296 * Event payload: `<event.fields. >`
1297
1298The target dynamic scope must be specified explicitly when referring to
1299a field outside of the static scope (absolute scope reference). No
1300conflict can occur between relative and dynamic paths, because the
1301keywords `trace`, `stream`, and `event` are reserved, and thus not
1302permitted as field names. It is recommended that field names clashing
1303with CTF and C99 reserved keywords use an underscore prefix to
1304eliminate the risk of generating a description containing an invalid
1305field name. Consequently, fields starting with an underscore should have
1306their leading underscore removed by the CTF trace readers.
1307
1308The information available in the dynamic scopes can be thought of as the
1309current tracing context. At trace production, information about the
1310current context is saved into the specified scope field levels. At trace
1311consumption, for each event, the current trace context is therefore
1312readable by accessing the upper dynamic scopes.
1313
1314
1315### 7.4 TSDL examples
1316
1317The grammar representing the TSDL metadata is presented in
1318[TSDL grammar](#specC). This section presents a rather lighter reading that
1319consists in examples of TSDL metadata, with template values.
1320
1321The stream ID can be left out if there is only one stream in the
1322trace. The event `id` field can be left out if there is only one event
1323in a stream.
1324
1325~~~ tsdl
1326trace {
1327 major = /* value */; /* CTF spec version major number */
1328 minor = /* value */; /* CTF spec version minor number */
1329 uuid = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; /* Trace UUID */
1330 byte_order = /* be OR le */; /* Endianness (required) */
1331 packet.header := struct {
1332 uint32_t magic;
1333 uint8_t uuid[16];
1334 uint32_t stream_id;
1335 };
1336};
1337
1338/*
1339 * The "env" (environment) scope contains assignment expressions. The
1340 * field names and content are implementation-defined.
1341 */
1342env {
1343 pid = /* value */; /* example */
1344 proc_name = "name"; /* example */
1345 /* ... */
1346};
1347
1348stream {
1349 id = /* stream_id */;
1350 /* Type 1 - Few event IDs; Type 2 - Many event IDs. See section 6.1. */
1351 event.header := /* event_header_1 OR event_header_2 */;
1352 event.context := struct {
1353 /* ... */
1354 };
1355 packet.context := struct {
1356 /* ... */
1357 };
1358};
1359
1360event {
1361 name = "event_name";
1362 id = /* value */; /* Numeric identifier within the stream */
1363 stream_id = /* stream_id */;
1364 loglevel = /* value */;
1365 model.emf.uri = "string";
1366 context := struct {
1367 /* ... */
1368 };
1369 fields := struct {
1370 /* ... */
1371 };
1372};
1373
1374callsite {
1375 name = "event_name";
1376 func = "func_name";
1377 file = "myfile.c";
1378 line = 39;
1379 ip = 0x40096c;
1380};
1381~~~
1382
1383More detail on [types](#spec4):
1384
1385~~~ tsdl
1386/*
1387 * Named types:
1388 *
1389 * Type declarations behave similarly to the C standard.
1390 */
1391
1392typedef aliased_type_specifiers new_type_declarators;
1393
1394/* e.g.: typedef struct example new_type_name[10]; */
1395
1396/*
1397 * typealias
1398 *
1399 * The "typealias" declaration can be used to give a name (including
1400 * pointer declarator specifier) to a type. It should also be used to
1401 * map basic C types (float, int, unsigned long, ...) to a CTF type.
1402 * Typealias is a superset of "typedef": it also allows assignment of a
1403 * simple variable identifier to a type.
1404 */
1405
1406typealias type_class {
1407 /* ... */
1408} := type_specifiers type_declarator;
1409
1410/*
1411 * e.g.:
1412 * typealias integer {
1413 * size = 32;
1414 * align = 32;
1415 * signed = false;
1416 * } := struct page *;
1417 *
1418 * typealias integer {
1419 * size = 32;
1420 * align = 32;
1421 * signed = true;
1422 * } := int;
1423 */
1424
1425struct name {
1426 /* ... */
1427};
1428
1429variant name {
1430 /* ... */
1431};
1432
1433enum name : integer_type {
1434 /* ... */
1435};
1436~~~
1437
1438Unnamed types, contained within compound type fields, `typedef` or
1439`typealias`:
1440
1441~~~ tsdl
1442struct {
1443 /* ... */
1444}
1445~~~
1446
1447~~~ tsdl
1448struct {
1449 /* ... */
1450} align(value)
1451~~~
1452
1453~~~ tsdl
1454variant {
1455 /* ... */
1456}
1457~~~
1458
1459~~~ tsdl
1460enum : integer_type {
1461 /* ... */
1462}
1463~~~
1464
1465~~~ tsdl
1466typedef type new_type[length];
1467
1468struct {
1469 type field_name[length];
1470}
1471~~~
1472
1473~~~ tsdl
1474typedef type new_type[length_type];
1475
1476struct {
1477 type field_name[length_type];
1478}
1479~~~
1480
1481~~~ tsdl
1482integer {
1483 /* ... */
1484}
1485~~~
1486
1487~~~ tsdl
1488floating_point {
1489 /* ... */
1490}
1491~~~
1492
1493~~~ tsdl
1494struct {
1495 integer_type field_name:size; /* GNU/C bitfield */
1496}
1497~~~
1498
1499~~~ tsdl
1500struct {
1501 string field_name;
1502}
1503~~~
1504
1505
1506## 8. Clocks
1507
1508Clock metadata allows to describe the clock topology of the system, as
1509well as to detail each clock parameter. In absence of clock description,
1510it is assumed that all fields named `timestamp` use the same clock
1511source, which increments once per nanosecond.
1512
1513Describing a clock and how it is used by streams is threefold: first,
1514the clock and clock topology should be described in a `clock`
1515description block, e.g.:
1516
1517~~~ tsdl
1518clock {
1519 name = cycle_counter_sync;
1520 uuid = "62189bee-96dc-11e0-91a8-cfa3d89f3923";
1521 description = "Cycle counter synchronized across CPUs";
1522 freq = 1000000000; /* frequency, in Hz */
1523 /* precision in seconds is: 1000 * (1/freq) */
1524 precision = 1000;
1525 /*
1526 * clock value offset from Epoch is:
1527 * offset_s + (offset * (1/freq))
1528 */
1529 offset_s = 1326476837;
1530 offset = 897235420;
1531 absolute = FALSE;
1532};
1533~~~
1534
1535The mandatory `name` field specifies the name of the clock identifier,
1536which can later be used as a reference. The optional field `uuid` is
1537the unique identifier of the clock. It can be used to correlate
1538different traces that use the same clock. An optional textual
1539description string can be added with the `description` field. The
1540`freq` field is the initial frequency of the clock, in Hz. If the
1541`freq` field is not present, the frequency is assumed to be 1000000000
1542(providing clock increment of 1 ns). The optional `precision` field
1543details the uncertainty on the clock measurements, in (1/freq) units.
1544The `offset_s` and `offset` fields indicate the offset from
1545POSIX.1 Epoch, 1970-01-01 00:00:00 +0000 (UTC), to the zero of value
1546of the clock. The `offset_s` field is in seconds. The `offset` field is
1547in (1/freq) units. If any of the `offset_s` or `offset` field is not
1548present, it is assigned the 0 value. The field `absolute` is `TRUE` if
1549the clock is a global reference across different clock UUID
1550(e.g. NTP time). Otherwise, `absolute` is `FALSE`, and the clock can
1551be considered as synchronized only with other clocks that have the same
1552UUID.
1553
1554Secondly, a reference to this clock should be added within an integer
1555type:
1556
1557~~~ tsdl
1558typealias integer {
1559 size = 64; align = 1; signed = false;
1560 map = clock.cycle_counter_sync.value;
1561} := uint64_ccnt_t;
1562~~~
1563
1564Thirdly, stream declarations can reference the clock they use as a
1565timestamp source:
1566
1567~~~ tsdl
1568struct packet_context {
1569 uint64_ccnt_t ccnt_begin;
1570 uint64_ccnt_t ccnt_end;
1571 /* ... */
1572};
1573
1574stream {
1575 /* ... */
1576 event.header := struct {
1577 uint64_ccnt_t timestamp;
1578 /* ... */
1579 };
1580 packet.context := struct packet_context;
1581};
1582~~~
1583
ed8509fc
MD
1584Within the stream event context, event context, and event payload,
1585fields of N-bit integer type referring to a clock, if the integer overflows
941a19cf
PP
1586compared to the N low order bits of the clock prior value found in the
1587same stream, then it is assumed that one, and only one, overflow
1588occurred. It is therefore important that events encoding time on a small
1589number of bits happen frequently enough to detect when more than one
1590N-bit overflow occurs.
1591
1592In a packet context, clock field names ending with `_begin` and `_end`
1593have a special meaning: this refers to the timestamps at, respectively,
ed8509fc
MD
1594the beginning and the end of each packet. Those are required to be
1595complete representations of the clock value.
941a19cf
PP
1596
1597## A. Helper macros
1598
1599The two following macros keep track of the size of a GNU/C structure
1600without padding at the end by placing HEADER_END as the last field.
1601A one byte end field is used for C90 compatibility (C99 flexible arrays
1602could be used here). Note that this does not affect the effective
1603structure size, which should always be calculated with the
1604`header_sizeof()` helper.
1605
1606~~~ c
1607#define HEADER_END char end_field
1608#define header_sizeof(type) offsetof(typeof(type), end_field)
1609~~~
1610
1611## B. Stream header rationale
1612
1613An event stream is divided in contiguous event packets of variable
1614size. These subdivisions allow the trace analyzer to perform a fast
1615binary search by time within the stream (typically requiring to index
1616only the event packet headers) without reading the whole stream. These
1617subdivisions have a variable size to eliminate the need to transfer the
1618event packet padding when partially filled event packets must be sent
1619when streaming a trace for live viewing/analysis. An event packet can
1620contain a certain amount of padding at the end. Dividing streams into
1621event packets is also useful for network streaming over UDP and flight
1622recorder mode tracing (a whole event packet can be swapped out of the
1623buffer atomically for reading).
1624
1625The stream header is repeated at the beginning of each event packet to
1626allow flexibility in terms of:
1627
1628 * streaming support
1629 * allowing arbitrary buffers to be discarded without making the trace
1630 unreadable
1631 * allow UDP packet loss handling by either dealing with missing event packet
1632 or asking for re-transmission
1633 * transparently support flight recorder mode
1634 * transparently support crash dump
1635
1636
1637## C. TSDL Grammar
1638
1639~~~ c
1640/*
1641 * Common Trace Format (CTF) Trace Stream Description Language (TSDL) Grammar.
1642 *
1643 * Inspired from the C99 grammar:
1644 * http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf (Annex A)
1645 * and c++1x grammar (draft)
1646 * http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3291.pdf (Annex A)
1647 *
1648 * Specialized for CTF needs by including only constant and declarations from
1649 * C99 (excluding function declarations), and by adding support for variants,
1650 * sequences and CTF-specific specifiers. Enumeration container types
1651 * semantic is inspired from c++1x enum-base.
1652 */
1653~~~
1654
1655
1656### C.1 Lexical grammar
1657
1658
1659#### C.1.1 Lexical elements
1660
1661~~~ text
1662token:
1663 keyword
1664 identifier
1665 constant
1666 string-literal
1667 punctuator
1668~~~
1669
1670#### C.1.2 Keywords
1671
1672~~~ text
1673keyword: is one of
1674
1675align
1676callsite
1677const
1678char
1679clock
1680double
1681enum
1682env
1683event
1684floating_point
1685float
1686integer
1687int
1688long
1689short
1690signed
1691stream
1692string
1693struct
1694trace
1695typealias
1696typedef
1697unsigned
1698variant
1699void
1700_Bool
1701_Complex
1702_Imaginary
1703~~~
1704
1705
1706#### C.1.3 Identifiers
1707
1708~~~ text
1709identifier:
1710 identifier-nondigit
1711 identifier identifier-nondigit
1712 identifier digit
1713
1714identifier-nondigit:
1715 nondigit
1716 universal-character-name
1717 any other implementation-defined characters
1718
1719nondigit:
1720 _
1721 [a-zA-Z] /* regular expression */
1722
1723digit:
1724 [0-9] /* regular expression */
1725~~~
1726
1727
1728#### C.1.4 Universal character names
1729
1730~~~ text
1731universal-character-name:
1732 \u hex-quad
1733 \U hex-quad hex-quad
1734
1735hex-quad:
1736 hexadecimal-digit hexadecimal-digit hexadecimal-digit hexadecimal-digit
1737~~~
1738
1739
1740##### C.1.5 Constants
1741
1742~~~ text
1743constant:
1744 integer-constant
1745 enumeration-constant
1746 character-constant
1747
1748integer-constant:
1749 decimal-constant integer-suffix-opt
1750 octal-constant integer-suffix-opt
1751 hexadecimal-constant integer-suffix-opt
1752
1753decimal-constant:
1754 nonzero-digit
1755 decimal-constant digit
1756
1757octal-constant:
1758 0
1759 octal-constant octal-digit
1760
1761hexadecimal-constant:
1762 hexadecimal-prefix hexadecimal-digit
1763 hexadecimal-constant hexadecimal-digit
1764
1765hexadecimal-prefix:
1766 0x
1767 0X
1768
1769nonzero-digit:
1770 [1-9]
1771
1772integer-suffix:
1773 unsigned-suffix long-suffix-opt
1774 unsigned-suffix long-long-suffix
1775 long-suffix unsigned-suffix-opt
1776 long-long-suffix unsigned-suffix-opt
1777
1778unsigned-suffix:
1779 u
1780 U
1781
1782long-suffix:
1783 l
1784 L
1785
1786long-long-suffix:
1787 ll
1788 LL
1789
1790enumeration-constant:
1791 identifier
1792 string-literal
1793
1794character-constant:
1795 ' c-char-sequence '
1796 L' c-char-sequence '
1797
1798c-char-sequence:
1799 c-char
1800 c-char-sequence c-char
1801
1802c-char:
1803 any member of source charset except single-quote ('), backslash
1804 (\), or new-line character.
1805 escape-sequence
1806
1807escape-sequence:
1808 simple-escape-sequence
1809 octal-escape-sequence
1810 hexadecimal-escape-sequence
1811 universal-character-name
1812
1813simple-escape-sequence: one of
1814 \' \" \? \\ \a \b \f \n \r \t \v
1815
1816octal-escape-sequence:
1817 \ octal-digit
1818 \ octal-digit octal-digit
1819 \ octal-digit octal-digit octal-digit
1820
1821hexadecimal-escape-sequence:
1822 \x hexadecimal-digit
1823 hexadecimal-escape-sequence hexadecimal-digit
1824~~~
1825
1826
1827#### C.1.6 String literals
1828
1829~~~ text
1830string-literal:
1831 " s-char-sequence-opt "
1832 L" s-char-sequence-opt "
1833
1834s-char-sequence:
1835 s-char
1836 s-char-sequence s-char
1837
1838s-char:
1839 any member of source charset except double-quote ("), backslash
1840 (\), or new-line character.
1841 escape-sequence
1842~~~
1843
1844
1845#### C.1.7 Punctuators
1846
1847~~~ text
1848punctuator: one of
1849 [ ] ( ) { } . -> * + - < > : ; ... = ,
1850~~~
1851
1852
1853### C.2 Phrase structure grammar
1854
1855~~~ text
1856primary-expression:
1857 identifier
1858 constant
1859 string-literal
1860 ( unary-expression )
1861
1862postfix-expression:
1863 primary-expression
1864 postfix-expression [ unary-expression ]
1865 postfix-expression . identifier
1866 postfix-expressoin -> identifier
1867
1868unary-expression:
1869 postfix-expression
1870 unary-operator postfix-expression
1871
1872unary-operator: one of
1873 + -
1874
1875assignment-operator:
1876 =
1877
1878type-assignment-operator:
1879 :=
1880
1881constant-expression-range:
1882 unary-expression ... unary-expression
1883~~~
1884
1885
1886#### C.2.2 Declarations:
1887
1888~~~ text
1889declaration:
1890 declaration-specifiers declarator-list-opt ;
1891 ctf-specifier ;
1892
1893declaration-specifiers:
1894 storage-class-specifier declaration-specifiers-opt
1895 type-specifier declaration-specifiers-opt
1896 type-qualifier declaration-specifiers-opt
1897
1898declarator-list:
1899 declarator
1900 declarator-list , declarator
1901
1902abstract-declarator-list:
1903 abstract-declarator
1904 abstract-declarator-list , abstract-declarator
1905
1906storage-class-specifier:
1907 typedef
1908
1909type-specifier:
1910 void
1911 char
1912 short
1913 int
1914 long
1915 float
1916 double
1917 signed
1918 unsigned
1919 _Bool
1920 _Complex
1921 _Imaginary
1922 struct-specifier
1923 variant-specifier
1924 enum-specifier
1925 typedef-name
1926 ctf-type-specifier
1927
1928align-attribute:
1929 align ( unary-expression )
1930
1931struct-specifier:
1932 struct identifier-opt { struct-or-variant-declaration-list-opt } align-attribute-opt
1933 struct identifier align-attribute-opt
1934
1935struct-or-variant-declaration-list:
1936 struct-or-variant-declaration
1937 struct-or-variant-declaration-list struct-or-variant-declaration
1938
1939struct-or-variant-declaration:
1940 specifier-qualifier-list struct-or-variant-declarator-list ;
1941 declaration-specifiers-opt storage-class-specifier declaration-specifiers-opt declarator-list ;
1942 typealias declaration-specifiers abstract-declarator-list type-assignment-operator declaration-specifiers abstract-declarator-list ;
1943 typealias declaration-specifiers abstract-declarator-list type-assignment-operator declarator-list ;
1944
1945specifier-qualifier-list:
1946 type-specifier specifier-qualifier-list-opt
1947 type-qualifier specifier-qualifier-list-opt
1948
1949struct-or-variant-declarator-list:
1950 struct-or-variant-declarator
1951 struct-or-variant-declarator-list , struct-or-variant-declarator
1952
1953struct-or-variant-declarator:
1954 declarator
1955 declarator-opt : unary-expression
1956
1957variant-specifier:
1958 variant identifier-opt variant-tag-opt { struct-or-variant-declaration-list }
1959 variant identifier variant-tag
1960
1961variant-tag:
1962 < unary-expression >
1963
1964enum-specifier:
1965 enum identifier-opt { enumerator-list }
1966 enum identifier-opt { enumerator-list , }
1967 enum identifier
1968 enum identifier-opt : declaration-specifiers { enumerator-list }
1969 enum identifier-opt : declaration-specifiers { enumerator-list , }
1970
1971enumerator-list:
1972 enumerator
1973 enumerator-list , enumerator
1974
1975enumerator:
1976 enumeration-constant
1977 enumeration-constant assignment-operator unary-expression
1978 enumeration-constant assignment-operator constant-expression-range
1979
1980type-qualifier:
1981 const
1982
1983declarator:
1984 pointer-opt direct-declarator
1985
1986direct-declarator:
1987 identifier
1988 ( declarator )
1989 direct-declarator [ unary-expression ]
1990
1991abstract-declarator:
1992 pointer-opt direct-abstract-declarator
1993
1994direct-abstract-declarator:
1995 identifier-opt
1996 ( abstract-declarator )
1997 direct-abstract-declarator [ unary-expression ]
1998 direct-abstract-declarator [ ]
1999
2000pointer:
2001 * type-qualifier-list-opt
2002 * type-qualifier-list-opt pointer
2003
2004type-qualifier-list:
2005 type-qualifier
2006 type-qualifier-list type-qualifier
2007
2008typedef-name:
2009 identifier
2010~~~
2011
2012
2013#### C.2.3 CTF-specific declarations
2014
2015~~~ text
2016ctf-specifier:
2017 clock { ctf-assignment-expression-list-opt }
2018 event { ctf-assignment-expression-list-opt }
2019 stream { ctf-assignment-expression-list-opt }
2020 env { ctf-assignment-expression-list-opt }
2021 trace { ctf-assignment-expression-list-opt }
2022 callsite { ctf-assignment-expression-list-opt }
2023 typealias declaration-specifiers abstract-declarator-list type-assignment-operator declaration-specifiers abstract-declarator-list
2024 typealias declaration-specifiers abstract-declarator-list type-assignment-operator declarator-list
2025
2026ctf-type-specifier:
2027 floating_point { ctf-assignment-expression-list-opt }
2028 integer { ctf-assignment-expression-list-opt }
2029 string { ctf-assignment-expression-list-opt }
2030 string
2031
2032ctf-assignment-expression-list:
2033 ctf-assignment-expression ;
2034 ctf-assignment-expression-list ctf-assignment-expression ;
2035
2036ctf-assignment-expression:
2037 unary-expression assignment-operator unary-expression
2038 unary-expression type-assignment-operator type-specifier
2039 declaration-specifiers-opt storage-class-specifier declaration-specifiers-opt declarator-list
2040 typealias declaration-specifiers abstract-declarator-list type-assignment-operator declaration-specifiers abstract-declarator-list
2041 typealias declaration-specifiers abstract-declarator-list type-assignment-operator declarator-list
2042~~~
This page took 0.161044 seconds and 4 git commands to generate.