Make struct event_packet_header explicitly declared

[ctf.git] / common-trace-format-proposal.txt

diff --git a/common-trace-format-proposal.txt b/common-trace-format-proposal.txt
index fcc3432ba54542df866759dab77abcb05f63ff27..35c1e71b4c662abdfb509149c2a7ca73ade1ef27 100644 (file)
--- a/common-trace-format-proposal.txt
+++ b/common-trace-format-proposal.txt
@@ -1,5 +1,5 @@
 
 

-RFC: Common Trace Format (CTF) Proposal (v1.6)
+RFC: Common Trace Format (CTF) Proposal (pre-v1.7)

 
 Mathieu Desnoyers, EfficiOS Inc.
 
 
 Mathieu Desnoyers, EfficiOS Inc.
 


@@ -7,7 +7,9 @@ The goal of the present document is to propose a trace format that suits the
 needs of the embedded, telecom, high-performance and kernel communities. It is
 based on the Common Trace Format Requirements (v1.4) document. It is designed to
 allow traces to be natively generated by the Linux kernel, Linux user-space
 needs of the embedded, telecom, high-performance and kernel communities. It is
 based on the Common Trace Format Requirements (v1.4) document. It is designed to
 allow traces to be natively generated by the Linux kernel, Linux user-space

-applications written in C/C++, and hardware components.
+applications written in C/C++, and hardware components. One major element of
+CTF is the Trace Stream Description Language (TSDL) which flexibility
+enables description of various binary trace stream layouts.

 
 The latest version of this document can be found at:
 
 
 The latest version of this document can be found at:
 


@@ -51,7 +53,8 @@ virtual file system. Because each event stream is appended to while a trace is
 being recorded, each is associated with a separate file for output.  Therefore,
 a stored trace can be represented as a directory containing one file per stream.
 
 being recorded, each is associated with a separate file for output.  Therefore,
 a stored trace can be represented as a directory containing one file per stream.
 

-A metadata event stream contains information on trace event types. It describes:
+A metadata event stream contains information on trace event types
+expressed in the Trace Stream Description Language (TSDL). It describes:

 
 - Trace version.
 - Types available.
 
 - Trace version.
 - Types available.


@@ -68,14 +71,10 @@ A metadata event stream contains information on trace event types. It describes:
 3. Event stream
 
 An event stream is divided in contiguous event packets of variable size. These
 3. Event stream
 
 An event stream is divided in contiguous event packets of variable size. These

-subdivisions have a variable size. An event packet can contain a certain amount
-of padding at the end. The rationale for the event stream design choices is
-explained in Appendix B. Stream Header Rationale.
-
-An event stream is divided in contiguous event packets of variable size. These
-subdivisions have a variable size. An event packet can contain a certain amount
-of padding at the end.  The stream header is repeated at the beginning of each
-event packet.
+subdivisions have a variable size. An event packet can contain a certain
+amount of padding at the end. The stream header is repeated at the
+beginning of each event packet. The rationale for the event stream
+design choices is explained in Appendix B. Stream Header Rationale.

 
 The event stream header will therefore be referred to as the "event packet
 header" throughout the rest of this document.
 
 The event stream header will therefore be referred to as the "event packet
 header" throughout the rest of this document.


@@ -83,9 +82,13 @@ header" throughout the rest of this document.
 
 4. Types
 
 
 4. Types
 

+Types are organized as type classes. Each type class belong to either of two
+kind of types: basic types or compound types.
+

 4.1 Basic types
 
 4.1 Basic types
 

-A basic type is a scalar type, as described in this section.
+A basic type is a scalar type, as described in this section. It includes
+integers, GNU/C bitfields, enumerations, and floating point values.

 
 4.1.1 Type inheritance
 
 
 4.1.1 Type inheritance
 


@@ -99,15 +102,16 @@ types, but must be derived into a type to be usable in an event field.
 
 We define "byte-packed" types as aligned on the byte size, namely 8-bit.
 We define "bit-packed" types as following on the next bit, as defined by the
 
 We define "byte-packed" types as aligned on the byte size, namely 8-bit.
 We define "bit-packed" types as following on the next bit, as defined by the

-"bitfields" section.
+"Integers" section.

 
 All basic types, except bitfields, are either aligned on an architecture-defined
 specific alignment or byte-packed, depending on the architecture preference.
 Architectures providing fast unaligned write byte-packed basic types to save
 space, aligning each type on byte boundaries (8-bit). Architectures with slow
 unaligned writes align types on specific alignment values. If no specific
 
 All basic types, except bitfields, are either aligned on an architecture-defined
 specific alignment or byte-packed, depending on the architecture preference.
 Architectures providing fast unaligned write byte-packed basic types to save
 space, aligning each type on byte boundaries (8-bit). Architectures with slow
 unaligned writes align types on specific alignment values. If no specific

-alignment is declared for a type nor its parents, it is assumed to be bit-packed
-for bitfields and byte-packed for other types.
+alignment is declared for a type, it is assumed to be bit-packed for
+integers with size not multiple of 8 bits and for gcc bitfields. All
+other types are byte-packed.

 
 Metadata attribute representation of a specific alignment:
 
 
 Metadata attribute representation of a specific alignment:
 


@@ -181,19 +185,19 @@ Metadata representation:
 
 Example of type inheritance (creation of a uint32_t named type):
 
 
 Example of type inheritance (creation of a uint32_t named type):
 

-typedef integer {
+typealias integer {

   size = 32;
   signed = false;
   align = 32;
   size = 32;
   signed = false;
   align = 32;

-} uint32_t;
+} := uint32_t;

 
 Definition of a named 5-bit signed bitfield:
 
 
 Definition of a named 5-bit signed bitfield:
 

-typedef integer {
+typealias integer {

   size = 5;
   signed = true;
   align = 1;
   size = 5;
   signed = true;
   align = 1;

-} int5_t;
+} := int5_t;

 
 4.1.6 GNU/C bitfields
 
 
 4.1.6 GNU/C bitfields
 


@@ -248,11 +252,11 @@ floating_point {
 
 Example of type inheritance:
 
 
 Example of type inheritance:
 

-typedef floating_point {
+typealias floating_point {

   exp_dig = 8;         /* sizeof(float) * CHAR_BIT - FLT_MANT_DIG */
   mant_dig = 24;       /* FLT_MANT_DIG */
   byte_order = native;
   exp_dig = 8;         /* sizeof(float) * CHAR_BIT - FLT_MANT_DIG */
   mant_dig = 24;       /* FLT_MANT_DIG */
   byte_order = native;

-} float;
+} := float;

 
 TODO: define NaN, +inf, -inf behavior.
 
 
 TODO: define NaN, +inf, -inf behavior.
 


@@ -268,14 +272,10 @@ description within the metadata. The mapping table maps inclusive value ranges
 values to strings.  An enumeration from the C language can be represented in
 this format by having the same start_value and end_value for each element, which
 is in fact a range of size 1. This single-value range is supported without
 values to strings.  An enumeration from the C language can be represented in
 this format by having the same start_value and end_value for each element, which
 is in fact a range of size 1. This single-value range is supported without

-repeating the start and end values with the value = string declaration. If the
-<integer_type> is omitted, the type chosen by the C compiler to hold the
-enumeration is used. The <integer_type> specifier can only be omitted for
-enumerations containing only simple "value -> string" mappings (compatible with
-C).
-
-enum <integer_type> name {
-  string              = start_value1 ... end_value1,
+repeating the start and end values with the value = string declaration.
+
+enum name : integer_type {
+  somestring          = start_value1 ... end_value1,

   "other string"      = start_value2 ... end_value2,
   yet_another_string,  /* will be assigned to end_value2 + 1 */
   "some other string" = value,
   "other string"      = start_value2 ... end_value2,
   yet_another_string,  /* will be assigned to end_value2 + 1 */
   "some other string" = value,


@@ -285,7 +285,7 @@ enum <integer_type> name {
 If the values are omitted, the enumeration starts at 0 and increment of 1 for
 each entry:
 
 If the values are omitted, the enumeration starts at 0 and increment of 1 for
 each entry:
 

-enum name {
+enum name : unsigned int {

   ZERO,
   ONE,
   TWO,
   ZERO,
   ONE,
   TWO,


@@ -297,12 +297,26 @@ Overlapping ranges within a single enumeration are implementation defined.
 
 A nameless enumeration can be declared as a field type or as part of a typedef:
 
 
 A nameless enumeration can be declared as a field type or as part of a typedef:
 

-enum <integer_type> {
+enum : integer_type {
+  ...
+}
+
+Enumerations omitting the container type ": integer_type" use the "int"
+type (for compatibility with C99). The "int" type must be previously
+declared. E.g.:
+
+typealias integer { size = 32; align = 32; signed = true } := int;
+
+enum {

   ...
 }
 
   ...
 }
 

+

 4.2 Compound types
 
 4.2 Compound types
 

+Compound are aggregation of type declarations. Compound types include
+structures, variant, arrays, sequences, and strings.
+

 4.2.1 Structures
 
 Structures are aligned on the largest alignment required by basic types
 4.2.1 Structures
 
 Structures are aligned on the largest alignment required by basic types


@@ -336,7 +350,125 @@ struct {
   ...
 }
 
   ...
 }
 

-4.2.2 Arrays
+4.2.2 Variants (Discriminated/Tagged Unions)
+
+A CTF variant is a selection between different types. A CTF variant must
+always be defined within the scope of a structure or within fields
+contained within a structure (defined recursively). A "tag" enumeration
+field must appear in either the same lexical scope, prior to the variant
+field (in field declaration order), in an uppermost lexical scope (see
+Section 7.3.1), or in an uppermost dynamic scope (see Section 7.3.2).
+The type selection is indicated by the mapping from the enumeration
+value to the string used as variant type selector. The field to use as
+tag is specified by the "tag_field", specified between "< >" after the
+"variant" keyword for unnamed variants, and after "variant name" for
+named variants.
+
+The alignment of the variant is the alignment of the type as selected by the tag
+value for the specific instance of the variant. The alignment of the type
+containing the variant is independent of the variant alignment.  The size of the
+variant is the size as selected by the tag value for the specific instance of
+the variant.
+
+A named variant declaration followed by its definition within a structure
+declaration:
+
+variant name {
+  field_type sel1;
+  field_type sel2;
+  field_type sel3;
+  ...
+};
+
+struct {
+  enum : integer_type { sel1, sel2, sel3, ... } tag_field;
+  ...
+  variant name <tag_field> v;
+}
+
+An unnamed variant definition within a structure is expressed by the following
+metadata:
+
+struct {
+  enum : integer_type { sel1, sel2, sel3, ... } tag_field;
+  ...
+  variant <tag_field> {
+    field_type sel1;
+    field_type sel2;
+    field_type sel3;
+    ...
+  } v;
+}
+
+Example of a named variant within a sequence that refers to a single tag field:
+
+variant example {
+  uint32_t a;
+  uint64_t b;
+  short c;
+};
+
+struct {
+  enum : uint2_t { a, b, c } choice;
+  variant example <choice> v[unsigned int];
+}
+
+Example of an unnamed variant:
+
+struct {
+  enum : uint2_t { a, b, c, d } choice;
+  /* Unrelated fields can be added between the variant and its tag */
+  int32_t somevalue;
+  variant <choice> {
+    uint32_t a;
+    uint64_t b;
+    short c;
+    struct {
+      unsigned int field1;
+      uint64_t field2;
+    } d;
+  } s;
+}
+
+Example of an unnamed variant within an array:
+
+struct {
+  enum : uint2_t { a, b, c } choice;
+  variant <choice> {
+    uint32_t a;
+    uint64_t b;
+    short c;
+  } v[10];
+}
+
+Example of a variant type definition within a structure, where the defined type
+is then declared within an array of structures. This variant refers to a tag
+located in an upper lexical scope. This example clearly shows that a variant
+type definition referring to the tag "x" uses the closest preceding field from
+the lexical scope of the type definition.
+
+struct {
+  enum : uint2_t { a, b, c, d } x;
+
+  typedef variant <x> {        /*
+                        * "x" refers to the preceding "x" enumeration in the
+                        * lexical scope of the type definition.
+                        */
+    uint32_t a;
+    uint64_t b;
+    short c;
+  } example_variant;
+
+  struct {
+    enum : int { x, y, z } x;  /* This enumeration is not used by "v". */
+    example_variant v;                 /*
+                                * "v" uses the "enum : uint2_t { a, b, c, d }"
+                                * tag.
+                                */
+  } a[10];
+}
+
+4.2.3 Arrays

 
 Arrays are fixed-length. Their length is declared in the type declaration within
 the metadata. They contain an array of "inner type" elements, which can refer to
 
 Arrays are fixed-length. Their length is declared in the type declaration within
 the metadata. They contain an array of "inner type" elements, which can refer to


@@ -352,7 +484,7 @@ A nameless array can be declared as a field type within a structure, e.g.:
   uint8_t field_name[10];
 
 
   uint8_t field_name[10];
 
 

-4.2.3 Sequences
+4.2.4 Sequences

 
 Sequences are dynamically-sized arrays. They start with an integer that specify
 the length of the sequence, followed by an array of "inner type" elements.
 
 Sequences are dynamically-sized arrays. They start with an integer that specify
 the length of the sequence, followed by an array of "inner type" elements.


@@ -369,7 +501,7 @@ long field_name[int];
 The length type follows the integer types specifications, and the sequence
 elements follow the "array" specifications.
 
 The length type follows the integer types specifications, and the sequence
 elements follow the "array" specifications.
 

-4.2.4 Strings
+4.2.5 Strings

 
 Strings are an array of bytes of variable size and are terminated by a '\0'
 "NULL" character.  Their encoding is described in the metadata. In absence of
 
 Strings are an array of bytes of variable size and are terminated by a '\0'
 "NULL" character.  Their encoding is described in the metadata. In absence of


@@ -377,9 +509,9 @@ encoding attribute information, the default encoding is UTF-8.
 
 Metadata representation of a named string type:
 
 
 Metadata representation of a named string type:
 

-typedef string {
+typealias string {

   encoding = UTF8 OR ASCII;
   encoding = UTF8 OR ASCII;

-} name;
+} := name;

 
 A nameless string type can be declared as a field type:
 
 
 A nameless string type can be declared as a field type:
 


@@ -450,12 +582,21 @@ Metadata-defined layout (event packet context):
 
 5.1 Event Packet Header Fixed Layout Description
 
 
 5.1 Event Packet Header Fixed Layout Description
 

+The event packet header layout is indicated by the trace packet.header
+field. Here is an example structure type for the packet header with the
+fields typically expected:
+

 struct event_packet_header {
   uint32_t magic;
   uint8_t  trace_uuid[16];
   uint32_t stream_id;
 };
 
 struct event_packet_header {
   uint32_t magic;
   uint8_t  trace_uuid[16];
   uint32_t stream_id;
 };
 

+trace {
+  ...
+  packet.header := struct event_packet_header;
+};
+

 5.2 Event Packet Context Description
 
 Event packet context example. These are declared within the stream declaration
 5.2 Event Packet Context Description
 
 Event packet context example. These are declared within the stream declaration


@@ -476,21 +617,31 @@ struct event_packet_context {
   uint8_t  stream_packet_count_bits;   /* Significant counter bits */
   uint8_t  compression_scheme;
   uint8_t  encryption_scheme;
   uint8_t  stream_packet_count_bits;   /* Significant counter bits */
   uint8_t  compression_scheme;
   uint8_t  encryption_scheme;

-  uint8_t  checksum;
+  uint8_t  checksum_scheme;

 };
 
 };
 

+

 6. Event Structure
 
 The overall structure of an event is:
 
 6. Event Structure
 
 The overall structure of an event is:
 

-  - Event Header (as specifed by the stream metadata)
-  - Extended Event Header (as specified by the event header)
-  - Event Context (as specified by the stream metadata)
-  - Event Payload (as specified by the event metadata)
+1 - Stream Packet Context (as specified by the stream metadata)
+ 2 - Event Header (as specified by the stream metadata)
+  3 - Stream Event Context (as specified by the stream metadata)
+   4 - Event Context (as specified by the event metadata)
+    5 - Event Payload (as specified by the event metadata)

 
 

+This structure defines an implicit dynamic scoping, where variants
+located in inner structures (those with a higher number in the listing
+above) can refer to the fields of outer structures (with lower number in
+the listing above). See Section 7.3 TSDL Scopes for more detail.

 
 6.1 Event Header
 
 
 6.1 Event Header
 

+Event headers can be described within the metadata. We hereby propose, as an
+example, two types of events headers. Type 1 accommodates streams with less than
+31 event IDs. Type 2 accommodates streams with 31 or more event IDs.
+

 One major factor can vary between streams: the number of event IDs assigned to
 a stream. Luckily, this information tends to stay relatively constant (modulo
 event registration while trace is being recorded), so we can specify different
 One major factor can vary between streams: the number of event IDs assigned to
 a stream. Luckily, this information tends to stay relatively constant (modulo
 event registration while trace is being recorded), so we can specify different


@@ -498,12 +649,8 @@ representations for streams containing few event IDs and streams containing
 many event IDs, so we end up representing the event ID and timestamp as densely
 as possible in each case.
 
 many event IDs, so we end up representing the event ID and timestamp as densely
 as possible in each case.
 

-We therefore provide two types of events headers. Type 1 accommodates streams
-with less than 31 event IDs. Type 2 accommodates streams with 31 or more event
-IDs.
-
-The "extended headers" are used in the rare occasions where the information
-cannot be represented in the ranges available in the event header. They are also
+The header is extended in the rare occasions where the information cannot be
+represented in the ranges available in the standard event header. They are also

 used in the rare occasions where the data required for a field could not be
 collected: the flag corresponding to the missing field within the missing_fields
 array is then set to 1.
 used in the rare occasions where the data required for a field could not be
 collected: the flag corresponding to the missing field within the missing_fields
 array is then set to 1.


@@ -515,89 +662,98 @@ Types uintX_t represent an X-bit unsigned integer.
 
   - Aligned on 32-bit (or 8-bit if byte-packed, depending on the architecture
     preference).
 
   - Aligned on 32-bit (or 8-bit if byte-packed, depending on the architecture
     preference).

-  - Fixed size: 32 bits.

   - Native architecture byte ordering.
   - Native architecture byte ordering.

+  - For "compact" selection
+    - Fixed size: 32 bits.
+  - For "extended" selection
+    - Size depends on the architecture and variant alignment.

 
 struct event_header_1 {
 
 struct event_header_1 {

-  uint5_t id;          /*
-                        * id: range: 0 - 30.
-                        * id 31 is reserved to indicate a following
-                        * extended header.
-                        */
-  uint27_t timestamp;
-};
-
-The end of a type 1 header is aligned on a 32-bit boundary (or packed).
-
-
-6.1.2 Extended Type 1 Event Header
-
-  - Follows struct event_header_1, which is aligned on 32-bit, so no need to
-    realign.
-  - Variable size (depends on the number of fields per event).
-  - Native architecture byte ordering.
-  - NR_FIELDS is the number of fields within the event.
-
-struct event_header_1_ext {
-  uint32_t id;                         /* 32-bit event IDs */
-  uint64_t timestamp;                  /* 64-bit timestamps */
-  uint1_t missing_fields[NR_FIELDS];   /* missing event fields bitmap */
+  /*
+   * id: range: 0 - 30.
+   * id 31 is reserved to indicate an extended header.
+   */
+  enum : uint5_t { compact = 0 ... 30, extended = 31 } id;
+  variant <id> {
+    struct {
+      uint27_t timestamp;
+    } compact;
+    struct {
+      uint32_t id;                      /* 32-bit event IDs */
+      uint64_t timestamp;               /* 64-bit timestamps */
+    } extended;
+  } v;

 };
 
 
 };
 
 

-6.1.3 Type 2 - Many event IDs
+6.1.2 Type 2 - Many event IDs

 
 

-  - Aligned on 32-bit (or 8-bit if byte-packed, depending on the architecture
+  - Aligned on 16-bit (or 8-bit if byte-packed, depending on the architecture

     preference).
     preference).

-  - Fixed size: 48 bits.

   - Native architecture byte ordering.
   - Native architecture byte ordering.

+  - For "compact" selection
+    - Size depends on the architecture and variant alignment.
+  - For "extended" selection
+    - Size depends on the architecture and variant alignment.

 
 struct event_header_2 {
 
 struct event_header_2 {

-  uint32_t timestamp;
-  uint16_t id;         /*
-                        * id: range: 0 - 65534.
-                        * id 65535 is reserved to indicate a following
-                        * extended header.
-                        */
-};
-
-The end of a type 2 header is aligned on a 16-bit boundary (or 8-bit if
-byte-packed).
-
-
-6.1.4 Extended Type 2 Event Header
-
-  - Follows struct event_header_2, which alignment end on a 16-bit boundary, so
-    we need to align on 64-bit integer architecture alignment (or 8-bit if
-    byte-packed).
-  - Variable size (depends on the number of fields per event).
-  - Native architecture byte ordering.
-  - NR_FIELDS is the number of fields within the event.
-
-struct event_header_2_ext {
-  uint64_t timestamp;                  /* 64-bit timestamps */ 
-  uint32_t id;                         /* 32-bit event IDs */
-  uint1_t missing_fields[NR_FIELDS];   /* missing event fields bitmap */
+  /*
+   * id: range: 0 - 65534.
+   * id 65535 is reserved to indicate an extended header.
+   */
+  enum : uint16_t { compact = 0 ... 65534, extended = 65535 } id;
+  variant <id> {
+    struct {
+      uint32_t timestamp;
+    } compact;
+    struct {
+      uint32_t id;                      /* 32-bit event IDs */
+      uint64_t timestamp;               /* 64-bit timestamps */ 
+    } extended;
+  } v;

 };
 
 
 6.2 Event Context
 
 The event context contains information relative to the current event. The choice
 };
 
 
 6.2 Event Context
 
 The event context contains information relative to the current event. The choice

-and meaning of this information is specified by the metadata "stream"
-information. For this trace format, event context is usually empty, except when
-the metadata "stream" information specifies otherwise by declaring a non-empty
-structure for the event context. An example of event context is to save the
-event payload size with each event, or to save the current PID with each event.
-These are declared within the stream declaration within the metadata.
+and meaning of this information is specified by the metadata "stream" and
+"event" information. The "stream" context is applied to all events within the
+stream. The "stream" context structure follows the event header. The "event"
+context is applied to specific events. Its structure follows the "stream"
+context stucture.

 
 

-An example event context type:
+An example of stream-level event context is to save the event payload size with
+each event, or to save the current PID with each event.  These are declared
+within the stream declaration within the metadata:

 
 

-      struct event_context {
+  stream {
+    ...
+    event {
+      ...
+      context := struct {

         uint pid;
         uint16_t payload_size;
       };
         uint pid;
         uint16_t payload_size;
       };

+    }
+  };
+
+An example of event-specific event context is to declare a bitmap of missing
+fields, only appended after the stream event context if the extended event
+header is selected. NR_FIELDS is the number of fields within the event (a
+numeric value).

 
 

+  event {
+    context = struct {
+      variant <id> {
+        struct { } compact;
+        struct {
+          uint1_t missing_fields[NR_FIELDS]; /* missing event fields bitmap */
+        } extended;
+      } v;
+    };
+    ...
+  }

 
 6.3 Event Payload
 
 
 6.3 Event Payload
 


@@ -623,45 +779,137 @@ The event payload is aligned on the largest alignment required by types
 contained within the payload. (This follows the ISO/C standard for structures)
 
 
 contained within the payload. (This follows the ISO/C standard for structures)
 
 

+7. Trace Stream Description Language (TSDL)
+
+The Trace Stream Description Language (TSDL) allows expression of the
+binary trace streams layout in a C99-like Domain Specific Language
+(DSL).
+
+
+7.1 Metadata
+
+The trace stream layout description is located in the trace meta-data.
+The meta-data is itself located in a stream identified by its name:
+"metadata".
+
+It is made of "event packets", which each start with an event packet
+header. The event type within the metadata stream have no event header
+nor event context. Each event only contains a "string" payload without
+any null-character. The events are packed one next to another. Each
+event packet start with an event packet header, which contains, amongst
+other fields, the magic number, trace UUID and packet length. In the
+event packet header, the trace UUID is represented as an array of bytes.
+Within the string-based metadata description, the trace UUID is
+represented as a string of hexadecimal digits and dashes "-".
+
+The metadata can be parsed by reading characters within the metadata
+stream, for each packet starting after the packet header, for the length
+of the packet payload specified in the header.  Text contained within
+"/*" and "*/", as well as within "//" and end of line, are treated as
+comments.  Boolean values can be represented as true, TRUE, or 1 for
+true, and false, FALSE, or 0 for false.
+
+
+7.2 Declaration vs Definition
+
+A declaration associates a layout to a type, without specifying where
+this type is located in the event structure hierarchy (see Section 6).
+This therefore includes typedef, typealias, as well as all type
+specifiers. In certain circumstances (typedef, structure field and
+variant field), a declaration is followed by a declarator, which specify
+the newly defined type name (for typedef), or the field name (for
+declarations located within structure and variants). Array and sequence,
+declared with square brackets ("[" "]"), are part of the declarator,
+similarly to C99. The enumeration base type is specified by
+": enum_base", which is part of the type specifier. The variant tag
+name, specified between "<" ">", is also part of the type specifier.
+
+A definition associates a type to a location in the event structure
+hierarchy (see Section 6). This association is denoted by ":=", as shown
+in Section 7.3.

 
 

-7. Metadata

 
 

-The meta-data is located in a stream named "metadata". It is made of "event
-packets", which each start with an event packet header. The event type within
-the metadata stream have no event header nor event context. Each event only
-contains a null-terminated "string" payload, which is a metadata description
-entry. The events are packed one next to another. Each event packet start with
-an event packet header, which contains, amongst other fields, the magic number
-and trace UUID.
+7.3 TSDL Scopes

 
 

-The metadata can be parsed by reading through the metadata strings, skipping
-newlines and null-characters. Type names may contain spaces.
+TSDL uses two different types of scoping: a lexical scope is used for
+declarations and type definitions, and a dynamic scope is used for
+variants references to tag fields.
+
+7.3.1 Lexical Scope
+
+Each of "trace", "stream", "event", "struct" and "variant" have their own
+nestable declaration scope, within which types can be declared using "typedef"
+and "typealias". A root declaration scope also contains all declarations
+located outside of any of the aforementioned declarations. An inner
+declaration scope can refer to type declared within its container
+lexical scope prior to the inner declaration scope. Redefinition of a
+typedef or typealias is not valid, although hiding an upper scope
+typedef or typealias is allowed within a sub-scope.
+
+7.3.2 Dynamic Scope
+
+A dynamic scope consists in the lexical scope augmented with the
+implicit event structure definition hierarchy presented at Section 6.
+The dynamic scope is only used for variant tag definitions. It is used
+at definition time to look up the location of the tag field associated
+with a variant.
+
+Therefore, variants in lower levels in the dynamic scope (e.g. event
+context) can refer to a tag field located in upper levels (e.g. in the
+event header) by specifying, in this case, the associated tag with
+<header.field_name>. This allows, for instance, the event context to
+define a variant referring to the "id" field of the event header as
+selector.
+
+The target dynamic scope must be specified explicitly when referring to
+a field outside of the local static scope. The dynamic scope prefixes
+are thus:
+
+ - Stream Packet Context: <stream.packet.context. >,
+ - Event Header: <stream.event.header. >,
+ - Stream Event Context: <stream.event.context. >,
+ - Event Context: <event.context. >,
+ - Event Payload: <event.fields. >.
+
+Multiple declarations of the same field name within a single scope is
+not valid. It is however valid to re-use the same field name in
+different scopes. There is no possible conflict, because the dynamic
+scope must be specified when a variant refers to a tag field located in
+a different dynamic scope.
+
+The information available in the dynamic scopes can be thought of as the
+current tracing context. At trace production, information about the
+current context is saved into the specified scope field levels. At trace
+consumption, for each event, the current trace context is therefore
+readable by accessing the upper dynamic scopes.
+
+
+7.4 TSDL Examples
+
+The grammar representing the TSDL metadata is presented in Appendix C.
+TSDL Grammar. This section presents a rather ligher reading that
+consists in examples of TSDL metadata, with template values:

 
 trace {
 
 trace {

-  major = value;       /* Trace format version */
+  major = value;                               /* Trace format version */

   minor = value;
   minor = value;

-  uuid = value;                /* Trace UUID */
-  word_size = value;
+  uuid = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa";       /* Trace UUID */
+  packet.header := struct {
+    uint32_t magic;
+    uint8_t  trace_uuid[16];
+    uint32_t stream_id;
+  };

 };
 
 stream {
   id = stream_id;
 };
 
 stream {
   id = stream_id;

-  event {
-    /* Type 1 - Few event IDs; Type 2 - Many event IDs. See section 6.1. */
-    header_type = event_header_1 OR event_header_2;
-    /*
-     * Extended event header type. Only present if specified in event header
-     * on a per-event basis.
-     */
-    header_type_ext = event_header_1_ext OR event_header_2_ext;
-    context_type = struct {
-      ...
-    };
+  /* Type 1 - Few event IDs; Type 2 - Many event IDs. See section 6.1. */
+  event.header := event_header_1 OR event_header_2;
+  event.context := struct {
+    ...

   };
   };

-  packet {
-    context_type = struct {
-      ...
-    };
+  packet.context := struct {
+    ...

   };
 };
 
   };
 };
 


@@ -669,7 +917,10 @@ event {
   name = event_name;
   id = value;                  /* Numeric identifier within the stream */
   stream = stream_id;
   name = event_name;
   id = value;                  /* Numeric identifier within the stream */
   stream = stream_id;

-  fields = struct {
+  context := struct {
+    ...
+  };
+  fields := struct {

     ...
   };
 };
     ...
   };
 };


@@ -679,44 +930,68 @@ event {
 /*
  * Named types:
  *
 /*
  * Named types:
  *

- * A named type can only have a prefix and postfix if it aliases a CTF basic
- * type. A type name aliasing another type name cannot have prefix nor postfix,
- * but the type aliased can have a prefix and/or postfix.
+ * Type declarations behave similarly to the C standard.

  */
 
  */
 

-typedef aliased_type_prefix aliased_type new_type aliased_type_postfix;
+typedef aliased_type_specifiers new_type_declarators;

 
 /* e.g.: typedef struct example new_type_name[10]; */
 
 
 /* e.g.: typedef struct example new_type_name[10]; */
 

-typedef type_class {
+/*
+ * typealias
+ *
+ * The "typealias" declaration can be used to give a name (including
+ * pointer declarator specifier) to a type. It should also be used to
+ * map basic C types (float, int, unsigned long, ...) to a CTF type.
+ * Typealias is a superset of "typedef": it also allows assignment of a
+ * simple variable identifier to a type.
+ */
+
+typealias type_class {

   ...
   ...

-} new_type_prefix new_type new_type_postfix;
+} := type_specifiers type_declarator;

 
 /*
  * e.g.: 
 
 /*
  * e.g.: 

- * typedef integer {
+ * typealias integer {

  *   size = 32;
  *   align = 32;
  *   signed = false;
  *   size = 32;
  *   align = 32;
  *   signed = false;

- * } struct page *;
+ * } := struct page *;
+ *
+ * typealias integer {
+ *  size = 32;
+ *  align = 32;
+ *  signed = true;
+ * } := int;

  */
 
 struct name {
   ...
 };
 
  */
 
 struct name {
   ...
 };
 

-enum <integer_type> name {
+variant name {

   ...
 };
 
   ...
 };
 

+enum name : integer_type {
+  ...
+};

 
 

-/* Unnamed types, contained within compound type fields or typedef. */
+
+/*
+ * Unnamed types, contained within compound type fields, typedef or typealias.
+ */

 
 struct {
   ...
 }
 
 
 struct {
   ...
 }
 

-enum <integer_type> {
+variant {
+  ...
+}
+
+enum : integer_type {

   ...
 }
 
   ...
 }
 


@@ -748,6 +1023,7 @@ struct {
   string field_name;
 }
 
   string field_name;
 }
 

+

 A. Helper macros
 
 The two following macros keep track of the size of a GNU/C structure without
 A. Helper macros
 
 The two following macros keep track of the size of a GNU/C structure without


@@ -786,3 +1062,383 @@ flexibility in terms of:
 
 The event stream header will therefore be referred to as the "event packet
 header" throughout the rest of this document.
 
 The event stream header will therefore be referred to as the "event packet
 header" throughout the rest of this document.

+
+
+C. TSDL Grammar
+
+/*
+ * Common Trace Format (CTF) Trace Stream Description Language (TSDL) Grammar.
+ *
+ * Inspired from the C99 grammar:
+ * http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf (Annex A)
+ * and c++1x grammar (draft)
+ * http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3291.pdf (Annex A)
+ *
+ * Specialized for CTF needs by including only constant and declarations from
+ * C99 (excluding function declarations), and by adding support for variants,
+ * sequences and CTF-specific specifiers. Enumeration container types
+ * semantic is inspired from c++1x enum-base.
+ */
+
+1) Lexical grammar
+
+1.1) Lexical elements
+
+token:
+       keyword
+       identifier
+       constant
+       string-literal
+       punctuator
+
+1.2) Keywords
+
+keyword: is one of
+
+const
+char
+double
+enum
+event
+floating_point
+float
+integer
+int
+long
+short
+signed
+stream
+string
+struct
+trace
+typealias
+typedef
+unsigned
+variant
+void
+_Bool
+_Complex
+_Imaginary
+
+
+1.3) Identifiers
+
+identifier:
+       identifier-nondigit
+       identifier identifier-nondigit
+       identifier digit
+
+identifier-nondigit:
+       nondigit
+       universal-character-name
+       any other implementation-defined characters
+
+nondigit:
+       _
+       [a-zA-Z]        /* regular expression */
+
+digit:
+       [0-9]           /* regular expression */
+
+1.4) Universal character names
+
+universal-character-name:
+       \u hex-quad
+       \U hex-quad hex-quad
+
+hex-quad:
+       hexadecimal-digit hexadecimal-digit hexadecimal-digit hexadecimal-digit
+
+1.5) Constants
+
+constant:
+       integer-constant
+       enumeration-constant
+       character-constant
+
+integer-constant:
+       decimal-constant integer-suffix-opt
+       octal-constant integer-suffix-opt
+       hexadecimal-constant integer-suffix-opt
+
+decimal-constant:
+       nonzero-digit
+       decimal-constant digit
+
+octal-constant:
+       0
+       octal-constant octal-digit
+
+hexadecimal-constant:
+       hexadecimal-prefix hexadecimal-digit
+       hexadecimal-constant hexadecimal-digit
+
+hexadecimal-prefix:
+       0x
+       0X
+
+nonzero-digit:
+       [1-9]
+
+integer-suffix:
+       unsigned-suffix long-suffix-opt
+       unsigned-suffix long-long-suffix
+       long-suffix unsigned-suffix-opt
+       long-long-suffix unsigned-suffix-opt
+
+unsigned-suffix:
+       u
+       U
+
+long-suffix:
+       l
+       L
+
+long-long-suffix:
+       ll
+       LL
+
+digit-sequence:
+       digit
+       digit-sequence digit
+
+hexadecimal-digit-sequence:
+       hexadecimal-digit
+       hexadecimal-digit-sequence hexadecimal-digit
+
+enumeration-constant:
+       identifier
+       string-literal
+
+character-constant:
+       ' c-char-sequence '
+       L' c-char-sequence '
+
+c-char-sequence:
+       c-char
+       c-char-sequence c-char
+
+c-char:
+       any member of source charset except single-quote ('), backslash
+       (\), or new-line character.
+       escape-sequence
+
+escape-sequence:
+       simple-escape-sequence
+       octal-escape-sequence
+       hexadecimal-escape-sequence
+       universal-character-name
+
+simple-escape-sequence: one of
+       \' \" \? \\ \a \b \f \n \r \t \v
+
+octal-escape-sequence:
+       \ octal-digit
+       \ octal-digit octal-digit
+       \ octal-digit octal-digit octal-digit
+
+hexadecimal-escape-sequence:
+       \x hexadecimal-digit
+       hexadecimal-escape-sequence hexadecimal-digit
+
+1.6) String literals
+
+string-literal:
+       " s-char-sequence-opt "
+       L" s-char-sequence-opt "
+
+s-char-sequence:
+       s-char
+       s-char-sequence s-char
+
+s-char:
+       any member of source charset except double-quote ("), backslash
+       (\), or new-line character.
+       escape-sequence
+
+1.7) Punctuators
+
+punctuator: one of
+       [ ] ( ) { } . -> * + - < > : ; ... = ,
+
+
+2) Phrase structure grammar
+
+primary-expression:
+       identifier
+       constant
+       string-literal
+       ( unary-expression )
+
+postfix-expression:
+       primary-expression
+       postfix-expression [ unary-expression ]
+       postfix-expression . identifier
+       postfix-expressoin -> identifier
+
+unary-expression:
+       postfix-expression
+       unary-operator postfix-expression
+
+unary-operator: one of
+       + -
+
+assignment-operator:
+       =
+
+type-assignment-operator:
+       :=
+
+constant-expression:
+       unary-expression
+
+constant-expression-range:
+       constant-expression ... constant-expression
+
+2.2) Declarations:
+
+declaration:
+       declaration-specifiers declarator-list-opt ;
+       ctf-specifier ;
+
+declaration-specifiers:
+       storage-class-specifier declaration-specifiers-opt
+       type-specifier declaration-specifiers-opt
+       type-qualifier declaration-specifiers-opt
+
+declarator-list:
+       declarator
+       declarator-list , declarator
+
+abstract-declarator-list:
+       abstract-declarator
+       abstract-declarator-list , abstract-declarator
+
+storage-class-specifier:
+       typedef
+
+type-specifier:
+       void
+       char
+       short
+       int
+       long
+       float
+       double
+       signed
+       unsigned
+       _Bool
+       _Complex
+       _Imaginary
+       struct-specifier
+       variant-specifier
+       enum-specifier
+       typedef-name
+       ctf-type-specifier
+
+struct-specifier:
+       struct identifier-opt { struct-or-variant-declaration-list-opt }
+       struct identifier
+
+struct-or-variant-declaration-list:
+       struct-or-variant-declaration
+       struct-or-variant-declaration-list struct-or-variant-declaration
+
+struct-or-variant-declaration:
+       specifier-qualifier-list struct-or-variant-declarator-list ;
+       declaration-specifiers storage-class-specifier declaration-specifiers declarator-list ;
+       typealias declaration-specifiers abstract-declarator-list := declaration-specifiers abstract-declarator-list ;
+       typealias declaration-specifiers abstract-declarator-list := declarator-list ;
+
+specifier-qualifier-list:
+       type-specifier specifier-qualifier-list-opt
+       type-qualifier specifier-qualifier-list-opt
+
+struct-or-variant-declarator-list:
+       struct-or-variant-declarator
+       struct-or-variant-declarator-list , struct-or-variant-declarator
+
+struct-or-variant-declarator:
+       declarator
+       declarator-opt : constant-expression
+
+variant-specifier:
+       variant identifier-opt variant-tag-opt { struct-or-variant-declaration-list }
+       variant identifier variant-tag
+
+variant-tag:
+       < identifier >
+
+enum-specifier:
+       enum identifier-opt { enumerator-list }
+       enum identifier-opt { enumerator-list , }
+       enum identifier
+       enum identifier-opt : declaration-specifiers { enumerator-list }
+       enum identifier-opt : declaration-specifiers { enumerator-list , }
+
+enumerator-list:
+       enumerator
+       enumerator-list , enumerator
+
+enumerator:
+       enumeration-constant
+       enumeration-constant = constant-expression
+       enumeration-constant = constant-expression-range
+
+type-qualifier:
+       const
+
+declarator:
+       pointer-opt direct-declarator
+
+direct-declarator:
+       identifier
+       ( declarator )
+       direct-declarator [ type-specifier ]
+       direct-declarator [ constant-expression ]
+
+abstract-declarator:
+       pointer-opt direct-abstract-declarator
+
+direct-abstract-declarator:
+       identifier-opt
+       ( abstract-declarator )
+       direct-abstract-declarator [ type-specifier ]
+       direct-abstract-declarator [ constant-expression ]
+       direct-abstract-declarator [ ]
+
+pointer:
+       * type-qualifier-list-opt
+       * type-qualifier-list-opt pointer
+
+type-qualifier-list:
+       type-qualifier
+       type-qualifier-list type-qualifier
+
+typedef-name:
+       identifier
+
+2.3) CTF-specific declarations
+
+ctf-specifier:
+       event { ctf-assignment-expression-list-opt }
+       stream { ctf-assignment-expression-list-opt }
+       trace { ctf-assignment-expression-list-opt }
+       typealias declaration-specifiers abstract-declarator-list := declaration-specifiers abstract-declarator-list ;
+       typealias declaration-specifiers abstract-declarator-list := declarator-list ;
+
+ctf-type-specifier:
+       floating_point { ctf-assignment-expression-list-opt }
+       integer { ctf-assignment-expression-list-opt }
+       string { ctf-assignment-expression-list-opt }
+
+ctf-assignment-expression-list:
+       ctf-assignment-expression
+       ctf-assignment-expression-list ; ctf-assignment-expression
+
+ctf-assignment-expression:
+       unary-expression assignment-operator unary-expression
+       unary-expression type-assignment-operator type-specifier
+       declaration-specifiers storage-class-specifier declaration-specifiers declarator-list
+       typealias declaration-specifiers abstract-declarator-list := declaration-specifiers abstract-declarator-list
+       typealias declaration-specifiers abstract-declarator-list := declarator-list