-Common Trace Format (CTF) Specification (pre-v1.8)
+Common Trace Format (CTF) Specification (v1.8.1)
Mathieu Desnoyers, EfficiOS Inc.
7.2 Declaration vs Definition
7.3 TSDL Scopes
7.3.1 Lexical Scope
- 7.3.2 Dynamic Scope
+ 7.3.2 Static and Dynamic Scopes
7.4 TSDL Examples
+8. Clocks
1. Preliminary definitions
3. Event stream
An event stream can be divided into contiguous event packets of variable
-size. These subdivisions have a variable size. An event packet can
-contain a certain amount of padding at the end. The stream header is
-repeated at the beginning of each event packet. The rationale for the
-event stream design choices is explained in Appendix B. Stream Header
-Rationale.
+size. An event packet can contain a certain amount of padding at the
+end. The stream header is repeated at the beginning of each event
+packet. The rationale for the event stream design choices is explained
+in Appendix B. Stream Header Rationale.
The event stream header will therefore be referred to as the "event packet
header" throughout the rest of this document.
4.1.3 Byte order
-By default, the native endianness of the source architecture the trace is used.
+By default, the native endianness of the source architecture is used.
Byte order can be overridden for a basic type by specifying a "byte_order"
attribute. Typical use-case is to specify the network byte order (big endian:
"be") to save data captured from the network into the trace without conversion.
};
If the values are omitted, the enumeration starts at 0 and increment of 1 for
-each entry:
+each entry. An entry with omitted value that follows a range entry takes
+as value the end_value of the previous range + 1:
enum name : unsigned int {
ZERO,
uint64_t second_field_name; /* Named type declared in the meta-data */
};
-The fields are placed in a sequence next to each other. They each possess a
-field name, which is a unique identifier within the structure.
+The fields are placed in a sequence next to each other. They each
+possess a field name, which is a unique identifier within the structure.
+The identifier is not allowed to use any reserved keyword
+(see Section C.1.2). Replacing reserved keywords with
+underscore-prefixed field names is recommended. Fields starting with an
+underscore should have their leading underscore removed by the CTF trace
+readers.
A nameless structure can be declared as a field type or as part of a typedef:
A CTF variant is a selection between different types. A CTF variant must
always be defined within the scope of a structure or within fields
contained within a structure (defined recursively). A "tag" enumeration
-field must appear in either the same lexical scope, prior to the variant
-field (in field declaration order), in an upper lexical scope (see
-Section 7.3.1), or in an upper dynamic scope (see Section 7.3.2). The
-type selection is indicated by the mapping from the enumeration value to
-the string used as variant type selector. The field to use as tag is
-specified by the "tag_field", specified between "< >" after the
-"variant" keyword for unnamed variants, and after "variant name" for
-named variants.
-
-The alignment of the variant is the alignment of the type as selected by the tag
-value for the specific instance of the variant. The alignment of the type
-containing the variant is independent of the variant alignment. The size of the
-variant is the size as selected by the tag value for the specific instance of
-the variant.
+field must appear in either the same static scope, prior to the variant
+field (in field declaration order), in an upper static scope , or in an
+upper dynamic scope (see Section 7.3.2). The type selection is indicated
+by the mapping from the enumeration value to the string used as variant
+type selector. The field to use as tag is specified by the "tag_field",
+specified between "< >" after the "variant" keyword for unnamed
+variants, and after "variant name" for named variants.
+
+The alignment of the variant is the alignment of the type as selected by
+the tag value for the specific instance of the variant. The size of the
+variant is the size as selected by the tag value for the specific
+instance of the variant.
+
+The alignment of the type containing the variant is independent of the
+variant alignment. For instance, if a structure contains two fields, a
+32-bit integer, aligned on 32 bits, and a variant, which contains two
+choices: either a 32-bit field, aligned on 32 bits, or a 64-bit field,
+aligned on 64 bits, the alignment of the outmost structure will be
+32-bit (the alignment of its largest field, disregarding the alignment
+of the variant). The alignment of the variant will depend on the
+selector: if the variant's 32-bit field is selected, its alignment will
+be 32-bit, or 64-bit otherwise. It is important to note that variants
+are specifically tailored for compactness in a stream. Therefore, the
+relative offsets of compound type fields can vary depending on
+the offset at which the compound type starts if it contains a variant
+that itself contains a type with alignment larger than the largest field
+contained within the compound type. This is caused by the fact that the
+compound type may contain the enumeration that select the variant's
+choice, and therefore the alignment to be applied to the compound type
+cannot be determined before encountering the enumeration.
+
+Each variant type selector possess a field name, which is a unique
+identifier within the variant. The identifier is not allowed to use any
+reserved keyword (see Section C.1.2). Replacing reserved keywords with
+underscore-prefixed field names is recommended. Fields starting with an
+underscore should have their leading underscore removed by the CTF trace
+readers.
+
A named variant declaration followed by its definition within a structure
declaration:
Example of a variant type definition within a structure, where the defined type
is then declared within an array of structures. This variant refers to a tag
-located in an upper lexical scope. This example clearly shows that a variant
+located in an upper static scope. This example clearly shows that a variant
type definition referring to the tag "x" uses the closest preceding field from
-the lexical scope of the type definition.
+the static scope of the type definition.
struct {
enum : uint2_t { a, b, c, d } x;
typedef variant <x> { /*
* "x" refers to the preceding "x" enumeration in the
- * lexical scope of the type definition.
+ * static scope of the type definition.
*/
uint32_t a;
uint64_t b;
4.2.4 Sequences
Sequences are dynamically-sized arrays. They refer to a a "length"
-unsigned integer field, which must appear in either the same lexical scope,
+unsigned integer field, which must appear in either the same static scope,
prior to the sequence field (in field declaration order), in an upper
-lexical scope (see Section 7.3.1), or in an upper dynamic scope (see
-Section 7.3.2). This length field represents the number of elements in
-the sequence. The sequence per se is an array of "inner type" elements.
+static scope, or in an upper dynamic scope (see Section 7.3.2). This
+length field represents the number of elements in the sequence. The
+sequence per se is an array of "inner type" elements.
TSDL meta-data representation for a sequence type definition:
include all event timestamps assigned to events contained within the packet.
- Events discarded count
- Snapshot of a per-stream free-running counter, counting the number of
- events discarded that were supposed to be written in the stream prior to
- the first event in the event packet.
- * Note: producer-consumer buffer full condition should fill the current
+ events discarded that were supposed to be written in the stream after
+ the last event in the event packet.
+ * Note: producer-consumer buffer full condition can fill the current
event packet with padding so we know exactly where events have been
- discarded.
+ discarded. However, if the buffer full condition chooses not
+ to fill the current event packet with padding, all we know
+ about the timestamp range in which the events have been
+ discarded is that it is somewhere between the beginning and
+ the end of the packet.
- Lossless compression scheme used for the event packet content. Applied
directly to raw data. New types of compression can be added in following
versions of the format.
uint32_t cpu_id;
uint32_t/uint16_t content_size;
uint32_t/uint16_t packet_size;
- uint8_t stream_packet_count_bits; /* Significant counter bits */
uint8_t compression_scheme;
uint8_t encryption_scheme;
uint8_t checksum_scheme;
network-readiness) for meta-data stream generated and transported by a
tracer.
-The text-only meta-data file is a plain text TSDL description.
+The text-only meta-data file is a plain-text TSDL description. This file
+must begin with the following characters to identify the file as a CTF
+TSDL text-based metadata file (without the double-quotes) :
+
+"/* CTF"
+
+It must be followed by a space, and the version of the specification
+followed by the CTF trace, e.g.:
+
+" 1.8"
+
+These characters allow automated discovery of file type and CTF
+specification version. They are interpreted as a the beginning of a
+comment by the TSDL metadata parser. The comment can be continued to
+contain extra commented characters before it is closed.
The packet-based meta-data is made of "meta-data packets", which each
start with a meta-data packet header. The packet-based meta-data
endianness of the architecture by trying to read the CTF magic number
and its counterpart in reversed endianness. The events within the
meta-data stream have no event header nor event context. Each event only
-contains a "sequence" payload, which is a sequence of bits using the
-"trace.packet.header.content_size" field as a placeholder for its length
-(the packet header size should be substracted). The formatting of this
-sequence of bits is a plain-text representation of the TSDL description.
-Each meta-data packet start with a special packet header, specific to
-the meta-data stream, which contains, exactly:
+contains a special "sequence" payload, which is a sequence of bits which
+length is implicitly calculated by using the
+"trace.packet.header.content_size" field, minus the packet header size.
+The formatting of this sequence of bits is a plain-text representation
+of the TSDL description. Each meta-data packet start with a special
+packet header, specific to the meta-data stream, which contains,
+exactly:
struct metadata_packet_header {
uint32_t magic; /* 0x75D11D57 */
uint8_t compression_scheme; /* 0 if unused */
uint8_t encryption_scheme; /* 0 if unused */
uint8_t checksum_scheme; /* 0 if unused */
+ uint8_t major; /* CTF spec version major number */
+ uint8_t minor; /* CTF spec version minor number */
};
The packet-based meta-data can be converted to a text-only meta-data by
-concatenating all the strings in contains.
+concatenating all the strings it contains.
In the textual representation of the meta-data, the text contained
within "/*" and "*/", as well as within "//" and end of line, are
7.3 TSDL Scopes
-TSDL uses two different types of scoping: a lexical scope is used for
-declarations and type definitions, and a dynamic scope is used for
-variants references to tag fields and for sequence references to length
-fields.
+TSDL uses three different types of scoping: a lexical scope is used for
+declarations and type definitions, and static and dynamic scopes are
+used for variants references to tag fields (with relative and absolute
+path lookups) and for sequence references to length fields.
7.3.1 Lexical Scope
-Each of "trace", "stream", "event", "struct" and "variant" have their own
-nestable declaration scope, within which types can be declared using "typedef"
-and "typealias". A root declaration scope also contains all declarations
-located outside of any of the aforementioned declarations. An inner
-declaration scope can refer to type declared within its container
-lexical scope prior to the inner declaration scope. Redefinition of a
-typedef or typealias is not valid, although hiding an upper scope
-typedef or typealias is allowed within a sub-scope.
+Each of "trace", "env", "stream", "event", "struct" and "variant" have
+their own nestable declaration scope, within which types can be declared
+using "typedef" and "typealias". A root declaration scope also contains
+all declarations located outside of any of the aforementioned
+declarations. An inner declaration scope can refer to type declared
+within its container lexical scope prior to the inner declaration scope.
+Redefinition of a typedef or typealias is not valid, although hiding an
+upper scope typedef or typealias is allowed within a sub-scope.
+
+7.3.2 Static and Dynamic Scopes
-7.3.2 Dynamic Scope
+A local static scope consists in the scope generated by the declaration
+of fields within a compound type. A static scope is a local static scope
+augmented with the nested sub-static-scopes it contains.
-A dynamic scope consists in the lexical scope augmented with the
+A dynamic scope consists in the static scope augmented with the
implicit event structure definition hierarchy presented at Section 6.
-The dynamic scope is used for variant tag and sequence length
-definitions. It is used at definition time to look up the location of
-the tag field associated with a variant, and to lookup up the location
-of the length field associated with a sequence.
-
-Therefore, variants (or sequences) in lower levels in the dynamic scope
-(e.g. event context) can refer to a tag (or length) field located in
-upper levels (e.g. in the event header) by specifying, in this case, the
-associated tag with <header.field_name>. This allows, for instance, the
-event context to define a variant referring to the "id" field of the
-event header as selector.
-The target dynamic scope must be specified explicitly when referring to
-a field outside of the local static scope (a local static scope contains
-all fields present within the same scope, at the same nesting level).
+Multiple declarations of the same field name within a local static scope
+is not valid. It is however valid to re-use the same field name in
+different local scopes.
+
+Nested static and dynamic scopes form lookup paths. These are used for
+variant tag and sequence length references. They are used at the variant
+and sequence definition site to look up the location of the tag field
+associated with a variant, and to lookup up the location of the length
+field associated with a sequence.
+
+Variants and sequences can refer to a tag field either using a relative
+path or an absolute path. The relative path is relative to the scope in
+which the variant or sequence performing the lookup is located.
+Relative paths are only allowed to lookup within the same static scope,
+which includes its nested static scopes. Lookups targeting parent static
+scopes need to be performed with an absolute path.
+
+Absolute path lookups use the full path including the dynamic scope
+followed by a "." and then the static scope. Therefore, variants (or
+sequences) in lower levels in the dynamic scope (e.g. event context) can
+refer to a tag (or length) field located in upper levels (e.g. in the
+event header) by specifying, in this case, the associated tag with
+<stream.event.header.field_name>. This allows, for instance, the event
+context to define a variant referring to the "id" field of the event
+header as selector.
+
The dynamic scope prefixes are thus:
+ - Trace Environment: <env. >,
- Trace Packet Header: <trace.packet.header. >,
- Stream Packet Context: <stream.packet.context. >,
- Event Header: <stream.event.header. >,
- Event Context: <event.context. >,
- Event Payload: <event.fields. >.
-Multiple declarations of the same field name within a single scope is
-not valid. It is however valid to re-use the same field name in
-different scopes. There is no possible conflict, because the dynamic
-scope must be specified when a variant refers to a tag field located in
-a different dynamic scope.
+
+The target dynamic scope must be specified explicitly when referring to
+a field outside of the static scope (absolute scope reference). No
+conflict can occur between relative and dynamic paths, because the
+keywords "trace", "stream", and "event" are reserved, and thus
+not permitted as field names. It is recommended that field names
+clashing with CTF and C99 reserved keywords use an underscore prefix to
+eliminate the risk of generating a description containing an invalid
+field name. Consequently, fields starting with an underscore should have
+their leading underscore removed by the CTF trace readers.
+
The information available in the dynamic scopes can be thought of as the
current tracing context. At trace production, information about the
in a stream.
trace {
- major = value; /* Trace format version */
- minor = value;
+ major = value; /* CTF spec version major number */
+ minor = value; /* CTF spec version minor number */
uuid = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; /* Trace UUID */
byte_order = be OR le; /* Endianness (required) */
packet.header := struct {
};
};
+/*
+ * The "env" (environment) scope contains assignment expressions. The
+ * field names and content are implementation-defined.
+ */
+env {
+ pid = value; /* example */
+ proc_name = "name"; /* example */
+ ...
+};
+
stream {
id = stream_id;
/* Type 1 - Few event IDs; Type 2 - Many event IDs. See section 6.1. */
};
event {
- name = event_name;
+ name = "event_name";
id = value; /* Numeric identifier within the stream */
stream_id = stream_id;
+ loglevel = value;
+ model.emf.uri = "string";
context := struct {
...
};
}
+8. Clocks
+
+Clock metadata allows to describe the clock topology of the system, as
+well as to detail each clock parameter. In absence of clock description,
+it is assumed that all fields named "timestamp" use the same clock
+source, which increments once per nanosecond.
+
+Describing a clock and how it is used by streams is threefold: first,
+the clock and clock topology should be described in a "clock"
+description block, e.g.:
+
+clock {
+ name = cycle_counter_sync;
+ uuid = "62189bee-96dc-11e0-91a8-cfa3d89f3923";
+ description = "Cycle counter synchronized across CPUs";
+ freq = 1000000000; /* frequency, in Hz */
+ /* precision in seconds is: 1000 * (1/freq) */
+ precision = 1000;
+ /*
+ * clock value offset from Epoch is:
+ * offset_s + (offset * (1/freq))
+ */
+ offset_s = 1326476837;
+ offset = 897235420;
+ absolute = FALSE;
+};
+
+The mandatory "name" field specifies the name of the clock identifier,
+which can later be used as a reference. The optional field "uuid" is the
+unique identifier of the clock. It can be used to correlate different
+traces that use the same clock. An optional textual description string
+can be added with the "description" field. The "freq" field is the
+initial frequency of the clock, in Hz. If the "freq" field is not
+present, the frequency is assumed to be 1000000000 (providing clock
+increment of 1 ns). The optional "precision" field details the
+uncertainty on the clock measurements, in (1/freq) units. The "offset_s"
+and "offset" fields indicate the offset from POSIX.1 Epoch, 1970-01-01
+00:00:00 +0000 (UTC), to the zero of value of the clock. The "offset_s"
+field is in seconds. The "offset" field is in (1/freq) units. If any of
+the "offset_s" or "offset" field is not present, it is assigned the 0
+value. The field "absolute" is TRUE if the clock is a global reference
+across different clock uuid (e.g. NTP time). Otherwise, "absolute" is
+FALSE, and the clock can be considered as synchronized only with other
+clocks that have the same uuid.
+
+
+Secondly, a reference to this clock should be added within an integer
+type:
+
+typealias integer {
+ size = 64; align = 1; signed = false;
+ map = clock.cycle_counter_sync.value;
+} := uint64_ccnt_t;
+
+Thirdly, stream declarations can reference the clock they use as a
+time-stamp source:
+
+struct packet_context {
+ uint64_ccnt_t ccnt_begin;
+ uint64_ccnt_t ccnt_end;
+ /* ... */
+};
+
+stream {
+ /* ... */
+ event.header := struct {
+ uint64_ccnt_t timestamp;
+ /* ... */
+ }
+ packet.context := struct packet_context;
+};
+
+For a N-bit integer type referring to a clock, if the integer overflows
+compared to the N low order bits of the clock prior value, then it is
+assumed that one, and only one, overflow occurred. It is therefore
+important that events encoding time on a small number of bits happen
+frequently enough to detect when more than one N-bit overflow occurs.
+
+In a packet context, clock field names ending with "_begin" and "_end"
+have a special meaning: this refers to the time-stamps at, respectively,
+the beginning and the end of each packet.
+
+
A. Helper macros
The two following macros keep track of the size of a GNU/C structure without
align
const
char
+clock
double
enum
+env
event
floating_point
float
variant identifier variant-tag
variant-tag:
- < identifier >
+ < unary-expression >
enum-specifier:
enum identifier-opt { enumerator-list }
2.3) CTF-specific declarations
ctf-specifier:
+ clock { ctf-assignment-expression-list-opt }
event { ctf-assignment-expression-list-opt }
stream { ctf-assignment-expression-list-opt }
+ env { ctf-assignment-expression-list-opt }
trace { ctf-assignment-expression-list-opt }
typealias declaration-specifiers abstract-declarator-list type-assignment-operator declaration-specifiers abstract-declarator-list
typealias declaration-specifiers abstract-declarator-list type-assignment-operator declarator-list