docs: whats-new.adoc: use headings for main topics

[barectf.git] / docs / modules / ROOT / pages / whats-new.adoc

= What's new in barectf{nbsp}3?

barectf{nbsp}3 is a major update of the project.

This page presents the main, user-visible changes.

== Support for user-defined static and dynamic array field types

You can now use your own xref:yaml:static-array-ft-obj.adoc[static] and
xref:yaml:dyn-array-ft-obj.adoc[dynamic array field types] within your
xref:yaml:dst-obj.adoc[data stream type] or
xref:yaml:ert-obj.adoc[event record type] root field types.

====
This xref:yaml:index.adoc[YAML configuration] has static and dynamic
array field type objects:

[source,yaml]
----
--- !<tag:barectf.org,2020/3/config>
trace:
  type:
    $include:
      - stdint.yaml
      - stdmisc.yaml
    native-byte-order: little-endian
    data-stream-types:
      my_stream:
        event-record-common-context-field-type:
          class: structure
          members:
            - src_ip_addr:
                field-type:
                  class: static-array
                  length: 4
                  element-field-type: uint8
        event-record-types:
          my_event:
            payload-field-type:
              class: structure
              members:
                - message_id: uint32
                - messages:
                    field-type:
                      class: dynamic-array
                      element-field-type: string
----

barectf generates the following
C{nbsp}xref:tracing-funcs:index.adoc[tracing function] for the
`my_event` event record type:

[source,c]
----
void barectf_my_stream_trace_my_event(struct barectf_my_stream_ctx *sctx,
                                      const uint8_t *cc_src_ip_addr,
                                      uint32_t p_message_id,
                                      uint32_t p___messages_len,
                                      const char * const *p_messages);
----
====

[[api]]
== Reworked Python API

The `barectf` Python API was revamped to:

* Match the recent CTF terminology and model
  (like https://babeltrace.org/[Babeltrace{nbsp}2] and the
  upcoming CTF{nbsp}2).

* Remove unneeded properties from some classes, like the encoding of an
  integer field type object or the byte order of any field type object
  (barectf{nbsp}3.0 always uses the configured native byte order).

* Have a better object model.
+
For example, a `barectf.SignedEnumerationFieldType` object is a
`barectf.SignedIntegerFieldType` object, which is a
`barectf._BitArrayFieldType` object, which is a
`barectf._FieldType` object.

* Decouple the xref:yaml:index.adoc#stages[effective YAML
  configuration object] printing operation from the parsing operation.
+
Now, `barectf.configuration_from_file()` returns a configuration object
from a YAML configuration while `barectf.effective_configuration_file()`
returns the effective version of a YAML configuration.

* Make the `barectf.configuration_from_file()`,
  `barectf.effective_configuration_file()`, and
  `barectf.configuration_file_major_version()` functions accept a
  https://docs.python.org/3/glossary.html#term-file-object[file object]
  instead of a file system path.
+
Those three functions can process barectf{nbsp}2 and{nbsp}<<yaml,3>>
YAML configurations.

* Offer all the API from the `barectf` package itself.
+
From which modules are the names exactly is now an implementation
detail: you only need to import `barectf`.

NOTE: The `barectf` Python package's documentation is not available yet.

[[yaml]]
== New YAML configuration schema

barectf{nbsp}3.0 comes with a new YAML configuration schema to
parallel the <<api,new Python API>>.

Rest assured that both the barectf Python API and the
<<cli,`barectf` CLI tool>> can still process a
barectf{nbsp}2 YAML configuration.

The most significant changes are:

* An updated terminology for property names and values.

* The xref:yaml:cfg-obj.adoc[configuration object] mapping (the YAML
  document) must be
  xref:yaml:yaml-primer.adoc#tag[tagged] with
  `tag:barectf.org,2020/3/config`:
+
.Configuration object
[source,yaml]
----
--- !<tag:barectf.org,2020/3/config>
# ...
----
+
This is how barectf identifies a barectf{nbsp}3 YAML configuration.
+
This YAML tag also makes it possible to insert a barectf configuration
object within another, unrelated YAML document if need be.

* The configuration object contains a xref:yaml:trace-obj.adoc[trace
  object] which itself contains a xref:yaml:trace-type-obj.adoc[trace
  type object]:
+
.Configuration object
[source,yaml]
----
--- !<tag:barectf.org,2020/3/config>
trace:
  type:
    native-byte-order: big-endian
    uuid: 296e1e04-91e4-4b54-b77c-19e776f1d3a7
    # ...
  environment:
    sys_id: 1983
    node_id: 17
    sys_name: tel/long-can
----
+
A trace can have an environment while a trace type can have a UUID.

* You can specify _two_ independent
  xref:yaml:cfg-obj.adoc#prefix-obj[prefixes] instead of a single one: a
  file name prefix and an identifier prefix:
+
.Configuration object
[source,yaml]
----
--- !<tag:barectf.org,2020/3/config>
options:
  code-generation:
    prefix:
      file-name: my-system
      identifier: msys_
trace:
  type:
    # ...
----
+
With barectf{nbsp}2, the single prefix `my_app_` used to become the file
name prefix `my_app` and the identifier prefix `my_app_`.

* You don't need to specify special CTF structure field type members,
  like `magic`, `stream_id`, and `timestamp`.
+
Instead, you specify xref:yaml:trace-type-obj.adoc#features-obj[trace
type] and xref:yaml:dst-obj.adoc#features-obj[data stream type
features]:
+
.Trace type object
[source,yaml]
----
$features:
  magic-field-type: true
  uuid-field-type: true
data-stream-types:
  my_stream:
    $features:
      packet:
        beginning-timestamp-field-type: true
        end-timestamp-field-type: false
        discarded-event-records-counter-snapshot-field-type: true
      event-record:
        timestamp-field-type: true
    # ...
  # ...
# ...
----
+
You can still control the exact field type of a feature:
+
.Trace type object
[source,yaml]
----
$features:
  data-stream-type-id-field-type: uint8
# ...
----

* A data stream type can have zero or one
  xref:yaml:dst-obj.adoc#def-clk-type-name-prop[default clock type]:
+
.Trace type object
[source,yaml]
----
clock-types:
  sys23:
    description: System clock (pin 23)
    frequency: 8000000
    origin-is-unix-epoch: false
data-stream-types:
  my_stream:
    $default-clock-type-name: sys23
    # ...
  # ...
# ...
----
+
When a data stream type has a default clock type, its timestamp integer
field types (packet beginning, packet end, and event record)
automatically refer to this specific clock type, effectively removing
the xref:yaml:int-ft-obj.adoc[integer field type object]'s
`property-mappings` property.

* The only way to make a data stream type the default one is with its
  xref:yaml:dst-obj.adoc#is-def-prop[`$is-default` boolean property].
+
.Trace type object
[source,yaml]
----
data-stream-types:
  my_stream:
    $is-default: true
    # ...
  # ...
# ...
----

* You cannot specify custom
  xref:how-barectf-works:ctf-primer.adoc#pkt[packet] header and
  xref:how-barectf-works:ctf-primer.adoc#er[event record] header field
  type members anymore.
+
The header field types only exist for the trace format itself.
+
Instead, xref:yaml:dst-obj.adoc#pkt-ctx-ft-extra-members-prop[append
user-defined members to the packet context field type] and use the
xref:yaml:dst-obj.adoc#er-common-ctx-ft-prop[event record common context
field type].

* You don't need to define
  xref:yaml:trace-type-obj.adoc#ft-aliases-prop[field type aliases]
  in any specific order:
+
.Trace type object
[source,yaml]
----
$field-type-aliases:
  user-id:
    $inherit: base-id
    size: 16
  base-id:
    class: unsigned-integer
    preferred-display-base: hexadecimal
# ...
----
+
This is also the case within a barectf{nbsp}2 YAML metadata object.

* An xref:yaml:int-ft-obj.adoc[integer field type] _is_ (conceptually
  inherits) a bit array field type.
+
A bit array field type has size and alignment properties. It doesn't
have a byte order property: as of barectf{nbsp}3.0, the generated tracer
always uses the configured
xref:yaml:trace-type-obj.adoc#native-bo-prop[native byte order].

* An integer field type object doesn't have a `signed` property:
  unsigned and signed integer field types are two different classes:
+
.xref:yaml:struct-ft-obj.adoc[Structure field type object]
[source,yaml]
----
class: structure
members:
  - unsigned_int:
      field-type:
        class: unsigned-integer
        size: 32
  - signed_int:
      field-type:
        class: signed-integer
        size: 16
----

* An xref:yaml:enum-ft-obj.adoc[enumeration field type] _is_ an integer
  field type; it doesn't have a `value-type` property anymore:
+
.Signed enumeration field type object
[source,yaml]
----
class: signed-enumeration
size: 16
alignment: 32
preferred-display-base: octal
# ...
----

* The xref:yaml:enum-ft-obj.adoc#mappings-prop[mappings] of an
  enumeration field type are now a YAML mapping of labels to sequences
  of integer ranges:
+
.Signed enumeration field type object
[source,yaml]
----
class: signed-enumeration
size: 16
mappings:
  Poly:
    - -23
    - [45, 1001]
  UdeM:
    - [2000, 3000]
  UQÀM:
    - [1, 5]
    - -40
----

* A xref:yaml:real-ft-obj.adoc[real field type] _is_ a bit array field
  type. Its `size` property indicates if it's single-precision
  or a double-precision:
+
.Real field type object
[source,yaml]
----
class: real
size: 64
----

* The xref:yaml:struct-ft-obj.adoc#members-prop[members] of a structure
  field type are a sequence instead of a mapping (YAML mappings are
  _not_ ordered):
+
.Structure field type object
[source,yaml]
----
class: structure
members:
  - msg: string
  - msg_id: uint8
  - exceptions:
      field-type:
        class: dynamic-array
        element-field-type: string
----
+
This sequence is considered to be an _ordered mapping_, similar to
YAML's https://yaml.org/type/omap.html[`+!!omap+`] type.

The xref:yaml:include.adoc#std[standard partial YAML files] were updated
to honour the new YAML configuration schema when a barectf{nbsp}3 YAML
configuration includes them.

[[cli]]
== Upgraded command-line interface

The xref:cli:usage.adoc[`barectf` CLI tool] now has a
https://git-scm.com/[Git]-like user interface with the following commands

xref:cli:usage.adoc#generate-command[`generate`]::
    Exactly the barectf{nbsp}2 command-line interface: generates
    the C{nbsp}source and CTF metadata stream files of a tracer
    from a xref:yaml:index.adoc[YAML configuration file].

xref:cli:usage.adoc#show-effective-configuration-command[`show-effective-configuration`]::
    Prints the xref:yaml:index.adoc#stages[_effective_] version of
    a YAML configuration file.

xref:cli:usage.adoc#show-configuration-version-command[`show-configuration-version`]::
    Prints the major version (2 or 3) of a YAML configuration file.

The `barectf` CLI tool remains backward compatible with its
barectf{nbsp}2 counterpart: the default command is `generate`.

== Improved generated C{nbsp}code

The generated C{nbsp}code is now ``const``-correct.

There are a few new public definition and function aliases to match the
<<api,Python API>>'s updated terminology:

* `_BARECTF_IDENTIFIER_PREFIX` is defined to the same value as
  `_BARECTF_PREFIX`.
+
See the code generation header options object's
xref:yaml:cfg-obj.adoc#iden-prefix-def-prop[`identifier-prefix-definition`
property].

* `_BARECTF_DEFAULT_DATA_STREAM_TYPE_NAME` is defined to the same
  value as `_BARECTF_DEFAULT_STREAM`.
+
See the code generation header options object's
xref:yaml:cfg-obj.adoc#def-dst-name-def-prop[`default-data-stream-type-name-definition`
property].

* xref:platform:api.adoc#barectf-disc-er-count-func[`+barectf_discarded_event_records_count()+`]
  returns the same thing as `+barectf_packet_events_discarded()+`.

* xref:platform:api.adoc#barectf-pkt-buf-addr-func[`+barectf_packet_buf_addr()+`]
  returns the same thing as `+barectf_packet_buf()+`.

The `barectf-bitfield.h` header only contains what's needed by the
target's xref:yaml:trace-type-obj.adoc#native-bo-prop[native byte
order]. Also, only `barectf.c` includes this header now, not leaking its
definitions through the public `barectf.h` header.