Add dynamic array tracing tests

[deliverable/barectf.git] / README.md

![Icon made by Freepik from www.flaticon.com](https://lttng.org/blog/images/barectf.png)

:bangbang: | ATTENTION: barectf is currently transitioning from v2 to v3. In the meantime, this README might be incorrect. New documentation will be written.
:---: | :---

[![](https://img.shields.io/pypi/v/barectf.svg)](https://pypi.python.org/pypi/barectf)
[![Jenkins](https://img.shields.io/jenkins/s/https/ci.lttng.org/barectf_master_build.svg)](https://ci.lttng.org/job/barectf_master_build)

_**barectf**_ is a command-line generator of ANSI C tracers which output
[Common Trace Format](http://diamon.org/ctf) packets natively.

You will find barectf interesting if:

  1. You need to trace an application.
  2. You need tracing to be efficient, yet flexible: record integers of
     custom sizes and alignments, floating point numbers, enumerations
     supported by a specific integer type, and null-terminated
     UTF-8/ASCII strings (C strings).
  3. You need to be able to convert the recorded binary events to
     human-readable text, as well as analyze them with Python scripts
     ([Babeltrace](http://diamon.org/babeltrace/) does all that,
     given a CTF input). [Trace Compass](http://tracecompass.org/) is
     another CTF-compatible application.
  4. You _cannot_ use [LTTng](http://lttng.org/), an efficient tracing
     framework for the Linux kernel and Linux/BSD user applications,
     which also outputs CTF traces.

The target audience of barectf is developers who need to trace [bare
metal](https://en.wikipedia.org/wiki/Bare_machine) systems. The code
produced by barectf is pure ANCI C (with one exception, see the current
limitations below) and can be lightweight enough to fit on a tiny
microcontroller.

**Key features**:

  * Single input: easy-to-write [YAML configuration
    file](https://github.com/efficios/barectf/wiki/Writing-the-YAML-configuration-file).
  * 1-to-1 mapping from tracing function parameters to event fields.
  * Custom and bundled
    [_platforms_](https://github.com/efficios/barectf/wiki/barectf-platform)
    hiding the details of opening/closing packets and writing them to a
    back-end (continuous tracing), getting the clock values, etc.:
    * _linux-fs_: basic Linux application tracing platform which writes
      stream files to the file system for demonstration purposes.
    * _parallella_: Adapteva
      Epiphany/[Parallella](http://parallella.org/) with host-side
      consumer.
  * CTF metadata is generated by the command-line tool (automatic trace
    UUID, stream IDs, and event IDs).
  * All basic CTF types are supported: integers, floating point numbers,
    enumerations, and null-terminated strings (C strings).
  * Binary streams produced by the generated tracer and metadata file
    produced by barectf are CTF 1.8-compliant.
  * Human-readable error reporting at generation time.
  * barectf is written in Python 3, hence you can run the tool on
    various platforms.
  * Generated tracers are known to build with `gcc` (tested with the
    IA-32, x86-64, MIPS, ARM, and AVR architectures), `g++`, `clang`,
    `clang++`, [`8cc`](https://github.com/rui314/8cc),
    [`tcc`](http://bellard.org/tcc/), VS2008 (with a custom `stdint.h`),
    and VS2010.

**Current limitations**:

As of this version:

  * All the generated tracing C functions, for a given barectf
    stream-specific context, need to be called from the same thread, and
    cannot be called from an interrupt handler, _unless_ a user-provided
    synchronization mechanism is used.
  * The generated C code needs the `stdint.h` header, which is new in
    C99. If your standard C library does not have this header,
    you can create one yourself and put it in one of your include
    directories to define the following types according to your
    architecture:
    * `int8_t`
    * `int16_t`
    * `int32_t`
    * `int64_t`
    * `uint8_t`
    * `uint16_t`
    * `uint32_t`
    * `uint64_t`
  * CTF compound types (array, sequence, structure, variant) are not
    supported yet, except at some very specific locations in the
    metadata.


## Installing

Make sure you have Python 3 and `pip` for Python 3 installed, then
install barectf.

Note that you may pass the `--user` argument to
`pip install` to install the tool in your home directory (instead of
installing globally).

**Ubuntu 14.04 and 16.04**:

It is recommended to use the
[barectf PPA](https://launchpad.net/~lttng/+archive/ubuntu/barectf),
which also installs the man page:

    sudo apt-add-repository ppa:lttng/barectf
    sudo apt-get update
    sudo apt-get install python3-barectf

Otherwise, you can always use `pip3`:

    sudo apt-get install python3-pip
    sudo pip3 install barectf

**Other, recent Ubuntu**:

    sudo apt-get install python3-pip
    sudo pip3 install barectf

**Ubuntu 12.04 and lower**:

    sudo apt-get install python3-setuptools
    sudo easy_install3 pip
    sudo pip3 install barectf

**Debian**:

    sudo apt-get install python3-pip
    sudo pip3 install barectf

**Fedora 20 and up**:

    sudo yum install python3-pip
    sudo pip3 install barectf

**Arch Linux**:

It is recommended to use the
[AUR package](https://aur.archlinux.org/packages/barectf/), which also
installs the man page. If you have
[yaourt](https://archlinux.fr/yaourt-en):

    sudo yaourt -Sy barectf

Otherwise, you can always use `pip`:

    sudo pacman -S python-pip
    sudo pip install barectf

**macOS (OS X)**:

With [Homebrew](http://brew.sh/):

    brew install python3
    pip3 install barectf


### Man page

Since the philosophy of setuptools packages is to include everything
within the package, the barectf man page is not installed on the system
when installing barectf with `pip` or with `setup.py`. This would be the
job of distribution packages.

You can install it manually:

    wget https://raw.githubusercontent.com/efficios/barectf/vVERSION/doc/man/barectf.1 -O /usr/local/man/man1/barectf.1

Replace `VERSION` with the desired version, for example:

    wget https://raw.githubusercontent.com/efficios/barectf/v2.1.4/doc/man/barectf.1 -O /usr/local/man/man1/barectf.1


## What is CTF?

See the [CTF in a nutshell](http://diamon.org/ctf/#ctf-in-a-nutshell)
section of CTF's website to understand the basics of this
trace format.

The most important thing to understand about CTF, for barectf use
cases, is the layout of a binary stream packet:

  * Packet header (defined at the trace level)
  * Packet context (defined at the stream level)
  * Sequence of events (defined at the stream level):
    * Event header (defined at the stream level)
    * Stream event context (defined at the stream level)
    * Event context (defined at the event level)
    * Event payload (defined at the event level)

The following diagram, stolen without remorse from CTF's website, shows
said packet layout:

![](http://diamon.org/ctf/img/ctf-stream-packet.png)

Any of those six dynamic scopes, if defined at all, has an associated
CTF type. barectf requires them to be structure types.


## Using

See the [project's wiki](https://github.com/efficios/barectf/wiki) which
contains all the information needed to use barectf.


## Testing

Bash is required for testing barectf.

The barectf tests execute the `barectf` command available in your
`$PATH`. The best way to test a specific version of barectf is to create
a Python 3 [virtual environment](https://virtualenv.pypa.io/en/latest/),
install the appropriate version, and then run the tests.

In the barectf source tree root, do:

    virtualenv --python=python3 virt
    . ./virt/bin/activate
    rehash # if using zsh
    ./setup.py install
    (cd tests && ./test.bash)

You can specify [Bats](https://github.com/sstephenson/bats) options to
`./test.bash`, like `--tap` to get a [TAP](https://testanything.org/)
output.

You can exit the virtual environment by running `deactivate`.


## Community

Since the barectf community is small, it's sharing the communication
channels of the [LTTng](http://lttng.org/) project,
as [EfficiOS](http://www.efficios.com/) is the main sponsor of both
projects. It goes like this:

| Item | Location | Notes |
| --- | --- | --- |
| Mailing list | [lttng-dev](https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev) (`lttng-dev@lists.lttng.org`) | Preferably, use the `[barectf]` subject prefix |
| IRC | [`#lttng`](irc://irc.oftc.net/lttng) on the OFTC network | More specifically, query `eepp` (barectf's maintainer) on this network or on freenode |
| Code contribution | Create a new GitHub [pull request](https://github.com/efficios/barectf/pulls) | |
| Bug reporting | Create a new GitHub [issue](https://github.com/efficios/barectf/issues/new) | |
| Continuous integration | [barectf item on LTTng's CI](https://ci.lttng.org/job/barectf/) | |
| Blog | The [LTTng blog](http://lttng.org/blog/) contains many posts about barectf | |