X-Git-Url: http://git.efficios.com/?a=blobdiff_plain;f=doc%2Fman%2Fbabeltrace2-intro.7.txt;h=a82c00bd6cd916c3354e73896fa25907ea5d159a;hb=2ae422dd2857c664d6657c388359034015d94315;hp=62cea7085e71e51056736874fba4c43f5650f8ee;hpb=a8be4294fc84cc1b81b1690412c22c9e06cdb8e3;p=babeltrace.git diff --git a/doc/man/babeltrace2-intro.7.txt b/doc/man/babeltrace2-intro.7.txt index 62cea708..a82c00bd 100644 --- a/doc/man/babeltrace2-intro.7.txt +++ b/doc/man/babeltrace2-intro.7.txt @@ -1,158 +1,292 @@ -babeltrace2-intro(7) -=================== -:manpagetype: man page -:revdate: 5 October 2017 += babeltrace2-intro(7) +:manpagetype: manual page +:revdate: 14 September 2019 -NAME ----- -babeltrace2-intro - Introduction to Babeltrace +== NAME + +babeltrace2-intro - Introduction to Babeltrace 2 -DESCRIPTION ------------ -This man page is an introduction to the Babeltrace project. +== DESCRIPTION -The <> section lists the parts of the -project and shows the major changes from Babeltrace{nbsp}1 to -Babeltrace{nbsp}2 while the <> section -defines the core concepts of Babeltrace. +This manual page is an introduction to the Babeltrace~2 project. -The <> section shows -how some <> are visually represented in other -Babeltrace man pages. +The <> section describes the +parts of the project and shows the major changes from Babeltrace~1 +to Babeltrace~2 while the <> section defines the core concepts of Babeltrace~2. + +The <> section +shows how some <> are visually represented in other +Babeltrace~2 manual pages. [[what-is]] -WHAT IS BABELTRACE? -------------------- -Babeltrace is an open-source software project of which the purpose is -to process or convert +== WHAT IS BABELTRACE~2? + +Babeltrace~2 is an open-source software project of which the +purpose is to process or convert https://en.wikipedia.org/wiki/Tracing_(software)[traces]. -The Babeltrace project includes the following parts: +The Babeltrace~2 project includes the following parts: -[[libbabeltrace2]]Babeltrace library (libbabeltrace2):: +[[libbabeltrace2]]Babeltrace~2 library (libbabeltrace2):: A shared library with a C API. + With libbabeltrace2, you can programmatically create <> -and <>, build and run <>, and more (see the <> section for -more details about those concepts). All the other Babeltrace parts rely -on this library. +and <>, build and run <>, and more (see the <> section for more details about those concepts). ++ +All the other Babeltrace~2 parts rely on this library. -[[babeltrace2-1]]`babeltrace2` command:: +[[babeltrace2]]`babeltrace2` command-line program:: A command-line interface which uses libbabeltrace2 to load plugins, - create a trace processing graph, create components, and run the - graph. + create a trace processing graph, create <>, connect + their <> correctly, and run the graph. + -You can also use `babeltrace2` to list the available plugins or to query -an object from a component class. +You can also use `babeltrace2` to list the available plugins or to +<> an object from a component class. + See man:babeltrace2(1). -[[python-bindings]]Babeltrace Python bindings:: - A Python{nbsp}3 package which offers a Pythonic interface of +[[python-bindings]]Babeltrace~2 Python bindings:: + A Python~3 package (`bt2`) which offers a Pythonic interface of libbabeltrace2. + -You can perform the same operations which are available in libbabeltrace2 -with the Python bindings, but in a really easier way and with less code. +You can perform the same operations which are available in +libbabeltrace2 with the Python bindings, but more conveniently and with +less code. However, the Python bindings are less performant than +libbabeltrace2. -Babeltrace project's plugins:: - The Babeltrace <> shipped with the project. +Babeltrace~2 project's plugins:: + The Babeltrace~2 <> shipped with the project. + -Those plugins are not special, in that they only rely on libbabeltrace2 -and you don't need them to use libbabeltrace2, man:babeltrace2(1), or the -Python bindings. +Those plugins are not special in that they only rely on libbabeltrace2 +and you don't need them to use libbabeltrace2, man:babeltrace2(1), or +the Python bindings. However, the project's plugins provide many widely +used trace format encoders/decoders as well as common <> utilities. + -The Babeltrace project's plugins are: +The Babeltrace~2 project's plugins are: + -- `ctf`:: - Common Trace Format input/output, including the LTTng live source. + https://diamon.org/ctf/[Common Trace Format] (CTF) input/output, + including the LTTng live source. + See man:babeltrace2-plugin-ctf(7). `lttng-utils`:: - Graph utilities specific to http://lttng.org/[LTTng] traces. + Graph utilities specific to https://lttng.org/[LTTng] traces. + See man:babeltrace2-plugin-lttng-utils(7). `text`:: - Text input/output. + Plain text input/output. + See man:babeltrace2-plugin-text(7). `utils`:: - Graph utilities (muxer, trimmer, counter, dummy sink). + Common graph utilities (muxer, trimmer, counter, dummy sink). + See man:babeltrace2-plugin-utils(7). -- -Python plugin provider:: - A shared library which libbabeltrace2 tries to load to add support - for Babeltrace plugins written in Python. + +=== Changes since Babeltrace~1 + +This manual page is an introduction to Babeltrace~2, a rewrite of +Babeltrace~1 with a focus on extensibility, flexibility, and +interoperability. + +Babeltrace~1 exists since 2010. + +You can install both projects on the same file system as there are no +file name conflicts. + +The major improvements brought by Babeltrace~2 are: + +General:: ++ +-- +* Full <> support: any user can distribute a Babeltrace~2 + plugin and, as long as <> finds it, any + application linked to libbabeltrace2 can load it and use it. ++ +Plugins are not just trace format encoders and decoders: they package +source, filter, and sink <> so that you can +connect specialized, reusable <> together in a +<> to create a customized trace conversion +or analysis device. ++ +This modular strategy is much like how the +https://www.ffmpeg.org/[FFmpeg], +https://gstreamer.freedesktop.org/[GStreamer], and +https://en.wikipedia.org/wiki/DirectShow[DirectShow] projects approach +media stream processing. + +* All the parts of the Babeltrace~2 project run on the major + operating systems, including Windows and macOS. + +* Some <>, such as compcls:sink.text.pretty + (similar to the `text` output format of man:babeltrace(1)) and + compcls:sink.text.details, can write color codes to the standard + output when it's connected to a color-enabled terminal. ++ +The Babeltrace~2 log, printed to the standard output, can also be +colorized. +-- + +Command-line interface:: + -The package you use to write a Python Babeltrace plugin is the one -provided by the Python bindings. +-- +* Whereas you can convert traces from one format to another with + Babeltrace~1's CLI tool, man:babeltrace(1), you can also execute a + custom trace manipulation task with man:babeltrace2(1) thanks to the + man:babeltrace2-run(1) command. +* The man:babeltrace2-convert(1) command features an automatic source + component discovery algorithm to find the best suited components to + create for a given non-option argument (file or directory path, or + custom string like an https://lttng.org[LTTng live] URL). ++ +For example: ++ +[role="term"] +---- +$ babeltrace2 /path/to/ctf/trace +---- ++ +[role="term"] +---- +$ babeltrace2 net://localhost/host/myhost/my-session +---- +-- + +https://diamon.org/ctf/[CTF] input/output:: ++ +-- +* The compcls:source.ctf.fs component class, which is more or less the + equivalent of Babeltrace~1's `ctf` input format, has features + not found in Babeltrace~1: ++ +-- +** The component handles many trace quirks which are the results of + known tracer bugs and corner cases (LTTng-UST, LTTng-modules, and + https://github.com/efficios/barectf[barectf]), making it possible to + decode malformed packets. + +** The component merges CTF traces sharing the same UUID into a single, + logical trace. ++ +This feature supports LTTng~2.11's tracing session rotation trace +chunks. +-- -Changes since Babeltrace{nbsp}1 -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This man page is an introduction to Babeltrace{nbsp}2, a rewrite of -Babeltrace{nbsp}1 with a focus on extensibility and flexibility. +* With a compcls:sink.ctf.fs component, you can create CTF traces on + the file system. ++ +With man:babeltrace2(1), you can use the `--output-format=ctf` and +manopt:babeltrace2-convert(1):--output options to create an implicit +compcls:sink.ctf.fs component. ++ +For example: ++ +[role="term"] +---- +$ babeltrace2 /path/to/input/trace \ + --output-format=ctf --output=trace-dir +---- +-- -Babeltrace{nbsp}1 exists since 2010. The major improvements brought by -Babeltrace{nbsp}2 are: +https://lttng.org[LTTng live] input:: ++ +-- +* The man:babeltrace(1) command exits successfully when it cannot find + an LTTng live (`--input-format=lttng-live` option) tracing session. ++ +The manparam:source.ctf.lttng-live:session-not-found-action +initialization parameter controls what a compcls:source.ctf.lttng-live +<> does when it cannot find the remote +tracing session. ++ +If the action is `end`, the message iterator does like man:babeltrace(1) +and simply ends successfully. ++ +If the action is `continue` (the default), the message iterator never +ends: it keeps on trying until the tracing session exists, indeed +subscribing to the session. +-- -* Full plugin support: any user can distribute a Babeltrace plugin and, - as long as <> finds it, any application - linked to libbabeltrace2 can load it and use it. +Library:: ++ +-- +* <> shares nothing with libbabeltrace. ++ +The Babeltrace~2 library C API has features such as: + -Plugins are not just input and output formats: they provide source, -filter, and sink <> so that you can connect -specialized, reusable components together in a graph to create a -customized trace conversion or analysis device. +-- +* A single header file. +* Function precondition and postcondition checking. +* Object-oriented model with shared and unique objects. +* Strict C typing and `const` correctness. +* User-extensible classes. +* Rich, thread-safe error reporting. +* Per-<> and per-subsystem logging levels. +* Trace intermediate representation (IR) objects to make the API + trace-format-agnostic. +* A versioned protocol for message interchange between components to + enable forward and backward compatibility. +-- -* In order to support user components, all the objects of libbabeltrace2 - have a reference count. The possible reference cycles are handled - internally so that the library's API is clean and predictable. The two - reference counting functions, `bt_get()` and `bt_put()`, are all you - need to manage the lifetime of any Babeltrace object. +* You can build the library in developer mode to enable an extensive set + of function precondition and postcondition checks. ++ +The developer mode can help detect programming errors early when you +develop a Babeltrace~2 <> or an application using +libbabeltrace2. ++ +See the project's `README` for build-time requirements and detailed +build instructions. -* All the parts of the Babeltrace project run on the major operating - systems, including Windows and macOS. +-- [[concepts]] -BABELTRACE CONCEPTS -------------------- -This section defines the main concepts of the Babeltrace project. These -concepts translate into types and functions in +== BABELTRACE~2 CONCEPTS + +This section defines the main concepts of the Babeltrace~2 project. + +These concepts translate into types and functions in <> and its <>, but also as command-line actions and options in the -<>. The other Babeltrace man pages -assume that you are familiar with the following definitions. +<>. The other Babeltrace~2 +manual pages assume that you are familiar with the following +definitions. -Some Babeltrace concepts are interdependent: it is normal to jump from -one definition to another to understand the big picture. +Some Babeltrace~2 concepts are interdependent: it is normal to jump +from one definition to another to understand the big picture. [[comp-cls]]Component class:: - A reusable class from which you can instantiate one or more - <> instances. + A reusable class which you can instantiate as one or more + <> within a <>. + -There are three types of component classes used to instantiate the three -types of components (source, filter, and sink). +There are three types of component classes used to create the three +types of components: source, filter, and sink. + -A component class provides methods, one of which is an initialization +A component class implements methods, one of which is an initialization method, or constructor, to create a component. You pass _initialization parameters_ to this method to customize the created component. For -example, the initialization method of the compcls:src.ctf.fs component -class accepts a mandatory manparam:source.ctf.fs:path parameter which is -the file system path to the trace(s). It also accepts an optional -manparam:source.ctf.fs:clock-class-offset-ns parameter which is an -offset, in nanoseconds, to add to all the clock classes found in the -traces's metadata. +example, the initialization method of the compcls:source.ctf.fs +component class accepts a mandatory manparam:source.ctf.fs:inputs +parameter which is an array of file system path(s) to the CTF trace(s). +It also accepts an optional manparam:source.ctf.fs:clock-class-offset-ns +parameter which is an offset, in nanoseconds, to add to all the clock +classes (descriptors of stream clocks) found in the traces's metadata. ++ +A component class can have a description and a help text. [[comp]]Component:: A node within a <>. @@ -161,26 +295,25 @@ There are three types of components: + -- Source component:: - An input component which produces <>. + An input component which produces <>. + -Examples: CTF files input, log file input, LTTng-live input, random +Examples: CTF files input, log file input, LTTng live input, random event generator. Filter component:: - An intermediate component which can discard the notifications it - receives, transform them, augment them, sort them, or create new - ones. + An intermediate component which can transform the messages it + consumes, augment them, sort them, discard them, or create new ones. + -Examples: filter which removes notifications based on an expression, -filter which adds debugging information to selected events, notification -multiplexer, trace trimmer. +Examples: filter which removes messages based on an expression, +filter which adds debugging information to selected events, message +muxer, trace trimmer. Sink component:: - An output component which consumes notifications and usually writes - them to one or more formatted files. + An output component which consumes messages and usually writes them + to one or more formatted files. + -Examples: log file output, CTF files output, text output on the -console. +Examples: log file output, CTF files output, pretty-printed plain text +output. -- + Components are connected together within a <>. The terms -_component_ and _component instance_ are equivalent. +_component_ and _component class instance_ are equivalent. + Within a trace processing graph, each component has a unique name. This is not the name of its component class, but an instance name. If `human` -is a component class name, than `John` could be a component name. +is a component class name, than `Nancy` and `John` could be component +names. ++ +Once a <> is configured (the first time it runs), you +cannot add components to it for the remaining graph's lifetime. [[port]]Port:: A connection point, on a <>, from which are sent or - to which are received <> when the <> is running. + where are received <> when the <> runs. + -An output port is where notifications are sent. An input port is where -notifications are received. Source components have output ports, sink +An output port is from where messages are sent. An input port is where +messages are received. Source components have output ports, sink components have input ports, and filter components have both. + -An output port can only be connected to a single input port at a given -time. +You can only connect an output port to a single input port. ++ +All ports do not need to be connected. + -A filter or sink component receiving notifications from its input ports -is said to _consume_ notifications. +A filter or sink component receiving messages from its input ports +is said to _consume_ messages. + The link between an output port and input port is a <>. + -A component can dynamically add and remove ports while a graph is -running. For example, a compcls:filter.utils.muxer component always -makes sure that it has at least one available input port. +Once a <> is configured (the first time it runs), you +cannot connect ports for the remaining graph's lifetime. [[conn]]Connection:: The link between an output <> and an input port through - which <> flow when a <> is running. + which <> flow when a <> runs. + +[[msg-iter]]Message iterator:: + An iterator on an input <> of which the returned elements + are <>. ++ +A <> or another message iterator can create many message +iterators on a single input port, before or while the <> runs. -[[notif]]Notification:: - An atomic element sent from an output <> to an - input port. +[[msg]]Message:: + The element of a <>. + -A source <> produces notifications, while a sink -component consumes them. A filter component can both consume and -produce notifications. +Messages flow from output <> to input ports. + -The main types of notifications are: +A source <> <> produces +messages, while a sink component consumes them. A filter component +message iterator can both consume and produce messages. ++ +The main types of messages are: + -- Event:: - A trace event record within a packet. + A trace event record within a packet or within a stream. Packet beginning:: The beginning of a packet within a stream. + -A packet is a container of events. +A packet is a conceptual container of events. Packet end:: The end of a packet within a stream. @@ -246,10 +392,10 @@ Packet end:: Stream beginning:: The beginning of a stream. + -A stream is a container of packets. +A stream is a conceptual container of packets and/or events. + -Usually, a given source component's output port sends packet and -event notifications which belong to a single stream. +Usually, a given source component's output port sends packet and event +messages which belong to a single stream, but it's not required. Stream end:: The end of a stream. @@ -265,61 +411,65 @@ Discarded packets:: [[graph]]Trace processing graph:: A https://en.wikipedia.org/wiki/Filter_graph[filter graph] where - nodes are <> and <> flow from + nodes are <> and <> flow from output <> to input ports. + You can build a trace processing graph with -<>, with the <>, or with the man:babeltrace2-run(1) and -man:babeltrace2-convert(1) commands. +<>, with the +<>, or with the +man:babeltrace2-run(1) and man:babeltrace2-convert(1) CLI commands. + -When you _run_ a trace processing graph, the sink components consume -notifications from their input ports, making all the graph's components -work one notification at a time to perform the trace conversion or -analysis. +When a trace processing graph _runs_, the sink components consume +messages from their input ports, making all the graph's +<> work one message at a time to perform the +trace conversion or analysis duty. [[plugin]]Plugin:: - A container of <> as a shared library. + A container, or package, of <> as a + shared library or Python module. + Each component class within a plugin has a type (source, filter, or sink) and a name. The type and name pair is unique within a given plugin. + -<> can load a plugin (`.so` or `.dll` file) -at run time: the result is a plugin object in which you can find a -specific component class and instantiate it within a <> as a <>. +<> can load a plugin (`.so`, `.dll`, or +`.py` file) at run time: the result is a plugin object in which you can +find a specific component class and instantiate it within a +<> as a <>. + -The <> uses the -'TYPE.PLUGIN.COMPCLS' format to identify a specific component -class within a specific plugin. 'TYPE' is either `source`, `filter`, -or `sink`. +The <> uses the +'COMP-CLS-TYPE.PLUGIN-NAME.COMP-CLS-NAME' format to identify a specific +component class within a specific plugin. 'COMP-CLS-TYPE' is either +`source` (or `src`), `filter` (or `flt`), or `sink`. + -You can list the available Babeltrace plugins with the +You can list the available Babeltrace~2 plugins with the man:babeltrace2-list-plugins(1) command. [[query]]Query:: An operation with which you can get a named object from a - <>, possibly with the help of query - parameters. + <>, possibly with custom query parameters. + The plain text metadata stream of a CTF trace and the available LTTng -live sessions of a given LTTng relay daemon are examples of queries. +live sessions of a given LTTng relay daemon are examples of query +objects. + -You can use the man:babeltrace2-query(1) command to query a component -class's object. +You can use <>, the +<>, or the +man:babeltrace2-query(1) CLI command to query a component class's +object. [[graph-repr]] -TRACE PROCESSING GRAPH REPRESENTATION -------------------------------------- -In the Babeltrace man pages, a component is represented with a box. The -box has the <> type, <> name, -and component class name at the top. Just below, between square -brackets, is its component instance name within the <> type, +<> name, and component class name at the top. Just below, +between square brackets, is its component name within the <>. Each <> is represented with an `@` symbol -on the edge of the component box with its name inside the box. Output -ports are on the right edge while input ports are on the left edge. +on the border(s) of the component box with its name inside the box. +Output ports are on the box's right border while input ports are on the +box's left border. For example, here's a source component box: @@ -334,9 +484,9 @@ For example, here's a source component box: +------------+ ---- -This one is an instance of the compcls:src.ctf.fs component class named -`my-src`. It has three output ports named `stream0`, `stream1`, and -`stream2`. +This one is an instance of the compcls:source.ctf.fs component class +named `my-src`. It has three output ports named `stream0`, `stream1`, +and `stream2`. A trace processing graph is represented with multiple component boxes connected together. The <> are arrows from output @@ -356,28 +506,28 @@ For example, here's a simple conversion graph: +-----------------+ ---- -Note that input port `in3` of component `muxer` is not currently -connected in this example. +Note that input port `in3` of component `muxer` is not connected in this +example. Sometimes, we symbolically represent other resources which are consumed from or produced by components. In this case, arrows are used, but they -do not go to or from port symbols (`@`). For example, in the graph above, -the `ctf` component consumes a CTF trace and the `text` component -prints to the console, so here's a more complete diagram: +do not go to or from port symbols (`@`), except for messages. For +example, in the graph above, the `ctf` source component consumes a CTF +trace and the `text` sink component prints plain text to the terminal, +so here's a more complete diagram: ---- - CTF trace - | -.------' -| +------------+ +-----------------+ +------------------+ -| | src.ctf.fs | | flt.utils.muxer | | sink.text.pretty | -'->| [ctf] | | [muxer] | | [text] | - | | | | | | - | stream0 @--->@ in0 out @--->@ in | - | stream1 @--->@ in1 | +--+---------------+ - | stream2 @--->@ in2 | | - +------------+ @ in3 | '---> Console - +-----------------+ +CTF trace + | + | +------------+ +-----------------+ +------------------+ + | | src.ctf.fs | | flt.utils.muxer | | sink.text.pretty | + '-->| [ctf] | | [muxer] | | [text] | + | | | | | | + | stream0 @--->@ in0 out @--->@ in | + | stream1 @--->@ in1 | +-----+------------+ + | stream2 @--->@ in2 | | + +------------+ @ in3 | '--> Terminal + +-----------------+ ---- Here's another example of a more complex graph which splits a specific @@ -410,6 +560,6 @@ stream using some criteria: include::common-footer.txt[] -SEE ALSO --------- +== SEE ALSO + man:babeltrace2(1)