doc/man/babeltrace2-intro.7.txt

   1 = babeltrace2-intro(7)
   2 :manpagetype: manual page
   3 :revdate: 14 September 2019
   4
   5
   6 == NAME
   7
   8 babeltrace2-intro - Introduction to Babeltrace 2
   9
  10
  11 == DESCRIPTION
  12
  13 This manual page is an introduction to the Babeltrace~2 project.
  14
  15 The <<what-is,``WHAT IS BABELTRACE~2?''>> section describes the
  16 parts of the project and shows the major changes from Babeltrace~1
  17 to Babeltrace~2 while the <<concepts,``BABELTRACE~2
  18 CONCEPTS''>> section defines the core concepts of Babeltrace~2.
  19
  20 The <<graph-repr,``TRACE PROCESSING GRAPH REPRESENTATION''>> section
  21 shows how some <<concepts,concepts>> are visually represented in other
  22 Babeltrace~2 manual pages.
  23
  24
  25 [[what-is]]
  26 == WHAT IS BABELTRACE~2?
  27
  28 Babeltrace~2 is an open-source software project of which the
  29 purpose is to process or convert
  30 https://en.wikipedia.org/wiki/Tracing_(software)[traces].
  31
  32 The Babeltrace~2 project includes the following parts:
  33
  34 [[libbabeltrace2]]Babeltrace~2 library (libbabeltrace2)::
  35     A shared library with a C API.
  36 +
  37 With libbabeltrace2, you can programmatically create <<plugin,plugins>>
  38 and <<comp-cls,component classes>>, build and run <<graph,trace
  39 processing graphs>>, and more (see the <<concepts,``BABELTRACE~2
  40 CONCEPTS''>> section for more details about those concepts).
  41 +
  42 All the other Babeltrace~2 parts rely on this library.
  43
  44 [[babeltrace2]]`babeltrace2` command-line program::
  45     A command-line interface which uses libbabeltrace2 to load plugins,
  46     create a trace processing graph, create <<comp,components>>, connect
  47     their <<port,ports>> correctly, and run the graph.
  48 +
  49 You can also use `babeltrace2` to list the available plugins or to
  50 <<query,query>> an object from a component class.
  51 +
  52 See man:babeltrace2(1).
  53
  54 [[python-bindings]]Babeltrace~2 Python bindings::
  55     A Python~3 package (`bt2`) which offers a Pythonic interface of
  56     libbabeltrace2.
  57 +
  58 You can perform the same operations which are available in
  59 libbabeltrace2 with the Python bindings, but more conveniently and with
  60 less code. However, the Python bindings are less performant than
  61 libbabeltrace2.
  62
  63 Babeltrace~2 project's plugins::
  64     The Babeltrace~2 <<plugin,plugins>> shipped with the project.
  65 +
  66 Those plugins are not special in that they only rely on libbabeltrace2
  67 and you don't need them to use libbabeltrace2, man:babeltrace2(1), or
  68 the Python bindings. However, the project's plugins provide many widely
  69 used trace format encoders/decoders as well as common <<graph,trace
  70 processing graph>> utilities.
  71 +
  72 The Babeltrace~2 project's plugins are:
  73 +
  74 --
  75 `ctf`::
  76   https://diamon.org/ctf/[Common Trace Format] (CTF) input/output,
  77   including the LTTng live source.
  78 +
  79 See man:babeltrace2-plugin-ctf(7).
  80
  81 `lttng-utils`::
  82   Graph utilities specific to https://lttng.org/[LTTng] traces.
  83 +
  84 See man:babeltrace2-plugin-lttng-utils(7).
  85
  86 `text`::
  87   Plain text input/output.
  88 +
  89 See man:babeltrace2-plugin-text(7).
  90
  91 `utils`::
  92   Common graph utilities (muxer, trimmer, counter, dummy sink).
  93 +
  94 See man:babeltrace2-plugin-utils(7).
  95 --
  96
  97
  98 === Changes since Babeltrace~1
  99
 100 This manual page is an introduction to Babeltrace~2, a rewrite of
 101 Babeltrace~1 with a focus on extensibility, flexibility, and
 102 interoperability.
 103
 104 Babeltrace~1 exists since 2010. The major improvements brought by
 105 Babeltrace~2 are:
 106
 107 * Full plugin support: any user can distribute a Babeltrace~2
 108   plugin and, as long as <<libbabeltrace2,libbabeltrace2>> finds it, any
 109   application linked to libbabeltrace2 can load it and use it.
 110 +
 111 Plugins are not just trace format encoders and decoders: they provide
 112 source, filter, and sink <<comp-cls,component classes>> so that you can
 113 connect specialized, reusable components together in a trace processing
 114 graph to create a customized trace conversion or analysis device.
 115
 116 * In order to support user components, many of the objects of
 117   libbabeltrace2 have a reference count. The possible reference cycles
 118   are handled internally so that the library's API is clean and
 119   predictable.
 120 +
 121 Objects which are often used on the "fast path" (for example, events,
 122 fields, and clock snapshots) are unique: they have no reference count.
 123
 124 * All the parts of the Babeltrace~2 project run on the major
 125   operating systems, including Windows and macOS.
 126
 127
 128 [[concepts]]
 129 == BABELTRACE~2 CONCEPTS
 130
 131 This section defines the main concepts of the Babeltrace~2 project.
 132
 133 These concepts translate into types and functions in
 134 <<libbabeltrace2,libbabeltrace2>> and its <<python-bindings,Python
 135 bindings>>, but also as command-line actions and options in the
 136 <<babeltrace2,`babeltrace2` program>>. The other Babeltrace~2
 137 manual pages assume that you are familiar with the following
 138 definitions.
 139
 140 Some Babeltrace~2 concepts are interdependent: it is normal to jump
 141 from one definition to another to understand the big picture.
 142
 143 [[comp-cls]]Component class::
 144     A reusable class which you can instantiate as one or more
 145     <<comp,components>> within a <<graph,trace processing graph>>.
 146 +
 147 There are three types of component classes used to create the three
 148 types of components: source, filter, and sink.
 149 +
 150 A component class implements methods, one of which is an initialization
 151 method, or constructor, to create a component. You pass _initialization
 152 parameters_ to this method to customize the created component. For
 153 example, the initialization method of the compcls:source.ctf.fs
 154 component class accepts a mandatory manparam:source.ctf.fs:inputs
 155 parameter which is an array of file system path(s) to the CTF trace(s).
 156 It also accepts an optional manparam:source.ctf.fs:clock-class-offset-ns
 157 parameter which is an offset, in nanoseconds, to add to all the clock
 158 classes (descriptors of stream clocks) found in the traces's metadata.
 159 +
 160 A component class can have a description and a help text.
 161
 162 [[comp]]Component::
 163     A node within a <<graph,trace processing graph>>.
 164 +
 165 There are three types of components:
 166 +
 167 --
 168 Source component::
 169     An input component which produces <<msg,messages>>.
 170 +
 171 Examples: CTF files input, log file input, LTTng live input, random
 172 event generator.
 173
 174 Filter component::
 175     An intermediate component which can transform the messages it
 176     consumes, augment them, sort them, discard them, or create new ones.
 177 +
 178 Examples: filter which removes messages based on an expression,
 179 filter which adds debugging information to selected events, message
 180 muxer, trace trimmer.
 181
 182 Sink component::
 183     An output component which consumes messages and usually writes them
 184     to one or more formatted files.
 185 +
 186 Examples: log file output, CTF files output, pretty-printed plain text
 187 output.
 188 --
 189 +
 190 Components are connected together within a <<graph,trace processing
 191 graph>> through their <<port,ports>>. Source components have output
 192 ports, sink components have input ports, and filter components have
 193 both.
 194 +
 195 A component is the instance of a <<comp-cls,component class>>. The terms
 196 _component_ and _component class instance_ are equivalent.
 197 +
 198 Within a trace processing graph, each component has a unique name. This
 199 is not the name of its component class, but an instance name. If `human`
 200 is a component class name, than `Nancy` and `John` could be component
 201 names.
 202 +
 203 Once a <<graph,graph>> is configured (the first time it runs), you
 204 cannot add components to it for the remaining graph's lifetime.
 205
 206 [[port]]Port::
 207     A connection point, on a <<comp,component>>, from which are sent or
 208     where are received <<msg,messages>> when the <<graph,trace
 209     processing graph>> runs.
 210 +
 211 An output port is from where messages are sent. An input port is where
 212 messages are received. Source components have output ports, sink
 213 components have input ports, and filter components have both.
 214 +
 215 You can only connect an output port to a single input port.
 216 +
 217 All ports do not need to be connected.
 218 +
 219 A filter or sink component receiving messages from its input ports
 220 is said to _consume_ messages.
 221 +
 222 The link between an output port and input port is a <<conn,connection>>.
 223 +
 224 Once a <<graph,graph>> is configured (the first time it runs), you
 225 cannot connect ports for the remaining graph's lifetime.
 226
 227 [[conn]]Connection::
 228     The link between an output <<port,port>> and an input port through
 229     which <<msg,messages>> flow when a <<graph,trace processing
 230     graph>> runs.
 231
 232 [[msg-iter]]Message iterator::
 233     An iterator on an input <<port,port>> of which the returned elements
 234     are <<msg,messages>>.
 235 +
 236 A <<comp,component>> or another message iterator can create many message
 237 iterators on a single input port, before or while the <<graph,trace
 238 processing graph>> runs.
 239
 240 [[msg]]Message::
 241     The element of a <<msg-iter,message iterator>>.
 242 +
 243 Messages flow from output <<port,ports>> to input ports.
 244 +
 245 A source <<comp,component>> <<msg-iter,message iterator>> produces
 246 messages, while a sink component consumes them. A filter component
 247 message iterator can both consume and produce messages.
 248 +
 249 The main types of messages are:
 250 +
 251 --
 252 Event::
 253     A trace event record within a packet or within a stream.
 254
 255 Packet beginning::
 256     The beginning of a packet within a stream.
 257 +
 258 A packet is a conceptual container of events.
 259
 260 Packet end::
 261     The end of a packet within a stream.
 262
 263 Stream beginning::
 264     The beginning of a stream.
 265 +
 266 A stream is a conceptual container of packets and/or events.
 267 +
 268 Usually, a given source component's output port sends packet and event
 269 messages which belong to a single stream, but it's not required.
 270
 271 Stream end::
 272     The end of a stream.
 273
 274 Discarded events::
 275     A count of discarded events within a given time interval for a given
 276     stream.
 277
 278 Discarded packets::
 279     A count of discarded packets within a given time interval for a
 280     given stream.
 281 --
 282
 283 [[graph]]Trace processing graph::
 284     A https://en.wikipedia.org/wiki/Filter_graph[filter graph] where
 285     nodes are <<comp,components>> and <<msg,messages>> flow from
 286     output <<port,ports>> to input ports.
 287 +
 288 You can build a trace processing graph with
 289 <<libbabeltrace2,libbabeltrace2>>, with the
 290 <<python-bindings,Babeltrace~2 Python bindings>>, or with the
 291 man:babeltrace2-run(1) and man:babeltrace2-convert(1) CLI commands.
 292 +
 293 When a trace processing graph _runs_, the sink components consume
 294 messages from their input ports, making all the graph's
 295 <<msg-iter,message iterators>> work one message at a time to perform the
 296 trace conversion or analysis duty.
 297
 298 [[plugin]]Plugin::
 299     A container, or package, of <<comp-cls,component classes>> as a
 300     shared library or Python module.
 301 +
 302 Each component class within a plugin has a type (source, filter, or
 303 sink) and a name. The type and name pair is unique within a given
 304 plugin.
 305 +
 306 <<libbabeltrace2,libbabeltrace2>> can load a plugin (`.so`, `.dll`, or
 307 `.py` file) at run time: the result is a plugin object in which you can
 308 find a specific component class and instantiate it within a
 309 <<graph,trace processing graph>> as a <<comp,component>>.
 310 +
 311 The <<babeltrace2,`babeltrace2` program>> uses the
 312 'COMP-CLS-TYPE.PLUGIN-NAME.COMP-CLS-NAME' format to identify a specific
 313 component class within a specific plugin. 'COMP-CLS-TYPE' is either
 314 `source` (or `src`), `filter` (or `flt`), or `sink`.
 315 +
 316 You can list the available Babeltrace~2 plugins with the
 317 man:babeltrace2-list-plugins(1) command.
 318
 319 [[query]]Query::
 320     An operation with which you can get a named object from a
 321     <<comp-cls,component class>>, possibly with custom query parameters.
 322 +
 323 The plain text metadata stream of a CTF trace and the available LTTng
 324 live sessions of a given LTTng relay daemon are examples of query
 325 objects.
 326 +
 327 You can use <<libbabeltrace2,libbabeltrace2>>, the
 328 <<python-bindings,Babeltrace~2 Python bindings>>, or the
 329 man:babeltrace2-query(1) CLI command to query a component class's
 330 object.
 331
 332
 333 [[graph-repr]]
 334 == TRACE PROCESSING GRAPH REPRESENTATION
 335
 336 In the Babeltrace~2 manual pages, a component is represented with a
 337 box. The box has the <<comp-cls,component class>> type,
 338 <<plugin,plugin>> name, and component class name at the top. Just below,
 339 between square brackets, is its component name within the <<graph,trace
 340 processing graph>>. Each <<port,port>> is represented with an `@` symbol
 341 on the border(s) of the component box with its name inside the box.
 342 Output ports are on the box's right border while input ports are on the
 343 box's left border.
 344
 345 For example, here's a source component box:
 346
 347 ----
 348 +------------+
 349 | src.ctf.fs |
 350 |  [my-src]  |
 351 |            |
 352 |    stream0 @
 353 |    stream1 @
 354 |    stream2 @
 355 +------------+
 356 ----
 357
 358 This one is an instance of the compcls:source.ctf.fs component class
 359 named `my-src`. It has three output ports named `stream0`, `stream1`,
 360 and `stream2`.
 361
 362 A trace processing graph is represented with multiple component boxes
 363 connected together. The <<conn,connections>> are arrows from output
 364 ports to input ports.
 365
 366 For example, here's a simple conversion graph:
 367
 368 ----
 369 +------------+    +-----------------+    +------------------+
 370 | src.ctf.fs |    | flt.utils.muxer |    | sink.text.pretty |
 371 |    [ctf]   |    |     [muxer]     |    |      [text]      |
 372 |            |    |                 |    |                  |
 373 |    stream0 @--->@ in0         out @--->@ in               |
 374 |    stream1 @--->@ in1             |    +------------------+
 375 |    stream2 @--->@ in2             |
 376 +------------+    @ in3             |
 377                   +-----------------+
 378 ----
 379
 380 Note that input port `in3` of component `muxer` is not connected in this
 381 example.
 382
 383 Sometimes, we symbolically represent other resources which are consumed
 384 from or produced by components. In this case, arrows are used, but they
 385 do not go to or from port symbols (`@`), except for messages. For
 386 example, in the graph above, the `ctf` source component consumes a CTF
 387 trace and the `text` sink component prints plain text to the terminal,
 388 so here's a more complete diagram:
 389
 390 ----
 391 CTF trace
 392   |
 393   |   +------------+    +-----------------+    +------------------+
 394   |   | src.ctf.fs |    | flt.utils.muxer |    | sink.text.pretty |
 395   '-->|    [ctf]   |    |     [muxer]     |    |      [text]      |
 396       |            |    |                 |    |                  |
 397       |    stream0 @--->@ in0         out @--->@ in               |
 398       |    stream1 @--->@ in1             |    +-----+------------+
 399       |    stream2 @--->@ in2             |          |
 400       +------------+    @ in3             |          '--> Terminal
 401                         +-----------------+
 402 ----
 403
 404 Here's another example of a more complex graph which splits a specific
 405 stream using some criteria:
 406
 407 ----
 408 +------------+    +-----------------+    +------------------+
 409 | src.ctf.fs |    | flt.utils.muxer |    | sink.text.pretty |
 410 |  [ctf-in]  |    |     [muxer]     |    |      [text]      |
 411 |            |    |                 |    |                  |
 412 |    stream0 @--->@ in0         out @--->@ in               |
 413 |    stream1 @--->@ in1             |    +------------------+
 414 |    stream2 @-.  @ in2             |
 415 +------------+ |  +-----------------+      +-------------+
 416                |                           | sink.ctf.fs |
 417                |                           |  [ctf-out0] |
 418                |  +-------------------+    |             |
 419                |  | flt.some.splitter | .->@ in          |
 420                |  |     [splitter]    | |  +-------------+
 421                |  |                   | |
 422                '->@ in              A @-'  +-------------+
 423                   |                 B @-.  | sink.ctf.fs |
 424                   +-------------------+ |  |  [ctf-out1] |
 425                                         |  |             |
 426                                         '->@ in          |
 427                                            +-------------+
 428 ----
 429
 430
 431 include::common-footer.txt[]
 432
 433
 434 == SEE ALSO
 435
 436 man:babeltrace2(1)