| 1 | = babeltrace2-intro(7) |
| 2 | :manpagetype: manual page |
| 3 | :revdate: 14 September 2019 |
| 4 | |
| 5 | |
| 6 | == NAME |
| 7 | |
| 8 | babeltrace2-intro - Introduction to Babeltrace 2 |
| 9 | |
| 10 | |
| 11 | == DESCRIPTION |
| 12 | |
| 13 | This manual page is an introduction to the Babeltrace~2 project. |
| 14 | |
| 15 | The <<what-is,``WHAT IS BABELTRACE~2?''>> section describes the |
| 16 | parts of the project and shows the major changes from Babeltrace~1 |
| 17 | to Babeltrace~2 while the <<concepts,``BABELTRACE~2 |
| 18 | CONCEPTS''>> section defines the core concepts of Babeltrace~2. |
| 19 | |
| 20 | The <<graph-repr,``TRACE PROCESSING GRAPH REPRESENTATION''>> section |
| 21 | shows how some <<concepts,concepts>> are visually represented in other |
| 22 | Babeltrace~2 manual pages. |
| 23 | |
| 24 | |
| 25 | [[what-is]] |
| 26 | == WHAT IS BABELTRACE~2? |
| 27 | |
| 28 | Babeltrace~2 is an open-source software project of which the |
| 29 | purpose is to process or convert |
| 30 | https://en.wikipedia.org/wiki/Tracing_(software)[traces]. |
| 31 | |
| 32 | The Babeltrace~2 project includes the following parts: |
| 33 | |
| 34 | [[libbabeltrace2]]Babeltrace~2 library (libbabeltrace2):: |
| 35 | A shared library with a C API. |
| 36 | + |
| 37 | With libbabeltrace2, you can programmatically create <<plugin,plugins>> |
| 38 | and <<comp-cls,component classes>>, build and run <<graph,trace |
| 39 | processing graphs>>, and more (see the <<concepts,``BABELTRACE~2 |
| 40 | CONCEPTS''>> section for more details about those concepts). |
| 41 | + |
| 42 | All the other Babeltrace~2 parts rely on this library. |
| 43 | |
| 44 | [[babeltrace2]]`babeltrace2` command-line program:: |
| 45 | A command-line interface which uses libbabeltrace2 to load plugins, |
| 46 | create a trace processing graph, create <<comp,components>>, connect |
| 47 | their <<port,ports>> correctly, and run the graph. |
| 48 | + |
| 49 | You can also use `babeltrace2` to list the available plugins or to |
| 50 | <<query,query>> an object from a component class. |
| 51 | + |
| 52 | See man:babeltrace2(1). |
| 53 | |
| 54 | [[python-bindings]]Babeltrace~2 Python bindings:: |
| 55 | A Python~3 package (`bt2`) which offers a Pythonic interface of |
| 56 | libbabeltrace2. |
| 57 | + |
| 58 | You can perform the same operations which are available in |
| 59 | libbabeltrace2 with the Python bindings, but more conveniently and with |
| 60 | less code. However, the Python bindings are less performant than |
| 61 | libbabeltrace2. |
| 62 | |
| 63 | Babeltrace~2 project's plugins:: |
| 64 | The Babeltrace~2 <<plugin,plugins>> shipped with the project. |
| 65 | + |
| 66 | Those plugins are not special in that they only rely on libbabeltrace2 |
| 67 | and you don't need them to use libbabeltrace2, man:babeltrace2(1), or |
| 68 | the Python bindings. However, the project's plugins provide many widely |
| 69 | used trace format encoders/decoders as well as common <<graph,trace |
| 70 | processing graph>> utilities. |
| 71 | + |
| 72 | The Babeltrace~2 project's plugins are: |
| 73 | + |
| 74 | -- |
| 75 | `ctf`:: |
| 76 | https://diamon.org/ctf/[Common Trace Format] (CTF) input/output, |
| 77 | including the LTTng live source. |
| 78 | + |
| 79 | See man:babeltrace2-plugin-ctf(7). |
| 80 | |
| 81 | `lttng-utils`:: |
| 82 | Graph utilities specific to https://lttng.org/[LTTng] traces. |
| 83 | + |
| 84 | See man:babeltrace2-plugin-lttng-utils(7). |
| 85 | |
| 86 | `text`:: |
| 87 | Plain text input/output. |
| 88 | + |
| 89 | See man:babeltrace2-plugin-text(7). |
| 90 | |
| 91 | `utils`:: |
| 92 | Common graph utilities (muxer, trimmer, counter, dummy sink). |
| 93 | + |
| 94 | See man:babeltrace2-plugin-utils(7). |
| 95 | -- |
| 96 | |
| 97 | |
| 98 | === Changes since Babeltrace~1 |
| 99 | |
| 100 | This manual page is an introduction to Babeltrace~2, a rewrite of |
| 101 | Babeltrace~1 with a focus on extensibility, flexibility, and |
| 102 | interoperability. |
| 103 | |
| 104 | Babeltrace~1 exists since 2010. The major improvements brought by |
| 105 | Babeltrace~2 are: |
| 106 | |
| 107 | * Full plugin support: any user can distribute a Babeltrace~2 |
| 108 | plugin and, as long as <<libbabeltrace2,libbabeltrace2>> finds it, any |
| 109 | application linked to libbabeltrace2 can load it and use it. |
| 110 | + |
| 111 | Plugins are not just trace format encoders and decoders: they provide |
| 112 | source, filter, and sink <<comp-cls,component classes>> so that you can |
| 113 | connect specialized, reusable components together in a trace processing |
| 114 | graph to create a customized trace conversion or analysis device. |
| 115 | |
| 116 | * In order to support user components, many of the objects of |
| 117 | libbabeltrace2 have a reference count. The possible reference cycles |
| 118 | are handled internally so that the library's API is clean and |
| 119 | predictable. |
| 120 | + |
| 121 | Objects which are often used on the "fast path" (for example, events, |
| 122 | fields, and clock snapshots) are unique: they have no reference count. |
| 123 | |
| 124 | * All the parts of the Babeltrace~2 project run on the major |
| 125 | operating systems, including Windows and macOS. |
| 126 | |
| 127 | |
| 128 | [[concepts]] |
| 129 | == BABELTRACE~2 CONCEPTS |
| 130 | |
| 131 | This section defines the main concepts of the Babeltrace~2 project. |
| 132 | |
| 133 | These concepts translate into types and functions in |
| 134 | <<libbabeltrace2,libbabeltrace2>> and its <<python-bindings,Python |
| 135 | bindings>>, but also as command-line actions and options in the |
| 136 | <<babeltrace2,`babeltrace2` program>>. The other Babeltrace~2 |
| 137 | manual pages assume that you are familiar with the following |
| 138 | definitions. |
| 139 | |
| 140 | Some Babeltrace~2 concepts are interdependent: it is normal to jump |
| 141 | from one definition to another to understand the big picture. |
| 142 | |
| 143 | [[comp-cls]]Component class:: |
| 144 | A reusable class which you can instantiate as one or more |
| 145 | <<comp,components>> within a <<graph,trace processing graph>>. |
| 146 | + |
| 147 | There are three types of component classes used to create the three |
| 148 | types of components: source, filter, and sink. |
| 149 | + |
| 150 | A component class implements methods, one of which is an initialization |
| 151 | method, or constructor, to create a component. You pass _initialization |
| 152 | parameters_ to this method to customize the created component. For |
| 153 | example, the initialization method of the compcls:source.ctf.fs |
| 154 | component class accepts a mandatory manparam:source.ctf.fs:inputs |
| 155 | parameter which is an array of file system path(s) to the CTF trace(s). |
| 156 | It also accepts an optional manparam:source.ctf.fs:clock-class-offset-ns |
| 157 | parameter which is an offset, in nanoseconds, to add to all the clock |
| 158 | classes (descriptors of stream clocks) found in the traces's metadata. |
| 159 | + |
| 160 | A component class can have a description and a help text. |
| 161 | |
| 162 | [[comp]]Component:: |
| 163 | A node within a <<graph,trace processing graph>>. |
| 164 | + |
| 165 | There are three types of components: |
| 166 | + |
| 167 | -- |
| 168 | Source component:: |
| 169 | An input component which produces <<msg,messages>>. |
| 170 | + |
| 171 | Examples: CTF files input, log file input, LTTng live input, random |
| 172 | event generator. |
| 173 | |
| 174 | Filter component:: |
| 175 | An intermediate component which can transform the messages it |
| 176 | consumes, augment them, sort them, discard them, or create new ones. |
| 177 | + |
| 178 | Examples: filter which removes messages based on an expression, |
| 179 | filter which adds debugging information to selected events, message |
| 180 | muxer, trace trimmer. |
| 181 | |
| 182 | Sink component:: |
| 183 | An output component which consumes messages and usually writes them |
| 184 | to one or more formatted files. |
| 185 | + |
| 186 | Examples: log file output, CTF files output, pretty-printed plain text |
| 187 | output. |
| 188 | -- |
| 189 | + |
| 190 | Components are connected together within a <<graph,trace processing |
| 191 | graph>> through their <<port,ports>>. Source components have output |
| 192 | ports, sink components have input ports, and filter components have |
| 193 | both. |
| 194 | + |
| 195 | A component is the instance of a <<comp-cls,component class>>. The terms |
| 196 | _component_ and _component class instance_ are equivalent. |
| 197 | + |
| 198 | Within a trace processing graph, each component has a unique name. This |
| 199 | is not the name of its component class, but an instance name. If `human` |
| 200 | is a component class name, than `Nancy` and `John` could be component |
| 201 | names. |
| 202 | + |
| 203 | Once a <<graph,graph>> is configured (the first time it runs), you |
| 204 | cannot add components to it for the remaining graph's lifetime. |
| 205 | |
| 206 | [[port]]Port:: |
| 207 | A connection point, on a <<comp,component>>, from which are sent or |
| 208 | where are received <<msg,messages>> when the <<graph,trace |
| 209 | processing graph>> runs. |
| 210 | + |
| 211 | An output port is from where messages are sent. An input port is where |
| 212 | messages are received. Source components have output ports, sink |
| 213 | components have input ports, and filter components have both. |
| 214 | + |
| 215 | You can only connect an output port to a single input port. |
| 216 | + |
| 217 | All ports do not need to be connected. |
| 218 | + |
| 219 | A filter or sink component receiving messages from its input ports |
| 220 | is said to _consume_ messages. |
| 221 | + |
| 222 | The link between an output port and input port is a <<conn,connection>>. |
| 223 | + |
| 224 | Once a <<graph,graph>> is configured (the first time it runs), you |
| 225 | cannot connect ports for the remaining graph's lifetime. |
| 226 | |
| 227 | [[conn]]Connection:: |
| 228 | The link between an output <<port,port>> and an input port through |
| 229 | which <<msg,messages>> flow when a <<graph,trace processing |
| 230 | graph>> runs. |
| 231 | |
| 232 | [[msg-iter]]Message iterator:: |
| 233 | An iterator on an input <<port,port>> of which the returned elements |
| 234 | are <<msg,messages>>. |
| 235 | + |
| 236 | A <<comp,component>> or another message iterator can create many message |
| 237 | iterators on a single input port, before or while the <<graph,trace |
| 238 | processing graph>> runs. |
| 239 | |
| 240 | [[msg]]Message:: |
| 241 | The element of a <<msg-iter,message iterator>>. |
| 242 | + |
| 243 | Messages flow from output <<port,ports>> to input ports. |
| 244 | + |
| 245 | A source <<comp,component>> <<msg-iter,message iterator>> produces |
| 246 | messages, while a sink component consumes them. A filter component |
| 247 | message iterator can both consume and produce messages. |
| 248 | + |
| 249 | The main types of messages are: |
| 250 | + |
| 251 | -- |
| 252 | Event:: |
| 253 | A trace event record within a packet or within a stream. |
| 254 | |
| 255 | Packet beginning:: |
| 256 | The beginning of a packet within a stream. |
| 257 | + |
| 258 | A packet is a conceptual container of events. |
| 259 | |
| 260 | Packet end:: |
| 261 | The end of a packet within a stream. |
| 262 | |
| 263 | Stream beginning:: |
| 264 | The beginning of a stream. |
| 265 | + |
| 266 | A stream is a conceptual container of packets and/or events. |
| 267 | + |
| 268 | Usually, a given source component's output port sends packet and event |
| 269 | messages which belong to a single stream, but it's not required. |
| 270 | |
| 271 | Stream end:: |
| 272 | The end of a stream. |
| 273 | |
| 274 | Discarded events:: |
| 275 | A count of discarded events within a given time interval for a given |
| 276 | stream. |
| 277 | |
| 278 | Discarded packets:: |
| 279 | A count of discarded packets within a given time interval for a |
| 280 | given stream. |
| 281 | -- |
| 282 | |
| 283 | [[graph]]Trace processing graph:: |
| 284 | A https://en.wikipedia.org/wiki/Filter_graph[filter graph] where |
| 285 | nodes are <<comp,components>> and <<msg,messages>> flow from |
| 286 | output <<port,ports>> to input ports. |
| 287 | + |
| 288 | You can build a trace processing graph with |
| 289 | <<libbabeltrace2,libbabeltrace2>>, with the |
| 290 | <<python-bindings,Babeltrace~2 Python bindings>>, or with the |
| 291 | man:babeltrace2-run(1) and man:babeltrace2-convert(1) CLI commands. |
| 292 | + |
| 293 | When a trace processing graph _runs_, the sink components consume |
| 294 | messages from their input ports, making all the graph's |
| 295 | <<msg-iter,message iterators>> work one message at a time to perform the |
| 296 | trace conversion or analysis duty. |
| 297 | |
| 298 | [[plugin]]Plugin:: |
| 299 | A container, or package, of <<comp-cls,component classes>> as a |
| 300 | shared library or Python module. |
| 301 | + |
| 302 | Each component class within a plugin has a type (source, filter, or |
| 303 | sink) and a name. The type and name pair is unique within a given |
| 304 | plugin. |
| 305 | + |
| 306 | <<libbabeltrace2,libbabeltrace2>> can load a plugin (`.so`, `.dll`, or |
| 307 | `.py` file) at run time: the result is a plugin object in which you can |
| 308 | find a specific component class and instantiate it within a |
| 309 | <<graph,trace processing graph>> as a <<comp,component>>. |
| 310 | + |
| 311 | The <<babeltrace2,`babeltrace2` program>> uses the |
| 312 | 'COMP-CLS-TYPE.PLUGIN-NAME.COMP-CLS-NAME' format to identify a specific |
| 313 | component class within a specific plugin. 'COMP-CLS-TYPE' is either |
| 314 | `source` (or `src`), `filter` (or `flt`), or `sink`. |
| 315 | + |
| 316 | You can list the available Babeltrace~2 plugins with the |
| 317 | man:babeltrace2-list-plugins(1) command. |
| 318 | |
| 319 | [[query]]Query:: |
| 320 | An operation with which you can get a named object from a |
| 321 | <<comp-cls,component class>>, possibly with custom query parameters. |
| 322 | + |
| 323 | The plain text metadata stream of a CTF trace and the available LTTng |
| 324 | live sessions of a given LTTng relay daemon are examples of query |
| 325 | objects. |
| 326 | + |
| 327 | You can use <<libbabeltrace2,libbabeltrace2>>, the |
| 328 | <<python-bindings,Babeltrace~2 Python bindings>>, or the |
| 329 | man:babeltrace2-query(1) CLI command to query a component class's |
| 330 | object. |
| 331 | |
| 332 | |
| 333 | [[graph-repr]] |
| 334 | == TRACE PROCESSING GRAPH REPRESENTATION |
| 335 | |
| 336 | In the Babeltrace~2 manual pages, a component is represented with a |
| 337 | box. The box has the <<comp-cls,component class>> type, |
| 338 | <<plugin,plugin>> name, and component class name at the top. Just below, |
| 339 | between square brackets, is its component name within the <<graph,trace |
| 340 | processing graph>>. Each <<port,port>> is represented with an `@` symbol |
| 341 | on the border(s) of the component box with its name inside the box. |
| 342 | Output ports are on the box's right border while input ports are on the |
| 343 | box's left border. |
| 344 | |
| 345 | For example, here's a source component box: |
| 346 | |
| 347 | ---- |
| 348 | +------------+ |
| 349 | | src.ctf.fs | |
| 350 | | [my-src] | |
| 351 | | | |
| 352 | | stream0 @ |
| 353 | | stream1 @ |
| 354 | | stream2 @ |
| 355 | +------------+ |
| 356 | ---- |
| 357 | |
| 358 | This one is an instance of the compcls:source.ctf.fs component class |
| 359 | named `my-src`. It has three output ports named `stream0`, `stream1`, |
| 360 | and `stream2`. |
| 361 | |
| 362 | A trace processing graph is represented with multiple component boxes |
| 363 | connected together. The <<conn,connections>> are arrows from output |
| 364 | ports to input ports. |
| 365 | |
| 366 | For example, here's a simple conversion graph: |
| 367 | |
| 368 | ---- |
| 369 | +------------+ +-----------------+ +------------------+ |
| 370 | | src.ctf.fs | | flt.utils.muxer | | sink.text.pretty | |
| 371 | | [ctf] | | [muxer] | | [text] | |
| 372 | | | | | | | |
| 373 | | stream0 @--->@ in0 out @--->@ in | |
| 374 | | stream1 @--->@ in1 | +------------------+ |
| 375 | | stream2 @--->@ in2 | |
| 376 | +------------+ @ in3 | |
| 377 | +-----------------+ |
| 378 | ---- |
| 379 | |
| 380 | Note that input port `in3` of component `muxer` is not connected in this |
| 381 | example. |
| 382 | |
| 383 | Sometimes, we symbolically represent other resources which are consumed |
| 384 | from or produced by components. In this case, arrows are used, but they |
| 385 | do not go to or from port symbols (`@`), except for messages. For |
| 386 | example, in the graph above, the `ctf` source component consumes a CTF |
| 387 | trace and the `text` sink component prints plain text to the terminal, |
| 388 | so here's a more complete diagram: |
| 389 | |
| 390 | ---- |
| 391 | CTF trace |
| 392 | | |
| 393 | | +------------+ +-----------------+ +------------------+ |
| 394 | | | src.ctf.fs | | flt.utils.muxer | | sink.text.pretty | |
| 395 | '-->| [ctf] | | [muxer] | | [text] | |
| 396 | | | | | | | |
| 397 | | stream0 @--->@ in0 out @--->@ in | |
| 398 | | stream1 @--->@ in1 | +-----+------------+ |
| 399 | | stream2 @--->@ in2 | | |
| 400 | +------------+ @ in3 | '--> Terminal |
| 401 | +-----------------+ |
| 402 | ---- |
| 403 | |
| 404 | Here's another example of a more complex graph which splits a specific |
| 405 | stream using some criteria: |
| 406 | |
| 407 | ---- |
| 408 | +------------+ +-----------------+ +------------------+ |
| 409 | | src.ctf.fs | | flt.utils.muxer | | sink.text.pretty | |
| 410 | | [ctf-in] | | [muxer] | | [text] | |
| 411 | | | | | | | |
| 412 | | stream0 @--->@ in0 out @--->@ in | |
| 413 | | stream1 @--->@ in1 | +------------------+ |
| 414 | | stream2 @-. @ in2 | |
| 415 | +------------+ | +-----------------+ +-------------+ |
| 416 | | | sink.ctf.fs | |
| 417 | | | [ctf-out0] | |
| 418 | | +-------------------+ | | |
| 419 | | | flt.some.splitter | .->@ in | |
| 420 | | | [splitter] | | +-------------+ |
| 421 | | | | | |
| 422 | '->@ in A @-' +-------------+ |
| 423 | | B @-. | sink.ctf.fs | |
| 424 | +-------------------+ | | [ctf-out1] | |
| 425 | | | | |
| 426 | '->@ in | |
| 427 | +-------------+ |
| 428 | ---- |
| 429 | |
| 430 | |
| 431 | include::common-footer.txt[] |
| 432 | |
| 433 | |
| 434 | == SEE ALSO |
| 435 | |
| 436 | man:babeltrace2(1) |