X-Git-Url: http://git.efficios.com/?a=blobdiff_plain;f=doc%2Fbindings%2Fpython%2Fsource%2Fexamples.rst;h=21e5f438ab1b5f7674bbf8c22939f04f78cbb3ef;hb=ba64dfcccb1f1bd7a259dc5d563ba422b8375582;hp=317574762dd1160077d4157c54e414b1edd7e45d;hpb=4e5f388fcb9d27babc2e2a5bff5c904127a040cf;p=babeltrace.git diff --git a/doc/bindings/python/source/examples.rst b/doc/bindings/python/source/examples.rst index 31757476..21e5f438 100644 --- a/doc/bindings/python/source/examples.rst +++ b/doc/bindings/python/source/examples.rst @@ -1,703 +1,910 @@ +.. include:: common.rst + .. _examples: -******** Examples -******** - -This section presents a few short and straightforward examples -of Babeltrace legacy Python bindings usage. - -The examples are divided into two categories: those which demonstrate -the :ref:`reader API `, and those which demonstrate -the :ref:`CTF writer API `. - - -.. _reader-api-examples: - -Reader API examples -=================== - -The :ref:`reader API ` includes everything needed to open -traces and iterate on events in order. - - -Open one trace and print all event names ----------------------------------------- - -This example shows how to open a single CTF trace, iterate on all the -events, and print their names. - -.. code-block:: python - - import babeltrace.reader - import sys - - - # get the trace path from the first command line argument - trace_path = sys.argv[1] - - trace_collection = babeltrace.reader.TraceCollection() - - trace_collection.add_trace(trace_path, 'ctf') - - for event in trace_collection.events: - print(event.name) - - -Open multiple traces and print all event field names ----------------------------------------------------- - -This example opens multiple CTF traces (their paths are provided as -command line arguments), iterates on all their correlated events in -order, and prints a list of their field names. - -.. code-block:: python - - import babeltrace.reader - import sys - +======== +This section contains a few short and straightforward examples which +show how to use the Babeltrace |~| 2 Python bindings. - trace_collection = babeltrace.reader.TraceCollection() +The :mod:`bt2` package provides the Babeltrace |~| 2 Python bindings. +Note that the :mod:`babeltrace` package is part of the Babeltrace |~| 1 +project: it's somewhat out-of-date and not compatible with the +:mod:`bt2` package. - for path in sys.argv[1:]: - trace_collection.add_trace(path, 'ctf') +Assume that all the examples below are named :file:`example.py`. - for event in trace_collection.events: - print(', '.join(event.keys())) +.. _examples_tcmi: +Iterate trace events +-------------------- +The most convenient and high-level way to iterate the events of one or +more traces is with a :class:`bt2.TraceCollectionMessageIterator` +object. -Print a specific event field ----------------------------- +A :class:`bt2.TraceCollectionMessageIterator` object roughly offers the +same features as the ``convert`` command of the :command:`babeltrace2` +command-line program (see the :bt2man:`babeltrace2-convert(1)` manual +page), but in a programmatic, Pythonic way. -Reading the field value of an :class:`babeltrace.reader.Event` object -is done by using its :class:`dict`-like interface: +As of Babeltrace |~| |version|, the trace collection message iterator +class is a Python bindings-only feature: the Python code uses +libbabeltrace2 internally, but the latter does not offer this utility as +such. -.. code-block:: python +The :class:`bt2.TraceCollectionMessageIterator` interface features: - field_value = event['field_name'] +* **Automatic source component (trace format) discovery**. -As such, you can use Python's ``in`` keyword to verify if a given -event contains a given field name: + ``convert`` command equivalent example: -.. code-block:: python + .. code-block:: text - if 'field_name' in event: - # ... + $ babeltrace2 /path/to/my/trace -The following example iterates on the events of a trace, and prints the -value of the ``fd`` field if it's available. +* **Explicit component class instantiation**. -.. code-block:: python + ``convert`` command equivalent example: - import babeltrace.reader - import sys + .. code-block:: text - # get the trace path from the first command line argument - trace_path = sys.argv[1] + $ babeltrace2 --component=source.my.format - trace_collection = babeltrace.reader.TraceCollection() +* **Passing initialization parameters to both auto-discovered and + explicitly created components**. - trace_collection.add_trace(trace_path, 'ctf') + ``convert`` command equivalent example: - for event in trace_collection.events: - if 'fd' in event: - print(event['fd']) + .. code-block:: text -Beware that different fields of the same event may share the same name -if they are in different scopes. In this case, the :class:`dict`-like -interface prioritizes event payload fields before event context fields, -event context fields before stream event context fields, and so on -(see :class:`babeltrace.reader.Event` for this exact list of -priorities). It is possible to get the value of an event's field -within a specific scope using -:meth:`babeltrace.reader.Event.field_with_scope`: + $ babeltrace2 /path/to/my/trace --params=detailed=no \ + --component=source.ctf.fs \ + --params='inputs=["/path/to/my/trace"]' -.. code-block:: python +* **Trace event muxing**. - import babeltrace.reader - import babeltrace.common + The message iterator muxes (combines) the events from multiple + compatible streams into a single, time-sorted sequence of events. - # ... + .. code-block:: text - field_value = event.field_with_scope('field_name', - babeltrace.common.CTFScope.EVENT_CONTEXT) + $ babeltrace2 /path/to/trace1 /path/to/trace2 /path/to/trace3 +* **Stream intersection mode**. -Bonus: top 5 running processes using LTTng ------------------------------------------- + ``convert`` command equivalent example: -Since `LTTng `_ produces CTF traces, the -Babeltrace Python binding can read LTTng traces. + .. code-block:: text -This somewhat more complex example reads a whole LTTng Linux kernel -trace, and outputs the short names of the top 5 running processes on -CPU 0 during the whole trace. + $ babeltrace2 /path/to/my/trace --stream-intersection -.. code-block:: python +* **Stream trimming with beginning and/or end times**. - from collections import Counter - import babeltrace.reader - import sys + ``convert`` command equivalent example: + .. code-block:: text - # a trace collection holds one or more traces - col = babeltrace.reader.TraceCollection() + $ babeltrace2 /path/to/my/trace --begin=22:14:38 --end=22:15:07 - # add the trace provided by the user (first command line argument) - # (LTTng traces always have the 'ctf' format) - if col.add_trace(sys.argv[1], 'ctf') is None: - raise RuntimeError('Cannot add trace') +While the :command:`babeltrace2 convert` command creates a ``sink.text.pretty`` +component class (by default) to pretty-print events as plain text lines, +a :class:`bt2.TraceCollectionMessageIterator` object is a Python +iterator which makes its user a message consumer (there's no sink +component):: - # this counter dict will hold execution times: - # - # task command name -> total execution time (ns) - exec_times = Counter() + import bt2 - # this holds the last `sched_switch` timestamp - last_ts = None + for msg in bt2.TraceCollectionMessageIterator('/path/to/trace'): + if type(msg) is bt2._EventMessageConst: + print(msg.event.name) - # iterate on events - for event in col.events: - # keep only `sched_switch` events - if event.name != 'sched_switch': - continue +.. _examples_tcmi_autodisc: - # keep only events which happened on CPU 0 - if event['cpu_id'] != 0: - continue +Discover traces +~~~~~~~~~~~~~~~ +Pass one or more file paths, directory paths, or other strings when you +build a :class:`bt2.TraceCollectionMessageIterator` object to let it +automatically determine which source components to create for you. - # event timestamp - cur_ts = event.timestamp +If you pass a directory path, the message iterator traverses the +directory recursively to find traces, automatically selecting the +appropriate source component classes to instantiate. - if last_ts is None: - # we start here - last_ts = cur_ts +The :class:`bt2.TraceCollectionMessageIterator` object and the +:command:`babeltrace2 convert` CLI command share the same automatic +component discovery algorithm. See the +:bt2link:`Create implicit components from non-option arguments ` +section of the :bt2man:`babeltrace2-convert(1)` manual page for more +details. - # previous task command (short) name - prev_comm = event['prev_comm'] +The following example shows how to use a +:class:`bt2.TraceCollectionMessageIterator` object to automatically +discover one or more traces from a single path (file or directory). For +each trace event, the example prints its name:: - # initialize entry in our dict if not yet done - if prev_comm not in exec_times: - exec_times[prev_comm] = 0 + import bt2 + import sys - # compute previous command execution time - diff = cur_ts - last_ts + # Get the trace path from the first command-line argument. + path = sys.argv[1] - # update execution time of this command - exec_times[prev_comm] += diff + # Create a trace collection message iterator with this path. + msg_it = bt2.TraceCollectionMessageIterator(path) - # update last timestamp - last_ts = cur_ts + # Iterate the trace messages. + for msg in msg_it: + # `bt2._EventMessageConst` is the Python type of an event message. + if type(msg) is bt2._EventMessageConst: + # An event message holds a trace event. + event = msg.event - # print top 5 - for name, ns in exec_times.most_common(5): - s = ns / 1000000000 - print('{:20}{} s'.format(name, s)) + # Print event's name. + print(event.name) +Run this example: -Inspect event declarations and their field declarations -------------------------------------------------------- +.. code-block:: text -When :meth:`babeltrace.reader.TraceCollection.add_trace` is called -and a trace is successfully opened and added, a corresponding -:class:`babeltrace.reader.TraceHandle` object for this trace is -returned. It is then possible to iterate on the event declarations of -this trace handle using :attr:`babeltrace.reader.TraceHandle.events`. -Each generated :class:`babeltrace.reader.EventDeclaration` object -contains common properties for this type of event, including its -field declarations. This is useful for inspecting the available -events of a trace, and their "signature" in terms of fields, before -iterating its actual events. + $ python3 example.py /path/to/one/or/more/traces -This example adds a trace to a trace collection, and uses the returned -trace handle to iterate on its event declarations. The goal here is to -make sure the ``sched_switch`` event exists, and that it contains -at least the following fields: +Output example: -* ``prev_comm``, which should be an array of 8-bit integers -* ``prev_tid``, which should be an integer +.. code-block:: text -.. code-block:: python + kmem_kmalloc + kmem_kfree + kmem_cache_alloc_node + block_getrq + kmem_kmalloc + block_plug + kmem_kfree + block_rq_insert + kmem_kmalloc + kmem_kfree + kmem_kmalloc + kmem_kfree - import babeltrace.reader as btr - import sys +The example above is simplistic; it does not catch the exceptions that +some statements can raise: +* ``bt2.TraceCollectionMessageIterator(path)`` raises an exception if + it cannot find any trace. - def validate_sched_switch_fields(event_decl): - found_prev_comm = False - found_prev_tid = False +* Each iteration of the loop, or, more precisely, the + :meth:`bt2.TraceCollectionMessageIterator.__next__` method, raises + an exception if there's any error during the iteration process. - for field_decl in event_decl.fields: - if field_decl.name == 'prev_comm': - if isinstance(field_decl, btr.ArrayFieldDeclaration): - elem_decl = field_decl.element_declaration + For example, an internal source component message iterator can fail + when trying to decode a malformed trace file. + +.. _examples_tcmi_expl: + +Create explicit source components +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +If `automatic source component discovery <#examples-tcmi-autodisc>`_ +doesn't work for you (for example, because the source component class +you actually need to instantiate doesn't offer the +``babeltrace.support-info`` query object), create explicit source +components when you build a :class:`bt2.TraceCollectionMessageIterator` +object. + +The following example builds a trace collection message iterator to +explicitly instantiate a ``source.ctf.fs`` component class (found in the +``ctf`` plugin). Again, for each trace event, the example prints its +name:: + + import bt2 + import sys + + # Find the `ctf` plugin (shipped with Babeltrace 2). + ctf_plugin = bt2.find_plugin('ctf') + + # Get the `source.ctf.fs` component class from the plugin. + fs_cc = ctf_plugin.source_component_classes['fs'] + + # Create a trace collection message iterator, instantiating a single + # `source.ctf.fs` component class with the `inputs` initialization + # parameter set to open a single CTF trace. + msg_it = bt2.TraceCollectionMessageIterator(bt2.ComponentSpec(fs_cc, { + # Get the CTF trace path from the first command-line argument. + 'inputs': [sys.argv[1]], + })) + + # Iterate the trace messages. + for msg in msg_it: + # `bt2._EventMessageConst` is the Python type of an event message. + if type(msg) is bt2._EventMessageConst: + # Print event's name. + print(msg.event.name) - if isinstance(elem_decl, btr.IntegerFieldDeclaration): - if elem_decl.size == 8: - found_prev_comm = True - elif field_decl.name == 'prev_tid': - if isinstance(field_decl, btr.IntegerFieldDeclaration): - found_prev_tid = True +Run this example: + +.. code-block:: text - return found_prev_comm and found_prev_tid + $ python3 example.py /path/to/ctf/trace + +Output example: + +.. code-block:: text + + kmem_kmalloc + kmem_kfree + kmem_cache_alloc_node + block_getrq + kmem_kmalloc + block_plug + kmem_kfree + block_rq_insert + kmem_kmalloc + kmem_kfree + kmem_kmalloc + kmem_kfree + +The example above looks similar to the previous one using +`automatic source component discovery <#examples-tcmi-autodisc>`_, +but there are notable differences: + +* A ``source.ctf.fs`` component expects to receive the path to a + *single* `CTF `_ trace (a directory + containing a file named ``metadata``). + + Unlike the previous example, you must pass the exact + :abbr:`CTF (Common Trace Format)` trace directory path, *not* a + parent directory path. + +* Unlike the previous example, the example above can only read a single + trace. + + If you want to read multiple :abbr:`CTF (Common Trace Format)` traces + using explicit component class instantiation with a single trace + collection message iterator, you must create one ``source.ctf.fs`` + component per trace. +Note that the :class:`bt2.ComponentSpec` class offers the +:meth:`from_named_plugin_and_component_class` convenience static method +which finds the plugin and component class for you. You could therefore +rewrite the trace collection message iterator creation part of the +example above as:: - # get the trace path from the first command line argument - trace_path = sys.argv[1] + # Create a trace collection message iterator, instantiating a single + # `source.ctf.fs` component class with the `inputs` initialization + # parameter set to open a single CTF trace. + msg_it = bt2.TraceCollectionMessageIterator( + bt2.ComponentSpec.from_named_plugin_and_component_class('ctf', 'fs', { + # Get the CTF trace path from the first command-line argument. + 'inputs': [sys.argv[1]], + }) + ) + +.. _examples_tcmi_ev_field: + +Get a specific event field's value +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The :ref:`examples_tcmi_autodisc` and :ref:`examples_tcmi_expl` examples +show that a :class:`bt2.TraceCollectionMessageIterator` iterates the +time-sorted *messages* of one or more traces. - trace_collection = btr.TraceCollection() - trace_handle = trace_collection.add_trace(trace_path, 'ctf') - sched_switch_found = False +One specific type of message is :class:`bt2._EventMessageConst`, which +holds a trace event object. - for event_decl in trace_handle.events: - if event_decl.name == 'sched_switch': - if validate_sched_switch_fields(event_decl): - sched_switch_found = True - break +.. note:: - print('trace path: {}'.format(trace_handle.path)) + Everything you can find in the :mod:`bt2` package is publicly + accessible. - if sched_switch_found: - print('found sched_switch!') - else: - print('could not find sched_switch') + Names which start with ``_`` (underscore), like + :class:`bt2._EventMessageConst`, indicate that you can't + *instantiate* such a class (you cannot call the class). However, the + type itself remains public so that you can use its name to check an + object's type: + .. code-block:: python -.. _ctf-writer-api-examples: + if type(msg) is bt2._EventMessageConst: + # ... -CTF writer API examples -======================= + .. code-block:: python -The :ref:`CTF writer API ` is a set of classes which -allows a Python script to write complete -`CTF `_ (Common Trace Format) traces. + if isinstance(field, bt2._IntegerFieldConst): + # ... +Access an event object's field by using the event as a simple mapping +(like a read-only :class:`dict`), where keys are field names. The field +can belong to any part of the event (contexts or payload) and to its +packet's context, if any:: -One trace, one stream, one event, one field -------------------------------------------- + import bt2 + import sys -This is the most simple example of using the CTF writer API. It creates -one writer (responsible for writing one trace), then uses it to create -one stream. One event with a single field is appended to this single -stream, and everything is flushed. + # Create a trace collection message iterator from the first + # command-line argument. + msg_it = bt2.TraceCollectionMessageIterator(sys.argv[1]) -The trace is written in a temporary directory (its path is printed -at the beginning of the script). + # Iterate the trace messages. + for msg in msg_it: + # `bt2._EventMessageConst` is the Python type of an event message. + # Only keep such messages. + if type(msg) is not bt2._EventMessageConst: + continue -.. code-block:: python + # An event message holds a trace event. + event = msg.event - import babeltrace.writer as btw - import tempfile + # Only check `sched_switch` events. + if event.name != 'sched_switch': + continue + # In an LTTng trace, the `cpu_id` field is a packet context field. + # The mapping interface of `event` can still find it. + cpu_id = event['cpu_id'] - # temporary directory holding the CTF trace - trace_path = tempfile.mkdtemp() + # Previous and next process short names are found in the event's + # `prev_comm` and `next_comm` fields. + prev_comm = event['prev_comm'] + next_comm = event['next_comm'] - print('trace path: {}'.format(trace_path)) + # Print line, using field values. + msg = 'CPU {}: Switching process `{}` → `{}`' + print(msg.format(cpu_id, prev_comm, next_comm)) - # our writer - writer = btw.Writer(trace_path) +The example above assumes that the traces to open are +`LTTng `_ Linux kernel traces. - # create one default clock and register it to the writer - clock = btw.Clock('my_clock') - clock.description = 'this is my clock' - writer.add_clock(clock) +Run this example: - # create one default stream class and assign our clock to it - stream_class = btw.StreamClass('my_stream') - stream_class.clock = clock +.. code-block:: text - # create one default event class - event_class = btw.EventClass('my_event') + $ python3 example.py /path/to/one/or/more/lttng/traces - # create one 32-bit signed integer field - int32_field_decl = btw.IntegerFieldDeclaration(32) - int32_field_decl.signed = True +Output example: - # add this field declaration to our event class - event_class.add_field(int32_field_decl, 'my_field') +.. code-block:: text - # register our event class to our stream class - stream_class.add_event_class(event_class) + CPU 2: Switching process `Timer` → `swapper/2` + CPU 0: Switching process `swapper/0` → `firefox` + CPU 0: Switching process `firefox` → `swapper/0` + CPU 0: Switching process `swapper/0` → `rcu_preempt` + CPU 0: Switching process `rcu_preempt` → `swapper/0` + CPU 3: Switching process `swapper/3` → `alsa-sink-ALC26` + CPU 2: Switching process `swapper/2` → `Timer` + CPU 2: Switching process `Timer` → `swapper/2` + CPU 2: Switching process `swapper/2` → `pulseaudio` + CPU 0: Switching process `swapper/0` → `firefox` + CPU 1: Switching process `swapper/1` → `threaded-ml` + CPU 2: Switching process `pulseaudio` → `Timer` - # create our single event, based on our event class - event = btw.Event(event_class) +If you need to access a specific field, use: - # assign an integer value to our single field - event.payload('my_field').value = -23 +Event payload + :attr:`bt2._EventConst.payload_field` property. - # create our single stream - stream = writer.create_stream(stream_class) +Event specific context + :attr:`bt2._EventConst.specific_context_field` property. - # append our single event to our single stream - stream.append_event(event) +Event common context + :attr:`bt2._EventConst.common_context_field` property. - # flush the stream - stream.flush() +Packet context + :attr:`bt2._PacketConst.context_field` property. +Use Python's ``in`` operator to verify if: -Basic CTF fields ----------------- +A specific "root" field (in the list above) contains a given field by name + .. code-block:: python -This example writes a few events with basic CTF fields: integers, -floating point numbers, enumerations and strings. + if 'next_comm' in event.payload_field: + # ... -The trace is written in a temporary directory (its path is printed -at the beginning of the script). +Any of the root fields contains a given field by name + .. code-block:: python -.. code-block:: python - - import babeltrace.writer as btw - import babeltrace.common - import tempfile - import math + if 'next_comm' in event: + # ... +The following example iterates the events of a given trace, printing the +value of the ``fd`` payload field if it's available:: - trace_path = tempfile.mkdtemp() + import bt2 + import sys - print('trace path: {}'.format(trace_path)) + # Create a trace collection message iterator from the first command-line + # argument. + msg_it = bt2.TraceCollectionMessageIterator(sys.argv[1]) + # Iterate the trace messages. + for msg in msg_it: + # `bt2._EventMessageConst` is the Python type of an event message. + if type(msg) is bt2._EventMessageConst: + # Check if the `fd` event payload field exists. + if 'fd' in msg.event.payload_field: + # Print the `fd` event payload field's value. + print(msg.event.payload_field['fd']) - writer = btw.Writer(trace_path) +Output example: - clock = btw.Clock('my_clock') - clock.description = 'this is my clock' - writer.add_clock(clock) +.. code-block:: text - stream_class = btw.StreamClass('my_stream') - stream_class.clock = clock + 14 + 15 + 16 + 19 + 30 + 31 + 33 + 42 + 0 + 1 + 2 + 3 - event_class = btw.EventClass('my_event') +.. _examples_tcmi_ev_time: - # 32-bit signed integer field declaration - int32_field_decl = btw.IntegerFieldDeclaration(32) - int32_field_decl.signed = True - int32_field_decl.base = btw.IntegerBase.HEX +Get an event's time +~~~~~~~~~~~~~~~~~~~ +The time, or timestamp, of an event object belongs to its message as +a *default clock snapshot*. - # 5-bit unsigned integer field declaration - uint5_field_decl = btw.IntegerFieldDeclaration(5) - uint5_field_decl.signed = False +An event's clock snapshot is a *snapshot* (an immutable value) of the +value of the event's stream's clock when the event occurred. As of +Babeltrace |~| |version|, a stream can only have one clock: its default +clock. - # IEEE 754 single precision floating point number field declaration - float_field_decl = btw.FloatingPointFieldDeclaration() - float_field_decl.exponent_digits = btw.FloatingPointFieldDeclaration.FLT_EXP_DIG - float_field_decl.mantissa_digits = btw.FloatingPointFieldDeclaration.FLT_MANT_DIG +Use the :attr:`default_clock_snapshot` property of an event message +to get its default clock snapshot. A clock snapshot object offers, +amongst other things, the following properties: - # enumeration field declaration (based on the 5-bit unsigned integer above) - enum_field_decl = btw.EnumerationFieldDeclaration(uint5_field_decl) - enum_field_decl.add_mapping('DAZED', 3, 11) - enum_field_decl.add_mapping('AND', 13, 13) - enum_field_decl.add_mapping('CONFUSED', 17, 30) +:attr:`value` (:class:`int`) + Value of the clock snapshot in clock cycles. - # string field declaration - string_field_decl = btw.StringFieldDeclaration() - string_field_decl.encoding = babeltrace.common.CTFStringEncoding.UTF8 + A stream clock can have any frequency (Hz). - event_class.add_field(int32_field_decl, 'my_int32_field') - event_class.add_field(uint5_field_decl, 'my_uint5_field') - event_class.add_field(float_field_decl, 'my_float_field') - event_class.add_field(enum_field_decl, 'my_enum_field') - event_class.add_field(int32_field_decl, 'another_int32_field') - event_class.add_field(string_field_decl, 'my_string_field') +:attr:`ns_from_origin` (:class:`int`) + Number of nanoseconds from the stream clock's origin (often the Unix + epoch). - stream_class.add_event_class(event_class) +The following example prints, for each event, its name, its date/time, +and the difference, in seconds, since the previous event's time (if +any):: - stream = writer.create_stream(stream_class) + import bt2 + import sys + import datetime - # create and append first event - event = btw.Event(event_class) - event.payload('my_int32_field').value = 0xbeef - event.payload('my_uint5_field').value = 17 - event.payload('my_float_field').value = -math.pi - event.payload('my_enum_field').value = 8 # label: 'DAZED' - event.payload('another_int32_field').value = 0x20141210 - event.payload('my_string_field').value = 'Hello, World!' - stream.append_event(event) + # Create a trace collection message iterator from the first command-line + # argument. + msg_it = bt2.TraceCollectionMessageIterator(sys.argv[1]) - # create and append second event - event = btw.Event(event_class) - event.payload('my_int32_field').value = 0x12345678 - event.payload('my_uint5_field').value = 31 - event.payload('my_float_field').value = math.e - event.payload('my_enum_field').value = 28 # label: 'CONFUSED' - event.payload('another_int32_field').value = -1 - event.payload('my_string_field').value = trace_path - stream.append_event(event) + # Last event's time (ns from origin). + last_event_ns_from_origin = None - stream.flush() + # Iterate the trace messages. + for msg in msg_it: + # `bt2._EventMessageConst` is the Python type of an event message. + if type(msg) is bt2._EventMessageConst: + # Get event message's default clock snapshot's ns from origin + # value. + ns_from_origin = msg.default_clock_snapshot.ns_from_origin + # Compute the time difference since the last event message. + diff_s = 0 -Static array and sequence fields --------------------------------- + if last_event_ns_from_origin is not None: + diff_s = (ns_from_origin - last_event_ns_from_origin) / 1e9 -This example demonstrates how to write static array and sequence -fields. A static array has a fixed length, whereas a sequence reads -its length dynamically from another (integer) field. + # Create a `datetime.datetime` object from `ns_from_origin` for + # presentation. Note that such an object is less accurate than + # `ns_from_origin` as it holds microseconds, not nanoseconds. + dt = datetime.datetime.fromtimestamp(ns_from_origin / 1e9) -In this example, an event is appended to a single stream, in which -three fields are present: + # Print line. + fmt = '{} (+{:.6f} s): {}' + print(fmt.format(dt, diff_s, msg.event.name)) -* ``seqlen``, the dynamic length of the sequence ``seq`` (set to the - number of command line arguments) -* ``array``, a static array of 23 16-bit unsigned integers -* ``seq``, a sequence of ``seqlen`` strings, where the strings are - the command line arguments + # Update last event's time. + last_event_ns_from_origin = ns_from_origin + +Run this example: + +.. code-block:: text + + $ python3 example.py /path/to/one/or/more/traces + +Output example: + +.. code-block:: text + + 2015-09-09 22:40:41.551451 (+0.000004 s): lttng_ust_statedump:end + 2015-09-09 22:40:43.003397 (+1.451946 s): lttng_ust_dl:dlopen + 2015-09-09 22:40:43.003412 (+0.000015 s): lttng_ust_dl:build_id + 2015-09-09 22:40:43.003861 (+0.000449 s): lttng_ust_dl:dlopen + 2015-09-09 22:40:43.003865 (+0.000004 s): lttng_ust_dl:build_id + 2015-09-09 22:40:43.003879 (+0.000014 s): my_provider:my_first_tracepoint + 2015-09-09 22:40:43.003895 (+0.000016 s): my_provider:my_first_tracepoint + 2015-09-09 22:40:43.003898 (+0.000003 s): my_provider:my_other_tracepoint + 2015-09-09 22:40:43.003922 (+0.000023 s): lttng_ust_dl:dlclose + +Bonus: Print top 5 running processes using LTTng +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +As :ref:`examples_tcmi_ev_field` shows, a +:class:`bt2.TraceCollectionMessageIterator` can read +`LTTng `_ traces. -The trace is written in a temporary directory (its path is printed -at the beginning of the script). +The following example is similar to :ref:`examples_tcmi_ev_time`: it +reads a whole LTTng Linux kernel trace, but instead of printing the time +difference for each event, it accumulates them to print the short names +of the top |~| 5 running processes on CPU |~| 0 during the whole trace. .. code-block:: python - import babeltrace.writer as btw - import babeltrace.common - import tempfile - import sys - + import bt2 + import sys + import collections - trace_path = tempfile.mkdtemp() + # Create a trace collection message iterator from the first command-line + # argument. + msg_it = bt2.TraceCollectionMessageIterator(sys.argv[1]) - print('trace path: {}'.format(trace_path)) + # This counter dictionary will hold execution times: + # + # Task command name -> Total execution time (ns) + exec_times = collections.Counter() + # This holds the last `sched_switch` event time. + last_ns_from_origin = None - writer = btw.Writer(trace_path) + for msg in msg_it: + # `bt2._EventMessageConst` is the Python type of an event message. + # Only keep such messages. + if type(msg) is not bt2._EventMessageConst: + continue - clock = btw.Clock('my_clock') - clock.description = 'this is my clock' - writer.add_clock(clock) + # An event message holds a trace event. + event = msg.event - stream_class = btw.StreamClass('my_stream') - stream_class.clock = clock + # Only check `sched_switch` events. + if event.name != 'sched_switch': + continue - event_class = btw.EventClass('my_event') + # Keep only events which occurred on CPU 0. + if event['cpu_id'] != 0: + continue - # 16-bit unsigned integer field declaration - uint16_field_decl = btw.IntegerFieldDeclaration(16) - uint16_field_decl.signed = False + # Get event message's default clock snapshot's ns from origin value. + ns_from_origin = msg.default_clock_snapshot.ns_from_origin - # array field declaration (23 16-bit unsigned integers) - array_field_decl = btw.ArrayFieldDeclaration(uint16_field_decl, 23) + if last_ns_from_origin is None: + # We start here. + last_ns_from_origin = ns_from_origin - # string field declaration - string_field_decl = btw.StringFieldDeclaration() - string_field_decl.encoding = babeltrace.common.CTFStringEncoding.UTF8 + # Previous process's short name. + prev_comm = str(event['prev_comm']) - # sequence field declaration of strings (length will be the `seqlen` field) - seq_field_decl = btw.SequenceFieldDeclaration(string_field_decl, 'seqlen') + # Initialize an entry in our dictionary if not done yet. + if prev_comm not in exec_times: + exec_times[prev_comm] = 0 - event_class.add_field(uint16_field_decl, 'seqlen') - event_class.add_field(array_field_decl, 'array') - event_class.add_field(seq_field_decl, 'seq') + # Compute previous process's execution time. + diff_ns = ns_from_origin - last_ns_from_origin - stream_class.add_event_class(event_class) + # Update execution time of this command. + exec_times[prev_comm] += diff_ns - stream = writer.create_stream(stream_class) + # Update last event's time. + last_ns_from_origin = ns_from_origin - # create event - event = btw.Event(event_class) + # Print top 5. + for comm, ns in exec_times.most_common(5): + print('{:20}{} s'.format(comm, ns / 1e9)) - # set sequence length field - event.payload('seqlen').value = len(sys.argv) +Run this example: - # get array field - array_field = event.payload('array') +.. code-block:: text - # populate array field - for i in range(array_field_decl.length): - array_field.field(i).value = i * i + $ python3 example.py /path/to/lttng/trace - # get sequence field - seq_field = event.payload('seq') +Output example: - # assign sequence field's length field - seq_field.length = event.payload('seqlen') +.. code-block:: text - # populate sequence field - for i in range(seq_field.length.value): - seq_field.field(i).value = sys.argv[i] + swapper/0 326.294314471 s + chromium 2.500456202 s + Xorg.bin 0.546656895 s + threaded-ml 0.545098185 s + pulseaudio 0.53677713 s - # append event - stream.append_event(event) +Note that ``swapper/0`` is the "idle" process of CPU |~| 0 on Linux; +since we weren't using the CPU that much when tracing, its first +position in the list makes sense. - stream.flush() +Inspect event classes +~~~~~~~~~~~~~~~~~~~~~ +Each event stream is a *stream class* instance. +A stream class contains *event classes*. A stream class's event classes +describe all the possible events you can find in its instances. Stream +classes and event classes form the *metadata* of streams and events. -Structure fields ----------------- +The following example shows how to list all the event classes of a +stream class. For each event class, the example also prints the names of +its payload field class's first-level members. -A CTF structure is an ordered map of field names to actual fields, just -like C structures. In fact, an event's payload is a structure field, -so structure fields may contain other structure fields, and so on. +.. note:: -This examples shows how to create a structure field from a structure -field declaration, populate it, and write an event containing it as -a payload field. + As of Babeltrace |~| |version|, there's no way to access a stream class + without consuming at least one message for one of its instances + (streams). -The trace is written in a temporary directory (its path is printed -at the beginning of the script). + A source component can add new event classes to existing stream + classes during the trace processing task. Therefore, this example + only lists the initial stream class's event classes. .. code-block:: python - import babeltrace.writer as btw - import babeltrace.common - import tempfile + import bt2 + import sys + # Create a trace collection message iterator from the first command-line + # argument. + msg_it = bt2.TraceCollectionMessageIterator(sys.argv[1]) - trace_path = tempfile.mkdtemp() + # Get the message iterator's first stream beginning message. + for msg in msg_it: + # `bt2._StreamBeginningMessageConst` is the Python type of a stream + # beginning message. + if type(msg) is bt2._StreamBeginningMessageConst: + break - print('trace path: {}'.format(trace_path)) + # A stream beginning message holds a stream. + stream = msg.stream + # Get the stream's class. + stream_class = stream.cls - writer = btw.Writer(trace_path) + # The stream class object offers a mapping interface (like a read-only + # `dict`), where keys are event class IDs and values are + # `bt2._EventClassConst` objects. + for event_class in stream_class.values(): + print('{}:'.format(event_class.name)) - clock = btw.Clock('my_clock') - clock.description = 'this is my clock' - writer.add_clock(clock) + # The `payload_field_class` property of an event class returns a + # `bt2._StructureFieldClassConst` object. This object offers a + # mapping interface, where keys are member names and values are + # `bt2._StructureFieldClassMemberConst` objects. + for member in event_class.payload_field_class.values(): + fmt = ' {}: `{}.{}`' + print(fmt.format(member.name, bt2.__name__, + member.field_class.__class__.__name__)) - stream_class = btw.StreamClass('my_stream') - stream_class.clock = clock +Run this example: - event_class = btw.EventClass('my_event') +.. code-block:: text - # 32-bit signed integer field declaration - int32_field_decl = btw.IntegerFieldDeclaration(32) - int32_field_decl.signed = True + $ python3 example.py /path/to/trace - # string field declaration - string_field_decl = btw.StringFieldDeclaration() - string_field_decl.encoding = babeltrace.common.CTFStringEncoding.UTF8 +Output example: - # structure field declaration - struct_field_decl = btw.StructureFieldDeclaration() +.. code-block:: text - # add field declarations to our structure field declaration - struct_field_decl.add_field(int32_field_decl, 'field_one') - struct_field_decl.add_field(string_field_decl, 'field_two') - struct_field_decl.add_field(int32_field_decl, 'field_three') + sched_migrate_task: + comm: `bt2._StringFieldClassConst` + tid: `bt2._SignedIntegerFieldClassConst` + prio: `bt2._SignedIntegerFieldClassConst` + orig_cpu: `bt2._SignedIntegerFieldClassConst` + dest_cpu: `bt2._SignedIntegerFieldClassConst` + sched_switch: + prev_comm: `bt2._StringFieldClassConst` + prev_tid: `bt2._SignedIntegerFieldClassConst` + prev_prio: `bt2._SignedIntegerFieldClassConst` + prev_state: `bt2._SignedIntegerFieldClassConst` + next_comm: `bt2._StringFieldClassConst` + next_tid: `bt2._SignedIntegerFieldClassConst` + next_prio: `bt2._SignedIntegerFieldClassConst` + sched_wakeup_new: + comm: `bt2._StringFieldClassConst` + tid: `bt2._SignedIntegerFieldClassConst` + prio: `bt2._SignedIntegerFieldClassConst` + target_cpu: `bt2._SignedIntegerFieldClassConst` - event_class.add_field(struct_field_decl, 'my_struct') - event_class.add_field(string_field_decl, 'my_string') +.. _examples_graph: - stream_class.add_event_class(event_class) +Build and run a trace processing graph +-------------------------------------- +Internally, a :class:`bt2.TraceCollectionMessageIterator` object (see +:ref:`examples_tcmi`) builds a *trace processing graph*, just like the +:bt2man:`babeltrace2-convert(1)` CLI command, and then offers a +Python iterator interface on top of it. - stream = writer.create_stream(stream_class) +See the :bt2man:`babeltrace2-intro(7)` manual page to learn more about +the Babeltrace |~| 2 project and its core concepts. - # create event - event = btw.Event(event_class) +The following examples shows how to manually build and then run a trace +processing graph yourself (like the :bt2man:`babeltrace2-run(1)` CLI +command does). The general steps to do so are: - # get event's structure field - struct_field = event.payload('my_struct') +#. Create an empty graph. - # populate this structure field - struct_field.field('field_one').value = 23 - struct_field.field('field_two').value = 'Achilles Last Stand' - struct_field.field('field_three').value = -1534 +#. Add components to the graph. - # set event's string field - event.payload('my_string').value = 'Tangerine' + This process is also known as *instantiating a component class* + because the graph must first create the component from its class + before adding it. - # append event - stream.append_event(event) + A viable graph contains at least one source component and one sink + component. - stream.flush() +#. Connect component ports. + On initialization, components add input and output ports, depending + on their type. -Variant fields --------------- + You can connect component output ports to input ports within a graph. -The CTF variant is the most versatile field type. It acts as a -placeholder for any other type. Which type is selected depends on the -current value of an outer enumeration field, known as a *tag* from the -variant's point of view. +#. Run the graph. -Variants are typical constructs in communication protocols with -dynamic types. For example, `BSON `_, -the protocol used by `MongoDB `_, has specific -numeric IDs for each element type. - -This examples shows how to create a CTF variant field. The tag, an -enumeration field, must also be created and associated with the -variant. In this case, the tag selects between three types: a -32-bit signed integer, a string, or a floating point number. - -The trace is written in a temporary directory (its path is printed -at the beginning of the script). + This is a blocking operation which makes each sink component consume + some messages in a round robin fashion until there are no more. .. code-block:: python - import babeltrace.writer as btw - import babeltrace.common - import tempfile - - - trace_path = tempfile.mkdtemp() - - print('trace path: {}'.format(trace_path)) - - - writer = btw.Writer(trace_path) - - clock = btw.Clock('my_clock') - clock.description = 'this is my clock' - writer.add_clock(clock) - - stream_class = btw.StreamClass('my_stream') - stream_class.clock = clock - - event_class = btw.EventClass('my_event') - - # 32-bit signed integer field declaration - int32_field_decl = btw.IntegerFieldDeclaration(32) - int32_field_decl.signed = True - - # string field declaration - string_field_decl = btw.StringFieldDeclaration() - string_field_decl.encoding = babeltrace.common.CTFStringEncoding.UTF8 - - # IEEE 754 single precision floating point number field declaration - float_field_decl = btw.FloatingPointFieldDeclaration() - float_field_decl.exponent_digits = btw.FloatingPointFieldDeclaration.FLT_EXP_DIG - float_field_decl.mantissa_digits = btw.FloatingPointFieldDeclaration.FLT_MANT_DIG - - # enumeration field declaration (variant's tag) - enum_field_decl = btw.EnumerationFieldDeclaration(int32_field_decl) - enum_field_decl.add_mapping('INT', 0, 0) - enum_field_decl.add_mapping('STRING', 1, 1) - enum_field_decl.add_mapping('FLOAT', 2, 2) - - # variant field declaration (variant's tag field will be named `vartag`) - variant_field_decl = btw.VariantFieldDeclaration(enum_field_decl, 'vartag') + import bt2 + import sys + + # Create an empty graph. + graph = bt2.Graph() + + # Add a `source.text.dmesg` component. + # + # graph.add_component() returns the created and added component. + # + # Such a component reads Linux kernel ring buffer messages (see + # `dmesg(1)`) from the standard input and creates corresponding event + # messages. See `babeltrace2-source.text.dmesg(7)`. + # + # `my source` is the unique name of this component within `graph`. + comp_cls = bt2.find_plugin('text').source_component_classes['dmesg'] + src_comp = graph.add_component(comp_cls, 'my source') + + # Add a `sink.text.pretty` component. + # + # Such a component pretty-prints event messages on the standard output + # (one message per line). See `babeltrace2-sink.text.pretty(7)`. + # + # The `babeltrace2 convert` CLI command uses a `sink.text.pretty` + # sink component by default. + comp_cls = bt2.find_plugin('text').sink_component_classes['pretty'] + sink_comp = graph.add_component(comp_cls, 'my sink') + + # Connect the `out` output port of the `source.text.dmesg` component + # to the `in` input port of the `sink.text.pretty` component. + graph.connect_ports(src_comp.output_ports['out'], + sink_comp.input_ports['in']) + + # Run the trace processing graph. + graph.run() + +Run this example: + +.. code-block:: text + + $ dmesg -t | python3 example.py + +Output example: + +.. code-block:: text + + string: { str = "ata1.00: NCQ Send/Recv Log not supported" } + string: { str = "ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET FEATURES) succeeded" } + string: { str = "ata1.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) filtered out" } + string: { str = "ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET FEATURES) filtered out" } + string: { str = "ata1.00: NCQ Send/Recv Log not supported" } + string: { str = "ata1.00: configured for UDMA/133" } + string: { str = "ata1.00: Enabling discard_zeroes_data" } + string: { str = "OOM killer enabled." } + string: { str = "Restarting tasks ... done." } + string: { str = "PM: suspend exit" } + +Query a component class +----------------------- +Component classes, provided by plugins, can implement a method to +support *query operations*. + +A query operation is similar to a function call: the caller makes a +request (a query) with parameters and the component class's query +method returns a result object. + +The query operation feature exists so that you can benefit from a +component class's implementation to get information about a trace, a +stream, a distant server, and so on. For example, the +``source.ctf.lttng-live`` component class (see +:bt2man:`babeltrace2-source.ctf.lttng-live(7)`) offers the ``sessions`` +object to list the available +`LTTng live `_ tracing +session names and other properties. + +The semantics of the query parameters and the returned object are +completely defined by the component class implementation: the library +and its Python bindings don't enforce or suggest any layout. +The best way to know which objects you can query from a component class, +what are the expected and optional parameters, and what the returned +object contains is to read this component class's documentation. + +The following example queries the "standard" ``babeltrace.support-info`` +query object (see +:bt2man:`babeltrace2-query-babeltrace.support-info(7)`) from the +``source.ctf.fs`` component class +(see :bt2man:`babeltrace2-source.ctf.fs(7)`) and +pretty-prints the result. The ``babeltrace.support-info`` query object +indicates whether or not a given path locates a +:abbr:`CTF (Common Trace Format)` trace directory:: + + import bt2 + import sys + + # Get the `source.ctf.fs` component class from the `ctf` plugin. + comp_cls = bt2.find_plugin('ctf').source_component_classes['fs'] + + # The `babeltrace.support-info` query operation expects a `type` + # parameter (set to `directory` here) and an `input` parameter (the + # actual path or string to check, in this case the first command-line + # argument). + # + # See `babeltrace2-query-babeltrace.support-info(7)`. + params = { + 'type': 'directory', + 'input': sys.argv[1], + } + + # Create a query executor. + # + # This is the environment in which query operations happens. The + # queried component class has access to this executor, for example to + # retrieve the query operation's logging level. + query_exec = bt2.QueryExecutor(comp_cls, 'babeltrace.support-info', + params) + + # Query the component class through the query executor. + # + # This method returns the result. + result = query_exec.query() + + # Print the result. + print(result) - # register selectable fields to variant - variant_field_decl.add_field(int32_field_decl, 'INT') - variant_field_decl.add_field(string_field_decl, 'STRING') - variant_field_decl.add_field(float_field_decl, 'FLOAT') +As you can see, no trace processing graph is involved (like in +:ref:`examples_tcmi` and :ref:`examples_graph`): a query operation +is *not* a sequential trace processing task, but a simple, atomic +procedure call. - event_class.add_field(enum_field_decl, 'vartag') - event_class.add_field(variant_field_decl, 'var') +Run this example: - stream_class.add_event_class(event_class) +.. code-block:: text - stream = writer.create_stream(stream_class) + $ python3 example.py /path/to/ctf/trace - # first event: integer is selected - event = btw.Event(event_class) - tag_field = event.payload('vartag') - tag_field.value = 0 - event.payload('var').field(tag_field).value = 23 - stream.append_event(event) +Output example: - # second event: string is selected - event = btw.Event(event_class) - tag_field = event.payload('vartag') - tag_field.value = 1 - event.payload('var').field(tag_field).value = 'The Battle of Evermore' - stream.append_event(event) +.. code-block:: text + + {'group': '21c63a42-40bc-4c08-9761-3815ae01f43d', 'weight': 0.75} + +This result indicates that the component class is 75 |~| % confident that +:file:`/path/to/ctf/trace` is a CTF trace directory path. It also shows +that this specific CTF trace belongs to the +``21c63a42-40bc-4c08-9761-3815ae01f43d`` group; a single component can +handle multiple traces which belong to the same group. + +Let's try the sample example with a path that doesn't locate a CTF +trace: + +.. code-block:: text + + $ python3 example.py /etc + +Output: - # third event: floating point number is selected - event = btw.Event(event_class) - tag_field = event.payload('vartag') - tag_field.value = 2 - event.payload('var').field(tag_field).value = -15.34 - stream.append_event(event) +.. code-block:: text + + {'weight': 0.0} - stream.flush() +As expected, the zero weight indicates that ``/etc`` isn't a CTF trace +directory path.