babeltrace.git
5 years agolib: remove bt_self_port{,_input,_output}
Simon Marchi [Wed, 11 Sep 2019 15:43:19 +0000 (11:43 -0400)] 
lib: remove bt_self_port{,_input,_output}

They are equivalent to/redundant with

- bt_self_component_port
- bt_self_component_port_input
- bt_self_component_port_output

Change-Id: I2c466d829f2fecd3567aea195aa08045be9b6f53
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2030
CI-Build: Francis Deslauriers <francis.deslauriers@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agobt2: template borrow_user_attributes method in _TraceClassConst, _StreamClassConst...
Simon Marchi [Wed, 11 Sep 2019 15:35:50 +0000 (11:35 -0400)] 
bt2: template borrow_user_attributes method in _TraceClassConst, _StreamClassConst and _ClockClassConst

Change-Id: I74fc9a490278930f980ed8994b98fd7af37c959b
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2029
CI-Build: Francis Deslauriers <francis.deslauriers@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agobt2: add `__hash__()` method on hashable fields
Francis Deslauriers [Wed, 11 Sep 2019 21:07:31 +0000 (17:07 -0400)] 
bt2: add `__hash__()` method on hashable fields

Implement the `__hash__()` method of selected fields so that they can be
used as keys in dictionaries. This commit only implements the
`__hash__()` method of _NumericFieldConst and _StringFieldConst fields
after considering the following facts:
  1. "[...] the implementation of hashable collections requires that a key’s
    hash value is immutable (if the object’s hash value changes, it will be
    in the wrong hash bucket)." [1],
  2. Python built-in types bool, int, float, and string are hashable,
      and sequences and dictionaries are not.

So I suggest that only const fields should be hashable as they are
immutable, and that Container fields (variant, static array, etc.) should not
be hashable (even if they are const) to be consistent with Python
built-in types.

[1]: https://docs.python.org/3/reference/datamodel.html?highlight=__hash__#object.__hash__

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I3dc246802d628603a3f06ecd3a87b7e38b3f1e27
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2032
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
5 years agoFix: src.ctf.lttng-live: no stream beginning/end messages
Francis Deslauriers [Tue, 10 Sep 2019 01:27:24 +0000 (21:27 -0400)] 
Fix: src.ctf.lttng-live: no stream beginning/end messages

Issue
=====
`src.ctf.lttng-live` components don't emit stream beginning and end
messages. This is due to the `bt_msg_iter::emit_stream_begin_msg` field
being set to false at allocation time.

Solution
========
Set it to true right after the allocation.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Iae6b174e10070b32a877bb9ba7c1e637491ee15d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2022
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agobt2: Makefile: track dependencies of native_bt.c
Simon Marchi [Tue, 10 Sep 2019 15:08:43 +0000 (11:08 -0400)] 
bt2: Makefile: track dependencies of native_bt.c

Building native_bt.c involves including many .h files from include/.
Those dependencies are not tracked by the Makefile, which means that if
you build, modify a .h (which can be simply checking out a different
commit) and build again, native_bt.c won't be re-built (even though it
should).

SWIG has the same kind of features to make dependency tracking easy as
gcc.  The -MD switch generates a bt2/native_bt.d file containing a make
target with all files involved in generating bt2/native_bt.c.  All we
have to do is to include that file in our Makefile.  `-include` is used
so that it's not an error if the file does not exist (which happens the
first time you build).

Change-Id: Iedb187657d42770fa2caeadfa07079351af8a485
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2023
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agobt2: tests: Assert CC type, not if a subclass
Francis Deslauriers [Fri, 6 Sep 2019 20:27:21 +0000 (16:27 -0400)] 
bt2: tests: Assert CC type, not if a subclass

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I807e3a2946741263e2cd3cf0b48c4cddbf281632
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2010
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agobt2: Add `Const` suffix to non-user component-related classes and adapt tests
Francis Deslauriers [Fri, 6 Sep 2019 16:45:20 +0000 (12:45 -0400)] 
bt2: Add `Const` suffix to non-user component-related classes and adapt tests

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I2d6703707c55f1d420027cd83fa9124b83467938
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2009
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agobt2: Add `Const` suffix to `_*Port` classes and adapt tests
Francis Deslauriers [Fri, 6 Sep 2019 15:49:57 +0000 (11:49 -0400)] 
bt2: Add `Const` suffix to `_*Port` classes and adapt tests

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ic71e640abb206c249c63316af6ff536308291301
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2008
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agobt2: Add `Const` suffix to `_Connection` class and adapt tests
Francis Deslauriers [Fri, 6 Sep 2019 15:00:02 +0000 (11:00 -0400)] 
bt2: Add `Const` suffix to `_Connection` class and adapt tests

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I9f695ece01ba48a1fc7008b0ff2ac036fc6e0a7f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2007
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agobt2: Add remaining trace-ir `*Const` classes and adapt tests
Francis Deslauriers [Tue, 27 Aug 2019 19:23:15 +0000 (15:23 -0400)] 
bt2: Add remaining trace-ir `*Const` classes and adapt tests

Split Python classes into Const and Non-Const classes to mimic the type
safety offered by the C api. Const classes offer a read-only view of the
data. Non-Const classes subclass their respective Const classes.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I82b10d48bc9183d9a6bdcc6006ef0a886f9def83
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1924
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agobt2: Add `_Clock*Const` classes and adapt tests
Francis Deslauriers [Fri, 30 Aug 2019 16:20:05 +0000 (12:20 -0400)] 
bt2: Add `_Clock*Const` classes and adapt tests

Split Python Clock Class related classes to mimic the type safety
offered by the C api. Const classes offer a read-only view of the data.
Non-Const classes subclass their respective Const classes.

Makes all `_ClockSnapshot` object Const

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I81e4643c80e6e94445ae0e4948279891416a38ca
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1996
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agobt2: Add `*ValueConst` classes and adapt tests
Francis Deslauriers [Tue, 27 Aug 2019 16:14:45 +0000 (12:14 -0400)] 
bt2: Add `*ValueConst` classes and adapt tests

Split Python Value classes to mimic the type safety offered by the C
api. Const classes offer a read-only view of the data. Non-Const classes
subclass their respective Const classes.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I479db305a281063c7ee8aee2890c33c0b22e7153
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1983
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agobt2: field.py: add index check in `selected_option_index` setter of `_VariantField`
Francis Deslauriers [Thu, 5 Sep 2019 22:12:44 +0000 (18:12 -0400)] 
bt2: field.py: add index check in `selected_option_index` setter of `_VariantField`

Also, add corresponding test case.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ia0b585adba02d71008f9f63298eb017db4a4fabb
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2006
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agobt2: field.py: add `_count` method to `_VariantField`
Francis Deslauriers [Thu, 5 Sep 2019 22:10:54 +0000 (18:10 -0400)] 
bt2: field.py: add `_count` method to `_VariantField`

Also, add corresponding test case.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I536526aa0fa1c196056f35d6a74464b076e4b9d4
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2005
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agolib: add non-const FC borrow functions for container FCs
Francis Deslauriers [Thu, 5 Sep 2019 16:48:32 +0000 (12:48 -0400)] 
lib: add non-const FC borrow functions for container FCs

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I015c3038d505873cf0bdf2ce0673f92d757aeb19
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2002
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agolib: add `bt_field_borrow_class()` function declaration
Francis Deslauriers [Thu, 5 Sep 2019 16:47:08 +0000 (12:47 -0400)] 
lib: add `bt_field_borrow_class()` function declaration

Also, fix the its definition by removing the `const`.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I0ad6261e0890c7a9eab6241245bc99f532e714a9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2001
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agolib: fix can_seek_beginning method type cast
Simon Marchi [Mon, 9 Sep 2019 19:42:35 +0000 (15:42 -0400)] 
lib: fix can_seek_beginning method type cast

Use the right type for the cast.  It works right now because
can_seek_beginning and seek_beginning have the same signature, but
the can_seek_beginning signature is set to change very soon.

Change-Id: I9b51c1bba43fecd1d857578d28cae5b5ff058059
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
5 years agobt2: fix _seek_beginning -> _user_seek_beginning in comment
Simon Marchi [Mon, 9 Sep 2019 17:04:17 +0000 (13:04 -0400)] 
bt2: fix _seek_beginning -> _user_seek_beginning in comment

Change-Id: I93aaea82db86e1e8e37b49f5c26662c82337cb97
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
5 years agobt2: fix trivial typo
Simon Marchi [Mon, 9 Sep 2019 01:45:17 +0000 (21:45 -0400)] 
bt2: fix trivial typo

Change-Id: Iff1c9f8315625f63863763e1fbb35b86d6d54166
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
5 years agolib: make message iterator creation functions return a status
Simon Marchi [Thu, 5 Sep 2019 18:44:50 +0000 (14:44 -0400)] 
lib: make message iterator creation functions return a status

These two functions:

- bt_self_component_port_input_message_iterator_create_from_message_iterator
- bt_self_component_port_input_message_iterator_create_from_sink_component

call some user code at some point (the upstream message iterator's init
function), which can possibly fail (return ERROR).  However, they
currently just return the pointer to the created message iterator, no
status.  So on failure (NULL return value), the caller has to assume
that it translates to a MEMORY_ERROR, given that this is the semantic
for all creation functions that just return a pointer.

Therefore, change them to return a status, and return the pointer by
output parameter.

Wrappers for the Python bindings are necessary, because these functions
offer no guarantee about the value of the output pointer on failure.
But the argout typemap checks whether the pointer is NULL to know if the
function has failed or not.

A test is added in Python to check that if
_create_input_port_message_iterator (both in _UserMessageIterator and
_UserSinkComponent) fails, the error is propagated correctly.

Change-Id: I8151b7e9702d5210c28e2c4a9f323f1d990233ed
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2004
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agorename `babeltrace.trace-info` to `babeltrace.trace-infos`, `streams` to `stream...
Simon Marchi [Tue, 3 Sep 2019 19:28:20 +0000 (15:28 -0400)] 
rename `babeltrace.trace-info` to `babeltrace.trace-infos`, `streams` to `stream-infos`

Because the query response is a list of potentially multiple trace
infos, use the plural form for the object name.

Change-Id: I6b742b4993d1a1552c1d189c54759821d4fd069c
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1999
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agocli: improve error logging in set_stream_intersections
Simon Marchi [Thu, 5 Sep 2019 16:29:21 +0000 (12:29 -0400)] 
cli: improve error logging in set_stream_intersections

I experienced some errors in set_stream_intersections, and found the
error messages to be very generic and non-helpful.  This patch makes the
function use BT_CLI_LOGE_APPEND_CAUSE when possible and produce more
precise error messages.

Change-Id: I4db2859f8f76de95de12c4139c621b068e22d3ba
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/2003
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agobt2: pass all params to `babeltrace.trace-info` query when computing stream intersection
Simon Marchi [Tue, 3 Sep 2019 14:35:23 +0000 (10:35 -0400)] 
bt2: pass all params to `babeltrace.trace-info` query when computing stream intersection

TraceCollectionMessageIterator currently makes it mandatory to pass an
`inputs` parameter to source components when using the stream
intersection mode.  It then picks out that parameter and passes just by
itself to the `babeltrace.trace-info` query.

This is not right for two reasons:

1. The `babeltrace.trace-info` query should be executed with the same
   parameters as what the component will be created with.  This is because
   some parameters can influence the response of the query, such as
   `clock-class-offset-ns` for `src.ctf.fs`.
2. The `inputs` parameter is not mandatory.  It is a convention to which
   source component classes can adhere to work with auto source discovery,
   but it's not mandatory.  Stream intersection mode should work even with
   a source component class that doesn't use it.

A test is added, where a CTF trace is read using the stream intersection
mode, as well as a `clock-class-offset-s` parameter.  Without this
patch, the test would fail because the trimmer components would be
created with a range that doesn't consider the offset.  Therefore all
messages produced by the source (which have the offset applied) fall
outside the trimmers' ranges.  With this patch, the
`babeltrace.trace-info` query returns stream ranges with the offset
applied, so the trimmers have the correct ranges.

Change-Id: I79c86cafafe123c6d306d196d5dde71002b711f7
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1998
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: remove `intersection-range-ns` from `babeltrace.trace-info` query
Simon Marchi [Wed, 28 Aug 2019 20:31:12 +0000 (16:31 -0400)] 
ctf: remove `intersection-range-ns` from `babeltrace.trace-info` query

The `intersection-range-ns` field of the `src.ctf.fs` component class
response to the `babeltrace.trace-info` query is not essential, since it
can easily be computed from the stream ranges also in the response.
Remove it, and adjust users to compute the intersection themselves.

One user is the CLI and the other is the TraceCollectionMessageIterator
class in the Python bindings.

There is a small implementation detail change in
TraceCollectionMessageIterator.  Previously, we would do one
`babeltrace.trace-info` query per port.  With this patch, we pre-compute
the intersection range for each source component port and store it in a
port name to intersection mapping.  When we create the trimmers, we just look
up that value.

It results in less queries and avoids computing the intersections many
times.  It is also in line with how the CLI does.

Change-Id: Ic5ebded4d810ef992594941dde3316613909c98c
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1994
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: remove unnecessary assignment
Simon Marchi [Tue, 3 Sep 2019 19:19:17 +0000 (15:19 -0400)] 
ctf: remove unnecessary assignment

result is already set to NULL by the BT_VALUE_PUT_REF_AND_RESET just
above, so there is no need to set it again.

Change-Id: I9ab4839a8f9011fde846a0ee9cdb30d28786a362
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1997
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: make src.ctf.fs handle a single trace
Simon Marchi [Wed, 28 Aug 2019 19:01:10 +0000 (15:01 -0400)] 
ctf: make src.ctf.fs handle a single trace

Now that traces with different UUIDs get grouped in different src.ctf.fs
instances, it's not (as) useful for one src.ctf.fs component to support
reading multiple traces at the same time.

This patch simplifies the src.ctf.fs component class to make it handle
only a single logical trace (which can actually be composed of multiple
traces with the same UUID).

Internally, it works the same way as before: we create one `struct
ctf_fs_trace` per input path.  We then merge all of them to a single
one, the one that has the most expansive metadata.

Change-Id: I5a395cac3c379dedc06d98ccffa02b3750a35173
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1993
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: make src.ctf.fs not recurse
Simon Marchi [Wed, 28 Aug 2019 17:13:07 +0000 (13:13 -0400)] 
ctf: make src.ctf.fs not recurse

This patch makes the src.ctf.fs component class not recurse into the
directories passed as `inputs`.  This affects instantion of a new
component as well as `babeltrace.trace-info` queries.

Recursion is no longer needed, because it is handled by the automatic
source discovery feature at the CLI and TraceCollectionMessageIterator
levels.

This means that the elements of the `inputs` array must point directly
to traces, i.e. the directory that directly contains the `metadata`
file.

Providing an input path that does not point to such a directory is
unacceptable and will be met with an error of this class:

    CAUSED BY [src.ctf.fs: 'source.ctf.fs'] (/home/smarchi/src/babeltrace/src/plugins/ctf/fs-src/fs.c:1140)
      Path is not a CTF trace (does not contain a metadata file): `/home/smarchi/src/babeltrace/tests/data/ctf-traces/succeed/session-rotation`.

The bulk of the change is mostly in the
ctf_fs_component_create_ctf_fs_traces_one_root, which used to do the
recursion.  It now handles a single path, and is therefore renamed
ctf_fs_component_create_ctf_fs_traces_one_path.

Change-Id: Idb0bbf11f822c50f467fa4e4f661e65242ea78e2
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1989
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: make src.ctf.fs group inputs by UUID in `babeltrace.support-info` query
Simon Marchi [Thu, 8 Aug 2019 19:22:02 +0000 (15:22 -0400)] 
ctf: make src.ctf.fs group inputs by UUID in `babeltrace.support-info` query

When replying to the `babeltrace.support-info` query, the src.ctf.fs component
class currently groups all recognized CTF inputs in a single `ctf`
group.  It then internally groups traces with the same UUID together to
process them as a single trace (to support LTTng's session rotation
feature).

This commit makes it group the inputs by UUID, which will naturally
assign all inputs beloning to one logical trace to one instance of
src.ctf.fs.  Input traces without UUID are not placed in a group,
meaning that they will have their own src.ctf.fs component.

This paves the way to simplifying src.ctf.fs to support only a single
(logical) trace.

I added a test CTF trace for this, `session-rotation`, which consists of
two traces with three rotation chunks each.  We therefore expect to have
two groups with three inputs each.

The new Python test checks the `babeltrace.support-info` query directly.
I also added a new .expect file for this new test trace, it will at
least verify that we have the right number of streams (which will be off
if the grouping doesn't work as intended).

Change-Id: I13374fff9bafa94c5d8f4b6bac1c64afadef7a46
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1844
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
5 years agoctf: add internal ctf_metadata_decoder_get_trace_class_uuid()
Philippe Proulx [Wed, 10 Jul 2019 00:18:43 +0000 (20:18 -0400)] 
ctf: add internal ctf_metadata_decoder_get_trace_class_uuid()

This new internal function gets the trace class's UUID from the AST,
without having to create trace IR and CTF IR trace classes, which is a
much slower process (many allocations and checks).

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I04d58c66a461219ca01bf120417d629d9a1b9d10
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1843
CI-Build: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
5 years agoctf: refactor metadata decoder to always have an instance
Philippe Proulx [Tue, 9 Jul 2019 23:40:45 +0000 (19:40 -0400)] 
ctf: refactor metadata decoder to always have an instance

This patch changes the internal `ctf` plugin's metadata decoder API so
that you need to create a decoder instance to read properties instead of
calling ctf_metadata_decoder_packetized_file_stream_to_buf().

Now, you create a metadata decoder, you call
ctf_metadata_decoder_append_content() (similar to what was called
ctf_metadata_decoder_decode()), and then you can get the decoder's
current properties:

* ctf_metadata_decoder_get_ir_trace_class()
* ctf_metadata_decoder_borrow_ctf_trace_class()
* ctf_metadata_decoder_get_byte_order()
* ctf_metadata_decoder_get_uuid()
* ctf_metadata_decoder_get_text()

The decoder's configuration contains two new flags:

`create_trace_class`:
    Controls whether or not ctf_metadata_decoder_append_content()
    invokes the AST visitor to generate trace class objects.

    Trace classes are not needed, for example, for the `metadata-info`
    query.

`keep_plain_text`:
    Controls whether or not the decoder accumulates the metadata
    stream's plain text when you call
    ctf_metadata_decoder_append_content().

    This needs to be true to use ctf_metadata_decoder_get_text().

The goal of all this is to eventually add new metadata property getters
which do not need to create trace classes or to keep the metadata plain
text.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: If1095da91aee0f446d7f5efcec765e5abbb2b30a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1842
CI-Build: Simon Marchi <simon.marchi@efficios.com>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
5 years agoAdd bt_common_append_file_content_to_g_string()
Philippe Proulx [Tue, 9 Jul 2019 23:40:59 +0000 (19:40 -0400)] 
Add bt_common_append_file_content_to_g_string()

This new function appends the content of a file (`FILE *`) to a
`GString *` object.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I8552873d0c26baafe994e0294c3609f2a221c6e0
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1841
CI-Build: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
5 years agoctf: use create + append/insert functions when possible
Simon Marchi [Thu, 29 Aug 2019 03:22:04 +0000 (23:22 -0400)] 
ctf: use create + append/insert functions when possible

Make the query code use the container value functions that create a new
empty map/array and append it to the container at the same time.  It
requires less lines of code, but also makes ownership management easier:
from the start the created container is owned by the parent container.
So there is no need to track it and free it on failure.

Change-Id: Ie0e81c67f50c721ef62e159f6f1d28380078ca86
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1995
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agoctf: remove ctf_fs_trace::name
Simon Marchi [Wed, 28 Aug 2019 18:30:57 +0000 (14:30 -0400)] 
ctf: remove ctf_fs_trace::name

Now that we don't report the name of CTF traces in
`babeltrace.trace-info`, it is no longer useful to invent a name for
each trace.  So remove the ctf_fs_trace::name field, and everything
related to it.

Note that the name set on the bt_trace (in set_trace_name) does not
depend on this field.

Change-Id: Ia1d3e917e7b1c553c0da0c8365cda9ec600c854b
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1991
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agocli: remove unused path variable in set_stream_intersections
Simon Marchi [Wed, 28 Aug 2019 20:41:54 +0000 (16:41 -0400)] 
cli: remove unused path variable in set_stream_intersections

The `path` variable is used in the error message, but never set, so it
will always appear as "(unknown)".

In the future, it would be nice if the error message gave more info to
the user.  But right now, it is pointless to keep `path` there, so
remove it.

Change-Id: If55d8246dde8cffe1b584477643f105acaaa4792
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1992
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agoctf: remove unused fields from `babeltrace.trace-info` query
Simon Marchi [Wed, 28 Aug 2019 17:44:20 +0000 (13:44 -0400)] 
ctf: remove unused fields from `babeltrace.trace-info` query

The response of the src.ctf.fs component class to the
`babeltrace.trace-info` query object contains a few fields that are not
used at the moment.  Remove them, before somebody uses them, making us
stuck with them for eternity.

The removed fields are:

- stream class ids
- stream ids
- stream paths
- trace path
- trace range-ns
- trace name

The field `intersection-range-ns` will be removed in a separate patch.

Change-Id: Iee673acbb766cd951e758b5186eabb6fc058b442
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1990
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agotests: fix 'expected 2 blank lines, found 1' warning in utils.py
Simon Marchi [Tue, 27 Aug 2019 05:00:24 +0000 (01:00 -0400)] 
tests: fix 'expected 2 blank lines, found 1' warning in utils.py

This is not very important, but I'm fixing warnings anyway.  Black
is fine with this change (it doesn't revert it).

Change-Id: I8e8cf58407d6309756c193d29d3447e9b16df63c
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reported-by: flake8
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1982
Tested-by: jenkins <jenkins@lttng.org>
5 years agobt2: add missing bt2 import in interrupter.py
Simon Marchi [Tue, 27 Aug 2019 04:58:32 +0000 (00:58 -0400)] 
bt2: add missing bt2 import in interrupter.py

Fix this:

  ./src/bindings/python/bt2/bt2/interrupter.py:34:19: F821 undefined name 'bt2'

Change-Id: I84d39d8ced0d75c3a1d9218a43c701f9f653d247
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reported-by: flake8
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1981
Tested-by: jenkins <jenkins@lttng.org>
5 years agopython: fix all "do not use bare 'except'" warnings
Simon Marchi [Tue, 27 Aug 2019 04:57:30 +0000 (00:57 -0400)] 
python: fix all "do not use bare 'except'" warnings

I wasn't sure why that mattered, but I found this explanation pretty
good:

  https://help.semmle.com/wiki/pages/viewpage.action?pageId=29394005

In short, we don't want to catch KeyboardInterrupt and SystemExit.

Change-Id: I0ee20f45ac715a0477c867960cfaea511518171f
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reported-by: flake8
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1980
Tested-by: jenkins <jenkins@lttng.org>
5 years agopython: fix all 'imported but unused' warnings
Simon Marchi [Tue, 27 Aug 2019 04:54:49 +0000 (00:54 -0400)] 
python: fix all 'imported but unused' warnings

Fix all instances of this warning (code F401) found in the repo by
flake8.

Change-Id: I49dc56715d7f23a8779b7bcaf819ee2c90523a6f
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reported-by: flake8
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1979
Tested-by: jenkins <jenkins@lttng.org>
5 years agopython: fix all 'is assigned to but never used' warnings
Simon Marchi [Tue, 27 Aug 2019 04:50:06 +0000 (00:50 -0400)] 
python: fix all 'is assigned to but never used' warnings

Fix all instances of this warning (code F841) found in the repo by
flake8.  In one instance (in test_plugin.py), I added an assertion to
use the variable.  In all others, I considered the variable not useful
and removed it.

Change-Id: I95c282a610d0e3b8adf7cee40ce5229744e8ebc0
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reported-by: flake8
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1978
Tested-by: jenkins <jenkins@lttng.org>
5 years agoautodisc: don't accept NULL values for `support-info` query `group` key
Simon Marchi [Mon, 26 Aug 2019 20:52:33 +0000 (16:52 -0400)] 
autodisc: don't accept NULL values for `support-info` query `group` key

It is not necessary to support that, and it doesn't help much in
implementing component classes.  To keep the support-info protocol a bit
simpler, let's make it mandatory that if you have a `group` key, it must
be a string.

Change-Id: I45aa0a6fb57727d73d9fea48fbebc04dc9c0d5f5
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1977
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoautodisc: improve logging
Simon Marchi [Mon, 26 Aug 2019 20:24:21 +0000 (16:24 -0400)] 
autodisc: improve logging

Improve the logging in the support_info_query_all_sources function:

- Include the group after a successful query
- Include the group when announcing the winner
- Format messages according to our standard, "message: key=value, ...".

Change-Id: I9833ea63314cde912c1da56d6ed2483f34783f17
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1976
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agobt2: make ComponentSpec take a component class object
Simon Marchi [Mon, 12 Aug 2019 22:31:15 +0000 (18:31 -0400)] 
bt2: make ComponentSpec take a component class object

... rather than a plugin and component class name.  It can be a user
component class:

    class MySource(bt2._UserSourceComponent, ...):
        ...

    spec = bt2.ComponentSpec(MySource)

or a component class from a plugin:

    cc = bt2.find_plugin('my_plugin').source_component_classes['cerfeuil']
    spec = bt2.ComponentSpec(cc)

This is a bit more general compared to taking a plugin name and a
component class name, since it allows passing an component class that
does not come from a plugin.  It also allows the user to use component
classes from plugins in non-standard locations, by looking up plugins
manually with bt2.find_plugins_in_path.

For convenience, there is a new static method
bt2.ComponentSpec.from_named_plugin_and_component_class that creates a
ComponentSpec from a plugin and component class names.

Change-Id: I39e3fec6bbc7e7a9ba375065c9318b61b1791e35
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1890
CI-Build: Francis Deslauriers <francis.deslauriers@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agoautodisc: cast when assigning to different enum type
Simon Marchi [Mon, 26 Aug 2019 20:26:33 +0000 (16:26 -0400)] 
autodisc: cast when assigning to different enum type

Gets rid of these warnings with clang:

    /home/smarchi/src/babeltrace/src/autodisc/autodisc.c:766:13: warning: implicit conversion from enumeration type 'auto_source_discovery_internal_status' (aka 'enum auto_source_discovery_internal_status') to different enumeration type 'auto_source_discovery_status' (aka 'enum auto_source_discovery_status') [-Wenum-conversion]
                            status = internal_status;
                                   ~ ^~~~~~~~~~~~~~~
    /home/smarchi/src/babeltrace/src/autodisc/autodisc.c:778:13: warning: implicit conversion from enumeration type 'auto_source_discovery_internal_status' (aka 'enum auto_source_discovery_internal_status') to different enumeration type 'auto_source_discovery_status' (aka 'enum auto_source_discovery_status') [-Wenum-conversion]
                            status = internal_status;
                                   ~ ^~~~~~~~~~~~~~~

Change-Id: Id4cdaaf20da1789e604d346ff2c08239352eb381
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1975
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agoautodisc: make it possible to interrupt auto source discovery
Simon Marchi [Tue, 20 Aug 2019 00:56:26 +0000 (20:56 -0400)] 
autodisc: make it possible to interrupt auto source discovery

It is currently not possible to interrupt babeltrace2 while it is doing
auto source discovery.  However, it can be a quite long process that the
user might want to interrupt using ctrl-C.  This patch makes it possible
to do that.

An interrupter object is taken by the auto source discovery code and
passed all the way to support_info_query_all_sources.  In the CLI, the
global `the_interrupter` object, which is set on sigint, is passed.

Because support_info_query_all_sources (and other internal functions)
can now return a few different values (success with a match, success
without a match, error and interrupted), I changed it to return an enum
type.  The entry point of auto source discovery,
auto_discover_source_components, can now return "interrupted" as well.
However, since that function doesn't return "no match", I didn't want to
make it use the same return type as support_info_query_all_sources.  I
therefore made an "internal" and public version of the status enum.

With this patch, there is no way to interrupt an auto source discovery
started from Python.

Change-Id: I0040c7fab1887c4e05decd0659fa8ff3c85a16a1
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1972
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agoMake bitfield.h C++-friendly
Simon Marchi [Tue, 20 Aug 2019 01:54:07 +0000 (21:54 -0400)] 
Make bitfield.h C++-friendly

This patch changes bitfield.h to be usable in C++11.

It will probably never be compiled as C++ in the context of
babeltrace, but this is just to keep things sync'ed across projects.

Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Change-Id: Ib0fa31ee08d0f691841f96676556d2aa9fcd2426
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1965
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agotrimmer: make filter.utils.trimmer append error causes
Simon Marchi [Mon, 19 Aug 2019 14:46:55 +0000 (10:46 -0400)] 
trimmer: make filter.utils.trimmer append error causes

Change-Id: I6e4f291a8194691faa78b9e497b7563ec32eb7eb
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1967
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agobt2: make _create_event_message's parent parameter mandatory
Simon Marchi [Sat, 17 Aug 2019 05:24:46 +0000 (01:24 -0400)] 
bt2: make _create_event_message's parent parameter mandatory

The `parent` parameter of _create_event_message currently has the None
default value.  However, None is never a valid value, as parent must
either be a Stream or a Packet.  Having a None default value makes it
look like the parameter is optional, but is actually always required.

Therefore, remove the None default value from the parameter.

Change-Id: Id0f8d64006ca25281529fc6bb3b9912365799f44
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1962
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: make src.ctf.fs append error causes
Simon Marchi [Tue, 13 Aug 2019 15:56:18 +0000 (11:56 -0400)] 
ctf: make src.ctf.fs append error causes

This patch makes the src.ctf.fs plugin code append error causes whenever
an error status is returned.

A few new macros are added to comp-logging.h to support that.

Some functions are called both in component context and component class
context.  When appending error causes, we need to call the
function/macro for the right context.  My solution to this is to pass
both bt_self_component and bt_self_component_class pointers, but expect
only one of them to be set.  The new macro
BT_COMP_OR_COMP_CLASS_LOGE_APPEND_CAUSE takes care of calling the right
function for whichever is set.

Change-Id: I8d54f463d9a13066abdc140ef41f374e4daed303
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1966
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoFix: cli: check return value of bt_value_array_append_string_element
Simon Marchi [Sat, 17 Aug 2019 23:29:42 +0000 (19:29 -0400)] 
Fix: cli: check return value of bt_value_array_append_string_element

Found by coverity.

Change-Id: I7830920e70457d8281f57f4447d81fe194399821
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1963
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoFix: ctf: fix possible use-after-free in ctf_fs_component_create
Simon Marchi [Wed, 21 Aug 2019 23:47:32 +0000 (19:47 -0400)] 
Fix: ctf: fix possible use-after-free in ctf_fs_component_create

The error path destroys the ctf_fs_component, but doesn't reset the
variable, so we return a pointer to free'd memory.  We should be
returning NULL in the error case.  Fix it by assigning to NULL after
destroying the ctf_fs_component.

Change-Id: Ib7afd03009dc646460f77fae331920307229220a
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1973
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoFix: cli: replace remaining log level `V` (verbose) -> `T` (trace)
Philippe Proulx [Sat, 17 Aug 2019 04:45:52 +0000 (00:45 -0400)] 
Fix: cli: replace remaining log level `V` (verbose) -> `T` (trace)

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I0649be8f1121ee145163c026847f1117d965b728
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1958
Tested-by: jenkins <jenkins@lttng.org>
5 years agoFix: sink.ctf.fs: dereference before NULL check
Francis Deslauriers [Fri, 23 Aug 2019 18:38:03 +0000 (11:38 -0700)] 
Fix: sink.ctf.fs: dereference before NULL check

  CID 1404550 (#1 of 1): Dereference before null check (REVERSE_INULL)
  check_after_deref: Null-checking tsdl suggests that it may be null,
  but it has already been dereferenced on all paths leading to the
  check.

Reported-by: Coverity - Dereference before null check (REVERSE_INULL)
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I7566cbd74529e93587079d163bb1691e420f723f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1974
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoFix: autodisc: possible use-after-free
Francis Deslauriers [Wed, 21 Aug 2019 17:09:32 +0000 (10:09 -0700)] 
Fix: autodisc: possible use-after-free

Reported-by: scan-build: Memory error Use-after-free
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Icfb8a6640a096369d51ba1c04ce43d043af7af59
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1971
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
5 years agoFix: sink.utils.counter: possible NULL pointer dereference
Francis Deslauriers [Tue, 20 Aug 2019 03:33:12 +0000 (23:33 -0400)] 
Fix: sink.utils.counter: possible NULL pointer dereference

If the null check at line 148 is true, the `counter` pointer might be
used will be used while being NULL by the
`destroy_private_counter_data()` function.

scan-build report:
  Access to field 'msg_iter' results in a dereference of a null pointer
  (loaded from variable 'counter')

Reported-by: scan-build - Logic error Dereference of null pointer
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I50bb981433ef3f5083613b36bbef54743cddb4c2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1970
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agosink.ctf.fs: cleanup: duplicated `:` in log statements
Francis Deslauriers [Tue, 20 Aug 2019 03:28:33 +0000 (23:28 -0400)] 
sink.ctf.fs: cleanup: duplicated `:` in log statements

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Id0cc8d4ee400e3615e4a3e231c56c5d7315f7605
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1969
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agoFix: sink.ctf.fs: possible use-after-free
Francis Deslauriers [Tue, 20 Aug 2019 03:21:57 +0000 (23:21 -0400)] 
Fix: sink.ctf.fs: possible use-after-free

Issue
=====
We might use of `trace` pointer after freeing it in the error path.

Solution
========
Move the `fclose()` call (and surroundings) before the `end` label as
the `fh` pointer is only initialized after the only possible `goto end`.

Reported-by: scan-build - Use of memory after it is freed
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I8f346b45a76ce976019931f9c63c20dd18a88d86
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1968
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agolib: make g++ happy with plugin-dev.h macros
Simon Marchi [Sun, 18 Aug 2019 16:42:35 +0000 (12:42 -0400)] 
lib: make g++ happy with plugin-dev.h macros

g++ doesn't like the syntax used for assigning structures in the plugin
macros:

  /tmp/babeltrace/include/babeltrace2/plugin/plugin-dev.h:599:3: error: expected primary-expression before ‘.’ token
     .value._attr_name = _x,     \
     ^
  /tmp/babeltrace/include/babeltrace2/plugin/plugin-dev.h:871:2: note: in expansion of macro ‘__BT_PLUGIN_COMPONENT_CLASS_DESCRIPTOR_ATTRIBUTE’
    __BT_PLUGIN_COMPONENT_CLASS_DESCRIPTOR_ATTRIBUTE(source_msg_iter_init_method, BT_PLUGIN_COMPONENT_CLASS_DESCRIPTOR_ATTRIBUTE_TYPE_MSG_ITER_INIT_METHOD, _id, _comp_class_id, source, _x)
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /tmp/babeltrace/include/babeltrace2/plugin/plugin-dev.h:1356:2: note: in expansion of macro ‘BT_PLUGIN_SOURCE_COMPONENT_CLASS_MESSAGE_ITERATOR_INIT_METHOD_WITH_ID’
    BT_PLUGIN_SOURCE_COMPONENT_CLASS_MESSAGE_ITERATOR_INIT_METHOD_WITH_ID(auto, _name, _x)
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  can.cpp:829:1: note: in expansion of macro ‘BT_PLUGIN_SOURCE_COMPONENT_CLASS_MESSAGE_ITERATOR_INIT_METHOD’
   BT_PLUGIN_SOURCE_COMPONENT_CLASS_MESSAGE_ITERATOR_INIT_METHOD(CANSource, can_iter_init);
   ^

Changing

    .value._attr_name = _x

to

    .value = { ._attr_name = _x }

makes it work. There are a few instances of this, I changed all I found.

Built-tested with gcc 7.4.0 and clang-9, and verified that a plugin compiled
with g++ 7.4.0 works.

Change-Id: I17cb4edf1082003dc0bb9d0fae906c391daf3da4
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1964
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
5 years agobt2: don't show warning when query method raises a non-error exception
Simon Marchi [Fri, 16 Aug 2019 03:05:13 +0000 (23:05 -0400)] 
bt2: don't show warning when query method raises a non-error exception

I noticed that when a Python query raised bt2.UnknownObject, we show
this warning:

W BT2-PY component_class_query@native_bt_component_class.i.h:802 Failed to call Python class's _bt_query_from_native() method: py-cls-addr=0x6190001090a0

In my opinion, a query returning "unknown object" is part of the normal
course of operation, and we should not show a warning.  So, only show it
when the exception gets translated to a status < 0.  Also, append an
error cause while at it in case of error.

Change-Id: I079f67b74df881ab000c6e223667ca72dcd52713
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1952
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agobt_common_parse_lttng_live_url(): don't accept `net://RDHOST/host/TGTHOST`
Philippe Proulx [Sat, 17 Aug 2019 00:02:33 +0000 (20:02 -0400)] 
bt_common_parse_lttng_live_url(): don't accept `net://RDHOST/host/TGTHOST`

This patch makes the bt_common_parse_lttng_live_url() function not
accept LTTng live URLs with the form:

    net[4]://RDHOST[:RDPORT]/host/TGTHOST

This was previously accepted for the session listing feature, but in
fact the `babeltrace` program (Babeltrace 1) only accepts this form:

    net[4]://RDHOST[:RDPORT]

and the list of URLs that the `babeltrace2` program prints with the
first form is invalid.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I68a804c69ec557ed9cc77c9020afbbe2f79a7c43
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1961
Tested-by: jenkins <jenkins@lttng.org>
5 years agosrc.ctf.lttng-live: return unsigned integers in `sessions` query object
Philippe Proulx [Fri, 16 Aug 2019 23:41:46 +0000 (19:41 -0400)] 
src.ctf.lttng-live: return unsigned integers in `sessions` query object

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I1a0f505ee17f5657395d358dfcaa09fdc8e2afa1
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1960
Tested-by: jenkins <jenkins@lttng.org>
5 years agoFix: pass `inputs` parameter to implicit `src.ctf.lttng-live`, not `url`
Philippe Proulx [Fri, 16 Aug 2019 22:06:41 +0000 (18:06 -0400)] 
Fix: pass `inputs` parameter to implicit `src.ctf.lttng-live`, not `url`

Since the `src.ctf.lttng-live` component class started to handle the
`babeltrace.support-info` query object, its component accepts the
`inputs` array parameter, not the `url` string parameter.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I5d74a16c50ba16518a12be9064afd604c456e3fb
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1959
Tested-by: jenkins <jenkins@lttng.org>
5 years agocli: allow map values in --params arguments
Simon Marchi [Thu, 15 Aug 2019 04:00:15 +0000 (00:00 -0400)] 
cli: allow map values in --params arguments

This patch makes it possible to use map values in --params arguments.
For example:

    --params 'a={hi="mlady",mapping={tete="epaule",genou="orteil"}}'

The chosen syntax uses curly braces to delimit the map.  It uses an
equal sign between the key an value, to be consistent with the syntax of
the implicit map at the top-level of `--params` values.

Speaking of which: it's not done in this patch, but it should be
possible to make the new function a bit more generic and re-use it for
parsing the top-level dict of `--params` values.

Change-Id: I1400ff6a2cf6c9132a9c5dd731a462f43225146b
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1936
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agotests: add tests for CLI params parsing
Simon Marchi [Thu, 15 Aug 2019 15:17:11 +0000 (11:17 -0400)] 
tests: add tests for CLI params parsing

Add tests that checks various forms of valid parameters passed to the
CLI.

Change-Id: I81eb0ceefb6e4326fa001febbecfee5c66ad18c1
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1943
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agocli: exit with error if a plugin fails to load
Simon Marchi [Thu, 15 Aug 2019 00:06:21 +0000 (20:06 -0400)] 
cli: exit with error if a plugin fails to load

The CLI is currently permissive when plugins fails to load, it will
display an error but carry on.  To make it easier for users to catch
configuration and programming problems, this patch changes that be
strict.  If a plugin fails to load (for example, Python syntax error),
the CLI stops and the error is reported to the user.

If this ever becomes a problem (a user would like to carry on even
though some plugin fails to load), we can add a switch like
`--ignore-plugin-loading-error` to have a permissive mode.

Change-Id: Ic89d4e044ecdc34a6e169309479e17bb947ae637
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1937
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agocli: set log levels earlier
Simon Marchi [Wed, 14 Aug 2019 20:47:26 +0000 (16:47 -0400)] 
cli: set log levels earlier

The problem this patch is aiming to solve is that the effective log
level is not set when executing bt_config_convert_from_args.  For
example, bt_python_bindings_bt2_log_level is still NONE, only to be set
later.  So if Python plugin fails to be imported during the auto source
discovery phase, nothing will be logged, making debugging quite
difficult.  Any argument passed (top level --log-level, --debug or
--verbose) won't help because they will take effect later, when we call
set_auto_log_levels.

This patch moves the call to set_auto_log_levels earlier, as soon as we
know what the default log level will be.  For all commands but
`convert`, it is after we are done parsing the top-level arguments.  The
default log level will be the most verbose any --debug, --verbose or
--log-level argument passed.

With the `convert` command, it is possible to pass --debug and
--verbose, with the same effect as the top-level equivalents.  So for
this command, we call set_auto_log_levels just after parsing its
arguments.

The --debug argument sets the log level to TRACE and the --verbose
argument sets it to INFO.  If more than one of --debug, --verbose and
--log-level are specified, the most verbose level of all specified
arguments is chosen.

Note that the --log-level argument of the convert command is not the
same as the top-level's --log-level argument.  The argument of the
convert command sets the --log-level of a specific component or
auto-source-discovery input.

The behavior of the environment variables LIBBABELTRACE2_INIT_LOG_LEVEL,
BABELTRACE_PLUGIN_CTF_METADATA_LOG_LEVEL and
BABELTRACE_PYTHON_BT2_LOG_LEVEL is kept.  If they are specified, they
force the log level of the corresponding subsystem, overriding the
log level specified on the command line.

Change-Id: I9ac558737df4789fc8f5bd8d0240887b3b11b295
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1935
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agocli: return if bt_plugin_find_all_from_dir fails in load_dynamic_plugins
Simon Marchi [Wed, 14 Aug 2019 17:49:51 +0000 (13:49 -0400)] 
cli: return if bt_plugin_find_all_from_dir fails in load_dynamic_plugins

If the call to bt_plugin_find_all_from_dir in load_dynamic_plugins
returns an error (a negative status code), we currently continue
execution as if nothing happened.

This patch makes it so that we exit the loop and report the error to the
caller.

Change-Id: Ib22813b92fe06156cde1681c55651a294cd67909
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1934
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoFix: lib: increment refcount of bt_value_null when copying it
Simon Marchi [Thu, 15 Aug 2019 15:56:14 +0000 (11:56 -0400)] 
Fix: lib: increment refcount of bt_value_null when copying it

Value copy functions must return a new reference.  This is not done for
bt_value_null_copy, which causes refcount imbalance when a null value
gets copied.  A bug caused by this can be triggered with:

    $ git:(3e3535450850) ✗ LIBBABELTRACE2_INIT_LOG_LEVEL=W ~/build/babeltrace/src/cli/babeltrace2 -c src.ctf.fs --params='yo=null,madame=null,la=null'
    ...
    08-15 11:58:25.070  2757  2757 W LIB/VALUE bt_value_null_instance_release_func@value.c:72 Releasing the null value singleton: addr=0x7f070e1fc8e0
    08-15 11:58:25.070  2757  2757 F LIB/VALUE bt_object_put_ref@object.h:367 Babeltrace 2 library precondition not satisfied; error is:
    08-15 11:58:25.070  2757  2757 F LIB/VALUE bt_object_put_ref@object.h:367 Decrementing a reference count set to 0: addr=0x7f070e1fc8e0, ref-count=0
    08-15 11:58:25.070  2757  2757 F LIB/VALUE bt_object_put_ref@object.h:367 Aborting...

Change-Id: I338e6700201892cbe582719bf349041f316d78d8
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1942
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoFix: cli: increment bt_value_null ref count when returning null value
Simon Marchi [Thu, 15 Aug 2019 15:24:24 +0000 (11:24 -0400)] 
Fix: cli: increment bt_value_null ref count when returning null value

The ini_parse_value function returns a new reference to a value.
When return a null value, it needs to acquire a new reference to
bt_value_null to account for that.

A crash can be triggered because of this bug with:

    $ LIBBABELTRACE2_INIT_LOG_LEVEL=W ~/build/babeltrace/src/cli/babeltrace2 -c src.ctf.fs --params='yo=null,madame=null,la=null'
    08-15 11:27:23.947  8793  8793 W LIB/VALUE bt_value_null_instance_release_func@value.c:72 Releasing the null value singleton: addr=0x7fb46d3038e0
    08-15 11:27:23.947  8793  8793 F LIB/VALUE bt_object_put_ref@object.h:367 Babeltrace 2 library precondition not satisfied; error is:
    08-15 11:27:23.947  8793  8793 F LIB/VALUE bt_object_put_ref@object.h:367 Decrementing a reference count set to 0: addr=0x7fb46d3038e0, ref-count=0
    08-15 11:27:23.947  8793  8793 F LIB/VALUE bt_object_put_ref@object.h:367 Aborting...

Change-Id: I6748fd9d8a7ba89728f0e4ddb9ea62bf423fdc91
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1941
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoFix: field-class.c: dereference before `NULL` check
Philippe Proulx [Fri, 16 Aug 2019 18:24:27 +0000 (14:24 -0400)] 
Fix: field-class.c: dereference before `NULL` check

Found by Coverity Scan.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Iaf80ac22f38e3a9dc5d09a69ac5729ece7cdd2f3
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1957
Tested-by: jenkins <jenkins@lttng.org>
5 years agoTests: ctf: add basic `metadata-info` query test
Francis Deslauriers [Fri, 16 Aug 2019 15:43:58 +0000 (11:43 -0400)] 
Tests: ctf: add basic `metadata-info` query test

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I1dcd08a024581a078c163cb09a3372bf2a7ce35f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1955
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agobt2: add `StringValue.__contains__()` method
Francis Deslauriers [Fri, 16 Aug 2019 15:43:00 +0000 (11:43 -0400)] 
bt2: add `StringValue.__contains__()` method

This is useful to use the `in` Python operator like we do with regular
Python `str` objects:

  if 'my_env_var' in env_string_value:
    ...

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I8f7f5c02d71bc089e14c3b8f2ffa81045a9e8f15
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1954
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
5 years agocli: "Babeltrace configuration" -> "Babeltrace CLI configuration"
Philippe Proulx [Fri, 16 Aug 2019 16:30:01 +0000 (12:30 -0400)] 
cli: "Babeltrace configuration" -> "Babeltrace CLI configuration"

Make it clear that it's the configuration of the CLI program.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I972a8555f3f0bd8e7f97191a241df50d75349251
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1956
Tested-by: jenkins <jenkins@lttng.org>
5 years agocli: `convert` command: do not allow more than one sink component
Philippe Proulx [Fri, 16 Aug 2019 03:24:12 +0000 (23:24 -0400)] 
cli: `convert` command: do not allow more than one sink component

It's a nonsense to allow more than one component sink because, starting
from the muxer's single output port, there's a chain of components until
it reaches a sink. We don't have any tee filter, so we can't have more
than one sink: the second one won't get connected.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ibec54ae29b64bb0fb8637f1b37131f05abc53b77
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1953
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agocli: make --plugin-path and --omit-{system,home}-plugin-path global opts
Philippe Proulx [Fri, 16 Aug 2019 03:04:06 +0000 (23:04 -0400)] 
cli: make --plugin-path and --omit-{system,home}-plugin-path global opts

This patch makes the `--plugin-path`, `--omit-system-plugin-path`, and
`--omit-home-plugin-path` options global options, in that you must
specify them before the command's name.

For example, this used to work:

    $ babeltrace2 list-plugins --plugin-path=mein-plugins

With this patch it doesn't, as `--plugin-path` must come before the
command name:

    $ babeltrace2 --plugin-path=mein-plugins list-plugins

This change exists because those three options were handled by all five
commands anyway, so there was a lot of code redundancy.

Tests are adapted accordingly. Everything about those three options in
`test_convert_args` is removed as they will never be part of the
`--run-args` output now.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I6169b01c2c3943bc5ba13ff0be68f8f8a1853da2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1951
Tested-by: jenkins <jenkins@lttng.org>
5 years agoAdd all CLI `convert` command's tests to project's test suite
Philippe Proulx [Fri, 16 Aug 2019 03:03:25 +0000 (23:03 -0400)] 
Add all CLI `convert` command's tests to project's test suite

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ie9b49855198bf37e10257203d6d62919fe68ca39
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1950
Tested-by: jenkins <jenkins@lttng.org>
5 years agoFix: test_convert_args: use correct path to `utils.sh`
Philippe Proulx [Fri, 16 Aug 2019 02:48:38 +0000 (22:48 -0400)] 
Fix: test_convert_args: use correct path to `utils.sh`

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Iec66644f14468a5cbc931e71fc1ae2f21ddc6497
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1949
Tested-by: jenkins <jenkins@lttng.org>
5 years agolive_viewer_connection_create(): fix leak on connection error
Philippe Proulx [Fri, 16 Aug 2019 00:21:33 +0000 (20:21 -0400)] 
live_viewer_connection_create(): fix leak on connection error

g_free() is not enough here: the `url` member is not freed on error.
Just destroy the whole connection object cleanly.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I98184be066a9155e08d792d55fc4b8489a15bb30
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1946
Tested-by: jenkins <jenkins@lttng.org>
5 years agoFix: bt_common_string_until(): reset returned variable on error
Philippe Proulx [Fri, 16 Aug 2019 00:21:01 +0000 (20:21 -0400)] 
Fix: bt_common_string_until(): reset returned variable on error

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I5675899ea57e52eff8492b7444e2216c97f38ad7
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1945
Tested-by: jenkins <jenkins@lttng.org>
5 years agosrc.ctf.lttng-live: handle `babeltrace.support-info` query object
Philippe Proulx [Fri, 16 Aug 2019 00:05:29 +0000 (20:05 -0400)] 
src.ctf.lttng-live: handle `babeltrace.support-info` query object

This patch makes a `src.ctf.lttng-live` component class handle the
`babeltrace.support-info` query object. The query method replies a
weight of 0.75 (arbitrary, but it leaves some room for other component
classes to recognize this very specific URL format) when the input
string matches

    net://HOST[:PORT]/host/TGTHOST/SESSION

This makes it possible to create a conversion graph with a
`src.ctf.lttng-live` source like this:

    babeltrace2 net://localhost/host/qualinet/pour-un-travail-sans-retouche

Note that this will use the component's default "session not found"
action which is to continue, so the `src.ctf.lttng-live` message
iterator will never end. The legacy mode still works, however; when you
do:

    babeltrace2 -i lttng-live net://localhost/host/tgthost/session

the `convert` command adds the `session-not-found-action=end` parameter.

As such, when you do:

    babeltrace2 -i lttng-live net://localhost/host/tgthost

the `convert` command queries the `sessions` object to list the
available remote tracing sessions.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I73e38df1dfe9bed6d8e31015197a34945af026e5
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1944
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agolib: make empty array/map appending/inserting functions return new object
Philippe Proulx [Fri, 16 Aug 2019 01:01:49 +0000 (21:01 -0400)] 
lib: make empty array/map appending/inserting functions return new object

This makes the bt_value_array_append_empty_array_element(),
bt_value_array_append_empty_map_element(),
bt_value_map_insert_empty_array_entry(), and
bt_value_map_insert_empty_map_entry() accept a new optional output
parameter to return the created empty value object.

The goal of appending/inserting an empty array/map value object is
typically to fill it afterwards, so it makes this less painful, for
example:

    bt_value *inner_arr_obj;

    status = bt_value_array_append_empty_array_element(outer_array_obj,
        &inner_arr_obj);
    assert(status == 0);

    status = bt_value_array_append_bool_element(inner_arr_obj, true);
    assert(status == 0);

    status = bt_value_array_append_string_element(inner_arr_obj, "id");
    assert(status == 0);

You can pass `NULL` as this new parameter.

The returned reference is borrowed.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I2cd3359f357a3eb1da52c0e1426ef7804a4b13aa
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1948
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
5 years agolib: rename `BABELTRACE_DISABLE_PYTHON_PLUGINS` -> `LIBBABELTRACE2_...`
Philippe Proulx [Fri, 16 Aug 2019 00:38:15 +0000 (20:38 -0400)] 
lib: rename `BABELTRACE_DISABLE_PYTHON_PLUGINS` -> `LIBBABELTRACE2_...`

This is typically only used for debugging, so change the name to
`LIBBABELTRACE2_DISABLE_PYTHON_PLUGINS`.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I00f54f2a5c1d9abf9f0487ea6d8cd927037f0c0c
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1947

5 years agoTests: ctf: add tracers timestamp quirks workaround test cases
Francis Deslauriers [Fri, 21 Jun 2019 20:46:37 +0000 (16:46 -0400)] 
Tests: ctf: add tracers timestamp quirks workaround test cases

This commit adds test traces and test cases to cover the `src.ctf.fs`
workarounds used to support timestamp corner cases or bugs in the
tracers.

Note: I manually modified the test traces to contain problematic
timestamp values using an hex editor as if there would be created by an
affected tracer.

Test traces
===========
* `event-after-packet` lttng test trace
  This trace `event-after-packet` was modified manually to simulate the
  hard to reproduce `event-after-packet` LTTng tracers bug.

  When the bug occurs, the timestamp of the last event of a packet is larger than
  the `timestamp_end` of it's own packet. It makes it look like the event
  happened after the packet was closed which is non compliant with the CTF
  specification.

  To create a trace exhibiting this bug, the `timestamp_end` field of
  the last 2 packets was made one clock cycle smaller than the timestamp
  of the last event of the said packet.

  This bug was fixed in the follow versions of the LTTng tracers:
  - lttng-ust 2.11.0
  - lttng-module 2.11.0
  - lttng-module 2.10.10
  - lttng-module 2.9.13

* `lttng-crash` test trace
  This trace was modified manually to simulate a trace resulting from the
  lttng-crash utility.

  When extracting a trace using the lttng-crash utility the `timestamp_end` of
  some packets may be left uninitialized with the default value of 0.

  To create a trace exhibiting this behavior, the `timestamp_end` of the last
  three packets were set to 0.

  This is _not_ a bug and all LTTng tracers can produce such trace.

* `event-before-packet` barectf test trace
  This trace `event-before-packet` exhibits a barectf bug where the first
  event of the second packet has a timestamp smaller than the
  `timestamp_begin` field of its own packet.

  This bug was fixed in barectf tracer version 2.3.1.

Test cases
==========

* `trace-info` query
  - Add test case for `event-after-packet` LTTng quirk,
  - Add test case for `lttng-crash` workaround.

  Both of this test traces used in these test cases are based on the same
  trace and the end result on the `trace-info` query is the same. We use
  the same code for both test cases.

* Trace decoding
  -Add `test_succeed` cases for all three of the above test traces.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Icfdb8052aa2a88038e22ad6a58f391b6d8dbd3df
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1542
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: decoding: accommodate barectf `event-before-packet` timestamp quirk
Francis Deslauriers [Wed, 26 Jun 2019 22:17:18 +0000 (18:17 -0400)] 
ctf: decoding: accommodate barectf `event-before-packet` timestamp quirk

See prior commit titled:
      src.ctf.fs: index: accommodate bug in barectf tracer
for an explanation of the non-compliance of some barectf traces.

How this commit fixes the decoding for the affected traces
==========================================================
When about to emit a packet beginning message, check if the said packet
might be affected by barectf's `event-before-packet` bug by looking at
the version. If it's the case, delay the emission of the packet
beginning message until we decoded the first event of the packet. We
then check if the event is indeed before the packet.
For affected packets, we create packet beginning message using the
timestamp of the first event.

If there is event in the packet, there is nothing to fix so we emit the
packet beginning message when we are about to emit the packet end
message.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ia94e94edef27971c4acf7ab62463c80411ff1978
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1552
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: decoding: accommodate LTTng `event-after-packet` timestamp quirk
Francis Deslauriers [Wed, 26 Jun 2019 22:06:23 +0000 (18:06 -0400)] 
ctf: decoding: accommodate LTTng `event-after-packet` timestamp quirk

See prior commit titled:
  src.ctf.fs: index: accommodate bug in LTTng tracers
for an explanation of the non-compliance of some LTTng traces.

How this commit fixes the decoding for the affected traces
==========================================================
When about to emit a packet end message, check if the said packet is
affect by LTTng's `event-after-packet` bug. If it's the case, omit to
update the default_clock_snapshot to the `timestamp_end` of the packet.
This will expand the packet time span to the contain all its events.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I205caf0243c1d6f270bdb1b6508bb55a0806ba19
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1550
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: decoding: accommodate lttng-crash timestamp quirk
Francis Deslauriers [Wed, 26 Jun 2019 20:04:01 +0000 (16:04 -0400)] 
ctf: decoding: accommodate lttng-crash timestamp quirk

See previous commit titled:
  src.ctf.fs: index: accommodate lttng-crash timestamp quirk
for an explanation of the non-compliance of lttng-crash traces.

How this commit fixes the decoding for the affected traces
==========================================================
When about to emit a packet end message, check if the said packet is
affected by the trace-crash non-compliance. If it's the case, omit to
update the default_clock_snapshot to the `timestamp_end` of the packet.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: If965a424afa160692c2ec148fc8f60d3d9ac1273
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1548
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: index: accommodate lttng-crash timestamp quirk
Francis Deslauriers [Fri, 14 Jun 2019 19:56:50 +0000 (15:56 -0400)] 
ctf: index: accommodate lttng-crash timestamp quirk

This commit is to accommodate the indexing of non-compliant CTF traces
produced by LTTng tracers and extracted by the lttng-crash utility.

What makes those traces non-compliant ?
=======================================
The lttng-crash is used to extract a trace from the tracing buffers from
non-volatile memory following a system crash. Because a system crash can
happen at any time, it's possible that some tracing packets might not
have been closed properly. It's possible to observe this by looking at
the `timestamp_end` field of the packets. They might be left to their
initial value of 0 when their `timestamp_begin` are often larger than 0.

This has the higher level effect that affected packets have a
`timestamp_end` smaller than their `timestamp_begin`, as if their end
time was before the beginning time.

How this commit fixes the index for the affected traces
=====================================================
For each index entry, if the entry's `timestamp_end` is 0 and the
`timestamp_begin` is not 0:
 - If it's the stream file's last packet: set the packet index entry's end
   time to the packet's last event's time, if any, or to the packet's
   beginning time otherwise.
 - If it's not the stream file's last packet: set the packet index
   entry's end time to the next packet's beginning time.

Affected versions
=================
All current and future lttng-ust and lttng-modules versions are affect
because this really cannot be fixed by the tracers because lttng-crash
only tries to make the most of a critical situation (i.e. a system
crash).

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ib6ff9683b676983d8691f514f4d6c3e5c8c7e071
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1436
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: index: set `beginning` and `end` clock snapshot
Francis Deslauriers [Fri, 14 Jun 2019 21:14:41 +0000 (17:14 -0400)] 
ctf: index: set `beginning` and `end` clock snapshot

Currently only the timestamps since origin are set when creating the
index from the stream files.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I3a0cf2c0004deb4a2663cff91932fa33aea9cbb9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1438
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: index: accommodate bug in barectf tracer
Francis Deslauriers [Fri, 14 Jun 2019 16:46:32 +0000 (12:46 -0400)] 
ctf: index: accommodate bug in barectf tracer

This commit is to accommodate the indexing of non-compliant CTF traces
produced by the barectf tracer.

What makes those traces non-compliant ?
=======================================
A bug in barectf leads to events being emitted with timestamp less than
their packet's `timestamp_begin` field.

At a higher level, such event would appear as before the beginning of
its own containing packet and not within the range of any packet of the
trace.

How this commit fixes the index for the affected traces
=====================================================
For each index entry excluding the first:
  - Decode the packet until the first event and use the timestamp of
    this event as the `timestamp_begin` of this index entry.
  - Set the previous index entry's `timestamp_end` to the
    `timestamp_end` of the current index entry.

Affected versions
=================
  - before barectf 2.3.1

Tracer commit fixing the issue
===============================
barectf master branch: 27bc6f1e6265a0da01d8f1cf2d669f1a1537c47e

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I1e683467bda15d586a063ca9231cc536e143b346
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1435
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: index: accommodate bug in LTTng tracers
Francis Deslauriers [Fri, 14 Jun 2019 13:58:14 +0000 (09:58 -0400)] 
ctf: index: accommodate bug in LTTng tracers

This commit is to accommodate the indexing of non-compliant CTF traces
produced by LTTng tracers.

What makes those traces non-compliant ?
=======================================
A bug in lttng tracers may lead them to emit events with timestamps that
are greater than the timestamp_end of the their packets.

At a higher level, such event would appear as not within the range of a
packet in the trace.

How this commit fixes the index for the affected traces
=======================================================
For each index entry:
 - If it's the stream file's last packet: set the packet index entry's end
   time to the packet's last event's time by decoding the packet, if
   any, or to the packet's beginning time otherwise.
 - If it's not the stream file's last packet: set the packet index
   entry's end time to the next packet's beginning time.

Affected versions
=================
  - before lttng-ust 2.11.0
  - before lttng-module 2.11.0
  - before lttng-module 2.10.10
  - before lttng-module 2.9.13

Tracer commits fixing the issue
===============================
LTTng-UST master branch: 6c737d0594cac0d969e1948ea1ed55c15be9cec8
LTTng-Modules master branch: 2485a4307679e11711443483a5eb307639f955ed
LTTng-Modules stable-2.10 branch: a1817dcb28265d0f8db10c6bbe98fe8db2c53a8a
LTTng-Modules stable-2.9 branch: 266cef247ceb5d1b4d266a987b8bfcc53d8965ca

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I4c93d05a4e3f85b70657561e9a89cfa2542d49b8
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1420
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: add weak ref to stream file in index entry
Francis Deslauriers [Fri, 14 Jun 2019 16:23:36 +0000 (12:23 -0400)] 
ctf: add weak ref to stream file in index entry

This will be useful to create `bt_msg_iter` without having access to the
`ctf_fs_ds_file_info` struct owning the path to the stream file.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ie7173af60330b601bd755822198ed650e3b9b9b4
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1434
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: add tracer version extraction function
Francis Deslauriers [Wed, 12 Jun 2019 19:58:20 +0000 (15:58 -0400)] 
ctf: add tracer version extraction function

This commit adds a utility function to extract tracer version of a given
trace.

Because, it was discovered that some tracers may produce non-compliant
CTF traces more-or-less frequently. To accommodate those external tools,
we will add ad-hoc fixes for the affected version. We thus need a way to
check the tracer versions of the traces.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I67e73a5928dfac0cebc1135552f5525af930ebfc
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1418
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: msg-iter: add event timestamp accessors
Francis Deslauriers [Fri, 14 Jun 2019 13:44:41 +0000 (09:44 -0400)] 
ctf: msg-iter: add event timestamp accessors

Adds functions to extract the timestamp of the first and last event of
the current packet.
  BT_HIDDEN
  enum bt_msg_iter_status bt_msg_iter_first_event_clock_snapshot(
         struct bt_msg_iter *notit, uint64_t *first_event_cs);

  BT_HIDDEN
  enum bt_msg_iter_status bt_msg_iter_last_event_clock_snapshot(
         struct bt_msg_iter *notit, uint64_t *last_event_cs);

Those functions use the `decode_until_state()` function to decode the
current packet until one of two specific states and then used the clock
value of the default clock class.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I18736151fd8367fb33c6b46d81a860f54432ad55
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1432
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: msg-iter: extract func to decode packet until specific states
Francis Deslauriers [Fri, 14 Jun 2019 13:34:28 +0000 (09:34 -0400)] 
ctf: msg-iter: extract func to decode packet until specific states

This now generic function will be used by future commits needing to
decode packet until after the first or last event of a packet.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I34d86c078e2e512c5491eed978ecc4cc8adcb3a8
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1431
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: introduce dry-run mode for `bt_msg_iter`
Francis Deslauriers [Wed, 12 Jun 2019 18:15:42 +0000 (14:15 -0400)] 
ctf: introduce dry-run mode for `bt_msg_iter`

Dry-run mode can be used when component classes need to decode CTF
packets but do not want the `bt_msg_iter` to create and use the
Babeltrace library objects (bt_field, bt_message_*, etc.).

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I336ec4c893835e7b1d6f32262841793f7543aa06
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1417
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agoctf: only reset state of `bt_msg_iter` on _reset()
Francis Deslauriers [Wed, 12 Jun 2019 17:28:32 +0000 (13:28 -0400)] 
ctf: only reset state of `bt_msg_iter` on _reset()

The `bt_msg_iter` configuration should be left untouched by the reset.

The `bt_msg_iter_reset()` is used at the creation of the `bt_msg_iter`
(`bt_msg_iter_create()`) and when seeking with it
(`bt_msg_iter_seek()`). It's also used by `src.ctf.fs` message iterators
when seeking is done.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I2ab948858aa5b955a3063174c9887c54d98bad5d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1416
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
5 years agolib: remove unneeded `enum` tags in public headers
Philippe Proulx [Thu, 15 Aug 2019 02:27:14 +0000 (22:27 -0400)] 
lib: remove unneeded `enum` tags in public headers

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: If63cd9599deeee84e599ba055db477c4011e1db1
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1933
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agolib: assign values to all public enumerators
Philippe Proulx [Wed, 14 Aug 2019 19:03:17 +0000 (15:03 -0400)] 
lib: assign values to all public enumerators

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I18db5294b67335504d303ea57e1214d8dc5f76ee
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1932
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agolib: standardize status of bt_util_clock_cycles_to_ns_from_origin()
Philippe Proulx [Wed, 14 Aug 2019 19:02:50 +0000 (15:02 -0400)] 
lib: standardize status of bt_util_clock_cycles_to_ns_from_origin()

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ibfe6f7e345f838503b94140e30ec7a7b35866595
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1931
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
5 years agolib: graph.h: fix formatting
Philippe Proulx [Wed, 14 Aug 2019 18:50:11 +0000 (14:50 -0400)] 
lib: graph.h: fix formatting

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Idef225fc3c580bf9f90bd77b3e0a88c7a76267df
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1930
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
This page took 0.052951 seconds and 4 git commands to generate.