Fix: flt.utils.muxer: Undefined or garbage value returned
Reported-by: scan-build - Undefined or garbage value returned to caller Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I46c6e431ae4dcb68367dea84971779af017ecded
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1310 Tested-by: jenkins Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Fix: src.ctf.fs: possible use-after-free in the error path
Reported-by: scan-build - Use of memory after it is freed Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I5c19a45b0ddd11eed3872295efeb7b5201967419
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1303 Tested-by: jenkins Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
This removal should have been done in an earlier commit modifying this
loop:
Author: Francis Deslauriers <francis.deslauriers@efficios.com>
Date: Tue Apr 30 19:01:17 2019 -0400
src.ctf.fs: compute stream range using entire file info group
Given that index entries are sorted within their file info structure and
that file info structures are sorted by time within the file info group,
we can simply compute the range of the stream by using the absolute
first index entry and the absolute last index entry of the entire file
info group.
Add `BT_ASSERT()` to enforce invariants in developer mode.
No need to check if `trace->metadata_path` is null at the point because
it has been used by `fopen()` already and `abort()` is called if
`fopen()` fails.
CID 1401238 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking trace->metadata_path suggests that it
may be null, but it has already been dereferenced on all paths leading
to the check.
Fix: sink.ctf.fs: possible uses after free because of unchecked return values
Uses after free are possible because we ignore the return value of the
`bt_ctfser_write_*()` function calls. To avoid this, check the return
values and log an error message.
CID 1401217 (#4 of 4): Use after free (USE_AFTER_FREE)
11. deref_arg: Calling bt_ctfser_write_byte_aligned_unsigned_int
dereferences freed pointer stream->ctfser.base_mma
CID 1401193 (#2 of 2): Unchecked return value (CHECKED_RETURN)
8. check_return: Calling bt_ctfser_write_byte_aligned_unsigned_int
without checking return value (as is done elsewhere 8 out of 10
times).
Reported-by: Coverity (1401217) Use after free Reported-by: Coverity (1401193) Unchecked return value Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I7581c0e549d7bf916f42afe306976baa9822897a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1286 Tested-by: jenkins Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
CID 1401190 (#1 of 1): Useless call (USELESS_CALL)
side_effect_free: Calling bt_common_color_bold() is only useful for
its return value, which is ignored.
side_effect_free: Calling bt_common_color_fg_red() is only useful for
its return value, which is ignored.
Add -Werror=implicit-function-declaration to AM_CFLAGS
Implicit declaration of function is most likely always a bad thing.
Implicit declaration of function is particularly problematic with
Babeltrace 2 because of its plugin system. Currently, when building a
plugin shared object, the compiler issues a warning when it encounters
an implicit declaration but will create the .so just fine but with the
implicit function marked as undefined, as expected by the standard prior
to C99.
When trying to load a Babeltrace plugin with an undefined symbol, glib's
`g_module_open()` fails and Babeltrace then considers that this .so is
not a valid Babeltrace plugin.
A failure to load a plugin is not a error because we want Babeltrace to
be able to list/use every plugins in a given directory by looking at
every .so file. So we can't simply return an error in those cases.
Since Babeltrace marks plugins with undefined symbols as invalid they
can't be used in a graph.
This problem is caused when a typo in a function name finds its way into
the code and a warning message is ignored. It can also happen when the
code uses a function of more recent library than what is currently
installed on the system.
So, to ensure this never happens, we promote the
`-Wimplicit-function-declaration` warning to the rank of compilation
error.
This is diagnostic has been supported by GCC since v4.8 and Clang since
v4 at least so we believe it's safe to enable it by default.
flt.lttng-utils.debug-info: fd-cache: log to `debug` severity on stat() error
Issue
=====
A `debug-info` component tries to open multiple files in its search to
find the debugging information necessary to resolve the addresses
contained in the trace.
Early in the `bt_fd_cache_get_handle()` function, a `stat()`
is done on the path to get the inode number and device number. This
`stat()` returns an error if the file is absent. Currently, in those
cases, an message is logged at the `error` severity level (BT_LOGE_*)
resulting in the printing of an error message. Since multiple files are
tired until the right one is found (if any), the user can see multiple
error messages even if the right file may be found later.
This is undesirable because it mislead the user into thinking that an
error occurred when in fact it's completely normal.
Solution
========
Log the failure of this `stat()` call to the `debug` severity level.
Fix: sink.ctf.fs: writing 64bit real number as 32bit
Both 32bit and 64bit Trace IR real numbers are written as 32bit CTF
floats with the `bt_ctfser_write_float32()`. This is most probably a
copy-paste error.
To fix this, use the `bt_ctfser_write_float64()` in the else branch instead.
Coverity report:
CID 1401207 (#1 of 1): Identical code for different branches (IDENTICAL_BRANCHES)
identical_branches: The same code is executed regardless of whether
fc->base.size == 32U is true, because the 'then' and 'else' branches
are identical. Should one of the branches be modified, or the entire
'if' statement replaced?
Reported-by: Coverity (1401207) Identical code for different branches Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I462fd47e968e39f5878431d5ac6cbcbbf1d2ffcf
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1268 Tested-by: jenkins Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Fix: flt.lttng-utils.debug-info: `ip` field is 32bit on 32bit cpus
Issue
=====
The debug-info component enforces that the `ip` field used to resolve
addresses is a 64bit integer, but the LTTng UST `ip` context field is
32bit on a 32bit cpu. A debug-info component thus fail to recognize that
the trace has all the necessary information to resolve the addresses.
Here is the metadata representing the context on a 32bit system:
event.context := struct {
integer { size = 32; align = 8; signed = 1; encoding = none; base = 10; } _vpid;
integer { size = 32; align = 8; signed = 0; encoding = none; base = 16; } _ip;
};
Here is the metadata representing the context on a 64bit system:
event.context := struct {
integer { size = 32; align = 8; signed = 1; encoding = none; base = 10; } _vpid;
integer { size = 64; align = 8; signed = 0; encoding = none; base = 16; } _ip;
};
Solution
========
Remove the integer size check in the
`is_event_common_ctx_dbg_info_compatible()` function.
CID 1401244 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking
graph->listeners.source_output_port_added suggests that it may be null,
but it has already been dereferenced on all paths leading to the check.
CID 1401260 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking
graph->listeners.filter_output_port_added suggests that it may be null,
but it has already been dereferenced on all paths leading to the check.k
CID 1401243 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking
graph->listeners.filter_input_port_added suggests that it may be null,
but it has already been dereferenced on all paths leading to the check.
CID 1401185 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking graph->listeners.sink_input_port_added
suggests that it may be null, but it has already been dereferenced on
all paths leading to the check.
CID 1401262 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking
graph->listeners.source_filter_ports_connected suggests that it may be
null, but it has already been dereferenced on all paths leading to the
check.
CID 1401200 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking
graph->listeners.filter_filter_ports_connected suggests that it may be
null, but it has already been dereferenced on all paths leading to the
check.
CID 1401252 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking
graph->listeners.source_sink_ports_connected suggests that it may be
null, but it has already been dereferenced on all paths leading to the
check.
CID 1401264 (#1 of 1): Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking
graph->listeners.filter_sink_ports_connected suggests that it may be
null, but it has already been dereferenced on all paths leading to the
check.
Reported-by: Coverity (various ids) Dereference before null check Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I88c8172e006e4bae2c7b5e29b501e1a36f19ae5c
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1262 Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
The gelf_getshdr() function return a pointer to the second parameter on
success and NULL on error. This means that on we overwrite the
`curr_section_hdr` variable and leak memory.
To fix it, we simply change the variable from a heap allocated variable
to a stack allocated variable this has the effect of making code simpler.
Found using scan-build:
line 334, column 7
Potential leak of memory pointed to by 'curr_section_hdr'
Fix: sink.text.pretty: Unsigned compared against 0
CID 1401251 (#1 of 1): Unsigned compared against 0 (NO_EFFECT)
unsigned_compare: This less-than-zero comparison of an unsigned value is
never true. nr_fields < 0UL.
Reported-by: Coverity (1401251) Unsigned compared against 0 Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I8aa8a03a6b03ca1640570d70a51729d001af3b66
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1263 Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Philippe Proulx [Sun, 5 May 2019 11:53:50 +0000 (07:53 -0400)]
cli/babeltrace.c: update known log level environment variable names
Eventually we could find another way to synchronize the environment
variable names we use throughout the project with the CLI. We want to
decouple the plugins from the CLI (plugins do not know CLI, but CLI
knows plugins) so I don't want the plugins to support an environment
variable defined by the CLI for example.
Philippe Proulx [Fri, 3 May 2019 19:14:18 +0000 (15:14 -0400)]
lib: add unsigned and signed integer value API
This patch replaces the integer value API with the more specific
unsigned and signed integer value API. This opens the whole unsigned
64-bit set for component initialization parameters, query parameters,
queyr results, and, eventually, custom metadata attributes. This is also
more in line with the field API, which has both the unsigned and signed
versions.
Library
=======
The API is simply split in two, where an unsigned integer value contains
a `uint64_t` raw value and a signed integer value is the equivalent of
the previous integer value (`int64_t` raw value).
The types are `BT_VALUE_TYPE_UNSIGNED_INTEGER` and
`BT_VALUE_TYPE_SIGNED_INTEGER`.
There's no interaction between unsigned and signed integer values, in
that, for example, you cannot compare an unsigned integer value with a
signed integer value with bt_value_compare().
Behind the scenes, I kept a single `struct bt_value_integer` for
simplicity. It contains a union of `uint64_t` and `int64_t`. Most
functions call a generic unsigned version, casting to `uint64_t` when
necessary. For example, there's a common bt_value_integer_compare()
which compares the `uint64_t` values.
CLI
===
Before this patch, the `--params` option's format makes any integer
constant a signed integer value object. With this patch, you can create
an unsigned integer value by prepending the constant with `+`. For
example:
hello=+293, meow=+0x388ab88fd, uint-zero=+0
I wanted to use the `U` suffix, like in C/C++, but the GLib lexical
scanner won't allow a constant to be followed by an identifier without a
whitespace.
The documentation about the format of `--params` is updated for the
CLI's `--help` option and in the man pages.
Plugins
=======
It's up to each plugin to accept, for a given parameter, an unsigned
integer value, a signed integer value, or both. A plugin can be very
strict and it's not a bad thing: there are situations where an unsigned
integer is conceptually required.
The individual changes are:
`src.ctf.fs`:
In the `trace-info` query result, create unsigned integer values for
the `id` (stream ID) and `class-id` (stream class ID) entries. A CTF
metadata ID is never negative.
`src.ctf.lttng-live`:
In the `sessions` query result, the `timer-us`, `stream-count`, and
`client-count` entries are still signed integer values. They could
probably be unsigned, but I want to confirm this with developers
more involved with this component class before doing the change. For
the moment, the query operation is unchanged.
`sink.utils.counter`:
The `step` initialization parameter is now expected to be an
unsigned integer value. This is never negative.
I also made the initialization refuse parameters when they don't
have the expected type. There are new `logging.c` and `logging.h`
files for this component class because I used logging to communicate
said errors.
Library tests
=============
The integer value tests now test the signed value API.
I added equivalent unsigned value object tests.
Python bindings
===============
`_IntegerValue` is now the base class of `UnsignedIntegerValue` and
`SignedIntegerValue`. `_IntegerValue` contains the whole public
interface, using template methods with specialized parts defined in
subclasses.
`_IntegerValue` is imported in the `bt2` package for this use case:
if isinstance(val, bt2._IntegerValue):
do_something_with(val)
because `int(val)` is always valid.
The bt2.create_value() function still creates a `SignedIntegerValue`
object with an `int` value. This is the safest, as there's no way to
know that this value will be changed to a negative value in the future,
before the value object is frozen. For example, this would not be
possible with a conversion to an `UnsignedIntegerValue`:
val = bt2.create_value(47)
val.value = -23
whereas it is possible with a `SignedIntegerValue`. You can still
explicitly create an unsigned integer value:
Python bindings tests
=====================
`test_value.py` is updated to also test the `UnsignedIntegerValue`
class. _inject_numeric_testing_methods() now accepts an optional
`has_neg` parameter which, if false, does not inject tests which involve
negative integer values.
There is a common `_TestIntegerValue` class which
`SignedIntegerValueTestCase` and `UnsignedIntegerValueTestCase` inherit.
The tests are almost the same, but `SignedIntegerValueTestCase` adds a
few tests related to negative values. Both specific test cases set the
`_CLS` class attribute to the value object class to test, either
`UnsignedIntegerValue` or `SignedIntegerValue`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ic96ef9e1e16883cb5c59844c6ba5a060936efdb0
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1257 Tested-by: jenkins
Simon Marchi [Fri, 3 May 2019 21:25:14 +0000 (17:25 -0400)]
bt2: Make test_query_executor pass
This patch fixes and enables test_query_executor.py.
The only important change is in the query method: it is necessary to
check for `is_canceled` prior to calling the API function, since it
asserts that the executor is not cancelled.
I have removed the __eq__ method and tests, since it's quite unlikely to
be useful for the QueryExecutor class.
Change-Id: If88c2552627358b06f7d6068126034a20ebf88e8 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1252 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Fri, 3 May 2019 19:58:04 +0000 (15:58 -0400)]
bt2: Fix Makefile dependency tracking when building out of tree
The problem, from the point of view of the developer:
1. Build out of tree, python bindings enabled.
2. Modify a bindings source file (e.g. trace_class.py,
native_bt_trace.i).
3. Type "make" in the builddir.
4. The bindings are not rebuilt from the new source files.
5. Sad.
When we build out of tree, it's done in two steps (assume $builddir is
$top_builddir/bindings/python):
1. Copy all the static files (those checked in the repo, not generated
at runtime) from $srcdir/bt2 to $builddir/bt2. Those files are placed
next to other files that are generated (__init__.py, setup.py).
Together, they form the complete source of the bt2 package.
2. Run setup.py, which runs SWIG and outputs the built package in
$builddir/build_lib.
The problem described above happens because of how the dependencies are
stated for step #1:
When nothing exists yet in the build directory, bt2/native_bt_trace.i is
resolved to the version in the srcdir (because of make's VPATH, I
believe), so it works fine. But once the copied files exist, and you
try to change the source files and "make" again, the dependency
bt2/native_bt_trace.i will be resolved to the version in the build dir.
Since that version is not newer than copy-static-deps.stamp, nothing is
done.
The fix it, we can force the dependency to be the one in the source
directory, by prepending $(srcdir) to each file.
Change-Id: Ic09d430e53b59e5afa9ebb19d98ba219dff16537 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1250 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Fri, 3 May 2019 19:06:45 +0000 (15:06 -0400)]
fix: g_hash_table_insert prior to glib 2.40 returns void
Even the post glib 2.40 version will only return false if the value is
already present in the hashtable but will not fail to add it. The old
value will be handled according to the configuration of the hashtable
which should not be an error.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I5512c921ce220bf8a956c61f594d6a27e6ceb41e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1247 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Fri, 3 May 2019 23:51:11 +0000 (19:51 -0400)]
tests/plugins/ctf/Makefile.am: do not set `TESTS` in this file
We want the project's `make check` to continue even if specific tests
fail, so we can't define `TESTS` in a subdirectory's `Makefile.am`,
because if the test fails, `make` fails, and subsequent tests are not
executed.
There are ways to circumvent this, for example using `make`'s
`--always-make` option, but it was decided previously that we want
`make check` to run all tests as is, without any option.
Therefore we set `TESTS` to all tests in `tests/Makefile.am`.
To execute tests specific to a subdirectory, we have the following
targets:
Fix: flt.lttng-utils.debug-info: build id note section non-native byte order
Issue
=====
The build id comparison code does not take into account that the target
binary file could be compiled for another architecture and thus could be
of a different byte order. This makes the build_id testcase fail on
PowerPC. The artefacts used for this testcase are compiled on x86 and
are thus in little endian as opposed to PowerPC's big endianness.
Solution
========
Parse the Elf file to extract the endianness of the binary and use that
information to change the byte order of the note section if necessary.
Drawbacks
=========
None.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Id5d761832f9463b38ea0452da36053e885929aa9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1246 Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com> Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Fix: common: va_list type is implementation dependant
Issue
=====
BT_ASSERT(*args) happens to work on some library implementations because
the va_list type happens to be a pointer but this is not true with all
libraries. For example, compiling on arm64 results in the following
error:
In file included from common.c:34:0:
common.c: In function ‘bt_common_custom_vsnprintf’:
../include/babeltrace/assert-internal.h:42:7: error: wrong type
argument to unary exclamation mark
if (!(_cond)) { \
^
common.c:1524:2: note: in expansion of macro ‘BT_ASSERT’
BT_ASSERT(*args);
^~~~~~~~~
Solution
========
Remove that BT_ASSERT() call as its intent is unclear anyway.
Drawback
========
None.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Iada5189ef4580e48caefed12835bfab7c23eaf73
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1253
CI-Build: Jérémie Galarneau <jeremie.galarneau@efficios.com> Reviewed-by: Michael Jeanson <mjeanson@efficios.com> Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
translate-trace-ir-to-ctf-ir.c:935:16: warning: ‘var_fc’ may be used
uninitialized in this function [-Wmaybe-uninitialized]
named_fc =
fs_sink_ctf_field_class_variant_borrow_option_by_index(
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
var_fc, i);
I don't think this "condition" is reachable, but it is, in principle
possible. Initializing var_fc to NULL silences this warning.
Simon Marchi [Thu, 2 May 2019 16:21:20 +0000 (12:21 -0400)]
src.ctf.fs: make trace-info query accept clock-class-offset-{s,ns} parameters
The src.ctf.fs components accept clock-class-offset-s and
clock-class-offset-ns parameters, allowing to add an offset to the
clock.
The trace-info query currently doesn't accept the same parameters. In
general, I think we'll want the components and trace-info to accept
pretty much the same set of parameters, so that the information reported
by trace-info accurately represents how an eventual component reading
the same traces will work.
This patch splits the parameter handling of the component in a
standalone function re-used by the query. The advantage of sharing a
function like this is that if we add a parameter to the component, we'll
be forced to think about what happens to the trace-info query.
One behavior change is that trace_info_query now returns
BT_QUERY_STATUS_INVALID_PARAMS if the passed parameters are invalid.
A test is added to test the new functionality.
Change-Id: I3d917b8713fce4ce56a79a6e7c7bdc3e32d9ec0c Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1244 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Thu, 2 May 2019 20:53:30 +0000 (16:53 -0400)]
ctf: Use g_time_val_from_iso8601 instead of g_date_time_new_from_iso8601
g_date_time_new_from_iso8601 was introduced in glib 2.56, which is quite
recent. To support build with older glibs, change the code to use
g_time_val_from_iso8601 instead, introduced in glib 2.12.
The code in question only uses this function to validate that the passed
string is ISO8601-compliant, it doesn't use the resulting time value.
The problem of g_time_val_from_iso8601 is that its result is not year
2038-safe on 32-bit systems. We can expect that by then, that function
will have been removed from glib, forcing us to update our code.
Change-Id: I2f4fdfce9020ecefcfb5d653a3885d343e49bd76 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1245 Reviewed-by: Michael Jeanson <mjeanson@efficios.com> Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
... so they should be removed from EXTRA_DIST as well. Currently, it
causes a "make dist" failure:
make[4]: *** No rule to make target 'test_trace.py', needed by 'distdir'. Stop.
The tests/lib/trace-ir directory is currently not very useful, as it
contains no tests at all, so we could remove it in theory. But I
presume we'll add something in there in not too long, so I think we can
leave it there.
Change-Id: Iec44efd7c52e84b35a5a55206055b5d63635fbe1 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1241 Tested-by: jenkins Reviewed-by: Jonathan Rajotte Julien <jonathan.rajotte-julien@efficios.com> Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Thu, 2 May 2019 18:36:57 +0000 (14:36 -0400)]
include: add missing files to dist tarball
When making a tarball (make dist) and trying to build from it, it fails
because of missing header files. This patch adds all the necessary
headers to the dist tarball to make "./configure && make" from the
tarball succeed.
Change-Id: I89051cb060e911eac154ef1726034e905b49b085 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1243 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com> Reviewed-by: Jonathan Rajotte Julien <jonathan.rajotte-julien@efficios.com>
Simon Marchi [Wed, 1 May 2019 22:00:50 +0000 (18:00 -0400)]
bt2: Add wrapper for bt_plugin_get_version
There is a subtle bug with our use of the "char **OUT" typemap for the
bt_plugin_get_version function.
THe SWIG wrapper does this:
char *temp_value5;
arg5 = &temp_value5;
The initial content of temp_value5 is unknown, assume that it points to
unreadable memory. If the call to bt_plugin_get_version returns
PROPERTY_AVAILABILITY_NOT_AVAILABLE, the value of temp_value5 is
unchanged (but really, it could be anything, again we should assume it
points to unreadable memory). However, the epilogue of the typemap will
still try to transform the C string to a Python str object. Doing so
will cause a crash when trying to read the C string.
The fix is to make our own wrapper to bt_plugin_get_version, which
guarantees that if it returns PROPERTY_AVAILABILITY_NOT_AVAILABLE, it
will force the value of the output parameter to be NULL (which is
handled by the epilogue and transformed to None).
Setting the temporary variable to an invalid but non-zero pointer value
((void *) 1) makes this bug consistently reproducible and not rely on
random memory contents. I suggest we keep it there, in order to quickly
find other instances of this bug in the future.
Change-Id: Idd12ca4c4d4d732b841c1e12420e46e6a5a25874
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1221 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Fix: bt2: enum-conversion warning in native_bt_component_class.i
clang warns about the following enumeration conversion.
bt2/native_bt_wrap.c:3896:12: warning: implicit conversion from
enumeration type 'enum bt_message_iterator_status' to different
enumeration type 'enum bt_self_message_iterator_status'
[-Wenum-conversion]
status = BT_MESSAGE_ITERATOR_STATUS_END;
~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bt2/native_bt_wrap.c:3898:12: warning: implicit conversion from
enumeration type 'enum bt_message_iterator_status' to different
enumeration type 'enum bt_self_message_iterator_status'
[-Wenum-conversion]
status = BT_MESSAGE_ITERATOR_STATUS_AGAIN;
~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bt2/native_bt_wrap.c:3901:12: warning: implicit conversion from
enumeration type 'enum bt_message_iterator_status' to different
enumeration type 'enum bt_self_message_iterator_status'
[-Wenum-conversion]
status = BT_MESSAGE_ITERATOR_STATUS_ERROR;
~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bt2/native_bt_wrap.c:4513:43: warning: implicit conversion from
enumeration type 'enum bt_message_iterator_status' to different
enumeration type 'bt_self_message_iterator_status' (aka 'enum
bt_self_message_iterator_status') [-Wenum-conversion]
bt_self_message_iterator_status status = BT_MESSAGE_ITERATOR_STATUS_OK;
~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bt2/native_bt_wrap.c:4678:43: warning: implicit conversion from
enumeration type 'enum bt_message_iterator_status' to different
enumeration type 'bt_self_message_iterator_status' (aka 'enum
bt_self_message_iterator_status') [-Wenum-conversion]
bt_self_message_iterator_status status = BT_MESSAGE_ITERATOR_STATUS_OK;
~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Icb3bc0141cdd9f35a5a42420bb48167c12e70df0
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1183 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com> Tested-by: jenkins
Fix: bt2: int-to-pointer-cast warnings in SWIG utils functions
bt2/native_bt_wrap.c: In function ‘bt_py3_component_class_query’:
bt2/native_bt_wrap.c:4453:12: warning: cast to pointer from integer of
different size [-Wint-to-pointer-cast]
*result = (void *) PyLong_AsUnsignedLongLong(py_results_addr);
^
bt2/native_bt_wrap.c: In function ‘bt_py3_component_class_message_iterator_next’:
bt2/native_bt_wrap.c:4697:3: warning: cast to pointer from integer of
different size [-Wint-to-pointer-cast]
(const bt_message *) PyLong_AsUnsignedLongLong(
^
python-plugin-provider.c: In function ‘bt_plugin_from_python_plugin_info’:
python-plugin-provider.c:341:18: warning: cast to pointer from integer of
different size [-Wint-to-pointer-cast]
comp_class = (bt_component_class *)
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I44d15e5d072d993776539eddae5f3e37c6161cac
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1162
CI-Build: Francis Deslauriers <francis.deslauriers@efficios.com> Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
src.ctf.fs: compute stream range using entire file info group
Given that index entries are sorted within their file info structure and
that file info structures are sorted by time within the file info group,
we can simply compute the range of the stream by using the absolute
first index entry and the absolute last index entry of the entire file
info group.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I5b13451d72a4b1aeb9f31140ebfb025e2d712d7e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1128 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Wed, 1 May 2019 15:30:28 +0000 (11:30 -0400)]
bt2: update bindings to make test_component_class pass
This patch updates the binding with the goal of making
test_component_class pass.
Besides the obviously needed changes to the component class code, there
are a bunch of changes to things around it (component, graph), just
enough to make the test run.
One notable change is the _SharedObject._release method, used to make the
Python object stop manage the reference it currently owns.
Another functional change is how the component classes _query method
reports that it's not implemented. Currently, it default to return the
NotImplemented object. Instead, I made it raise a NotImplementedError.
Returning NotImplemented is useful if the caller has an alternative to
try when the method is not implemented. In our case, there is nothing
to do if the specific class doesn't implement _query. The exception is
caught in bt_py3_component_class_query, where other types of exceptions
need to be handled anyway.
Some changes are required to the library, to be able to get a specific
component class type from a specific component (e.g. a
bt_component_class_source from a bt_component_source).
Change-Id: I895817bc996c21b6a9c6d57605cc64f9ff61dfcc Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1083 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Wed, 1 May 2019 14:17:33 +0000 (10:17 -0400)]
lib: Add functions to borrow specialized component classes from specialized components
The API currently contains a function to obtain a 'bt_component_class *'
from a 'bt_component *'. This patch adds functions to obtain a
specialized component class from a specialized component. For example,
a 'bt_component_class_source *' from a 'bt_component_source *'.
Change-Id: Ide192c7759f630adbc884def2e5c462ce2b8f973 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1161 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Fri, 26 Apr 2019 19:41:24 +0000 (15:41 -0400)]
bt2: update value.py, make test_value pass
This patch updates value.py to match the current API.
The only known thing missing (to be done later) is to split classes in
their const and non-const variants (e.g. BoolValue and BoolValueConst),
where the methods that can modify values are only present on the
non-const versions.
The test is largely unchanged, except for the removal of things related
to frozen values.
Change-Id: I6424b0e6208ffe2cbe5e60fdd08579326abff29a Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1022 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 29 Apr 2019 16:12:36 +0000 (12:12 -0400)]
bt2: update object model
This patch updates the Babeltrace Python object model (what all
Babeltrace Python objects inherit from) to match the current status of
the API.
We have both Shared and Unique objects. Both types have a _ptr
member, a SWIG Python object wrapping a typed pointer (e.g. pointer to
struct bt_value).
A Shared object type is defined by inheriting from _SharedObject. It
wraps and owns a reference to a Babeltrace object that is reference
counted. The reference count is dropped when the Python object gets
deleted. The _SharedObject base class defines two class methods,
_get_ref and _put_ref, which must be implemented by subclasses. These
implementations must respectively get and put a reference of the
specialized pointer (_ptr) they wrap.
A Unique object type is defined by inheriting from _UniqueObject. It is
not refcounted. Rather, it is wholly owned by another object (the
owner) that is itself refcounted. A Unique object, in the Babeltrace
API, is destroyed when its parent is destroyed. There is therefore a
risk that someone would keep a (Python) reference to a unique object
that got destroyed, because its parent got destroyed, resulting in some
use-after-free bug. To avoid this, we make it so that when a Python
object representing a Babeltrace Unique object is created, that Python
object obtains a reference on the owner Shared object. When the Unique
Python object is destroyed, the reference on the Shared parent is
dropped.
The concepts of Freezable and Private objects don't exist anymore, so
they are removed. I have also removed the _PrivateConnection class,
since it won't be needed in the end and inherited from _PrivateObject.
For other classes representing Babeltrace concepts, I have made them
inherit the object type they'll need to inherit in the end (when all the
Python bindings will be fixed), even though none of these classes is
functional at the moment.
Change-Id: I9ab743dd7a28407913945648c1d74c467550df14 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1021 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Fix: may-uninitialized warning in trace-ir-metadata-field-class-copy.c
GCC warning:
In file included from ../../../include/babeltrace/common-internal.h:30:0,
from trace-ir-metadata-field-class-copy.c:31:
trace-ir-metadata-field-class-copy.c: In function ‘copy_field_class_content_internal’:
../../../include/babeltrace/trace-ir/field-class-const.h:196:3: error:
‘out_tag_field_class’ may be used uninitialized in this function
[-Werror=maybe-uninitialized]
bt_field_class_put_ref(_var); \
^~~~~~~~~~~~~~~~~~~~~~
trace-ir-metadata-field-class-copy.c:421:18: note: ‘out_tag_field_class’ was declared here
bt_field_class *out_tag_field_class;
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I73c6057f9795b153f95c1287f3f3f8e529328fbf
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1033 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Fix: maybe-uninitialized warnings in translate-trace-ir-to-ctf-ir.c
GCC warning example:
translate-trace-ir-to-ctf-ir.c:929:14: error: ‘struct_fc’ may be used
uninitialized in this function [-Werror=maybe-uninitialized]
named_fc = fs_sink_ctf_field_class_struct_borrow_member_by_index(
~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
struct_fc, i);
~~~~~~~~~~~~~
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I24ba8095f26cec640e0a24473b4fee05d1c404af
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1032 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Fix: maybe-uninitialized warning in create_relative_field_ref()
translate-trace-ir-to-ctf-ir.c: In function ‘resolve_field_class’:
translate-trace-ir-to-ctf-ir.c:302:6: error: ‘tgt_fc_name’ may be used
uninitialized in this function [-Werror=maybe-uninitialized]
g_string_assign(tgt_field_ref,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tgt_fc_name);
~~~~~~~~~~~~
translate-trace-ir-to-ctf-ir.c:177:14: note: ‘tgt_fc_name’ was declared here
const char *tgt_fc_name;
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I8d3398a852bf3ca412bb479eae72a81a3959d220
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1031 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Fix: unused-variable warnings in lib/graph/iterator.c
GCC warnings:
make[3]: Entering directory '/home/frdeso/projets/babeltrace/optimalize/lib/graph'
CC iterator.lo
iterator.c: In function ‘_set_iterator_state_after_seeking’:
iterator.c:889:59: error: variable ‘new_state’ set but not used [-Werror=unused-but-set-variable]
enum bt_self_component_port_input_message_iterator_state new_state = 0;
^~~~~~~~~
iterator.c: In function ‘find_message_ge_ns_from_origin’:
iterator.c:1159:59: error: unused variable ‘init_state’ [-Werror=unused-variable]
enum bt_self_component_port_input_message_iterator_state init_state =
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ia14a89f330198e4109af14f755aeed3378912256
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1030 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Fix: maybe-uninitialized warning in bt_ctf_stream_flush()
GCC warning:
In file included from ../../include/babeltrace/ctf-writer/fields-internal.h:41:0,
from ../../include/babeltrace/ctf-writer/field-wrapper-internal.h:26,
from ../../include/babeltrace/ctf-writer/event-internal.h:37,
from stream.c:32:
stream.c: In function ‘bt_ctf_stream_flush’:
../../include/babeltrace/ctfser-internal.h:568:36: error:
‘packet_context_offset_bits’ may be used uninitialized in this function
[-Werror=maybe-uninitialized]
ctfser->offset_in_cur_packet_bits = offset_bits;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~
stream.c:1608:11: note: ‘packet_context_offset_bits’ was declared here
uint64_t packet_context_offset_bits;
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ia7457b1a444290ba1a71459bfaf707812227899c
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1029 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Issue
=====
Omitting to pass the mandatory `path` parameter to the `metadata-info`
query results in a BT_ASSERT() failure when calling
bt_value_get_string(), or a NULL pointer dereference in non-DEV_MODE.
Similarly, setting the `path` to a non-string value would also result in
a BT_ASSERT() failure, or a invalid memory access in non-DEV_MODE
Solution
========
Confirm that the `path` parameter is present and is of the string type and
print error messages accordingly it's not.
Known drawbacks
===============
None.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I8182de2797c375262d77a4a8961bedb4c5ef9578
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1081 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Tue, 30 Apr 2019 22:29:01 +0000 (18:29 -0400)]
debug info: Skip test that depends on Python bindings
This test depends on the Python bindings, which don't work right now.
Skip it, so that it doesn't cause a "make check" failure while we are
repairing the bindings.
Change-Id: I8e9b1092e2794875b249b060581f4305be640a88 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1126 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Tue, 30 Apr 2019 21:52:12 +0000 (17:52 -0400)]
doc: Make python bindings doc build
Babeltrace 1 Python bindings (the babeltrace package) were removed in
commit
1c81bd8895c0 ("Remove babeltrace 1 Python bindings and tests")
However, the Python doc still references it. It tries to import it to
get the version, and therefore fails to build. Update it to import the
bt2 package instead.
Change-Id: I656a89ad83069f6794abf4c775a14fb9be2ecba0 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1124 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 8 Apr 2019 16:53:56 +0000 (12:53 -0400)]
python-plugin-provider: Make it build
This patch fixes building with --enable-python-plugins.
One problem is that the python plugin provider uses some
functions coming from the internal headers of the lib, which use some
logging functions that are internal to the lib, but not included by
those internal headers. Trying to add the relevant include file
(lib-logging-internal.h) conflicts with python-plugin-provider's own
logger.
python-plugin-provider doesn't really need to have its own logger. Even
though it is built as a separate shared object (so that we can package
it separately), it can be considered as internal to the library (and
that's why it includes some internal lib headers). Therefore, the
easiest way to fix this is to get rid of its logger and make the lib
header files that use lib logging include the required header file
(lib-logging-internal.h).
This requires the python-plugin-provider shared object to access a few
additional symbols not currently exposed by libbabeltrace, which
requires the removal of a few BT_HIDDEN.
I have also included fixes to the python-plugin-provider which I have
on hand. However, it won't work until the Python bindings are up and
running, so I have marked it as skipped.
Change-Id: I9b46d91fc635d670fef2e4d3c8b32203ec34443f Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1101 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Sat, 27 Apr 2019 21:04:45 +0000 (17:04 -0400)]
Fix: headers: make static inline upcasts compatible with C++
Issue
=====
There are many static inline functions which upcast a pointer from its
original type to another one known to be a parent. The functions upcast
with `(void *)` or `(const void *)`. While this works in C, it does not
in C++, which is more strict:
In file included from install/include/babeltrace/babeltrace.h:75,
from test.cpp:1:
install/include/babeltrace/trace-ir/field-class-const.h: In function
‘const bt_field_class_enumeration_mapping*
bt_field_class_unsigned_enumeration_mapping_as_mapping_const(const
bt_field_class_unsigned_enumeration_mapping*)’:
install/include/babeltrace/trace-ir/field-class-const.h:100:9:
error: invalid conversion from ‘const void*’ to ‘const
bt_field_class_enumeration_mapping*’ [-fpermissive]
return (const void *) mapping;
Solution
========
Use the new __BT_UPCAST() and __BT_UPCAST_CONST() macros to upcast
within headers. Those macros are defined in `babeltrace/types.h`. We use
static_cast<>() in C++ with the specific destination type, going through
`void *` or `const void *` to keep the same address.
Known drawbacks
===============
None.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Iaac8e55635eb2f358a546c32bd886a484f5ebfeb
Reviewed-on: https://review.gerrithub.io/c/eepp/babeltrace/+/452302 Reviewed-by: Simon Marchi <simon.marchi@polymtl.ca>
Simon Marchi [Fri, 26 Apr 2019 19:26:51 +0000 (15:26 -0400)]
bt2: Rename files to use singular form
I am mildly annoyed that most Python files use the singular form of the
concept they are related too (component.py, event.py) but others use the
plural form (fields.py, values.py).
Change this to consistently use the singular.
Change-Id: I1a33c47ec45ef3359025e0a66f01eaefc09fecdf Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.gerrithub.io/c/eepp/babeltrace/+/452257 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com> Tested-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Tue, 2 Apr 2019 18:03:30 +0000 (14:03 -0400)]
bt2: Mass notification -> message rename
As the "notification" concept has been renamed to "message", I decided
to do a mass rename (including renaming files). It's quite
straightforward and will reduce a little bit the noise in further
patches.
The changes were done mostly mechanically, but I verified them to make
sure it didn't go too wild (like changing "notifier" to "msgier", which
we wouldn't want).
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Change-Id: I6e6dec1234864400a2570a4726c32a1398cf42c0
Reviewed-on: https://review.gerrithub.io/c/eepp/babeltrace/+/452232 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com> Tested-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Fri, 26 Apr 2019 14:18:55 +0000 (10:18 -0400)]
bt2: Sync native_bt_field_class.i with field-class-const.h
Commit
2d97b62eea05 ("lib: have dedicated, unique unsigned and signed enum FC mapping objects")
Changed some declarations in field-class-const.h without doing the
corresponding changes in native_bt_field_class.i. I am now seeing this
build error:
bt2/native_bt_wrap.c: In function ‘_wrap_field_class_unsigned_enumeration_borrow_mapping_by_index_const’:
bt2/native_bt_wrap.c:11087:3: error: unknown type name ‘bt_field_class_unsigned_enumeration_mapping_ranges’; did you mean ‘bt_field_class_unsigned_enumeration_mapping’?
bt_field_class_unsigned_enumeration_mapping_ranges **arg4 = (bt_field_class_unsigned_enumeration_mapping_ranges **) 0 ;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
bt_field_class_unsigned_enumeration_mapping
Change-Id: I897749a6a7fe5bd1c3c16b1ea9625820e77f112b Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.gerrithub.io/c/eepp/babeltrace/+/452225 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com> Tested-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Wed, 24 Apr 2019 05:00:45 +0000 (01:00 -0400)]
lib: have dedicated, unique unsigned and signed enum FC mapping objects
This patch makes the enumeration field class mapping API aligned with
the structure field class member and variant field class option APIs. In
other words, a signed or an unsigned enumeration field class mapping is
now a dedicated type. An enumeration field class mapping object is
unique and always belongs to its parent field class.
bt_field_class_unsigned_enumeration_borrow_mapping_by_index_const() and
bt_field_class_signed_enumeration_borrow_mapping_by_index_const() now
return borrowed enumeration field class mapping objects.
You can use
bt_field_class_unsigned_enumeration_mapping_as_mapping_const() to change
a signed or unsigned enumeration field class mapping into a generic
enumeration field class mapping. The API for a generic enumeration field
class mapping is:
You cannot create an enumeration field class mapping object: you still
map a name to a specific range with
bt_field_class_unsigned_enumeration_map_range() and
bt_field_class_signed_enumeration_map_range().
This patch also makes the `flt.lttng-utils.debug-info` and `sink.ctf.fs`
component classes use the updated API.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Iba5da648ca69ec0ca0c5df09a9f383dd4b1e3972
Reviewed-on: https://review.gerrithub.io/c/eepp/babeltrace/+/451996
Simon Marchi [Wed, 3 Apr 2019 18:08:18 +0000 (14:08 -0400)]
tests/bindings/python: Mark all tests as skipped
This patch makes us skip all the Python bindings tests. The goal is for
these commands to run with success (even though they doesn't actually test
anything):
The strategy will be to gradually fix the bindings and their tests, and
gradually remove those skip annotations. At any point, we should be
able to run the testsuite with success, ensuring that we don't regress.
After this patch, "make check" is clean for me, with python bindings
enabled (--enable-python-bindings).
I am not sure what is still relevant in the python/babeltrace
directory, I suppose the ctf writer stuff still is. I think we can sort
this out later.
Change-Id: I1169c255c0c7c29fa5de75a358c02c4cd57aae20 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.gerrithub.io/c/eepp/babeltrace/+/452155 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com> Tested-by: Philippe Proulx <eeppeliteloop@gmail.com>
In some cases, I've replaced some references to concepts that don't
exist with the new concepts (e.g. some NOTIFICATION_TYPE_* to
MESSAGE_TYPE_*). In other cases, (e.g. fields.py and field_types.py),
it was easier to remove the code, knowing we'll add it back when
tackling the tests that require these files.
After this milestone, we'll be able to at least run some tests that
import the bt2 package, so we'll be able to make sure we don't regress
things while we fix others.
Change-Id: I930d5a4dd129777cb40c607670e27002bb727e39 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.gerrithub.io/c/eepp/babeltrace/+/452154 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com> Tested-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 1 Apr 2019 19:46:57 +0000 (15:46 -0400)]
Add env wrapper to facilitate importing the in-tree python bindings
The test_python_bt2 script sets up a special environment for running the
Python bindings tests. Crucially, it sets up PYTHONPATH so we are able
to "import bt2" from a Python interpreter. It can be difficult to
reproduce this environment exactly in an interactive shell, or when
trying to run a single test.
This patch introduces test_python_bt2_env, which can be used to run a
command in the same environment. This command can be anything from a
single test run to a sub-shell.
Also, we expect that many Babeltrace tests will test some non-Python
features using Python, for convenience. These tests will also need to
run in the right environment to be able to import the bt2 Python
package. Therefore, I think it makes sense to put it under tests/utils.
Once the bindings and their tests work better, we should make
the test_python_bt2 script use test_python_bt2_env, to avoid the duplication, but
for now I prefer to leave it as-is.
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Change-Id: I4e1655614756d638ef6bb9a4426719d4b754aa4b
Reviewed-on: https://review.gerrithub.io/c/eepp/babeltrace/+/452152 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com> Tested-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 22 Apr 2019 15:17:23 +0000 (11:17 -0400)]
bindings/python/bt2: Make the bt2 Python bindings build
This patch is just a little bit more than the bare minimum to make the
Python bindings build (that is, configure with --enable-python-bindings
and build). It consists of sync'ing the .i SWIG interface definition
files with the corresponding headers. It's possible that the interfaces
are incomplete, in which case they'll be adjusted in the a later pass.
For example, output parameters are currently not named in a way that
match the type maps.
For simplicity, I have included at the same time some other changes to
the .i files, such as adjusting the typemap definitions (in
native_bt.i). The helper functions (such as at the bottom of
native_bt_componentclass.i) are also changed to the "final" form they
have on my development branch. Even though these are not immediatly used,
they need to be changed for the bindings to build.
Signed-off-by: Simon Marchi <simon.marchi@efficios.com> Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I87a438a692690639f956b214f878df65885ed78c
Simon Marchi [Wed, 3 Apr 2019 16:08:11 +0000 (12:08 -0400)]
lib: Make bt_value_null a const pointer
We probably don't want anything to change the value of this pointer,
which points to the only instance of BT_VALUE_TYPE_NULL,
bt_value_null_instance. Therefore, declare it as a constant value.
It also helps a little bit the Python bindings: if we copy the
declaration verbatim to the .i file, without const, bt_value_null is
available through the special "cvar" global object (and can be
modified!). With const, it becomes read-only, so can be accessed
directly as native_bt.value_null.
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Simon Marchi [Thu, 25 Apr 2019 15:00:02 +0000 (11:00 -0400)]
ctf: Add dash to stream name suffixes
We add numerical suffixes to stream names, if needed, to make them
unique. For trace directories, we do something similar and decided to
format these suffixes with a dash to avoid confusion if the original
name ends with a number. This patch does the same thing but for stream
names.
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Change-Id: I3041fe98bbeb3c122d06e5fa54449ed6e55017bd
Reviewed-on: https://review.gerrithub.io/c/eepp/babeltrace/+/452110 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com> Tested-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Tue, 23 Apr 2019 13:32:58 +0000 (09:32 -0400)]
ctf: Use environment fields provided by LTTng to make output trace path
Starting with version 2.11, the LTTng tracer includes in the trace
environment all the information necessary to rebuild the output path
that it itself uses when outputting traces. This lets us recreate an
output path similar to what LTTng does when outputting ctf traces (with
the exception that we also add a level with the hostname, which LTTng
doesn't).
If the trace does not come from LTTng, or comes from LTTng < 2.11, we
keep using the same strategy: use a trace path based on the trace name,
which is itself based on the original directory.
This feature will especially be useful in conjunction with session
rotation: since the input trace is actually composed of multiple
traces/chunks, the trace name would otherwise be based on one particular
of these chunks. With the environment values, we can recreate a path
that describes correctly the trace as a whole. The output path is then
similar to what one would have been output by LTTng, if one had not used
the session rotation feature in the first place.
Change-Id: I03c0569869561ec11873b808c47d30196889cb36 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.gerrithub.io/c/eepp/babeltrace/+/451942 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com> Tested-by: Philippe Proulx <eeppeliteloop@gmail.com>
cli: mimic behavior of BT1 when trying to attach to live session
In Babeltrace 1, when trying to attach to a LTTng live session the
`babeltrace` command would exit with status 0 (SUCCESS) if it
successfully connected to the relay daemon _even if_ it did not find the
requested session.
To mimic the behavior of BT1, we set the `session-not-found-action`
parameter to "end". This makes the message iterator of an lttng-live
component return a END status if the requested session is not found
on the relay daemon and makes the cli return gracefully with SUCCESS.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I3b3810f97c090d102fd30902fda37434f62d5c01
Simon Marchi [Tue, 16 Apr 2019 20:16:01 +0000 (16:16 -0400)]
plugins/ctf/fs-src: Session rotation support: merge traces with same uuid
This patch makes the src.ctf.fs component merge traces that have the
same UUID, with the intent of supporting traces output by LTTng's
session rotation feature.
When using this feature of LTTng, the user ends up with multiple traces
that are independent (with their own metadata and data files), except
for the fact that they share the same trace uuid. The goal of this
patch is to make Babeltrace realize this and make all those traces
appear as a single trace.
The approach taken is to scan the search paths (passed as parameter to
the component) as usual and create a ctf_fs_trace for each found trace.
We then go over the list of traces in the component and merge those
having the same UUID.
Merging traces consists of merging corresponding data stream file groups
(ctf_fs_ds_file_group structs). Two ctf_fs_ds_file_group structs are
corresponding if they have the same stream_id (aka stream_instance_id in
CTF) and stream class id.
Merging two ctf_fs_ds_file_group structs consists of merging their list
of ctf_fs_ds_file_info structs, making sure to keep the resulting list
sorted by begin timestamp.
Since the trace-info query also uses create_ctf_fs_traces, its behavior
is updated as expected. Doing a trace-info on a directory containing
multiple traces with the same UUID will result in a single trace with
multiple files per stream. A little drawback/inconsistency that will be
address later is that the trace object in the output has a single
"path", which is set to the path of one of the original traces.
Ideally, it should have also have a list of paths and list all the
paths of the "physical" traces that form this "logical" trace.
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Change-Id: Ib290cf08119f9be4344057b2544b4627eea3f450
Simon Marchi [Tue, 23 Apr 2019 16:03:29 +0000 (12:03 -0400)]
cli, plugins/ctf/fs-src: Make src.ctf.fs accept multiple root paths
A following patch will make it possible to process multiple traces that
share the same UUID as a single trace. One requirement is to be able to
do this event when passing multiple arguments, like:
babeltrace path1 path2 ...
With the current code, this instantiates two src.ctf.fs components, and
therefore makes it impossible to do this merging by UUID.
This patch therefore changes the design so that we only instantiate a
single src.ctf.fs component and pass an array with the paths.
We also need to make the trace-info query support receiving multiple
paths. One reason is that in general, we want trace-info queries to
support the same parameters as the components. A more pragmatic reason
is that it will need to do the same kind of UUID-merging anyway. I
therefore made the trace-info query re-use more of the code used by the
component to locate and create the ctf_fs_trace list. One difference
between the two usages is that in the case of the query, we don't want
to create IR streams or component ports. I moved those outside of the
shared code path and made the component do that as a separate step.
One notable change in babeltrace.c is in set_stream_intersections.
We previously extracted the "path" parameter from the component creation
parameter, to make the query parameters. Instead, we now directly pass
the component creation parameters to the query. The idea is the same as
above, we want to pass the same parameters to the trace-info query as
what will be used to create the component, since it might alter how the
traces will be open, and therefore the results.
So in practice the changes boil down to:
- babeltrace-cfg-cli-args.c: Make it possible to serialize bt_value of
type BT_VALUE_TYPE_ARRAY, so we can build the list of paths in a
bt_value array, then serialize it to send it to "babeltrace run".
Make it instantiate a single ctf component, this even simplifies the
code a little bit.
- fs.c: Accept "paths" parameter with a list of paths, rather than
"path" with a single path, iterate on that list to find traces and
create trace objects just as before. Separate creation or IR streams
and component ports from the ctf_fs_trace creation.
- query.c: Re-use more of fs.c's functions to locate and create
ctf_fs_trace objects.
There are no functional changes intended, except if you are the kind of
person to pass params by hand to the src.ctf.fs component.
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Simon Marchi [Tue, 23 Apr 2019 15:16:00 +0000 (11:16 -0400)]
cli: Make append_parameter_to_args accept a bt_value value
append_parameter_to_args is used to append a --params=KEY=VALUE argument
to a list of arguments. It currently assumes that VALUE is a simple
string. We will want to generalize this to support arrays (in the form
of a bt_value of type array). This patch does a bit of refactoring in
preparation of that, making append_parameter_to_args accept a value of
type bt_value* instead of const char*.
append_string_parameter_to_args is introduced as a wrapper, for those
spots that still want to pass a simple string value. It converts the
string to a temporary bt_value of type string.
No functional changes intended.
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
projects / babeltrace.git / log
