Simon Marchi [Mon, 29 Jul 2019 15:42:58 +0000 (11:42 -0400)] 
cli: remove babeltrace2-log
babeltrace-log (now renamed to babeltrace2-log) is a legacy tool
provided by Babeltrace 1 used to transform `dmesg` output to CTF. The
version currently provided by Babeltrace 2 simply runs babeltrace2 with
a pre-defined graph topology. It does nothing that a user couldn't do
on the command line.
The equivalent now would be:
dmesg | babeltrace2 -c src.text.dmesg -o ctf -w /path/to/trace
It was therefore decided that it was no longer necessary to provide that
utility, and to remove it.
Change-Id: Id547bced56a8b9913cb152e1309919a15b6fbdd0
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1796
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sat, 27 Jul 2019 04:42:32 +0000 (00:42 -0400)]
cli: use argpar for parsing query command's arguments
Change-Id: Id53593e699e4b377ada5df46697e77e5a0e30721
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1794
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sat, 27 Jul 2019 04:23:58 +0000 (00:23 -0400)]
cli: use argpar for parsing list-plugin command's arguments
Change-Id: Ifc11c8652a23830cd5961e4df55a3a23817d1c95
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1793
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sat, 27 Jul 2019 03:37:22 +0000 (23:37 -0400)]
cli: use argpar for parsing help command's arguments
All straightforward, except that there is a functional change as part
of this patch. It was possible to pass multiple leftovers, and only the
first one was considered.
babeltrace2 help ctf tintin
did the same as
babeltrace2 help ctf
It now prints a beautiful error:
./src/cli/babeltrace2 help ctf tintin
ERROR: [Babeltrace CLI] (/home/smarchi/src/babeltrace/src/cli/babeltrace2.c:2564)
Command-line error: retcode=1
CAUSED BY [Babeltrace CLI] (/home/smarchi/src/babeltrace/src/cli/babeltrace2-cfg-cli-args.c:1461)
Extraneous command-line argument specified to `help` command: `tintin`.
Change-Id: I2c8f8610004d60a0844d18c9d622037cc7c1c04c
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1792
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sat, 27 Jul 2019 03:15:56 +0000 (23:15 -0400)]
cli: use argpar for parsing convert command's arguments
No functional changes intended, everything should be quite
straightforward.
Change-Id: I231c8f91aca6bef9327deee0c709b12ce5cea8db
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1791
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Fri, 26 Jul 2019 21:20:08 +0000 (17:20 -0400)]
cli: use argpar for parsing run command's arguments
A non-obvious change is the handling of the --retry-duration value.
Before this patch, it was automatically converted to a long by popt. We
now call g_ascii_strtoll and validate that it's parsable as an integer.
Also, introduce the help_option_is_specified function (which will be
used for other sub-commands as well). The goal is to check for --help
before everything else, such that doing:
$ babeltrace2 run hello --help
prints the help, rather than saying "Unexpected parameter `hello`".
Change-Id: Ib6873a2e2183e73f340cf25026924df298c8c1e9
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1790
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Wed, 24 Jul 2019 15:56:43 +0000 (11:56 -0400)]
cli: use argpar for top-level args
This patch replaces the hand-made parsing of the top-level arguments
with something based on the internal argpar library.
No functional changes expected from the point of view of the user.
One internal difference is that previously, the argc/argv passed to
sub-commands included an argv[0] that was ignored by popt. In the
subsequent patches, we replace popt with argpar, and argpar doesn't want
an argv[0] containing the name of the program. So instead of passing an
unnecessary argv[0], I have made it such that the argc/argv the
top-level passes to subcommands only represent the actual arguments. So
for example, with:
babeltrace2 convert -c src.ctf.fs
the bt_config_convert_from_args function used to receive this as argv:
["convert", "-c", "src.ctf.fs"]
With
babeltrace2 --debug -c src.ctf.fs
it used to receive
["--debug", "-c", "src.ctf.fs"]
With this patch, it will only receive `["-c", "src.ctf.fs"]`. So
functions bt_config_*_from_args are updated to cope with this
change. In particular, I passed the POPT_CONTEXT_KEEP_FIRST flag to
poptGetContext. But in practice, they will all disappear anyway, in
favor of argpar.
Change-Id: I9f1210f1c338c7eb39e228e20218c83e46961ee4
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1789
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Sun, 7 Jul 2019 03:15:58 +0000 (23:15 -0400)] 
Add internal command-line argument parser API
This patch adds an internal command-line argument parser API.
The exact API is well documented in `src/argpar/argpar.h`.
The features that I'm looking for are:
* C API.
* Support for ordered arguments in the results, even between options and
non-option arguments (sometimes called positional arguments).
* Support for GNU-style arguments, including "glued" short options and
long options with `=` to set the option's argument.
* Portable.
* Compatible license.
* No global variables (we're trying to avoid that programming style
throughout the project).
* Easy to use.
None of the popular libraries I looked at, including popt of course, met
all those specifications.
The goal of this is:
1. To simplify the parsing of general options (before the command name)
in bt_config_cli_args_create(). It is currently hard-coded.
This is not possible with popt because it would fail with an unknown
option, and you would not know the position of the command name's
argument within the array.
This is possible with g_option_context_parse().
2. To make it possible, in the `convert` command, to assign parameters
and other position-specific options to a non-option argument, for
example:
babeltrace2 /path/to/trace --params=some=param \
mein-other-trace --name=travel
This is not possible with popt as it collects all the non-option
arguments as an array of leftover arguments.
This is also not possible with g_option_context_parse() for the same
reasons as with popt.
getopt_long() could satisfy both 1. and 2., but it's somewhat a pain to
use and to maintain the available options, as you need to specify the
long options in a table and the equivalent short options as a single
string with a special encoding (the `convert` command's option string
would be particularily ugly). Also: getopt_long() plays a lot with
global variables; it's not thread-safe: the parser's state is global.
Also: the upstream getopt_long()'s (glibc) license is LGPL, which is
more restrictive than our MIT license, so I think we want to avoid that.
I believe having our own (tested) CLI argument parser is beneficial,
especially in the long term: we can drop a direct dependency (and popt
is getting old), not introduce a new one, it's about 450 lines of
documented C code, and if we ever need something very specific in the
argument parsing strategy in the future, we can add it directly. As a
reference, FFmpeg, a project which has complex argument parsing, similar
to Babeltrace's `convert` and `run` commands, has its own argument
parser (see parse_options() in [1]).
Only the CLI will use the bt_argpar_*() API, but to make the unit tests
simpler, I put it in its own convenience library.
[1]: https://github.com/FFmpeg/FFmpeg/blob/
b7b6ddd59693008c35b3247496ecc946331d0856/fftools/cmdutils.c
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Iff4fc305b9e9171c694e1e79428bd3838ddd989d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1646
CI-Build: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Wed, 31 Jul 2019 13:30:15 +0000 (09:30 -0400)] 
bt2: stream.py: add `trace` property to _Stream
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I434ebf2dd9c5a29bb8ef15ef7058479eba5ff9f6
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1807
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Tue, 30 Jul 2019 17:16:31 +0000 (13:16 -0400)] 
`ctf` plugins: port: cast to intmax_t all off_t variables when printing
This patch fixes new sites since the following commit:
commit
1974687e6b7a08d8383a4a5c75265d0ed3b8c5c9
Author: Michael Jeanson <mjeanson@efficios.com>
Date: Mon Jul 10 12:39:38 2017 -0400
Port: handle 'size_t' / 'off_t' on Solaris
The size of 'size_t' / 'off_t' vary on some platforms, use C99 portable
macros and print formats to handle them.
Print 'size_t' with '%zu' and use SIZE_MAX as max value.
Print 'off_t' casted to 'intmax_t' with '%jd'.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: I8c0677f5ff590c25df6257c2d3f5142de93d01b0
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1803
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Mon, 29 Jul 2019 19:32:08 +0000 (15:32 -0400)]
configure.ac: refine Python 3 interpreter and dev. libraries detection
This patch refines how the `configure` script detects the Python 3
interpreter and development libraries.
The goal is to clean up this part of `configure.ac` and to make the
Python 3 interpreter available as a general tool for testing, even
without building the Python bindings or the Python plugin provider.
Before this patch, `configure` ensures that the Python interpreter and
the development libraries exist and are valid if you use the
`--enable-python-bindings` or `--enable-python-plugins` option.
With this patch, `configure` always tries to find the Python
interpreter. If it finds it, it sets the local `have_python` shell
variable and defines the `HAVE_PYTHON` Automake definition.
If `configure` finds the Python interpreter, it always tries to find
`python-config`. It if finds it, it sets the local `have_python_dev`
shell variable and defines the `HAVE_PYTHON_DEV` Automake definition.
You can still override the include directories and linker flags with the
`PYTHON_INCLUDE` and `PYTHON_LDFLAGS` environment variables.
Now, if you specify the `--enable-python-bindings` or
`--enable-python-plugins` option, `$have_python_dev` must be `yes`.
What used to be `HAVE_PYTHON` (Automake definition) is renamed to
`ENABLE_PYTHON_COMMON_DEPS`: this is defined if we need to build any
common Python dependency. As of this patch, the only one is the internal
`py-common` convenience library.
In the final `configure` report, I added:
* Whether or not the Python interpreter was found and its path.
* Whether or not the `python-config` program (called "development
libraries") was found and its path.
This patch was only tested manually, locally removing the Python
interpreter, the `python-config` program, and passing the
`--enable-python-bindings` or `--enable-python-plugins` option to test
the different `configure` paths.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Id0ac60ebc97a9ce2951aee713eccdc24c19d3791
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1799
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Philippe Proulx [Mon, 29 Jul 2019 16:25:42 +0000 (12:25 -0400)]
src.ctf.lttng-live: live_..._list_sessions(): remove handshake
The only user of live_viewer_connection_list_sessions() is
lttng_live_query_list_sessions() which calls
live_viewer_connection_create() before. live_viewer_connection_create()
calls lttng_live_connect_viewer() which calls lttng_live_handshake().
There is no need to call lttng_live_handshake() twice. This makes the
relay daemon assign two viewer IDs to the CLI, as seen in Wireshark.
Babeltrace 1.5 does not send the "connect" command twice either.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I640b9397c21ca37ad5c841415b5ed0c12193825a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1798
Philippe Proulx [Mon, 29 Jul 2019 14:06:58 +0000 (10:06 -0400)]
Fix: live_viewer_connection_create(): `lttng_live_msg_iter` can be `NULL`
Because `lttng_live_msg_iter` can be `NULL` in
live_viewer_connection_create() when it's used from a query operation,
pass the log level to use directly and only read `self_comp` if
`lttng_live_msg_iter` exists.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ied5ec621bcd066df0bb312a6f81899616f9d1a9f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1797
Philippe Proulx [Mon, 29 Jul 2019 19:52:48 +0000 (15:52 -0400)]
Fix: tests/utils/utils.sh: export `BT_TESTS_PYTHON_CONFIG_BIN`
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I7e07fb028df066b130b1932c46d0c91d474172bf
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1800
CI-Build: Michael Jeanson <mjeanson@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Jonathan Rajotte [Tue, 23 Jul 2019 21:58:29 +0000 (17:58 -0400)]
`ctf` plugin: Use CTF_BYTE_ORDER_UNKNOWN in place of -1
Seen on clang-3.9, 4.0 and clang-1001.0.46.4.
Comparison of constant -1 with expression of type 'enum *****'
is always false [-Wtautological-constant-out-of-range-compare]
Note that the enum underlying type is implementation defined and left to
the choice of the compiler by the standard [1] (6.7.2.2 4). Most
compiler default to unsigned int. The use of -1 is not a problem per see
since wrap around of unsigned int behaviour is not undefined. Using -1
is the equivalent of assigning UINT_MAX here. This warning was removed
for later clang for these specific cases since the effect of always being
false is erroneous.
Still, it make more sense to use a specific enum for such case instead
of relying on the compiler type.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I5dcbaa2066cbab8c429ff54e8d8b8b295fc8baf1
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1764
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Mon, 29 Jul 2019 16:30:17 +0000 (12:30 -0400)] 
tests: fix test_auto_source_discovery on mingw
Add native path support and use the shared diff functions to properly
handle end of line characters.
Change-Id: Ie36e60b41e03a242161d1734f8cf9dadaef56a36
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1802
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Jonathan Rajotte [Tue, 23 Jul 2019 19:36:30 +0000 (15:36 -0400)]
`ctf` plugin: Use CTF_SCOPE_PACKET_UNKNOWN in place of -1
Seen on clang-3.9, 4.0 and clang-1001.0.46.4.
Comparison of constant -1 with expression of type 'enum *****'
is always false [-Wtautological-constant-out-of-range-compare]
Note that the enum underlying type is implementation defined and left to
the choice of the compiler by the standard [1] (6.7.2.2 4). Most
compiler default to unsigned int. The use of -1 is not a problem per see
since wrap around of unsigned int behaviour is not undefined. Using -1
is the equivalent of assigning UINT_MAX here. This warning was removed
for later clang for these specific cases since the effect of always being
false is erroneous.
Still, it make more sense to use a specific enum for such case instead
of relying on the compiler type.
get_root_scope_from_absolute_pathstr depends on the fact that
CTF_SCOPE_PACKET_HEADER is 0 to work.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: Ie13f6c4dbab5038eb7a7fea6d58ce4451bc551d1
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1762
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Wed, 24 Jul 2019 19:47:09 +0000 (15:47 -0400)]
lib: Explicit cast to bt_plugin_find_status
Clang 8.0:
error: implicit conversion from enumeration type 'bt_plugin_find_all_status' (aka 'enum bt_plugin_find_all_status') to different enumeration type 'enum bt_plugin_find_status' [-Werror,-Wenum-conversion
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: Ie8304eccaa329b953837eb24e5fe4c2b2b81f320
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1765
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Jonathan Rajotte [Tue, 23 Jul 2019 19:55:12 +0000 (15:55 -0400)]
src.ctf.lttng-live: Use SESSION_NOT_FOUND_ACTION_UNKNOWN in place of -1
Seen on clang-3.9, 4.0 and clang-1001.0.46.4.
Comparison of constant -1 with expression of type 'enum *****'
is always false [-Wtautological-constant-out-of-range-compare]
Note that the enum underlying type is implementation defined and left to
the choice of the compiler by the standard [1] (6.7.2.2 4). Most
compiler default to unsigned int. The use of -1 is not a problem per see
since wrap around of unsigned int behaviour is not undefined. Using -1
is the equivalent of assigning UINT_MAX here. This warning was removed
for later clang for these specific cases since the effect of always being
false is erroneous.
Still, it make more sense to use a specific enum for such case instead
of relying on the compiler type.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I447d04333ae104ea953437b095053f341f87bbb7
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1763
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Tue, 23 Jul 2019 19:23:02 +0000 (15:23 -0400)]
`ctf` plugin: Use is_log_level_set to infer ctf_event_class log_level validity
Seen on clang-3.9, 4.0 and clang-1001.0.46.4.
Comparison of constant -1 with expression of type 'enum *****'
is always false [-Wtautological-constant-out-of-range-compare]
Note that the enum underlying type is implementation defined and left to
the choice of the compiler by the standard [1] (6.7.2.2 4). Most
compiler default to unsigned int. The use of -1 is not a problem per see
since wrap around of unsigned int behaviour is not undefined. Using -1
is the equivalent of assigning UINT_MAX here. This warning was removed
for later clang for these specific cases since the effect of always
being false is erroneous.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I6ce97c2481f36573d1a221b59e2459f1fe35ded9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1761
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Wed, 24 Jul 2019 19:50:39 +0000 (15:50 -0400)]
sink.ctf.fs: remove unusued _is_variant_field_class_tag_valid()
Clang 8.0:
translate-trace-ir-to-ctf-ir.c:721:6: error: unused function '_is_variant_field_class_tag_valid' [-Werror,-Wunused-function]
bool _is_variant_field_class_tag_valid(
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I3794701f857a34d672cc7d297fd7bb7eb5ac31ad
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1766
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Thu, 25 Jul 2019 22:07:59 +0000 (18:07 -0400)]
lib: prepare the ground for stateful query operations
This patch changes the library (and everything which depends on it) so
that it is possible to make query operations stateful in the future if
needed.
Without this patch, a query operation can return the
`BT_COMPONENT_CLASS_QUERY_METHOD_STATUS_AGAIN` status. The query
executor's user can then retry the operation until it works, or abandon.
However, there's no way for this query function to keep state between
calls, so the process starts over every time whereas it could have
progressed otherwise (for example, a temporary unresponsive network).
Library changes
===============
This patch does two things to address the limitation above:
* It makes bt_query_executor_create() accept the component class, query
object, and query parameters. It also removes those parameters, as
well as the logging level, from bt_query_executor_query().
This ensures that a given query executor is dedicated to a specific
query operation. Once you create a query executor, you cannot change
the target component class, query object, and query parameters
afterwards.
The initial query executor's logging level is `BT_LOGGING_LEVEL_NONE`.
You can set it before you call bt_query_executor_query() with
bt_query_executor_set_logging_level().
Internally, the query executor keeps a reference on the component
class, a copy of the query object, and a reference on the query
parameters.
* It makes a query method receive a private query executor
(`bt_private_query_executor *`) object, and not receive the logging
level.
A private query executor is a private view of the query executor. This
is where we can eventually add bt_private_query_executor_set_data()
and bt_private_query_executor_get_data() to make the query operation
stateful. From the query method's point of view, the private query
executor is borrowed: there are no bt_private_query_executor_get_ref()
and bt_private_query_executor_put_ref() functions.
You can go from a private query executor to a constant query executor
with bt_private_query_executor_as_query_executor_const(). This is how
you can get the query executor's current logging level with
bt_query_executor_get_logging_level().
To keep the changes minimal, query methods still receive the object
and parameters every time. They are guaranteed, however, that for the
same query executor, they remain unchanged. This guarantee is not
relevant now anyway as there's no way to set and get private data.
Python bindings changes
=======================
* The new `_QueryExecutorCommon` class contains the common properties of
a query executor and a private query executor (`is_interrupted` and
`logging_level`). It requires its subclasses to implement the
_as_query_executor_ptr() method to get the query executor pointer.
* `QueryExecutor` inherits `_QueryExecutorCommon`.
You now build a query executor with the component class, object, and
(optional) query parameters:
query_exec = bt2.QueryExecutor(MySrc, 'avail-servers',
{'blacklist': ['node67']})
You now call the query() method without parameters:
query_exec.query()
You can set the logging level with:
query_exec.logging_level = bt2.LoggingLevel.TRACE
* The new `_PrivateQueryExecutor` class inherits `_QueryExecutorCommon`.
An instance of `_PrivateQueryExecutor` is passed to the query method
during a query operation.
Once the user's query method returns, _bt_query_from_native()
invalidates the private query executor in case the user kept a
reference somewhere. Calling any method of an invalid private query
executor raises `RuntimeError`.
CLI changes
===========
To centralize how query operations are performed in the CLI, all the
sites which need to query use the new cli_query(), implemented in
`src/cli/babeltrace2-query.c`.
cli_query() is similar to the static query() function in
`babeltrace2.c`, which already existed, but it accepts the logging level
directly and returns `bt_query_executor_query_status`.
The static query() function still exists, but calls cli_query() now.
The automatic source discovery feature now uses cli_query() instead of
creating its own query executor.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I3c254325817c2c0091b6c4f0030bc7020443b8e0
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1787
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Fri, 26 Jul 2019 18:56:31 +0000 (14:56 -0400)]
bt2: honor self component or query log level when logging
This patch makes the Python bindings native code honor the self
component or query log level in log statements when possible.
The locations where this is not possible are:
* Registration of a BT component class pointer to a Python component
class in global hash table.
* Lookup of a BT component class in global hash table.
* Module's destructor.
* Creation of a BT component class from the `_UserComponentType`
metaclass.
* Trace and trace class destruction listeners.
* Graph's port added and ports connected listeners.
All those sites still use the module's
`bt_python_bindings_bt2_log_level` hidden symbol which is initialized
from the `BABELTRACE_PYTHON_BT2_LOG_LEVEL` environment variable at
construction time.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I570f396c1d552b035e48269ce1f84e2e3044055c
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1786
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Fri, 26 Jul 2019 18:54:49 +0000 (14:54 -0400)]
Move `src/plugins/comp-logging.h` -> `src/logging/comp-logging.h`
Move this file so that other parts of the project than plugins, namely
Python bindings, can use it.
Also: in `comp-logging.h`, the requirement that `BT_LOG_OUTPUT_LEVEL`
and `BT_COMP_LOG_SELF_COMP` are set is removed because they are not
needed to use the generic BT_COMP_LOG_CUR_LVL() macro.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ic961c3a97863dcef63fb7ee673841650cf9ca9e9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1788
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Thu, 25 Jul 2019 18:02:58 +0000 (14:02 -0400)]
tests/bindings/python/bt2: remove unneeded `import` lines
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I05bb962fff357f362ee5d965dfc27d2eb5d58220
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1785
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Thu, 25 Jul 2019 18:00:33 +0000 (14:00 -0400)]
Test `bt2` public names (`test_package.py`)
This new test checks that the `bt2` package directly contains all the
names we want it to publicly expose.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I141b688622be7619a0b6b4e85f30845922864694
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1784
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Thu, 25 Jul 2019 16:47:00 +0000 (12:47 -0400)]
Add `babeltrace.` prefix to `trace-info` and `support-info` query objects
The `trace-info` and `support-info` query objects are supported by the
project's plugins, the project's Python bindings, and the project's CLI.
To avoid name clashes with third-party query objects, prepend
`babeltrace.` to the object names as a namespace.
The dot notation is not standardized in any way here: it is an arbitrary
format that the project chooses. We don't necessarily expect third-party
developers to follow this scheme.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I6aec9bf24f7a8a41796c3a1e84d982c99be6aee5
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1783
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Thu, 25 Jul 2019 16:42:00 +0000 (12:42 -0400)]
lib: standardize listener ID types with new `bt_listener_id` type
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ief13a1409b8f0038c0f035878fb49cfe6cf10207
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1782
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Simon Marchi [Wed, 24 Jul 2019 16:11:01 +0000 (12:11 -0400)]
lib: rename INVALID_OBJECT status to UNKNOWN_OBJECT
Rename INVALID_OBJECT to UNKNOWN_OBJECT, because it represents better
the situation: the passed object is not necessarily invalid (any string
is a valid object), it's just that the component class doesn't know that
object.
Change its numerical value to something positive, because it should not
be considered as an error (which negative numerical values represent).
A function returning UNKNOWN_OBJECT should not append an error cause.
Also, in the Python bindings, change the default implementation of
_query to raise bt2.UnknownObject instead of NotImplementedError.
NotImplementedError would result in an ERROR status code, whereas
bt2.UnknownObject becomes UNKNOWN_OBJECT. This matches the behavior of
a component class implemented in C, where if it doesn't provide a query
method, it will automatically return UNKNOWN_OBJECT.
Change-Id: Ica1418272b5bf2bbe8e96d639c16f56191151d79
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1760
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Wed, 24 Jul 2019 18:54:15 +0000 (14:54 -0400)]
bt2: move __version__ to version.py.in
The only bit of __init__.py that is set at configure time is the package
version. Move it to its own file, version.py.in, and make __init__.py
not generated. This will make life a bit easier when we need to modify
__init__.py.
Change-Id: Ibc9fb67a897b226c46b4fddb3af7c2219d1f94e9
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1759
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Wed, 24 Jul 2019 21:57:01 +0000 (17:57 -0400)]
*_string() enumerator to string functions: remove common prefix
Those functions are used for logging and error cause messages. Removing
this prefix makes the output less noisy. The logging statement typically
indicates what field is logged anyway, for example:
Destroying event message: addr=0x4f6c580, type=EVENT
Destroying port: addr=0x4f30c20, type=INPUT
Destroying structure field class object: addr=0x4eb2e30, type=STRUCTURE
Finalizing error cause: addr=0x4daa6f0, actor-type=UNKNOWN
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ieb0bda9287bafbd06e2aa0f077d5d631213e5803
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1780
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 21:42:25 +0000 (17:42 -0400)]
lib: rename `bt_scope` -> `bt_field_path_scope`
Also rename `bt2.Scope` to `bt2.FieldPathScope`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ieb4c13a336234bfc30223a4afed4e521472cefe3
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1779
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Thu, 25 Jul 2019 14:09:11 +0000 (10:09 -0400)]
cli: fix "permission denied" test after g_dir_open
When changing from opendir/readdir to g_dir_open/g_dir_read_name, I
forgot this check from reading errno to reading the error output
variable.
Change-Id: I95e22531c20ec90df0bc517bbf666f5698cddd15
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1781
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Wed, 24 Jul 2019 21:36:29 +0000 (17:36 -0400)]
tests/Makefile.am: add `test_auto_source_discovery` to suite
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I1ca07f16897524da7f93cc58fc8faad84d007ebf
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1778
CI-Build: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 21:27:22 +0000 (17:27 -0400)]
bt2: prepend `_user` to overridable protected methods
It is possible that, in the future, we add new, optional overridable
protected methods that the user can implement within a component class
or message iterator class to have new features.
There is the risk, however, that the user class already has a method
named as such. This would be considered a backward compatibility break
because the same user class with two different versions of Babeltrace 2
would not work the same way (and with the newest version, it probably
would not work at all).
To prevent this, this patch adds the `_user` namespace to all the
overridable protected methods. It will be documented that you cannot,
when you inherit one of the `bt2._User*` classes, have a method named
`_user_*()`, as this prefix is reserved for the future within the `bt2`
package.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I0c5ac29a21a83d6f644d0bb696f71d687ecfebae
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1777
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 21:07:31 +0000 (17:07 -0400)]
bt2: make `bt2._OverflowError` inherit `bt2._Error`
`bt2._OverflowError` represents the `__BT_FUNC_STATUS_OVERFLOW_ERROR`
status code, which is an error. The library appends an error cause when
returning this status, so `bt2._OverflowError` inherits `bt2._Error`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Iaf59bd604f595e8b57d46d7edec57b462e45416e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1776
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 21:05:22 +0000 (17:05 -0400)]
bt2: _init_and_register_exit(): remove unused `version` variable
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I7b5b14ec60b3ac20fa0cdfb422f909d506c8361a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1775
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 20:36:30 +0000 (16:36 -0400)]
bt2: clean available `bt2` package names
This patch makes the `bt2` package contain exactly the names that we
want to expose. When you do:
import bt2
dir(bt2)
the list, except for the standard names added by Python itself, contains
exactly what is meant to be used by the user. More specifically, it does
not contain the names of our internal modules, so that, for example,
`bt2.clock_class` is not available.
In `__init__.py`, doing
from bt2.field import _EnumerationField
binds the `field` name in the package to the `bt2.field` module [1]:
> When a submodule is loaded using any mechanism (e.g. importlib APIs,
> the import or import-from statements, or built-in __import__()) a
> binding is placed in the parent module’s namespace to the submodule
> object.
All the `from bt2.xyz import Something` lines eventually bind some or
all module names in the `bt2` package, so `__init__.py` explicitly tries
to delete the module names just after. `__init__.py` also deletes the
`_init_and_register_exit` name which is private and used a single time
to initialize the native part.
Internally, within a module file, because the module names are not
available anymore in the `bt2` package, we cannot do
import bt2.clock_class
as `clock_class` is not a name within `bt2`. This is why all those
private imports are replaced with, for example:
from bt2 import clock_class as bt2_clock_class
Now `bt2_clock_class` is a name bound to the `bt2.clock_class` module.
Note that we cannot use the
import bt2.clock_class as bt2_clock_class
because this leads to circular import issues with Python < 3.7 [2]. The
`from bt2 import clock_class` form does not create circular imports.
Doing this change in `__init__.py` revealed a lot of locations where
`bt2.some_module.SomeName` was used without explicitly importing
`bt2.some_module`. They are all fixed now.
[1]: https://docs.python.org/3/reference/import.html#submodules
[2]: https://bugs.python.org/issue30024
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ib7d2b3ef3283ace19cbcb811d537a63b34690ada
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1774
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 20:35:21 +0000 (16:35 -0400)]
test_error.py: remove dangling print()
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I4e48e3acb8f17620ea9987b94c741b491f488ad0
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1773
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 19:03:03 +0000 (15:03 -0400)]
bt2: move `_ListenerHandle` to `utils.py`
`_ListenerHandle` is not a name the package's user needs directly, so
move it to the internal `utils.py` instead of having it available as
`bt2._ListenerHandle`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I11b4476665b358d76379d73d8706dc6146d4b006
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1772
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 18:58:31 +0000 (14:58 -0400)]
bt2: prepend `_` to names of exception classes the user cannot call
`bt2._OverflowError` and `bt2._IncompleteUserClass` are classes the
package's user cannot instantiate, so this patch adds the typical `_`
prefix to indicate that.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I42cb22311dac6d36dd7e57be86cc71d27ebab6a2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1771
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 18:48:05 +0000 (14:48 -0400)]
lib: rename all `*_STATUS_OVERFLOW` -> `*_STATUS_OVERFLOW_ERROR`
All the error statuses must have the `_ERROR` suffix.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I2243c77ecb9723f3c97f7f9f9af09df646d48ae3
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1770
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 18:43:49 +0000 (14:43 -0400)]
lib: append error cause when returning `BT_FUNC_STATUS_OVERFLOW`
`BT_FUNC_STATUS_OVERFLOW` is an error status, so append an error cause
to make details available.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I1524a088698a7b8bf240e8b1206bf06d4b21870f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1769
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 18:31:59 +0000 (14:31 -0400)]
bt2: import public names into `__init__.py`
This patch updates `__init__.py.in` so that it imports exactly the names
which we want to make public (available directly in the `bt2` package).
Most public names which start with `_` exist to be able to test if a
given object is an instance of that type, for example:
if isinstance(val, bt2._IntegerValue):
# do something with `val` knowing it's an integer value
`bt2._Error` exists to catch it:
try:
# ...
except bt2._Error as exc:
# ...
`bt2._User*` names exist to inherit them.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I67c7bed00bdffcd0b1fe63603820ecbe2867ec7a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1768
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 18:04:07 +0000 (14:04 -0400)]
bt2: rename `_Generic*ComponentClass` -> `_*ComponentClass`
Those are public names, in that they exist within the `bt2` package, so
those simpler names feel more natural.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I9f762e44a8cb1560194c33a3f0be17eb3475f97c
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1767
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Wed, 24 Jul 2019 03:54:13 +0000 (23:54 -0400)]
lib: remove INVALID_PARAMS status
This patch removes the INVALID_PARAMS status. It is currently meant to
be used by component classes in query, if they don't get the parameters
they expect (mandatory parameter not present, wrong type, wrong value).
Since we now have a framework to describe errors precisely, this can be
accomplished more simply by returning ERROR and appending an error
cause.
So this patch removes INVALID_PARAMS, replacing all of its uses with
ERROR. It also remove other traces of that status, including in the
Python bindings.
- In lttng-live.c, we were logging a warning when getting invalid
parameters. Since we are now returning an error, I think it makes sense
to log an error. I also think it's something that deserves to be an
error anyway.
- Because queries now return ERROR without appending an error cause and
the library's bt_query_executor_query function doesn't append either, we
get in the case where we construct a bt2._Error and don't have an error
for the current thread. I added some error cause appending in
bt_query_executor_query to fix that, which I think is a valid
improvement on its own. Component classes should ideally also append an
error cause when they return ERROR, but that is left as an exercise for
later.
- The test_query_invalid_params test is removed. Changing raise
bt2.InvalidParams to raise ValueError would just make the test redundant
with test_query_gen_error.
Change-Id: I8c9277aa5e55bb8cde9b23a3aedf217cbe6a5849
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1756
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Wed, 24 Jul 2019 14:25:17 +0000 (10:25 -0400)]
cli: remove the global volatile `interrupted` variable
We can use the global interrupter's state instead, and we don't need to
check the interruption status before calling bt_query_executor_query()
and bt_graph_run() as both functions check their interruption state
initially.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I3818843c44b016cf2b872d879b461eae52730e6b
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1758
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 24 Jul 2019 14:25:05 +0000 (10:25 -0400)]
lib: bt_query_executor_query(): check interruption state before querying
This patch makes bt_query_executor_query() check its interruption state
(with bt_query_executor_is_interrupted()) before it calls the user
method. This is just a convenience so that the user does not need to
check bt_query_executor_is_interrupted() for every
bt_query_executor_query() call.
The comment `query-executor.c` explains why the function returns
`BT_QUERY_EXECUTOR_QUERY_STATUS_AGAIN` in that case.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I3bfceed538601d83c1ab627f5241c59f9f2e7c49
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1757
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sat, 20 Jul 2019 17:48:17 +0000 (13:48 -0400)]
lib: add bt_plugin_find_all()
This patch adds the public bt_plugin_find_all() function to find all the
plugins found in directories with the standard search order.
The motivation of this patch is to use bt_plugin_find_all() in the CLI
instead of duplicating the search algorithm (see load_all_plugins() in
`babeltrace2-plugins.c`).
bt_plugin_find_all() returns a plugin set. It takes four boolean
parameters to indicate whether or not it must search in:
* The directories specified by the standard environment variable
(`BABELTRACE_PLUGIN_PATH`).
* The system's plugin directory.
* The user's plugin directory.
* The built-in/static plugins.
All the plugins within a plugin set have unique names. When a plugin is
found in bt_plugin_find_all() and the output plugin set already has a
plugin with this name, it is discarded.
bt_plugin_find() is changed to use bt_plugin_find_all(). It also gains
the four boolean parameters to exclude a specific step of the search
algorithm.
In the `bt2` Python package:
* bt2.find_plugins() is renamed to bt2.find_plugins_in_path().
* bt2.find_plugins() wraps bt_plugin_find_all().
* Optional boolean parameters are added to bt2.find_plugin() to fully
wrap bt_plugin_find().
In `tests/lib/plugin.c`, `BT_FALSE` is passed to all the directory
exclusion parameters except for `find_in_std_env_var` to isolate the
search within those specific directories, without the system's or user's
plugin directory having an influence on the test.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Icad1387d64d15a639da107ab1ba30e5a53136ced
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1729
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sun, 21 Jul 2019 01:30:30 +0000 (21:30 -0400)]
lib: add bt_{graph,query_executor}_add_interrupter()
This patch makes it possible to add interrupter objects to a graph or to
a query executor.
To do this, you need to create an interrupter object
(bt_interrupter_create()), add it to a graph or to a query executor with
the new bt_graph_add_interrupter() and
bt_query_executor_add_interrupter() functions. Then you can interrupt
the object with bt_interrupter_set().
Within the whole project, the "cancel" terminology is renamed to
"interrupt", as many times it is possible to resume an interrupted
operation. For example, if you interrupt a running graph and
bt_graph_run() returns `BT_GRAPH_RUN_STATUS_AGAIN`, then you can resume
the graph's operation by calling bt_graph_run() again.
A graph or a query executor is deemed interrupted when any of its
interrupters is set. This makes it possible, for example, to add both a
global interrupter and a thread-specific interrupter to a graph object
so that either the thread-specific action or a global signal handler can
interrupt the graph.
As a convenience, bt_graph_interrupt() (which was bt_graph_cancel()) and
bt_query_executor_interrupt() (which was bt_query_executor_cancel()) are
kept: they set a default interrupter which is always part of the
object's interrupter set.
Conceptually, when a graph runs, each execution context is considered
independently interruptible:
* The execution of a message iterator.
* The execution of a sink component.
* The execution of the bt_graph_run() loop itself.
bt_graph_add_interrupter() conceptually adds the given interrupter to:
* All the graph's message iterators, current and future.
* All the graph's sink components.
* Itself, for bt_graph_run().
This is needed because message iterators typically won't check if they
are interrupted without performing "long" tasks. Because we cannot
guarantee that, bt_graph_run() checks if it's interrupted for each
sink consuming iteration.
This is why bt_graph_is_canceled() and bt_component_graph_is_canceled()
do not exist anymore, while the new
bt_self_message_iterator_is_interrupted() and
bt_self_component_sink_is_interrupted() functions are introduced.
However, because we don't need per-message iterator or per-sink
component interruption yet, bt_self_message_iterator_is_interrupted()
and bt_self_component_sink_is_interrupted() simply check their graph's
interrupters directly. The two functions exist so that, in the future,
we can add interrupters to a single message iterator (and its upstream
message iterators) or to a single sink component if needed.
As of this patch, you cannot remove an interrupter from a graph or from
a query executor as the project does not need this currently.
A message iterator, a sink component, a graph, or a query executor never
returns an "interrupted" status. It is the job of the actor which checks
the interruption status to return an appropriate status:
* Whenever possible, returning an "again" status is the cleanest
approach: this indicates that the operation can resume later, even if
it was interrupted.
* Otherwise, return an error status.
The distinction between a "real" "again" status and an "again" status
caused by an interruption is easy to make from the side controlling the
interrupter: check the interrupter's state to discover it. For example
(Python):
while True:
try:
my_graph.run()
except bt2.TryAgain:
if my_interrupter:
print('interrupted by user')
sys.exit(1)
else:
time.sleep(.1)
The same should be checked when getting an error, as an error can be the
consequence of an interruption (this is what `src.ctf.lttng-live` does
currently as it cannot safely interrupt some network interchanges and
resume them later).
The CLI is changed to have a single, global interrupter object. This
interrupter is added to any query executor or graph. The `SIGINT` signal
handler sets this interrupter. When getting an "again" or an error
status, the CLI checks the interrupter's state to discover if this is
the result of a user action, for example:
case BT_GRAPH_RUN_STATUS_AGAIN:
if (bt_interrupter_is_set(the_interrupter)) {
BT_CLI_LOGW_APPEND_CAUSE("Graph was interrupted by user.");
goto error;
}
/* ... */
In the `bt2` Python package:
* The `bt2.Canceled` exception is removed as it's not needed anymore.
The tests are adapted to demonstrate that.
* Graph.cancel() is changed to Graph.interrupt().
* Graph.is_canceled() is removed.
* There's a new Graph.add_interrupter() method.
* QueryExecutor.cancel() is changed to QueryExecutor.interrupt().
* There's a new QueryExecutor.add_interrupter() method.
* There's a new protected _UserMessageIterator._is_interrupted() method.
* There's a new protected _UserSinkComponent._is_interrupted() method.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I4d6631d39b585bd440457e7fea53a939ec9aaf3a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1735
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sat, 20 Jul 2019 22:47:17 +0000 (18:47 -0400)]
bt2: add interrupter support
This patch wraps the library's interrupter API within the `bt2` Python
package.
Tests are added to verify the new API.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I4d81759d1eaeac0b8036b8cbe9a4fd46ec57cbf1
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1734
Philippe Proulx [Sat, 20 Jul 2019 22:20:05 +0000 (18:20 -0400)]
lib: add interrupter API
This patch adds an interrupter API to the library.
An interrupter is a very simple shared object which you can set and
reset. On the const side, you can get whether or not the interrupter is
set.
The goal of this object is to be used as an async-signal-safe way to
interrupt a graph or a query executor. An interrupter object can
eventually be shared by many message iterators, interrupting all of them
at once when setting the interrupter. Moreover, a single message
iterator can have many interrupters, making it possible to interrupt one
or more specific message iterators from different interruption sources
(signal, graphical user interface input, thread-specific, etc.).
The user code is still responsible for checking the message iterator's
or sink component's interrupter's state at regular interval when
performing "long" operations.
You can create an interrupter object with bt_interrupter_create(). The
interrupter object is not set at creation time. You can set it with
bt_interrupter_set() and reset it with bt_interrupter_reset().
You can get whether or not the interrupter is set with
bt_interrupter_is_set().
This patch only adds the API: interrupter objects are not used anywhere
yet.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I610d48b70412d752b34b8d5760adf99a68d20704
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1733
Philippe Proulx [Sat, 20 Jul 2019 21:16:24 +0000 (17:16 -0400)]
lib: create input port msg iterator from self {msg iterator, sink comp.}
This patch makes an input port message iterator have a link to the
upstream message iterators it needs and vice versa.
The motivation behind this is to eventually be able to transmit some
state or property automatically to all the upstream message iterators,
recursively, of a given message iterator without any special user code.
For example, consider this graph of message iterators (arrow means "is
an upstream message iterator of"):
A <──┐
├──┬── C <──┬── F
B <──┘ │ │
│ E <──┘
D <─────┘
G <───── H
Here, setting the state/property on F would also set it on A, B, D, C,
and E. Setting it on C would set it on A, B, and D. Setting it on H
would only set it on G.
The message iterator graph, which can be considered like another,
"dynamic" graph on top of the static component graph used only to limit
a message iterator graph's topology, is always directed and acyclic,
where each node has a single parent. In other words, it is not permitted
that two message iterators use the same downstream message iterator, for
example:
A <──┐
├───── C <───── F
B <──┘ │ │
│ │
D <─────────┴────────┘
(D has both C and F as parents).
This is so that each message iterator and its upstream message iterators
constitute a single, private state.
This patch replaces
bt_self_component_port_input_message_iterator_create() with
bt_self_component_port_input_message_iterator_create_from_message_iterator()
and
bt_self_component_port_input_message_iterator_create_from_sink_component().
bt_self_component_port_input_message_iterator_create_from_sink_component()
does nothing with the self sink component parameter (except checking
that it's not `NULL`), but it's needed to make the functions type safe.
Internally, an input port message iterator contains both a weak pointer
to its downstream message iterator and an array of weak pointers to its
upstream message iterators.
When you create a message iterator with
bt_self_component_port_input_message_iterator_create_from_message_iterator(),
the function sets the new message iterator's downstream message iterator
to the provided self message iterator. It also adds the new message
iterator to the self message iterator's array of upstream message
iterators.
On message iterator finalization:
* The message iterator removes itself as the downstream message iterator
of all its upstream message iterators. It also clears the upstream
message iterator array.
* The message iterator removes itself as one of the upstream message
iterators of its downstream message iterator.
I didn't find any race condition regarding the new message iterator
links, but I could be wrong. The future and more tests will tell us.
In the `bt2` Python package:
* _UserComponentInputPort.create_message_iterator() is removed.
* _UserMessageIterator._create_input_port_message_iterator() is added
(accepts a user component input port).
* _UserSinkComponent._create_input_port_message_iterator() is added
(accepts a user component input port).
Tests are updated accordingly.
I added a test for
_UserMessageIterator._create_input_port_message_iterator() specifically
(needs a filter message iterator) because we use
_UserSinkComponent._create_input_port_message_iterator() in the current
tests.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I46cefca445353c453fbd9404b97687b81fa5f443
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1732
Simon Marchi [Tue, 23 Jul 2019 19:03:26 +0000 (15:03 -0400)]
bt2: fix some whitespace issues
Change-Id: Ic49586dd2e19ad175ad44190d519ce2e857605c1
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1747
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Wed, 24 Jul 2019 03:08:17 +0000 (23:08 -0400)]
lib: make default query implementation return INVALID_OBJECT, remove UNSUPPORTED status
Queries against component classes that don't implement the query method
currently return a special UNSUPPORTED status. There is however no need
for a different status than the one a query method returns when it
doesn't know about the object, INVALID_OBJECT.
Make the default implemention of "query" return INVALID_OBJECT, and
remove any trace of UNSUPPORTED, including in the Python bindings.
Change-Id: I43f59ef749f3803a832a1ad14e640d03424b3420
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1755
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Wed, 24 Jul 2019 02:49:46 +0000 (22:49 -0400)]
lib: remove LOADING_ERROR status
Remove the LOADING_ERROR status, use ERROR instead. It's not so useful
to return a separate kind of error, now that we have a way to
communicate detailed error causes.
Change-Id: I45682b12748fdf8a8466211323dd8563f01d423b
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1754
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Wed, 24 Jul 2019 02:38:20 +0000 (22:38 -0400)]
bt2: remove NonexistentClockSnapshot exception type
The NonexistentClockSnapshot exception type is used when trying to get a
clock snapshot from an object that doesn't have clock snapshots (because
of the way it's configured). It does not have much value over using a
standard exception type. Replace it with ValueError.
In the test add some validation about the error message, because it
would be possible for the test to be a false positive if some ValueError
is raised for another reason (a programming error on our part, for
example).
Change-Id: Iebf86bc53e578407bb60f2145246dbd110cafaac
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1753
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Wed, 24 Jul 2019 01:56:58 +0000 (21:56 -0400)]
bt2: prepend underscore to exceptions not meant to be raised by user
The exceptions Error, MemoryError and LoadingError are only meant to be
raised by the Python bindings, in response to corresponding status codes
from the Babeltrace API. As per our convention, names of classes not
meant to be instantiated directly by the user are prepended with an
underscore, so change these accordingly.
Note that they are still accessible to the user if they need to catch
an exception:
try:
...
except bt2._Error:
...
Change-Id: If094d817dac3c507b6bf3e1e794373f1c7fc33e4
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1752
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Tue, 23 Jul 2019 06:08:59 +0000 (02:08 -0400)]
bt2: rename CreationError to MemoryError, handle it in and out of Python bindings
Babeltrace API calls can explicitly return the MEMORY_ERROR status,
which is currently converted to a bt2.Error in Python. If this
bt2.Error goes back to the native side of the bindings, is is converted
back to the ERROR status. This is not ideal, as the status value should
stay MEMORY_ERROR when unwinding the stack.
Also, when creation functions fail (return NULL/None), we currently
raise a CreationError. Those creation functions only return NULL/None
if there were memory issues, there is not other way they can
legitimately fail (otherwise they would return a status). So it would
make sense for this to be reported as a MEMORY_ERROR.
To address these two issues, this patch:
- renames bt2.CreationError to bt2.MemoryError
- makes utils._handle_func_status raise a bt2.MemoryError when receiving
the MEMORY_ERROR status
- makes the native bindings return MEMORY_ERROR when catching a
bt2.MemoryError
Since there's no easy way to generate an actual MemoryError, I tested
this manually by hacking bt_graph_create to make it fail.
Change-Id: I4969e8ccd9618d8361fe45d0973df980bc9b7abd
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1751
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Wed, 17 Jul 2019 22:44:23 +0000 (18:44 -0400)]
bt2: make bt2.Error wrap current thread's error
This patch makes the Python bindings integrate better with the
Babeltrace error framework.
When a Babeltrace API call fails, it returns an ERROR or MEMORY_ERROR
status and appends a cause to the current thread's error object. When
that API call is made from the Python, we convert that to raising a
bt2.Error execption. With this patch, we now steal the current thread's
error object and give it to the bt2.Error exception object.
The Python code can catch this exception and consult the error causes
in the error object:
def _consume(self):
try:
something_that_raises_a_bt2_error()
except bt2.Error as exc:
for cause in exc:
print(cause)
If the Python code catches the exception and does nothing else with
it (as in the example above), the exception object is destroyed,
destroying the wrapped bt_error object with it. It can be seen as if
the Python code catches the error and recovers from it.
If the Python code lets the exception propagate back to the C code, like
this:
def _consume(self):
something_that_raises_a_bt2_error()
... the bt2.Error is converted back to an ERROR status. But we now also
take back the bt_error object from the bt2.Error exception object and
restore it as the current thread's error object, so that the error is
propagated. We also append a new cause to it, with the traceback of
where the bt2.Error exception was raised.
A more complex case is if the user raises their own exception from the
bt2.Error, using Python exception chaining:
def _consume(self):
try:
something_that_raises_a_bt2_error()
except bt2.Error as exc:
raise MyOwnExceptionType from exc
In this case, we start by restoring the bt_error as the thread's error
object, to put things back as they were before we called Python. We
then append one cause per exception in the chain, starting with the end
of the chain. That is, one cause for the bt2.Error (with the traceback
to where it was raised) and one cause for the MyOwnExceptionType
exception (with the traceback to where it was raised).
When it handles a bt2.Error, the Python code can obtain information
about the error, mainly to access the list of causes. It does so by
accessing the bt2.Error as a sequence of causes. Each cause kind (from
unknown actor, from component class actor, from component actor and from
message iterator actor) is represented by a class, that gives access to
the appropriate properties of the cause (e.g. component class name, port
name, etc).
Various design choices:
- Some causes have a `component_class_type` property, which returns the
type of component class in which the cause happened. This property
returns a value of the `bt_component_class_type` enum. The user can
compare the returned value against the properties of the newly
introduced ComponentClassType class.
- Because it's now non-trivial, the Error type was moved to the new
error.py file.
- Add BT_CURRENT_THREAD_MOVE_ERROR_AND_RESET macro, which calls
bt_current_thread_move_error and then clears the passed lvalue. It's not
essential, but may help avoid mistakes, since once you moved back the
error into place, it's no longer yours.
- Removed the raise bt2.Error in message._create_from_ptr. If we get an
unexpected message type, there's not a lot we can do, it's just a
programming error / bug that needs to be fixed. If it happens, we will
get a KeyError with the key that was not found, so it should be pretty
obvious.
Change-Id: Ibb63d57838a248fadda9cb741d920538fe588680
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1738
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Tue, 23 Jul 2019 04:32:05 +0000 (00:32 -0400)]
bt2: make bt2 add graph listener wrappers append error causes
If bt_bt2_graph_add_port_added_listener or
bt_bt2_graph_add_ports_connected_listener fail, it can be either because
one of the calls to the Babeltrace API fails or a Python object creation
fails. On the Python side, we raise a bt2.Error on failure. This seems
fine, although in a subsequent patch, it will become a pre-condition for
raising bt2.Error that we have an error set for the current thread.
Add some calls to BT_CURRENT_THREAD_ERROR_APPEND_CAUSE_FROM_UNKNOWN in
the errors paths of these wrappers that don't come from the library,
i.e. on which there would not already be an error cause appended.
On the Python side, use bt2.Error directly instead of
utils._raise_bt2_error. Since these were the last call sites of
utils._raise_bt2_error, remove it.
Change-Id: I1556b626a941e9cc127fb036d6e3b9b6346fde88
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1745
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Tue, 23 Jul 2019 04:11:44 +0000 (00:11 -0400)]
bt2: make bt_bt2_trace_{,class}_add_destruction_listener return a status
If the calls to bt_trace_class_add_destruction_listener and
bt_trace_add_destruction_listener in the Python bindings fail (due to an
hypothetical memory error that can't happen today), abort is called.
Strangely, back on the Python side, there is a "listener_id is None"
check that is a bit useless, since we would have aborted had there been
a failure.
Regularize the situation by making the
bt_bt2_trace_class_add_destruction_listener and
bt_bt2_trace_add_destruction_listener wrappers return a status, like any
other API function. We can then use utils._handle_func_status like
everywhere else.
Change-Id: I17901a0278f1615b45e6adc3b7223f1b3b3ff35e
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1744
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Tue, 23 Jul 2019 03:37:03 +0000 (23:37 -0400)]
bt2: remove utils._handle_ptr
A future patch will want to get rid of spots that raise bt2.Error, which
are not in direct response to a failed Babeltrace API call. The
utils._handle_ptr is one such spot. It turns out that the usages of
utils._handle_ptr are not useful anymore, because they check
situations that can either never happen, or are check by pre-condition
checks.
In connection.py, it's not possible anymore for
bt_connection_borrow_downstream_port_const or
bt_connection_borrow_upstream_port_const to return NULL/None, as
connections always have both ports set.
In field.py, the call in _create_field_from_ptr after
bt_field_borrow_class_const can be removed, as it's impossible for this
function to legitimately return NULL/None.
The call in _VariantField.selected_option is unnecessary, as it is after
a pre-condition check that the variant's selected option is non-NULL.
If we want to make Python bindings user-friendly (avoid hitting
pre-condition checks), we check before the call if the variant has a
currently selection option, and raise ValueError otherwise. However,
there does not seem to be an API for that at the moment.
Change-Id: I585e57030d58256cbf51161e75394503c281addf
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1743
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sun, 21 Jul 2019 16:49:05 +0000 (12:49 -0400)]
py-common: make bt_py_common_format_exception accept an arbitrary exception
A following patch will need to format each exception of the exception
chain independently. It will therefore need a function like
bt_py_common_format_exception, but that:
- takes the exception to format as a parameter, rather than relying on
the Python error indicator
- has an option for formatting just that exception, not following the
causes chain
This patch changes bt_py_common_format_exception to meet these
requirements. The behavior of the old bt_py_common_format_exception is
now offered by bt_py_common_format_current_exception, so the existing
callers are updated to use that one.
Change-Id: I7bbb442bc1dc363e5441b3b11ed6f0c6e213ba79
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1737
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sun, 21 Jul 2019 17:05:21 +0000 (13:05 -0400)]
Fix: bt2: Make bindings target depend on convenience libraries
Three convenience libraries are used to build the bindings' native
library. This dependency is not expressed in the Makefile, so if you
change one of them and re-run make, the bindings won't be rebuilt, and
the changes in that convenience lib won't be taken into account. For
example, if you touch src/py-common/.libs/libbabeltrace2-py-common.a and
re-run make, the bindings should (but currently don't) get re-built.
This patch adds the explicit dependency in the Makefile, so that any
change to one of these libs will trigger a rebuild of the bindings.
Change-Id: Iea47833b206bcd0f1cc0d3f72975b14fd6eded93
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1736
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Tue, 23 Jul 2019 16:15:08 +0000 (12:15 -0400)]
tests: fix Windows support in test_convert_args
Additionnal variables were added to the common case, add them for
Windows too.
Change-Id: I33ec919f305a3d69020ee906e29ce4c2636ae54f
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1750
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Tue, 23 Jul 2019 15:43:13 +0000 (11:43 -0400)]
tests: readlink 'canonicalize' is GNU specific
On platforms not using the GNU coreutils, namely MacOs, readlink doesn't
have a canonicalize option.
Use it only when availabe, otherwise use the relative paths.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I798e6320dda78c5d9b33a2568b663c57d35d42b3
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1749
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Mon, 22 Jul 2019 21:36:02 +0000 (17:36 -0400)]
tests: fix Python plugin provider tests on Windows
On Windows, the embedded Python interpreter in the Python plugin
provider can't automatically locate the path to it's own modules. Set
the PYTHONHOME variable to the proper value in 'run_python_bt2()'.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Ie30ac3212a208d77654add78604c534c05efe518
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1748
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Tue, 23 Jul 2019 16:23:35 +0000 (12:23 -0400)]
Standardize `!ptr` i/o `ptr == NULL`, `ptr` i/o `ptr != NULL`
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I3e47fe9ca121cf3fc4f71e6292b9fa9efc6df341
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1746
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Mon, 22 Jul 2019 17:45:22 +0000 (13:45 -0400)]
bt2: do not require sink component's _graph_is_configured() method
The "graph is configured" method is not required when you implement a
component class in C, as it's not strictly needed to make the graph
runnable, so it should not be required in Python either.
A default _UserSinkComponent._graph_is_configured() method is added to
make it optional.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I3ef4e3d159af61e2d06a0a47dddc1234aed93ae1
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1740
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Mon, 22 Jul 2019 17:42:26 +0000 (13:42 -0400)]
bt2: move _bt_graph_is_configured_from_native() to `_UserSinkComponent`
This method is only called on a sink user component, it does not need to
be in the base `_UserComponent`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ie537c6871d145d2cb3c96eaa0e95573b02731cc9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1739
Tested-by: jenkins <jenkins@lttng.org>
Michael Jeanson [Mon, 22 Jul 2019 21:39:31 +0000 (17:39 -0400)]
Update python bindings gitignore for native_bt.c
In
e7d63bf32268fcaf9e1f5724cf4def49f3f9c081, 'native_bt_wrap.c' was
renamed to 'native_bt.c' but the gitignore was not updated.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Ibd4186ea0ce12bb9078ceee18cff781649cdd2fe
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1742
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Fri, 5 Jul 2019 19:50:58 +0000 (15:50 -0400)]
cli: automatically detect sources for leftover arguments
This patch adds a source auto-discovery feature. The goal is for the
user to be able to just pass some inputs (which can be strings of a form
understood by a particular component class, or paths to
files/directories) and for babeltrace to figure out which component
classes are best suited to handle those inputs.
Currently, any leftover argument passed by the user is passed to a
src.ctf.fs instance. To use a different source component, the user must
use the --component argument. This system will therefore help usability
for users of non-src.ctf.fs sources. It will also allow splitting the
src.ctf.fs source into a "generic CTF" one and an "LTTng CTF" one, which
includes the fixups specific to CTF files produced by various LTTng
versions.
The big picture is that for each leftover argument (called `input`), we
ask all component classes if they can handle it (see `support-info
query` below), the component classes reply with a weight, and the input
is attributed to the component class that gave the largest weight.
More precisely, this is what we do for each leftover argument:
1. Ask all known source component classes if they recognize the
argument as an arbitrary string, which would be in a format that
makes sense to them (but doesn't point to a file or directory on
disk, unless it's a coincidence). The obvious example is
src.ctf.lttng-live, which would be apt to handle paths of the form
'net://...'. If some component classes claim to understand the
argument, the one with the largest weight is chosen, and a component
of that class will be instantiated.
2. If no component class has claimed the argument as an arbitrary
string and the argument points to a file or directory on disk, ask
them all if they recognize it as a file or directory they can handle.
If some component classes do, choose the one with the biggest weight.
3. If no component class has recognized it so far, and the input points
to a directory, we start to dig: for each child of that directory,
apply the sophisticated algorithm described in step 2.
If a leftover argument (including all its children, if it's a directory)
is not handled by any source component class, we show a warning. If
nothing is discovered and no explicit source is instantiated either, the
excecution fails on the "No source component" check that is already
there.
Component classes have the ability to "group" their inputs as they wish.
This means that multiple inputs attributed to a given component class
can be passed to a single instance of that class, all to separate
instances, or any combination in between. To achieve this, component
classes are able to also reply with a group key (a string of their
choice). Inputs with the same group key will be passed to the same
instance.
Implementation details and choices
----------------------------------
* Since leftovers are now passed to the source auto-discovery mechanism
as opposed to an implicit src.ctf.fs component previously, this
src.ctf.fs implicit component is no longer needed. This changes how we
handle some of the legacy (compatibility with babeltrace 1) flags.
If the user passes --clock-offset or --clock-offset-ns, we will search
in the auto-discovered sources and apply it to any src.ctf.fs instance
created. If no src.ctf.fs component would be instantiated, we issue
an error.
If the user passes --input-format ctf, we don't want other formats
possibly being read. We still use the auto-discovery mechanism, but
we restrict it to the src.ctf.fs component class (other component
classes won't be queried and therefore won't be instantiated).
* This also means that to keep the basic use case of "babeltrace2
<dir-with-ctf-traces>" working, we need to implement the support-info
query for the src.ctf.fs component class in the current patch. Not
doing so immediatly would break many tests. The simplest possible
implementation was added. It looks for inputs of type "directory",
which have a "metadata" file in it. It always reply with the same
group value, such that a single instance of the component class is
used (keeping the current behavior).
* Since the strings we pass to support-info queries (and eventually to
components we instantiate) are not necessarily paths to directories or
files on disk (they can be URLs, for example), we now use the term
"inputs" rather than "paths" for all of them.
* A support-info query returning ERROR aborts the auto-discovery
process, making it return an ERROR as well.
* If we can't open a directory because we don't have permission to read
it (EACCES), we log a warning and continue. Other errors abort the
auto-discovery process.
support-info query
------------------
The support-info query is a contract between the CLI and source
component classes. Source component classes that don't support it will
simply not be able to automatically discover inputs, and will have to be
explicitly instantiated using --component.
The parameters of a support-info query are a map containing these keys:
- `type` (string): possible values are "string", "directory" or "file",
indicating the nature of `input`, described below. The value "string"
means that the component class may try to interpret `input` as a
string with a format it can recognize. "directory" and "file"
respectively mean to interpret `input` as a directory and file. When
type is "directory" or "file", the component class can assume that
the corresponding directory or file exists on disk.
- input (string): input descriptor, to be interpreted according to
`type`.
A support-info response can be
- A real or integer value between 0 and 1, representing the weight
- A map value containing these keys:
- `weight` (real or integer), mandatory: between 0 and 1
- `group` (string or null), optional: a key by which to group inputs,
when instantiating components.
A component class that does not support a given must reply with a weight
of 0.
All inputs attributed to the same component class, sharing the same
group key, will be passed to the same component instance. inputs whose
group key is missing or null are not grouped with other inputs.
Components created by the auto-discovery mechanism are passed the
`inputs` parameter, an array of strings containing all inputs attributed
to this instance.
testing
-------
I have a brief catch-all test for this, it just covers a few important
cases. The test strategy is the following:
- Run babeltrace2 with an arbitrary string and a directory as
leftovers.
- One source recognizes the arbitrary string.
- Various sources recognizes files and directories inside the passed
directory.
- Each instantiated source outputs one line including its name and the
sorted list of its inputs.
- We sort the output of babeltrace and compare it with an expected
string. Since everything is sorted, the output should be stable.
Change-Id: I7f884551d7cb576974ea53420ead9c4a8005e99d
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1644
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 22 Jul 2019 18:35:14 +0000 (14:35 -0400)]
src.ctf.lttng-live: LOGI instead of LOGW when getting queried for an unknown object
Receiving a query about an unknown object happens in the normal course
of operations, and is not warning-worthy. In particular, a following
patch will query all source component classes using the `support-info`
object, to see if the source component class supports a given input.
Having a LOGW here would generate a lot of unimportant warnings.
Change-Id: I067424cb72f78aeda6ce6791b12fa00d7d90b5c0
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1741
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Mathieu Desnoyers [Mon, 17 Jun 2019 21:56:02 +0000 (17:56 -0400)] 
sink.text.pretty: don't use printf for binary values
printf is slow compared to simply appending '1' or '0' characters to the
GString.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I2605c1e6d2dc463be237e8eb00f70a84e4c5fd1a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1507
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Mathieu Desnoyers [Mon, 17 Jun 2019 21:05:58 +0000 (17:05 -0400)]
sink.text.pretty: use bt_common_g_string_append and bt_common_g_string_append_c
Use our own inline implementation of g_string_append and g_string_append_c
to improve text pretty-printing speed.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I39254ae912cb9bd227c68519552b4b610c15755f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1506
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Mathieu Desnoyers [Mon, 17 Jun 2019 20:12:49 +0000 (16:12 -0400)]
sink.text.pretty: remove field filtering
Field filtering uses GQuarks internally. Those generate g hash table
lookups, which appear at the top of perf reports.
This field filtering mechanism is not needed anymore, so remove it.
It's not needed anymore because the packet context fields and the event
fields are not special or associated to CTF anymore as the library's
trace IR is CTF-agnostic. So those special fields do not exist anyway in
the messages this sink consumes.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I7bf0238e6692fb277dafc32f864b0e20e89bf3d2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1505
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Mathieu Desnoyers [Mon, 17 Jun 2019 19:33:51 +0000 (15:33 -0400)]
sink.text.pretty: do not printf field name strings
Use g_string_append rather than the printf counterpart to output field
name strings. This significantly improves pretty printing speed
(34s -> 29s on reference trace).
This is a performance improvement mainly because only strings are passed
to printf in this case.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I4df52308c61346a81e0b79b079d53be52084707f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1504
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Mathieu Desnoyers [Mon, 17 Jun 2019 16:53:22 +0000 (12:53 -0400)]
sink.text.pretty: use bt_common_g_string_append_printf
g_string_append_printf() internally allocates a temporary buffer
through use of vasnprintf for each call. This clearly appears at
the top of perf report.
Use babeltrace's own bt_common_g_string_append_printf which operates
directly on the GString buffer, increasing its size to nearby next power
of two as needed.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I60b4dccfa19c4321eb5997233826cfabaf924ff2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1503
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Mathieu Desnoyers [Wed, 19 Jun 2019 18:23:09 +0000 (14:23 -0400)]
common: implement bt_common_g_string_append and bt_common_g_string_append_c
Implement inline versions of g_string_append and
g_string_append_c. Since those calls are performed very often within
sink.text.pretty, it is relevant for pretty-printing speed.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ib1069c7d21c6e707f75d839f333f034a50e2a43e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1515
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Mathieu Desnoyers [Wed, 19 Jun 2019 18:21:57 +0000 (14:21 -0400)]
common: implement bt_common_g_string_append_printf
g_string_append_printf() internally allocates a temporary buffer
through use of vasnprintf for each call. This clearly appears at
the top of perf report.
Implement our own private bt_g_string_append_printf which operates
directly on the GString buffer.
See proof of memory allocation/free near the top of this perf report
when converting a 400M LTTng trace to text (piped to /dev/null):
14.76% babeltrace2 libc-2.24.so [.] vfprintf
7.92% babeltrace2 [kernel.kallsyms] [k] vmacache_find
6.39% babeltrace2 libglib-2.0.so.0.5000.3 [.] g_string_insert_len
4.77% babeltrace2 babeltrace-plugin-ctf.so [.] bt_bfcr_start
4.23% babeltrace2 libc-2.24.so [.] _int_malloc
4.06% babeltrace2 libc-2.24.so [.] _IO_default_xsputn
3.60% babeltrace2 libc-2.24.so [.] strlen
3.60% babeltrace2 libc-2.24.so [.] __strchrnul
3.60% babeltrace2 libbabeltrace2.so.0.0.0 [.] bt_attributes_borrow_field_by_name
3.51% babeltrace2 libc-2.24.so [.] _int_free
2.88% babeltrace2 libc-2.24.so [.] __vasprintf_chk
2.88% babeltrace2 libc-2.24.so [.] _IO_str_init_static_internal
2.88% babeltrace2 libc-2.24.so [.] __strcmp_sse2_unaligned
2.16% babeltrace2 libc-2.24.so [.] malloc
2.16% babeltrace2 libc-2.24.so [.] __tzstring_len
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I956305ac8891d244dbdab6a5b0dde82ecc960e25
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1514
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sun, 21 Jul 2019 14:50:39 +0000 (10:50 -0400)]
tests/Makefile.am: remove needless comment
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I5e70b8b0fc3e5faf9663cfabf657d447fc5ed710
Simon Marchi [Sun, 21 Jul 2019 03:10:41 +0000 (23:10 -0400)]
Fix: bt2: incref Py_None in get_msg_range_common on error
When get_msg_range_common processes a result that is not OK, it
returns a tuple whose second element is None:
(status, None)
However, when setting the second element of the tuple to None, we are
missing an incref of Py_None. This incref is necessary, because
PyTuple_SET_ITEM steals the reference of the object we pass.
This patch fixes that.
Change-Id: I8a5af2853172a8399dd1779d90e0fcb8fc265032
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1731
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sat, 20 Jul 2019 22:18:40 +0000 (18:18 -0400)]
lib: logging: add `%!R` conv. specifier for integer range set and use it
This patch also renames "range set" to "integer range set" in logging
and precondition assertion messages.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I59aaf541d956d84fee5cad22393b230635558877
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1730
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sat, 20 Jul 2019 16:38:21 +0000 (12:38 -0400)]
lib: rename functions to clearly indicate API inheritance
This patch renames some functions of the library's API to show how
an API inherits another one, for example (before):
bt_field_signed_integer_get_value()
bt_field_unsigned_integer_get_value()
vs. (after):
bt_field_integer_signed_get_value
bt_field_integer_unsigned_get_value
The second version clearly shows that both are part of the
`bt_field_integer` API.
This patch aligns the names of those functions with other parts of the
API which already have the correct order of prefixes, for example:
bt_component_class_source_create()
bt_component_class_filter_create()
bt_component_class_sink_create()
bt_component_class_get_name()
bt_message_event_create()
bt_message_get_type()
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I5f7b637d1d72d610f88f3f7d0122c8975f26481b
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1728
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Sat, 13 Jul 2019 14:39:21 +0000 (10:39 -0400)]
Tests: flt.lttng-utils.debug-info: update debug-info tests
* Convert the currently disabled Python bindings based debug-info
testcase to a `sink.text.details` based test.
* Update debug-info test trace using a recently changed `libhello_so`
file.
* Move all debug-info test files to
`tests/plugins/flt.lttng-utils.debug-info/`
* Move `tests/data/debug-info/README.md` content to `CONTRIBUTING.adoc`
* Update `tests/data/plugins/flt.lttng-utils.debug-info/README.md` to
explain how to generate the debug-info test trace.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I67b8fa729e462137fb3e584c0f1eaf472256e3c9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1704
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Fri, 19 Jul 2019 19:14:45 +0000 (15:14 -0400)]
Fix: flt.utils.muxer: don't clear an empty `GPtrArray`
GLib < 2.48.0 asserts when clearing an empty `GPtrArray` with
g_array_remove_range():
GLib-CRITICAL **: g_ptr_array_remove_range: assertion
`index_ < array->len' failed
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: If9dad0869404ec72ee15724b72ad88780b671619
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1725
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Mon, 15 Jul 2019 21:50:11 +0000 (17:50 -0400)]
`test_plugin.py`: use absolute paths for plugin path comparison
Under a MinGW 64 test environment, the `BABELTRACE_PLUGIN_PATH`
environment variable gets translated to NT path by MinGW since the
passed paths fit the translation criteria. The relative part of the
paths also get thrown away and the relative paths are replaced by their
absolute equivalent.
Performing absolute path conversion on both test paths ensure
consistency across platforms of this test.
The error was:
error: PASS: bindings/python/bt2/test_python_bt2 6513 - test_name (test_plugin.PluginTestCase)
c:/users/joraj/babeltrace/src/plugins/ctf/babeltrace-plugin-ctf.la
c:/users/joraj/babeltrace/tests/../src/plugins
not ok 6514 - test_path (test_plugin.PluginTestCase)
FAIL: bindings/python/bt2/test_python_bt2 6514 - test_path (test_plugin.PluginTestCase)
Traceback (most recent call last):
bindings/python/bt2/test_python_bt2: Traceback (most recent call last):
File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 59, in testPartExecutor
bindings/python/bt2/test_python_bt2: File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 59, in testPartExecutor
yield
bindings/python/bt2/test_python_bt2: yield
File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 628, in run
bindings/python/bt2/test_python_bt2: File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 628, in run
testMethod()
bindings/python/bt2/test_python_bt2: testMethod()
File "C:/Users/joraj/babeltrace/tests/bindings/python/bt2\test_plugin.py", line 96, in test_path
bindings/python/bt2/test_python_bt2: File "C:/Users/joraj/babeltrace/tests/bindings/python/bt2\test_plugin.py", line 96, in test_path
self.assertTrue(plugin_path.startswith(plugin_path_env))
bindings/python/bt2/test_python_bt2: self.assertTrue(plugin_path.startswith(plugin_path_env))
File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 705, in assertTrue
bindings/python/bt2/test_python_bt2: File "C:/msys64/mingw64/lib/python3.7\unittest\case.py", line 705, in assertTrue
raise self.failureException(msg)
bindings/python/bt2/test_python_bt2: raise self.failureException(msg)
AssertionError: False is not true
bindings/python/bt2/test_python_bt2: AssertionError: False is not true
Change-Id: I4b3dc1c52a13dff5be88e417a9d1a877c81f66da
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1709
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Mon, 15 Jul 2019 21:49:56 +0000 (17:49 -0400)]
`test_query_trace_info.py`: adapt regex to NT path style
In a MinGW 64 test environment, the stream path is expressed using NT
path (`C:\\...`). In `test_query_trace_info.py`, adapt the regex
accordingly using Python's `pathlib` API.
Change-Id: Ieee3b9cac0881aec7086cef69ecf681ad71c0722
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1708
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Fri, 19 Jul 2019 21:49:39 +0000 (17:49 -0400)]
tests/plugins/flt.utils.trimmer/Makefile.am: remove useless HAVE_PYTHON guard
Philippe Proulx:
The `test_trimming` test does not depend on Python: it uses the CLI,
a `sink.text.details` component, and expectation strings to validate
that an `flt.utils.trimmer` message iterator trims as expected.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: If08b014259676a75c57db7eda109de55306aa062
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1727
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Fri, 19 Jul 2019 13:24:04 +0000 (09:24 -0400)]
flt.utils.muxer: support stream messages with default clock snapshot
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I46fef189f846999dbfedc3504a7ed5d908464e99
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1724
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Thu, 18 Jul 2019 18:06:03 +0000 (14:06 -0400)]
bt2: fix: don't return in bt_bool out typemap
I noticed this leak in the Valgrind output:
==11115== 5 bytes in 1 blocks are definitely lost in loss record 9 of 5,534
==11115== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11115== by 0x8EA64A8: SWIG_AsCharPtrAndSize (native_bt.c:3539)
==11115== by 0x8EDC722: _wrap_value_map_has_entry (native_bt.c:22039)
...
This is due to the fact that we return in the bt_bool out typemap, which
we shouldn't. In the case of the bt_value_map_has_entry wrapper, a
string is dynamically allocated for the "const char *" parameter. It is
normally freed at the end of the wrapper, except that because we return
early, it is never freed. The same problem is likely to happen with
other functions returning bt_bool.
The leak can be triggered by running this Python script:
from bt2 import value
m = value.MapValue()
m['allo'] = 2
print('allo' in m)
The generated code before this patch looks like:
result = (bt_bool)bt_value_map_has_entry((bt_value const *)arg1,(char const *)arg2);
{
if (result > 0) {
resultobj = Py_True;
} else {
resultobj = Py_False;
}
Py_INCREF(resultobj);
return resultobj;
}
if (alloc2 == SWIG_NEWOBJ) free((char*)buf2);
return resultobj;
fail:
if (alloc2 == SWIG_NEWOBJ) free((char*)buf2);
return NULL;
}
The memory leak is quite obvious. After this patch, it looks better:
result = (bt_bool)bt_value_map_has_entry((bt_value const *)arg1,(char const *)arg2);
{
if (result > 0) {
resultobj = Py_True;
} else {
resultobj = Py_False;
}
Py_INCREF(resultobj);
}
if (alloc2 == SWIG_NEWOBJ) free((char*)buf2);
return resultobj;
fail:
if (alloc2 == SWIG_NEWOBJ) free((char*)buf2);
return NULL;
}
Change-Id: Icdfcdf750610465331619cd9edab1e89dc930f64
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1723
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Thu, 18 Jul 2019 15:33:35 +0000 (11:33 -0400)]
bt2: remove unrelated comment from native_bt_integer_range_set.i
This was copied from native_bt_port.i, but does not apply to
native_bt_integer_range_set.i.
Change-Id: I13f900d531e202afd09e807a85f7f817def698a6
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1722
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Thu, 18 Jul 2019 14:11:55 +0000 (10:11 -0400)]
Apply black code formatter on all Python code
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I2b66d32d12c93d353097b25d80050e4dcfe5b1ff
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1642
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Thu, 11 Jul 2019 17:28:03 +0000 (13:28 -0400)]
Explicitly mention `black` in CodingStyle guidelines
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I782042dc24acb09506205abbbc7b6b22c2c0cb59
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1687
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Thu, 11 Jul 2019 17:17:05 +0000 (13:17 -0400)]
Tests: add `tap` utils code to Black formatter exclusion list
Those files are not written by us but rather copied into the project.
Let's not reformat them automatically so we can compare them with future
versions.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I98ccd8475b5367f1b3ed9084ba06965f28bc6f93
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1686
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Thu, 18 Jul 2019 05:33:32 +0000 (01:33 -0400)]
lib: remove `BT_ASSERT_PRE_FUNC`
`BT_ASSERT_PRE_FUNC` is defined to nothing, so remove it.
This should have been done in
bdb288b3e94e412a33c8647d44f6cfac66ca0665.
`BT_ASSERT_PRE_DEV_FUNC` is defined in non-developer mode to mark the
function as unused.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I382f87693a6079eeb13f537e2669402995c58f41
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1721
Philippe Proulx [Wed, 17 Jul 2019 22:12:39 +0000 (18:12 -0400)]
lib: bt_field_class_dynamic_array_create(): accept length FC parameter
This patch makes bt_field_class_dynamic_array_create() similar to
bt_field_class_variant_with_selector_create() for consistency: it
accepts the length field class on creation instead of setting it
afterwards.
You can pass `NULL` to create a dynamic array field class without a
length field class.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ib94adcc32128154b5979120ef0623be912f28734
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1718
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Fri, 12 Jul 2019 19:03:40 +0000 (15:03 -0400)]
lib: decouple variant FC option names from selector FC mapping names
This patch makes the options of a variant field class (FC) which has a
selector have their own range set, decoupled from any mapping of the
selector FC.
Motivation and solution
=======================
In CTF 1.8, structure FC member and variant FC option names which start
with `_` in the metadata stream must be "unescaped" by removing the `_`
prefix. For example:
struct {
string _salut;
string meow;
};
`_salut` becomes `salut` in trace IR, while `meow` stays `meow`.
CTF 1.8.2 specifies:
> Fields starting with an underscore should have their leading
> underscore removed by the CTF trace readers.
Here, we interpret "should" as "must" because Babeltrace 1, Trace
Compass, and other CTF consumers honor this recommandation.
It is not specified, however, that this strategy applies to enumeration
FC mapping labels. For example:
enum {
_SALUT,
MEOW,
};
In Babeltrace 1 (`text` output format and API), `_SALUT` and `MEOW`
remain as is, so Babeltrace 2 should do the same.
There's an issue however when an enumeration FC is used as a tag, or
selector, for a variant FC:
enum {
_salut,
_meow,
} tag;
variant <tag> {
string _salut;
int _meow;
};
This is valid TSDL 1.8, but once in trace IR, the enumeration FC mapping
labels need to stay as is, while the variant FC option names need to be
unescaped. Once in trace IR, the equivalent would be:
enum {
_salut,
_meow,
} tag;
variant <tag> {
string salut;
int meow;
};
Before this patch, this is not valid because, when a variant FC has a
selector FC, the option and mapping names must match exactly: we don't
want to bring this CTF-specific escaping logic into the CTF-agnostic
API.
The current hack to avoid this, performed by `src.ctf.fs`, is to also
unescape the mapping names, so as to get:
enum {
salut,
meow,
} tag;
variant <tag> {
string salut;
int meow;
};
However, this makes Babeltrace 2 not behave like Babeltrace 1 because
the enumeration FC mapping names are different. For example, the
`sink.text.pretty` outputs differ for the same CTF trace.
The solution brought by this patch to fix this issue is to make the
options of a variant FC have their own range set. Using the example
above, the layout in trace IR becomes:
enum {
_salut,
_meow,
} tag;
variant <tag> {
string salut {0};
int meow {1};
};
where `{0}` and `{1}` are the range sets associated to the option.
Here's another example:
enum {
_salut,
_meow = 12,
coucou = 17 ... 45,
_meow = 1 ... 5,
} tag;
variant <tag> {
string salut {0};
int meow {1 ... 5, 12};
int coucou[5] {17 ... 45};
};
This change allows `src.ctf.fs` to keep the enumeration FC mapping names
as is while properly escaping the variant FC option names.
Library changes
===============
The simple variant field class type is replaced with three new variant
field class types:
Variant without a selector:
You can create a variant FC without a selector FC, just like before,
with bt_field_class_variant_create(), passing `NULL` as the
`selector_field_class` parameter.
The field class's type is
`BT_FIELD_CLASS_TYPE_VARIANT_WITHOUT_SELECTOR`.
You can use bt_field_class_variant_without_selector_append_option()
to append an option (name and FC) to a variant FC without a
selector.
You can use bt_field_class_variant_borrow_option_by_index_const()
and bt_field_class_variant_borrow_option_by_name_const() to borrow
a variant FC (base) option from a variant FC without a selector.
Variant with a selector:
You can create a variant FC with a specific selector FC with
bt_field_class_variant_create().
The selector FC must be an _integer_ FC. This is less strict than
before: because each option has its own range set, the selector FC
does not need to be an enumeration FC. It can be an enumeration FC,
but there's no association between its mapping names and the variant
FC option names.
The field class types are
`BT_FIELD_CLASS_TYPE_VARIANT_WITH_UNSIGNED_SELECTOR` and
`BT_FIELD_CLASS_TYPE_VARIANT_WITH_SIGNED_SELECTOR`.
There's an unsigned and a signed type to make the API typing more
strict: a variant FC with an unsigned selector has specific option
objects from which you can borrow unsigned integer range sets.
You can use
bt_field_class_variant_with_unsigned_selector_append_option() and
bt_field_class_variant_with_signed_selector_append_option() to
append an option (name, FC, and range set) to a variant FC with a
selector.
You can use
bt_field_class_variant_with_unsigned_selector_borrow_option_by_index_const(),
bt_field_class_variant_with_unsigned_selector_borrow_option_by_name_const(),
bt_field_class_variant_with_signed_selector_borrow_option_by_index_const(),
or
bt_field_class_variant_with_signed_selector_borrow_option_by_name_const()
to borrow a variant FC with a selector option from a variant FC
with a selector.
The option object's type is either
`bt_field_class_variant_with_unsigned_selector_option` or
`bt_field_class_variant_with_signed_selector_option`. You can
convert it to a base option (to get its name and borrow its field
class) with
bt_field_class_variant_with_unsigned_selector_option_as_option_const()
or
bt_field_class_variant_with_signed_selector_option_as_option_const().
You can borrow the ranges of a variant FC with a selector option
with
bt_field_class_variant_with_unsigned_selector_option_borrow_ranges_const()
or
bt_field_class_variant_with_signed_selector_option_borrow_ranges_const().
You can use
bt_field_class_variant_with_selector_borrow_selector_field_path_const()
to borrow the selector field path from a variant FC with a selector.
I also added the following functions for convenience:
* bt_field_variant_borrow_selected_class_option_const()
* bt_field_variant_with_unsigned_selector_borrow_selected_class_option_const()
* bt_field_variant_with_signed_selector_borrow_selected_class_option_const()
For consistency, bt_field_variant_select_option_field() is renamed to
bt_field_variant_select_option_field_by_index().
This patch also makes an enumeration FC mapping contain an integer range
set object. This was planned anyway, and not doing it here would have
meant to duplicate and adapt code for the variant FC option ranges, for
example in `sink.text.details` to sort the ranges of a range set.
This means that bt_field_class_unsigned_enumeration_map_range() and
bt_field_class_signed_enumeration_map_range() are replaced with
bt_field_class_unsigned_enumeration_add_mapping() and
bt_field_class_signed_enumeration_add_mapping() which accept resp.
unsigned and signed integer range sets.
This also means that
bt_field_class_enumeration_mapping_get_range_count(),
bt_field_class_unsigned_enumeration_mapping_get_range_by_index(), and
bt_field_class_signed_enumeration_mapping_get_range_by_index() are
replaced with
bt_field_class_unsigned_enumeration_mapping_borrow_ranges_const() and
bt_field_class_signed_enumeration_mapping_borrow_ranges_const().
Because I needed it when adapting the project's plugins, I also added
the following functions for convenience:
* bt_field_class_unsigned_enumeration_borrow_mapping_by_label_const()
* bt_field_class_signed_enumeration_borrow_mapping_by_label_const()
Noteworthy plugin changes
=========================
`src.ctf.fs`:
* Enumeration FC mapping names are not unescaped anymore: they are
kept as is.
* A CTF IR named FC contains the original name and the escaped name.
The original name is used to find the ranges of a variant FC
option in the selector FC as the mapping names are not escaped
anymore, so they potentially do not match the variant FC option
names.
* When translating a CTF IR variant FC to a trace IR variant FC, the
trace IR selector (enumeration) FC's integer range set references
are reused directly to append the corresponding variant FC
options.
`sink.ctf.fs`:
* Enumeration FC mapping names are not escaped anymore: they are
kept as is.
* If a variant FC has a selector, then for each option, the
component finds the corresponding mapping in the selector FC _by
range set_ to know whether or not to escape the option name.
This is because this must work (from `src.ctf.fs`):
enum {
salut,
_meow,
} tag;
variant <tag> {
string salut;
int _meow;
};
Once in trace IR, the `_meow` option becomes `meow`, but the
`_meow` mapping keeps its original name. However, we know that,
for `src.ctf.fs`, the range sets match exactly. For the `meow`
option, the corresponding mapping is `_meow` because they both
have the range set with the single range [1, 1]. In that case,
when going back to TSDL, `sink.ctf.fs` writes `_meow` for the
option name, while `salut` remains `salut`.
I added new `sink.ctf.fs` tests, with new succeeding CTF traces,
to verify that the component works as expected with those specific
cases.
If there's any issue when doing this, `sink.ctf.fs` falls back to
creating a dedicated selector FC for the variant FC. For example,
in trace IR, this is totally valid:
enum {
a = 2,
b = 5,
d = 8,
} tag;
variant <tag> {
string a {2};
int b {11};
int c[22] {15 ... 19};
};
because there's no association between mapping names and option
names. This too:
int tag;
variant <tag> {
string a {2};
int b {11};
int c[22] {15 ... 19};
};
Those specimens cannot be translated to TSDL 1.8 however.
* Because of changes in the way TSDL identifers are protected and
validated, clock class names are not systematically escaped if
it's not needed. Therefore the clock class name `default` remains
`default`; it does not become `_default` like before.
`sink.text.details`:
* Variant FC option ranges are written next to the option's name:
var: Variant (unsigned selector) (3 options, Selector field path [Event payload: 0]):
COSSETTE: [0]: String
_PELCHAT: [1]: String
VOISINE: [2] [5, 19]: String
* Enumeration FC mapping ranges are written next to the option's
name instead of having one per line. I find this is more compact
and easier to read as mappings typically do not contain a lot of
ranges:
tag: Unsigned enumeration (8-bit, Base 10, 3 mappings):
COSSETTE: [0]
VOISINE: [2] [5, 19]
__PELCHAT: [1]
Python bindings changes
=======================
Essentially, the `bt2` Python package is updated to match the library's
API changes:
* `_EnumerationFieldClassMapping` is not a set anymore: it has
a `ranges` property (which is a set).
* _EnumerationFieldClass.map_range() is replaced with
_EnumerationFieldClass.add_mapping() to which you pass an integer
range set.
* _EnumerationFieldClass.labels_for_value() is replaced with
_EnumerationFieldClass.mappings_for_value() to get mappings instead of
simple labels.
* _EnumerationFieldClass.__getitem__() now uses
bt_field_class_unsigned_enumeration_borrow_mapping_by_label_const()
or
bt_field_class_signed_enumeration_borrow_mapping_by_label_const().
* The new `_VariantFieldClassWithSelectorOption` object inherits
`_VariantFieldClassOption`, adding a `ranges` property.
* With a `_VariantFieldClass`, you can borrow base options
(`_VariantFieldClassOption`) to get their names and field classes.
* The new types `_VariantFieldClassWithoutSelector`,
`_VariantFieldClassWithUnsignedSelector`, and
`_VariantFieldClassWithSignedSelector` are analogous to the new
types in the library's API.
You can create them with
_TraceClass.create_variant_field_class_without_selector() and
_TraceClass.create_variant_field_class_with_selector(). The selector
FC object must be an instance of `_IntegerFieldClass`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I084b03ea816ff8bee03ef5315c24fa24cfe74d80
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1717
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Fri, 12 Jul 2019 18:04:54 +0000 (14:04 -0400)]
bt2: add integer range set support
This patch adds `bt2` wrappers for the Babeltrace library integer range
and integer range set library objects.
The new `test_integer_range_set.py` file tests all the new objects.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I9e6852ca2ec6b04bcbc3b6622c03c3abcfa6e32f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1716
This page took 0.059262 seconds and 4 git commands to generate.
projects / babeltrace.git / log
