Jonathan Rajotte [Tue, 13 Aug 2019 22:34:20 +0000 (18:34 -0400)] 
bt_uuid_compare: return actual comparison
Previous implementation resulted in non-equal uuid_a being flagged as
smaller than uuid_b even when it was actually bigger.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I00dfe935f21514d78bd586bb52729c5e147c8a36
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1912
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Tue, 13 Aug 2019 18:20:34 +0000 (14:20 -0400)]
sink.ctf.fs: isa_length is now architecture_bit_width
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I30cac20017f7508ed170b936a7b90ebc2916b913
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1905
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Jonathan Rajotte [Tue, 13 Aug 2019 22:50:09 +0000 (18:50 -0400)]
src.ctf.lttng-live: consider youngest only when timestamp is smaller
When the timestamps are equals perform further comparison.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: Id7567019177ab2e2d4ad1ab1e8840aa07f9ba471
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1913
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Tue, 13 Aug 2019 22:51:32 +0000 (18:51 -0400)]
Use uuid as first comparison criteria
This is most important in the message ordering reproducibility against
multiple sources where some might populate the trace name and some don't.
A good example of a source not populating the trace name is lttng-live.
It facilitates the testing of lttng-live source plugin since it is
possible to compare it to the src.ctf.fs plugin directly.
Philippe adds: there's a trace UUID property in a CTF metadata stream,
but no trace name property. The trace name is entirely up to the
specific CTF source component class. Therefore it's safer to use the
UUID to distinguish unique traces.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I39f3c513e27bffd51ca02b089c097ac406e3316e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1914
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Mon, 12 Aug 2019 22:07:09 +0000 (18:07 -0400)] 
src.ctf.lttng-live: cleanup: rename functions and variables
Use term "youngest" whenever possible to signify our current best
candidate.
Rename `lttng_live_iterator_next_on_stream()` ->
`lttng_live_iterator_next_msg_on_stream()` to signify that it's
returning a message.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I7dcfccda30c87662209ac87e3bcda31699d3ea44
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1889
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Mon, 12 Aug 2019 21:35:01 +0000 (17:35 -0400)]
Fix: src.ctf.lttng-live: missing ordering within traces and streams
Issue
=====
Sorting of messages with the same timestamps are different from the
messages sorted by the a `flt.utils.muxer` component. This is caused by
the fact that we are not sorting messages within sessions and traces
but only across the different session currently handled by the
component.
Indeed, we need to sort messages at across three locations in the code
to ensure proper ordering: sessions, traces, and streams.
Solution
========
Handle messages with the same timestamps at each of these three
locations using the `common_muxing_compare_messages()` function.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Iadc66f3b6a12f34522f6364b3f47f15eefa87930
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1888
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Mon, 12 Aug 2019 03:01:00 +0000 (23:01 -0400)] 
bt2: `TraceCollectionMessageIterator`: find greatest MIP version
This patch makes the `TraceCollectionMessageIterator` object use
bt2.get_greatest_operative_mip_version() to find the greatest operative
MIP version of all the components to create, including the stream
intersection `flt.utils.trimmer` components, and then use the result to
create the graph with bt2.Graph().
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Id9024a2343d0e99384ae62bd77754912a4267dac
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1881
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sun, 11 Aug 2019 06:36:41 +0000 (02:36 -0400)]
cli: find greatest operative MIP version to create `run` command's graph
This patch makes the CLI's `run` command use
bt_get_greatest_operative_mip_version() to find the greatest operative
MIP version of all the components to create, including the stream
intersection `flt.utils.trimmer` components, and then use the result to
create the graph with bt_graph_create().
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I2e791ac9c8806457fa850d0dad72ea3ab6912f08
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1879
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sun, 11 Aug 2019 03:24:13 +0000 (23:24 -0400)]
lib: bt_graph_create(): accept MIP version
This patch makes bt_graph_create() accept the message interchange
protocol (MIP) version with which to configure the graph.
This MIP version will usually come from
bt_get_greatest_operative_mip_version() or bt_get_maximal_mip_version().
The components can access the configured MIP version with
bt_self_component_get_graph_mip_version().
As of this patch the only possible MIP version is 0. bt_graph_create()
does not accept a MIP version that is greater than
bt_get_maximal_mip_version() because almost all the library functions
need to know the configured MIP version to behave accordingly, should
the semantics change in the future.
In Python, Graph.__init__() accepts a `mip_version` parameter which
defaults to 0. A Python component can get the configured MIP version of
its operating graph with the `_UserComponent._graph_mip_version`
property.
All the sites where bt_graph_create() is called pass 0 to it. This is
correct for tests as we always control the component classes and
initialization parameters, so we know that 0 is always supported by
them. The locations where bt_get_greatest_operative_mip_version() must
be used are the CLI and `bt2.TraceCollectionMessageIterator`. Those
sites will be addressed by subsequent patches.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I324a317922a8caa5557043ae02b97e58fc91d92d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1878
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sun, 11 Aug 2019 02:38:39 +0000 (22:38 -0400)]
bt2: add bt2.get_{greatest_operative,maximal}_mip_version()
Those new functions wrap bt_get_greatest_operative_mip_version() and
bt_get_maximal_mip_version().
bt2.get_greatest_operative_mip_version() excepts a list of
`bt2.ComponentDescriptor` objects. The function builds a component
descriptor set (bt_component_descriptor_set_create()) and adds the
corresponding component descriptors to it and then calls
bt_get_greatest_operative_mip_version().
bt_get_greatest_operative_mip_version() translates
`BT_GET_OPERATIVE_MIP_VERSION_STATUS_NO_MATCH` into `None`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Iaeb36b976f3c651fa82a86de2e835f782d682299
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1877
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Mon, 12 Aug 2019 21:19:46 +0000 (17:19 -0400)]
bt2: add "get supported MIP versions" method support
This patch makes the `bt2` package wrap the library's "get supported MIP
versions" component class method.
In a Python component class, it is expected that the optional
_user_get_supported_mip_versions() method returns either a
`bt2.UnsignedIntegerRangeSet` object or what such an object needs to be
constructed. In other words, the method can return, for example:
return [0, 1, (3, 5)]
Like in the library, the default _user_get_supported_mip_versions()
returns 0 (only protocol 0 supported):
def _user_get_supported_mip_versions(cls, params, obj, log_level):
return [0]
_bt_get_supported_mip_versions_from_native() returns a new reference on
this unsigned integer range set. Then, the native
component_class_get_supported_mip_versions() copies the ranges from the
returned range set to the range set to fill. I believe this makes
_user_get_supported_mip_versions() easier to use than receiving a range
set object and filling it like it's done in C.
The feature is not tested in this patch because the "get supported MIP
versions" method is not used within the library.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I9e0c6f0fcac23d2224401c1fb1c258f061642767
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1875
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sat, 10 Aug 2019 19:46:37 +0000 (15:46 -0400)]
bt2: _IntegerRangeSet.add(): make it possible to add a single value
This patch makes it possible to do:
my_range_set.add(23)
which is the equivalent of:
my_range_set.add((23, 23))
It also makes it possible to do:
bt2.SignedIntegerRangeSet(-17)
bt2.UnsignedIntegerRangeSet(17)
which is the equivalent of:
bt2.SignedIntegerRangeSet(-17, -17)
bt2.UnsignedIntegerRangeSet(17, 17)
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I44bf1a257422fb9e74835a300eeda948a8e5d067
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1874
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sat, 10 Aug 2019 18:53:32 +0000 (14:53 -0400)]
bt2: add `ComponentDescriptor` class and test
The goal of a `ComponentDescriptor` object is to eventually be used
within an array of descriptors passed to a function which computes an
effective message interchange protocol version with the help of version
support component class methods.
I choose not to wrap the `bt_component_descriptor_set` API as this is
so simple and will be used at a single location.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ifef9b4fb0a7cf0278dbf8bed2d18d0b47934a272
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1873
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sat, 10 Aug 2019 22:40:50 +0000 (18:40 -0400)]
lib: add bt_get_{greatest_operative,maximal}_mip_version() functions
This patch adds the bt_get_greatest_operative_mip_version() and
bt_get_maximal_mip_version() functions.
The bt_get_greatest_operative_mip_version() accepts a set of component
descriptors and returns, on success, the (greatest) operative message
interchange protocol (MIP) version for all those descriptors.
bt_get_greatest_operative_mip_version() can fail if any component class's "get
supported MIP versions" fails. It can also return
`BT_GET_OPERATIVE_MIP_VERSION_STATUS_NO_MATCH` if any component
descriptor does not support MIP version 0, the only MIP version
currently supported by the the library.
bt_get_maximal_mip_version() returns the maximal MIP version supported
by the library (currently 0).
The purpose of both functions is to be used eventually to create a graph
of which all the components and the library itself operate using a
specific MIP. You can use bt_get_greatest_operative_mip_version() to find the
greatest compatible MIP version to be used by that set of component
descriptors, or you can use bt_get_maximal_mip_version() to get the
greatest MIP version possible for that library.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: If3875b3d76ddca2b4d58a18a829394eadb3bcd39
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1872
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sat, 10 Aug 2019 22:27:55 +0000 (18:27 -0400)]
Fix: {graph,query-executor}.h: missing `STATUS_` before `MEMORY_ERROR`
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I24307beb9896b754f13c0d887ef3ed03ee6596b2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1871
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sat, 10 Aug 2019 19:18:17 +0000 (15:18 -0400)]
lib: add "get supported MIP versions" method support
This patch adds a new optional method to component classes: get
supported message interchange protocol (MIP) versions.
As of this patch, you can set the "get supported MIP versions" method of
a component class, but it's not used.
The "message interchange protocol" term is very long so I chose to use
the acronym "mip" in the API. The API documentation will make this more
clear.
The method is considered a "class method"; no component exists yet when
calling it (like the query method). The method receives the component
class, the initialization parameters, the initalization method custom
data, and an existing, empty unsigned integer range set. The method's
purpose is to fill this integer range set with all the MIP versions it
supports depending on the initialization configuration. This makes it
possible to support specific MIP versions based on an input path or URI,
for example.
It is a postcondition that the method adds at least one MIP version to
the unsigned integer range set.
The "get supported MIP versions" method is optional: not implementing it
corresponds to adding [0, 0] to the unsigned integer range set. This
makes it possible to implement a component class of which the instance
operates according to the MIP protocol of Babeltrace 2.0.
The "get supported MIP versions" method can fail, but it cannot return
a "try again" status.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ib26ec9e90b47a3d3e51cc56465a264de9d02b97c
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1870
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sat, 10 Aug 2019 18:56:34 +0000 (14:56 -0400)]
lib: graph.c: add missing `enum` and `const` keywords
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I87eb15507f671073d3bea4d2dbb0d8dea2a63f85
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1869
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sat, 10 Aug 2019 18:52:21 +0000 (14:52 -0400)]
lib: add component descriptor set API
This patch adds an API to the library to create an empty set of
component descriptors and to add descriptors to it.
A component descriptor contains:
* A component class.
* Component initialization parameters.
* Component initialization custom data.
The motivation is to eventually be able to get an effective/compatible
message interchange protocol version number from such a set.
As of this patch, the API is available, but it's not used anywhere.
A component descriptor set is a shared object.
You add a component descriptor to a component descriptor set with
bt_component_descriptor_set_add_descriptor(). Internally, the descriptor
gets appended to one of the source, filter, or sink descriptor array to
eventually have bt_component_descriptor_get_*_descriptor_count() and
bt_component_descriptor_set_borrow_*_descriptor_by_index_const()
functions if needed.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I337d4c596bd988b20849700daf9747a3f188450d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1868
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Mon, 12 Aug 2019 19:46:05 +0000 (15:46 -0400)]
bt2: py_plugin.py: cache `_PluginInfo` objects
This patch makes _try_load_plugin_module() cache the `_PluginInfo`
objects it creates. The cache, `_plugin_infos`, maps a Python plugin
absolute path to a `_PluginInfo` object.
This avoids loading the Python module twice, which can create problems,
for example destroying component classes earlier than expected when the
new module replaces the old one in `sys.modules`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I55397a1d5e65de1f6495dccc23e1134bd1127631
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1887
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Mon, 12 Aug 2019 18:59:13 +0000 (14:59 -0400)] 
bt2: remove unused imports
I ran flake8 and removed unused import warnings in all files, except
__init__.py, where the imports are present but not used in that file on
purpose.
Change-Id: Ib422a25b042eb1ba0a68e5c254947b5cfe3b7398
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1886
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 12 Aug 2019 18:37:58 +0000 (14:37 -0400)]
bt2: raise an exception if an AutoSourceComponentSpec produces no component
We don't do anything special at the moment if an AutoSourceComponentSpec
passed to TraceCollectionMessageIterator produces no component. The
created graph will have no source component and the message iterator
will produce no message:
In [5]: list(bt2.TraceCollectionMessageIterator('/yomadame'))
08-12 14:43:03.938 20284 20284 W CLI-CFG-SRC-AUTO-DISC auto_discover_source_components@autodisc.c:755 No trace was found based on input `/yomadame`.
Out[5]: []
This patch changes the behavior to be a bit more strict. If any
AutoSourceComponentSpec passed to TraceCollectionMessageIterator ends up
producing no component, an exception is raised. In the following example, the
directory `/home/smarchi/lttng-traces` does contain some valid traces, and
therefore produces some components, whereas the string `/yomadame` does not
produce any.
In [2]: bt2.TraceCollectionMessageIterator(['/home/smarchi/lttng-traces', '/yomadame'])
08-12 14:45:06.433 24428 24428 W CLI-CFG-SRC-AUTO-DISC auto_discover_source_components@autodisc.c:755 No trace was found based on input `/yomadame`.
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-2-
107461e1b36b> in <module>
----> 1 bt2.TraceCollectionMessageIterator(['/home/smarchi/lttng-traces', '/yomadame'])
~/build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/trace_collection_message_iterator.py in __init__(self, source_component_specs, filter_component_specs, stream_intersection_mode, begin, end, plugin_set)
292 ]
293 self._src_comp_specs += _auto_discover_source_component_specs(
--> 294 auto_src_comp_specs, plugin_set
295 )
296
~/build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/trace_collection_message_iterator.py in _auto_discover_source_component_specs(auto_source_comp_specs, plugin_set)
194
195 msg = 'Some auto source component specs did not produce any component: ' + ', '.join(unused_inputs)
--> 196 raise RuntimeError(msg)
197
198 return comp_specs
RuntimeError: Some auto source component specs did not produce any component: /yomadame
Change-Id: I6c6df9fad18915e294559948812a8213596affdf
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1885
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 12 Aug 2019 17:43:32 +0000 (13:43 -0400)]
tests: remove bt_diff_cli_sorted from utils.sh
This function is not used anymore.
Change-Id: Iebca03d9193a4da20bb7a4511345c39bf62d31b0
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1884
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 12 Aug 2019 16:25:59 +0000 (12:25 -0400)]
tests: use os.pathsep when appending to BABELTRACE_PLUGIN_PATH
On Windows, we expect BABELTRACE_PLUGIN_PATH entries to be separated
with ';', not ':' (like PATH). Python conveniently provides an
OS-dependent definition for this.
Change-Id: Ie53af04e933258963b8495c72e7607a623e83ddb
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1883
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 12 Aug 2019 16:13:58 +0000 (12:13 -0400)]
cli: mention path separator for Windows in comment
This function uses bt_common_append_plugin_path_dirs, which uses
G_SEARCHPATH_SEPARATOR to split up paths. G_SEARCHPATH_SEPARATOR is ';'
on Windows.
Change-Id: If2a88d5cd138d0e1ab186e1e33fda9bb3ceaec5a
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1882
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Sun, 11 Aug 2019 15:33:40 +0000 (11:33 -0400)]
test_plugin.py: do not search in user/system directories for tests
In `test_plugin.py`, make bt2.find_plugin() not search in user and
system directories so that it needs to find the specific plugin in the
test's search path (given by `BABELTRACE_PLUGIN_PATH`) or fail.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I9bad521d079ce602fa152d9e24ab6b145e8edce9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1880
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Sun, 11 Aug 2019 02:31:41 +0000 (22:31 -0400)]
Fix: bt2: replace bt2.CreationError instances with bt2._MemoryError
Commit
4acc866e8298 ("bt2: rename CreationError to MemoryError, handle it in and out of Python bindings")
removed bt2.CreationError in favor of bt2.MemoryError (now
bt2._MemoryError). However, some usages of bt2.CreationError were
introduced afterwards, probably by patches that were in the pipeline.
Replace them with bt2._MemoryError.
Change-Id: Id3cb6542e507b1e41c165ca61df21069d72fb44f
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1867
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 5 Aug 2019 18:45:38 +0000 (14:45 -0400)]
bt2: add auto source discovery support to TraceCollectionMessageIterator
This patch makes TraceCollectionMessageIterator capable of automatically
detecting source components from given input strings, similar to how the
command line interface does.
A new type AutoSourceComponentSpec is introduced. The user can
instantiate this type, passing a mandatory input string (equivalent to a
non-option argument on the CLI). The user can then pass instances of
AutoSourceComponentSpec in the source_component_specs list when creating
a TraceCollectionMessageIterator. In
TraceCollectionMessageIterator.__init_, we run the automatic source
algorithm on all inputs passed through AutoSourceComponentSpec
objects. We create ComponentSpec from the results of the auto discovery
and append them to the source component spec list. The rest of the
TraceCollectionMessageIterator then works as before.
The user can attach params and log levels to AutoSourceComponentSpec
inputs. The semantic for those is the same as for the CLI. There is
also an `obj` parameter, allowing to pass an arbitrary Python object.
The semantic for this one is the same as the log level: if an
auto-discovered component comes from multiple inputs, it will receive
the obj of the last input that contributed to getting it instantiated
that has a non-None `obj`.
For convenience, some shorthands when creating a
TraceCollectionMessageIterator are equivalent to using an
AutoSourceComponentSpec. For example, these three forms are equivalent:
- TraceCollectionMessageIterator('foo')
- TraceCollectionMessageIterator(AutoSourceComponentSpec('foo'))
- TraceCollectionMessageIterator([AutoSourceComponentSpec('foo')])
Implementation details
----------------------
New test cases are added to test the new feature. They are based on
the corresponding CLI tests, in that they test equivalent scenarios
(same inputs, expecting the same results).
In order for TraceCollectionMessageIterator to find the test plugin
(both bt_plugin_test.py files), the only way I found was to amend
BABELTRACE_PLUGIN_PATH for the duration of the test.
Change-Id: I0e1100d850c920723861609cfa14707f8c669892
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1826
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Fri, 9 Aug 2019 21:57:56 +0000 (17:57 -0400)]
tests: move auto source discovery test artifacts
These test artifacts will soon be used to test the auto source discovery
feature in the Python bindings, so move them out of the `cli` directory.
Change-Id: Ia6763a42657c988442436e088fbf0e96de5baf49
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1865
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Mon, 5 Aug 2019 15:05:08 +0000 (11:05 -0400)]
Move autodisc to its own convenience library
Since we'll want to make TraceCollectionMessageIterator have an
automatic source discovery feature, like the CLI has, move the auto
discovery code to its own library, which doesn't have any dependency on
the CLI.
To avoid dependencies on the CLI, here are the changes done to the auto
discovery code:
- To avoid depending on cli_query, use a simplified version,
simple_query, included in the auto discovery code. It doesn't handle
retrying if the status is TRY_AGAIN, as it is not very likely that a
babeltrace2.support-info query implementation will need to do that, as
we want them to be very fast (we can always revisit it in the future).
- Avoid using BT_CLI_LOGE_APPEND_CAUSE, replace with the new
BT_AUTODISC_LOGE_AND_APPEND.
- Don't use require_loaded_plugins, get_loaded_plugins_count and
borrow_loaded_plugin, as those are CLI-specific. Instead, the caller of
auto_discover_source_components must provide an array of plugins.
- plugin_restrict, used to filter down to a single plugin by name, is
removed. Instead, it is the caller's responsibility to pass a single
plugin to `plugins` if it wants to restrict the search to a single
plugin.
A side effect of moving the code is that a warning is now printed by
default when no input is found from non-option argument. I think this
is desirable to have this warning printed, so I have opted to update the
test case to expect it.
Change-Id: Id4009976430126d926eac46ff044365a6b2bc006
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1825
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Fri, 9 Aug 2019 21:17:35 +0000 (17:17 -0400)]
tests: make auto-source-discovery-{log_level,params} tests use sink.text.details
For the same rationale as the previous patch, change how the test
component classes for test_auto_source_discovery_params and
test_auto_source_discovery_log_level report what they receive from a
print to stdout to stream name.
Change-Id: I62405ccd1fbdbe708fbec5dacc52894d4060dfb0
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1864
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Fri, 9 Aug 2019 20:08:37 +0000 (16:08 -0400)]
tests: make auto-source-discovery-grouping test use sink.text.details
This patch changes the way that test component classes for the auto
source discovery grouping CLI test reports the inputs passed to each
instantiated component.
Currently, they print to stdout the input names
in the components' __init__. That is then check by the test script
(test_auto_source_discovery_grouping). The
TraceCollectionMessageIterator class in the Python bindings will soon
have the capability to automatically discover source components.
However, it's not practical to use the same test method in Python.
To facilitate this, make the test component classes report the inputs
they receive by sending a stream beginning message, where the stream
name contains the input names. The eventual Python test will be able to
inspect the stream name to validate that the right inputs were passed to
each component.
Since the information is now passed in messages, the
test_auto_source_discovery_grouping test now uses the details sink.
Change-Id: Ie5c12e390f5b68fb50a6d18c53463860fb7a1c99
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1863
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Thu, 8 Aug 2019 17:58:47 +0000 (13:58 -0400)]
tests: print executed CLI command line in bt_cli
When debugging a test, it is often useful to have the complete
CLI command line executed, so we can copy paste it and reproduce
the failure by hand.
This patch makes bt_cli echo the executed CLI command line to
stderr, such that it will show up when running one test by hand and in
the test logs when running "make check".
This allows simplifying bt_diff: we no longer have to pass the
CLI arguments to have them included in the error message, since
the full CLI command line will typically be printed just above.
Change-Id: I64f8c80f648d06319efe4e7544447183ba8069cc
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1839
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Thu, 8 Aug 2019 15:50:46 +0000 (11:50 -0400)]
tests: make bt_diff check a single file pair
bt_diff currently accepts 4 file arguments:
- stdout expected
- stdout actual
- stderr expected
- stderr actual
To allow for more flexibility, change it to check a single pair. Tests
that need to check for both stdout and stderr will need to call it
twice.
Change-Id: I400d251e77931a3a457214fabdc59f4c56be7ce1
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1838
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Thu, 8 Aug 2019 15:09:11 +0000 (11:09 -0400)]
tests: add bt_cli util function
Add a function that simply runs the CLI, redirecting stdout and
stderr to the specified files. This will allow more flexible tests than
the existing "diff" function.
Change-Id: I8319c4a74a0baa3b40afb871bfb0605ac81c3270
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1837
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Sat, 10 Aug 2019 18:48:23 +0000 (14:48 -0400)]
m4/pprint.m4: check that the terminal supports at least 8 colors
Checking for 8 colors is enough here and makes rudimentary terminals
have colors in the report.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Icf49c3f24aa3546a81a52f4627b9f39ab88f929c
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1866
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Fri, 9 Aug 2019 20:35:54 +0000 (16:35 -0400)]
Fix: flt.utils.muxer: Explicit null dereferenced
Fixes coverity #
1404006
ID
1404006 (#1 of 1): Explicit null dereferenced (FORWARD_NULL)
18. var_deref_op: Dereferencing null pointer *muxer_upstream_msg_iter
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I055200226ab13653b219a084f38a2dd9bb16664b
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1859
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Fri, 9 Aug 2019 20:30:09 +0000 (16:30 -0400)]
Fix: muxer: check for MUXER_MSG_ITER_CLOCK_CLASS_EXPECTATION_NONE in validate_new_stream_clock_class
There is a problem currently when trying to send multiple streams that
don't have a clock class (which is a supported use case) to a muxer
instance.
The first time we enter validate_new_stream_clock_class without a clock
class, we set the expectation to
MUXER_MSG_ITER_CLOCK_CLASS_EXPECTATION_NONE. The second time we enter
without a clock class, we wrongfully go to the error case. We are
missing a check for MUXER_MSG_ITER_CLOCK_CLASS_EXPECTATION_NONE, which
is the expected and valid case. This patch adds it.
Also, the error message is erroneous. If we are in that situation, we
expect a stream class _without_ a clock class, but are met with a stream
class _with_ a clock class. Fix that at the same time.
Change-Id: Iea692578498ea87a33ecba5d2fd54b2980056cd0
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1862
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Fri, 9 Aug 2019 20:52:42 +0000 (16:52 -0400)]
Tests: flt.utils.muxer: run `black` on bt_plugin_muxer_test.py
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I83faf79c295130976aa741af080e59253a3083f7
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1860
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Fri, 9 Aug 2019 18:40:23 +0000 (14:40 -0400)]
src.ctf.lttng-live: use `common_muxing_compare_messages()` to break TS ties
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I2dc0087d995ea06e7ac9c11e4e5cd54958e932aa
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1848
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Fri, 9 Aug 2019 17:14:34 +0000 (13:14 -0400)]
Add message compare common func/library; make flt.utils.muxer use it
Change-Id: I8efe1e22fa7badd1328eb46f35315bb40936c46a
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1847
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Fri, 9 Aug 2019 21:24:59 +0000 (17:24 -0400)]
logging/log.h: always write the source file name and line number
We don't need to protect our source file names and line numbers in the
logs of a production build, so always write them.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I837dd0cfc4cf3279aa26f813f3ebae2b7f4d4d55
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1861
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Fri, 9 Aug 2019 19:01:32 +0000 (15:01 -0400)]
Fix: tests: flt.utils.muxer: missing '../' in path to utils.sh
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ia4b6d3275ea7d81a9fce770f34be700fe8c99e48
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1849
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Fri, 9 Aug 2019 19:11:52 +0000 (15:11 -0400)]
Fix: visitor-generate-ir.c: dereference after `NULL` check
Found by Coverity Scan.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I8b0e43d57b8a4bdfb39632216b2090e2f13af6c4
Philippe Proulx [Fri, 9 Aug 2019 19:09:32 +0000 (15:09 -0400)]
Fix: plugin-so.c: dereference after `NULL` check
Found by Coverity Scan.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I44f05edbeec6541b6b65f1f5ead55511d1085a0e
Philippe Proulx [Fri, 9 Aug 2019 19:06:37 +0000 (15:06 -0400)]
Fix: plugin-so.c: dereference before `NULL` check
Found by Coverity Scan.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I36ae818417b9f9902a561641e83d6e24837be462
Philippe Proulx [Fri, 9 Aug 2019 19:04:48 +0000 (15:04 -0400)]
Fix: common.c: logically dead code
Found by Coverity Scan.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ib870cf62d6028bb26259d448e5dbee59dacb9759
Philippe Proulx [Fri, 9 Aug 2019 19:03:24 +0000 (15:03 -0400)]
Fix: cli/babeltrace2.c: logically dead code
Found by Coverity Scan.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Icc8f3d6736cc086a18764d8c671683c226404e25
Philippe Proulx [Fri, 9 Aug 2019 19:00:35 +0000 (15:00 -0400)]
Fix: dmesg.c: deference before `NULL` check
Found by Coverity Scan.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I841e1434341037ddf482445464c3fd428f08edd7
Philippe Proulx [Fri, 9 Aug 2019 18:57:00 +0000 (14:57 -0400)]
Fix: translate-trace-ir-to-ctf-ir.c: dereference after `NULL` check
We only need to call must_protect_variant_option_name() when the tag
field class is not created by the component (`fc->tag_is_before`).
Furthermore, the local `must_protect` variable is not used, so the
corresponding must_protect_variant_option_name() parameter is not
either. They are both removed. Because the
must_protect_variant_option_name() function actually sets the name of
the option to use (protecting it or not), it's renamed to
maybe_protect_variant_option_name().
Found by Coverity Scan.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ieccc0bcddc5570f473d4303f7cc470a56af65a9e
Philippe Proulx [Fri, 9 Aug 2019 18:44:44 +0000 (14:44 -0400)]
Fix: argpar.c: logically dead code
Found by Coverity Scan.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Id67092bcdb0a0ebf7aa29186668457596c177ca6
Philippe Proulx [Fri, 9 Aug 2019 18:42:53 +0000 (14:42 -0400)]
Fix: babeltrace2-cfg-cli-args.c: uninitialized pointer read
Found by Coverity Scan.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Iaf8c35247a61562ab745b734d523831f2ea68be8
Francis Deslauriers [Thu, 8 Aug 2019 17:17:40 +0000 (13:17 -0400)]
Fix: src.ctf.lttng-live: omitting stream end on stream hang up
Issue
=====
When receiving a LTTNG_VIEWER_INDEX_HUP request status, a
`src.ctf.lttng-live` component quickly removes the associated stream
iterator and move on to the next request without sending the stream end
message to properly close the stream. This breaks the guarantee that
stream end messages are always present.
Solution
========
Whenever we receive an HUP status from the server, we trigger the
`bt_msg_iter` code to hit an BT_MSG_ITER_MEDIUM_STATUS_EOF so it
generates the stream end message that closes the stream.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I89924ee95b35ca62c103651ad44c9eaf71f6df53
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1840
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Fri, 9 Aug 2019 13:44:51 +0000 (09:44 -0400)]
Fix: muxer.c: compare_events(): initialize `ret` to 0
GCC complains when this `ret` is not initialized:
muxer.c: In function ‘compare_events’:
muxer.c:993:9: error: ‘ret’ may be used uninitialized in this
function [-Werror=maybe-uninitialized]
return ret;
It reaches `return ret` with an uninitialized `ret` when all the tests
fail to find a difference (which can happen).
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ib0541150173dbc845ba5784abfc5e131412d2663
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1846
Tested-by: jenkins <jenkins@lttng.org>
Jérémie Galarneau [Thu, 8 Aug 2019 04:45:56 +0000 (00:45 -0400)] 
src.ctf.lttng-live fix: remove overly strict check for session name
The live component checks that the path of a stream contains a
session's name. While this check works when interacting with pre-2.11
peers, there is no reason for a session's name to necessarily be in a
stream's path.
The LTTng relay daemon, as of 2.11, now expresses its path relative to
a session's output path, the session's name no longer appears in a
stream's path.
There are no comments explaining the reason for this check and it is
not present in babeltrace 1.5.x. Hence, it appears safe to remove it.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: Ic5499976ed45f1f4ae17056b9e881a8455898a44
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1845
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Fri, 19 Jul 2019 21:46:35 +0000 (17:46 -0400)]
Tests: flt.utils.muxer: add message ordering tests
* Add a test that checks the basic ordering of messages based on
timestamps.
* Add a test that checks the basic ordering of messages with the same
timestamps but different trace name, stream class id, or stream id.
* Add tests that check the ordering of messages with the same timestamps
and types.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I217119a00828288d0a2471c26b387a670e62608a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1726
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Francis Deslauriers [Wed, 17 Jul 2019 22:23:35 +0000 (18:23 -0400)]
flt.utils.muxer: deterministically order messages of same timestamps
Issue
=====
The ordering of messages with the exact same timestamp is runtime
dependent. It is causing flakiness in tests using a `sink.text.details`
sink component where such differences in ordering is checked.
Note: It's common to have messages with the same timestamps because that
messages with no clock snapshot are assumed to happen at the same time
that of the previous message.
Solution
========
We need to order messages deterministically even when timestamps are the
same. This commit adds functions to order, in a predictable manner,
messages that have the same timestamp (or no timestamp at all) so that
they are sorted in the same way every time a trace is read.
With this commit:
* We first try to sort the messages by their trace name, stream class
id, or stream id,
* If not possible, we check if the messages have different types and if
they do, we sort them using the following scheme:
SB < PB < EV < DE < MI < PE < DP < SE
* If they are of the same type, we sort them comparing their properties.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Id6fc1fe8b19d02ec1804e54255a73c64820d1e13
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1719
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jérémie Galarneau [Thu, 8 Aug 2019 02:21:02 +0000 (22:21 -0400)]
Fix: extra '%' causes babeltrace2 to abort() in logging code
An extra '%' in a TRACE logging statement causes
bt_common_custom_vsnprintf() to interpret '!' as a conversion
specifier. This causes handle_conversion_specifier_bt() to invoke
abort().
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: Ia077616ea84559dfc3abb57d6404d06a6e9b7baf
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1836
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Wed, 7 Aug 2019 03:48:44 +0000 (23:48 -0400)]
Fix: bt2: use unsigned iteration variable
This fixes the following compiler diagnostic (gcc 9.1.0):
bt2/native_bt.c: In function ‘_wrap_field_class_enumeration_unsigned_get_mapping_labels_for_value’:
bt2/native_bt.c:9387:25: error: comparison of integer expressions of different signedness: ‘int’ and ‘uint64_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
9387 | for (int i = 0; i < *arg4; i++) {
| ^
Change-Id: I543c61d64c0cf7b42cb242bdbb7b71cbf67dbe48
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1834
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Tue, 6 Aug 2019 16:49:54 +0000 (12:49 -0400)]
Fix: `ctf` plugin: msg-iter.c: do not switch packet twice
Issue
=====
In read_packet_header_begin_state(), bt_msg_iter_switch_packet() is
called systematically at the beginning of the state. However,
buf_ensure_available_bits() can fail below (the medium can return
`BT_MSG_ITER_MEDIUM_STATUS_AGAIN`), and read_packet_header_begin_state()
will be called again next time, calling bt_msg_iter_switch_packet()
twice for the same packet.
One of the problems of calling bt_msg_iter_switch_packet() twice is
that, the first time, the current snapshots are copied to the previous
snapshots and the current snapshots are reset; the second time, the
current (reset) snapshots are copied to the previous snapshots. This
shatters the whole snapshot states. Because the medium installed by
`src.ctf.lttng-live` can return `BT_MSG_ITER_MEDIUM_STATUS_AGAIN`, all
the messages that depend on the previous snapshots, like the discarded
events/packets messages, are wrong.
Solution
========
In `msg-iter.c`, add a new `STATE_SWITCH_PACKET` state which is
dedicated to doing what bt_msg_iter_switch_packet() used to do. Once it
completes, the next state is `STATE_DSCOPE_TRACE_PACKET_HEADER_BEGIN`.
read_packet_header_begin_state() does not switch packets itself now (the
`STATE_SWITCH_PACKET` state already occured at this point), so it can
safely occur twice.
Known drawbacks
===============
There's one more state to handle per packet, but this have a significant
performance hit.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reported-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I24dd2482762c61e339ab0310fddee6c9aa69d438
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1833
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Tue, 6 Aug 2019 16:47:26 +0000 (12:47 -0400)]
src.ctf.lttng-live: `live_viewer_connection`: make `obj` first member
This fixes a leak. The `bt_object` member must always be the first
because the internal API casts any `void *` parameter to `bt_object *`.
The object API was not getting the correct reference count here.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I15ccf573e0b7f5078d434852e915f90d685e65e1
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1832
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Tue, 6 Aug 2019 00:31:35 +0000 (20:31 -0400)]
lib, bt2: make query method receive custom data
Just like a component class initialization method can receive custom
user data (`init_method_data`) along with generic parameters, this patch
makes this possible for a query operation.
Library changes
===============
On the library side, this patch introduces the new
bt_query_executor_create_with_method_data() function which is like
bt_query_executor_create(), but accepts a `void *` user data parameter.
Internally, bt_query_executor_create() calls
bt_query_executor_create_with_method_data() with `NULL` as the
`method_data` parameter.
The component class query method type is changed to:
typedef bt_component_class_query_method_status
(*bt_component_class_*_query_method)(
bt_self_component_class_* *comp_class,
bt_private_query_executor *query_executor,
const char *object, const bt_value *params,
void *method_data, const bt_value **result);
The component class is guaranteed, for a given query executor, to always
being passed the same `method_data` value to its query method.
Python bindings changes
=======================
QueryExecutor.__init__() accepts a new, optional `method_obj` parameter
which defaults to `None`.
`None` gets translated to `NULL` as the `method_data` parameter, and
vice versa when a Python query method is called from the native part.
QueryExecutor.__init__() validates that you can only pass a Python
method object when the component class you query is a Python component
class.
Because you build the query executor and then make a query operation in
two steps, a Python reference to the method object must exist as long as
the query executor exists. This patch makes it reside within the Python
query executor wrapper (`_method_obj` attribute) as there's no way
currently to lose the Python wrapper and retrieve it through a library
call. If this becomes possible in the future, then we can introduce
query executor destruction listeners and add one from the Python
bindings to put the `PyObject *` reference when the query executor is
destroyed. A new test checks this
(test_query_with_method_obj_del_ref()), deleting the test's reference
between the query executor's construction and the actual query
operation. I tried without making the query executor keep a strong
reference and the test causes a segmentation fault, as expected, which
shows that the test is valuable.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I454cdad039fab5b7d46e46762243f212c200f114
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1831
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Mon, 5 Aug 2019 22:01:09 +0000 (18:01 -0400)]
Add bt_graph_add_simple_sink_component() tests
As of this patch, the tests are basic and only check that the statuses
returned by the user functions are correctly translated and returned by
bt_graph_run_once(). The user initialization function status is not
returned by bt_graph_add_simple_sink_component() because the
initialization function is called within the sink component's "graph is
configured" method.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Id8ccfd7df4e4a8f0a39af386ed4ad13fe657f86a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1830
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Philippe Proulx [Mon, 5 Aug 2019 21:57:27 +0000 (17:57 -0400)]
Fix: src/lib/graph/graph.h: do not override `comp_status` value
This made, for example, bt_graph_run() return
`BT_GRAPH_RUN_STATUS_ERROR` when the sink component's "graph is
configured" method returned
`BT_COMPONENT_CLASS_SINK_GRAPH_IS_CONFIGURED_METHOD_STATUS_MEMORY_ERROR`,
therefore not percolating the correct status.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ic89d9315f8bc1c3226a14955c9e6419462d5e588
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1829
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Philippe Proulx [Mon, 5 Aug 2019 20:15:32 +0000 (16:15 -0400)]
lib: add bt_graph_add_simple_sink_component()
This patch adds bt_graph_add_simple_sink_component(), an easy way to add
a simple sink component which has a single message iterator iterating a
single input port named `in` and which calls user-provided functions at
specific locations.
This makes it possible to create a sink component without having to:
* Create a sink component class with initialization, "graph is
configured", "consume", and finalization methods.
* Create an input port in the component class's initialization method.
* Create an input port message iterator in the component class's "graph
is configured" method.
* Put the input port message iterator's reference in the component
class's finalization method.
The goal of this new function is to make it easy to get messages at the
sink endpoint of a graph, just like we used to do with the output port
message iterator concept (but the graph model is honored now). The user
must still call bt_graph_run() or bt_graph_run_once() to make her
consume function (see details below) called: the added simple sink
component is not special in any way.
bt_graph_add_simple_sink_component() receives three function pointers
(and custom user data):
Initialize (optional):
Called after the simple sink component creates the input port
message iterator in the "graph is configured" method.
The user function receives the message iterator to perform any
initial task.
Consume:
Called for each "consume" method call of the simple sink component.
The user function receives the message iterator and can get the next
messages with bt_self_component_port_input_message_iterator_next()
as usual.
Finalize (optional):
Called when the simple sink component is finalized.
The message iterator is not available at this point.
I'm not wrapping this one in Python because it's so easy to replicate
with our bindings:
class _SimpleSink:
def __init__(self, params, consume_cb):
self._consume_cb = consume_cb
self._add_input_port('in')
def _user_graph_is_configured(self):
self._msg_iter = self._create_input_port_message_iterator(
self._input_ports['in']
)
def _consume(self):
self._consume_cb(self._msg_iter)
def _mein_consume(msg_iter):
...
...
graph.add_component(_SimpleSink, 'simple', _mein_consume)
...
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I02aaae16215160cd861c2a76793adddf808202d6
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1828
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Mon, 5 Aug 2019 20:35:56 +0000 (16:35 -0400)]
lib: remove colander sink component class
This is not used anymore.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ib0592d3adf02f82159e262184d92e6eaf67f2386
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1827
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sun, 4 Aug 2019 04:24:06 +0000 (00:24 -0400)]
cli, tests: rename "leftover" terms to "non-option"
"Leftover" is a popt term, and we don't use popt anymore. Arguments
which are not options are not leftover anymore: they are treated within
the same loop as option arguments, and their position relative to option
arguments are important, so we simply call them "non-option arguments".
Note that "non-option" is also a term that getopt(1) and getopt(3) use.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I6c27e6a18f9daa4a9d2e5f79058a94c926102da3
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1817
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sun, 4 Aug 2019 14:07:18 +0000 (10:07 -0400)]
test_try_again_many_times(): use three times `None`'s ref count
100,000 iterations was arbitrary anyway, so let's just depend on the
initial reference count of `None` and iterate three times that since
this is the object we want to avoid destroy.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I3d587c730b4587d5381d07541d0d86ccff344858
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1822
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sun, 4 Aug 2019 14:04:03 +0000 (10:04 -0400)]
lib: remove output port message iterator
The output port message iterator concept was not a good idea in the
beginning: it breaks the graph model, making it possible to iterate a
disconnected output port (which is weird from the component's
perspective), blocking the graph (user cannot call bt_graph_run()), and
forcing the graph not to contain any sink component. This looks and
feels like a hack.
For the C part, a subsequent patch should implement an easy way to add a
simple sink component to a graph based on a simple callback and custom
user data instead of going through the inconvenience of creating a sink
component class, setting the methods manually, creating an input port
message iterator once the graph is configured, etc.
For the Python part, we'll focus on `TraceCollectionMessageIterator` to
replace `_OutputPortMessageIterator`. `TraceCollectionMessageIterator`
should cover most of the use cases and is easier to use: you don't need
to set up your graph, add your own `flt.utils.muxer`, etc.
For more advanced use cases in Python, it's always possible to create a
"proxy sink component", just like `TraceCollectionMessageIterator` does
internally, to get full control on the input port message iterator.
To adapt the current tests, `TestOutputPortMessageIterator` in
`utils.py` can be used like an output port message iterator. Such an
iterator cannot seek however, so `test_message_iterator.py` needed
special treatments to make the eventual input port message iterator
seek.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I92d432fb33d35ae3c0262b723cdfeae82c6633c9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1821
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sun, 4 Aug 2019 05:13:15 +0000 (01:13 -0400)]
bt2: make `TraceCollectionMessageIterator` not use an output port msg iter
Because there's a plan to drop the output port message iterator concept
altogether, make `TraceCollectionMessageIterator` not use any output
port message iterator.
Instead, a `TraceCollectionMessageIterator` instance adds a proxy sink
component (`_TraceCollectionMessageIteratorProxySink`) to its graph,
connecting it to the last filter component in the chain, and sharing
with it a list having a single item.
When the sink consumes a message from its upstream message iterator, it
places it in the shared list as the first item.
TraceCollectionMessageIterator.__next__() is changed so that it calls
Graph.run_once() to make the proxy sink consume, and then reads the
consumed message from the shared list.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I4b561837a9c23d3d758ea089193cfdabf99fc27e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1820
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sun, 4 Aug 2019 05:11:03 +0000 (01:11 -0400)]
bt2: wrap bt_graph_run_once() (Graph.run_once())
As opposed to Graph.run() which simply returns `None` when it's finished
running, Graph.run_once() needs a way to indicate that the whole graph
is finished. In that case, the method raises `bt2.Stop`. Otherwise,
Graph.run_once() returns `None`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I8c5908b7b1a265486534a3ec01c9eb813c2cde7c
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1819
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Sun, 4 Aug 2019 05:09:28 +0000 (01:09 -0400)]
lib: rename bt_graph_consume() -> bt_graph_run_once()
While bt_graph_run() makes the graph run until the end,
bt_graph_run_once() makes it run a single time, that is, a single sink's
(next in the queue) "consume" operation.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I65dc06c93425d5b5aa1f9e31edf3e22bbeaf77f0
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1818
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sat, 3 Aug 2019 18:36:42 +0000 (14:36 -0400)]
bt2: pass custom Python object to Python component's __init__()
Just like you can pass custom data (`void *`) to the initialization
function of a component class written in C with the
bt_graph_add_*_component_with_init_method_data() functions, this patch
makes it possible to pass any Python object to the __init__() method of
a component class written in Python with the Graph.add_component()
method.
This patch installs a mechanism to share Python data between a Python
graph user and the methods of a Python component without relying on
`nonlocal`, global variables, or other such hacks. This data can be as
simple as an integer and as complex as a database connection, for
example.
The __init__() method of a Python component used to look like:
def __init__(self, params):
...
It's now:
def __init__(self, params, obj):
...
When you pass any Python object to Graph.add_component() as its `obj`
parameter, the Python component's __init__() method eventually receives
it.
Graph.add_component() ensures that the component class to instantiate is
a Python component class if `obj` is not `None`.
Internally, `None` gets converted to `NULL` as the `init_method_data`
parameter of the bt_graph_add_*_component_with_init_method_data()
functions, and then `NULL` gets converted back to `Py_None` in
component_class_init().
Now there is the risk that you call
bt_graph_add_*_component_with_init_method_data() in C with a Python
component class and pass a non-`NULL`, non-`PyObject *`
`init_method_data` parameter. However I consider an insignificant
drawback as you're not supposed to use the `init_method_data` with a
component class you don't know.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ib879ece9e423b3495b9449ca73674082020865c5
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1815
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Sat, 3 Aug 2019 18:29:27 +0000 (14:29 -0400)]
bt2: remove BT CC entry from global HT in _UserComponentType.__del__()
The global `bt_cc_ptr_to_py_cls` hash table maps BT component class
pointers to Python component classes (`PyObject *`) for component
classes created in Python. The key and value are weak references.
When the Python side calls native_bt.bt2_component_class_*_create() with
a Python component class, an entry is added to `bt_cc_ptr_to_py_cls` on
success. This works most of the time because all Python classes are
normally destroyed when the interpreter is finalized, but there could be
issues with specific/unusual import patterns.
To avoid potential issues, remove an entry from `bt_cc_ptr_to_py_cls` in
_UserComponentType.__del__(), where a strong BT component class
reference is finally released.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I450a1c6f179f352e758b9e3e5cac8f4711aa3c88
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1816
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Simon Marchi [Sun, 4 Aug 2019 15:00:22 +0000 (11:00 -0400)]
cli: free auto_source_discovery_result::original_input_indices
This field is never freed, causing a memory leak.
Change-Id: I8d118226c2b491ae6ac3237250ab6ce3928e8ac2
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1824
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sun, 4 Aug 2019 14:58:58 +0000 (10:58 -0400)]
cli: free leftover_params and leftover_loglevels
They are never freed, causing a memory leak.
Change-Id: I66e12face4e83ecf98c780e8f20b97b9669e04b0
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1823
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Thu, 1 Aug 2019 22:43:31 +0000 (18:43 -0400)]
cli: apply log levels (`--log-level` option) to leftovers
It is currently not possible to apply --log-level after a leftover.
This patch proposes a way to allow it and to deal with the ambiguity
that it poses.
In the simple case, we have:
babeltrace2 my-traces --log-level=TRACE
All source components auto-discovered from the `my-traces` leftover will
have the TRACE log level.
If we have more than one leftover, but _no_ cross-leftover grouping,
then it is also intuitive:
babeltrace2 my-traces-1 --log-level=TRACE my-traces-2 --log-level=DEBUG
... all source components discovered from `my-traces-1` will have log
level TRACE and all source components discovered from `my-traces-2` will
have log level DEBUG.
It becomes less obvious when components are given inputs coming from
multiple leftovers (because of the auto-discovery grouping feature):
which log level do they receive? For example, if the following line:
babeltrace2 my-traces-1 --log-level=TRACE my-traces-2 --log-level=DEBUG
leads to these components getting instantiated, with these inputs:
* Source component X with inputs `my-traces-1/x` and `my-traces-2/x`.
* Source component Y with input `my-traces-1/y`
In this case, each component receives (on the `run` command line) the
log level options of all leftovers that contributed to its inputs, in
the same order as they are provided on the command line. The resulting
`run` command line for the example above could therefore look like:
... --component x:src.my.comp --log-level=TRACE --log-level=DEBUG \
--component y:src.my.comp --log-level=TRACE
resulting in these effective log levels:
* Source component X: log level DEBUG
* Source component Y: log level TRACE
Change-Id: I5e1bf9e1b4dd139ff7900d81b302a1eda72fb37f
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1810
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Wed, 31 Jul 2019 21:40:56 +0000 (17:40 -0400)]
cli: apply parameters (`--params` option) to leftovers
It is currently not possible to apply --params after a leftover.
This patch proposes a way to allow it and to deal with the ambiguity
that it poses.
In the simple case, we have:
babeltrace2 my-traces --params=foo=2
All source components auto-discovered from the `my-traces` leftover will
receive the `foo=2` parameter.
If we have more than one leftover, but _no_ cross-leftover grouping,
then it is also intuitive:
babeltrace2 my-traces-1 --params=foo=2 my-traces-2 --params=bar=3
... all source components discovered from `my-traces-1` will receive
`foo=2` and all source components discovered from `my-traces-2` will
receive `bar=3`.
It becomes less obvious when components are given inputs coming from
multiple leftovers (because of the auto-discovery grouping feature):
which parameters do they receive? For example, if the following line:
babeltrace2 my-traces-1 --params=foo=2,bar=3 my-traces-2 --params=foo=4
leads to these components getting instantiated, with these inputs:
* Source component X with inputs `my-traces-1/x` and `my-traces-2/x`.
* Source component Y with input `my-traces-1/y`
In this case, each component receives the parameters of all leftovers
that contributed to its inputs, in the same order as they are provided
on the command line. The resulting `run` command line for the example
above could therefore look like:
... --component x:src.my.comp --params=foo=2,bar=3 --params=foo=4 \
--component y:src.my.comp --params=foo=2,bar=3
resulting in these parameters being passed to the components:
* Source component X receives parameters `foo=4,bar=3`
* Source component Y receives parameters `foo=2,bar=3`
Implementation details
----------------------
The auto discovery mechanism now returns, for each result, which
input from the passed `inputs` array contributed to that result.
This allows us, for the component that we create from a given result, to
get the parameters from the leftovers that have contributed to it.
Change-Id: Ic048e4e137c2e1f93b6da13a62629343500cb75a
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1809
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Fri, 2 Aug 2019 17:43:59 +0000 (13:43 -0400)]
bt2: move common internal functions to their own files
This patch moves C code from `native_bt_component_class.i` to the new
`native_bt_bt2_objects.h` and `native_bt_log_and_append_error.h` files,
namely:
* bt_bt2_init_from_bt2(), which was bt_bt2_cc_init_from_bt2()
* bt_bt2_exit_handler(), which was bt_bt2_cc_exit_handler()
* The common logging and error appending functions.
This is just cleaner as those functions are not strictly related to the
component class interface.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ic9d7fed993b94b45f624e28e4c512b5ccab776bd
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1814
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Fri, 2 Aug 2019 17:31:12 +0000 (13:31 -0400)]
bt2: native_bt_version.i: use `%include <babeltrace2/version.h>`
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Idd8cbba1f1189c59b0679c0e8757c4e9c845bd4e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1813
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Fri, 2 Aug 2019 17:25:42 +0000 (13:25 -0400)]
bt2: put SWIG interface file C code in separate files
This patch moves any code in `%{ %}` blocks within SWIG interface files
(`.i`) to their own files. For a SWIG interface file named `file.i`, the
code is moved to `file.i.h`. In `file.i`, there is:
%{
#include "file.i.h"
%}
This helps with text editors which do not always know how to highlight
the code of a `.i` file.
In `Makefile.am`, the `.i.h` files are added to the
`SWIG_INTERFACE_FILES` variable as they can be considered part of the
SWIG interface files.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I28975c9b3c827e65ba77af9134521ce1b920d740
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1812
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Thu, 1 Aug 2019 14:57:22 +0000 (10:57 -0400)]
tests: reorganize CLI's `convert` tests
Create a `convert` directory under tests/cli, so we can have
multiple separate tests targeting specific features of the convert
command.
Move test_convert_args in there, this is the file containing the things
that are easy to test, comparing the output of --run-args with an
expected output.
Move the auto source discovery test in there too, as it's a feature of
the convert command. Rename it to "auto source discovery grouping",
since that is mostly what it exercises, and we'll want to add tests for
other areas of auto source discovery.
Change-Id: I20e5085d6b41eec322b777f45be1fee87894fe91
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1808
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Mon, 29 Jul 2019 18:42:23 +0000 (14:42 -0400)]
cli: handle leftovers in the same loop as components
[this patch is better viewed with `git show -w`]
Handle leftovers (put them in the leftovers array) at the same time as
we process --component options and things that apply to components
(--params and --log-level). When we handle a leftover, assign the
current_item_type variable, to override the previous current item.
This change makes it so that we prevent processing --params or
--log-level after a leftover. Previously, this command line:
babeltrace2 -c src.bon.jour some-leftover --params=a=2
Would apply the `a=2` param to the `src.bon.jour` instance, since it was
the last component declared. This is confusing, however, because of the
some-leftover leftover in between.
With this patch, the `some-leftover` leftover becomes the "currently"
processed item when we reach it. And since it's not possible (for the
moment) to apply --params or --log-level to a leftover, the command line
shown above now result in the error:
No current component of which to set parameters:
a=2
I improved test_convert_args to be able to check the error message of a
bt2 run we expect to fail. This lets us avoid cases where babeltrace
fails for an unexpected reason, but the test still passes.
Change-Id: I0e065c1cd5f32f59292c9a40c6a8077a52d35237
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1806
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins <jenkins@lttng.org>
Simon Marchi [Tue, 30 Jul 2019 04:34:58 +0000 (00:34 -0400)]
cli: remove `run` and `convert` commands' --name option
There are currently two ways of setting a component's name:
- Using the --name option
- Using the NAME: prefix to the --component option
Having two ways of doing the same thing is not really useful here.
Supporting the --name option complicates the code a little bit: we have
to track whether the current component's name has been set, so that when
we start a new --component, we can generate a name if needed.
This patch removes the --name option in favor of always using the NAME:
prefix of --component. This way, as soon as we handle --component, we
know if the user has provided a name or not, and if not, we can generate
one immediatly (applies only in the convert command; in the run command, the
user must provide a name).
Change-Id: Id278e9d110e4ea415f66538ac55e9459e49b1d0e
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1805
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Tue, 30 Jul 2019 01:29:51 +0000 (21:29 -0400)]
cli: remove `convert` command's --path and --url options
It was decided that these two options can be removed. They are
shortcuts for things that can be set using --params.
The --path option, in particular, is not useful as it used to be: the
de-facto standard parameter that sources are going to accept for their
input data is no longer `path` (it is now `inputs`).
Change-Id: I023e9cdaf319d7a9d2951315ce595ce099648391
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1804
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sat, 27 Jul 2019 05:10:16 +0000 (01:10 -0400)]
Remove popt from project's dependencies
All of the argument parsing having been moved to use the internal argpar
library, we can now stop linking with popt. And since it was the last
use of popt in the repo, remove anything related to it.
Change-Id: Id2aee4869780e586833d6b6646e12e19c6d78f6a
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1795
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Mon, 29 Jul 2019 15:42:58 +0000 (11:42 -0400)]
cli: remove babeltrace2-log
babeltrace-log (now renamed to babeltrace2-log) is a legacy tool
provided by Babeltrace 1 used to transform `dmesg` output to CTF. The
version currently provided by Babeltrace 2 simply runs babeltrace2 with
a pre-defined graph topology. It does nothing that a user couldn't do
on the command line.
The equivalent now would be:
dmesg | babeltrace2 -c src.text.dmesg -o ctf -w /path/to/trace
It was therefore decided that it was no longer necessary to provide that
utility, and to remove it.
Change-Id: Id547bced56a8b9913cb152e1309919a15b6fbdd0
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1796
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sat, 27 Jul 2019 04:42:32 +0000 (00:42 -0400)]
cli: use argpar for parsing query command's arguments
Change-Id: Id53593e699e4b377ada5df46697e77e5a0e30721
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1794
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sat, 27 Jul 2019 04:23:58 +0000 (00:23 -0400)]
cli: use argpar for parsing list-plugin command's arguments
Change-Id: Ifc11c8652a23830cd5961e4df55a3a23817d1c95
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1793
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sat, 27 Jul 2019 03:37:22 +0000 (23:37 -0400)]
cli: use argpar for parsing help command's arguments
All straightforward, except that there is a functional change as part
of this patch. It was possible to pass multiple leftovers, and only the
first one was considered.
babeltrace2 help ctf tintin
did the same as
babeltrace2 help ctf
It now prints a beautiful error:
./src/cli/babeltrace2 help ctf tintin
ERROR: [Babeltrace CLI] (/home/smarchi/src/babeltrace/src/cli/babeltrace2.c:2564)
Command-line error: retcode=1
CAUSED BY [Babeltrace CLI] (/home/smarchi/src/babeltrace/src/cli/babeltrace2-cfg-cli-args.c:1461)
Extraneous command-line argument specified to `help` command: `tintin`.
Change-Id: I2c8f8610004d60a0844d18c9d622037cc7c1c04c
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1792
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Sat, 27 Jul 2019 03:15:56 +0000 (23:15 -0400)]
cli: use argpar for parsing convert command's arguments
No functional changes intended, everything should be quite
straightforward.
Change-Id: I231c8f91aca6bef9327deee0c709b12ce5cea8db
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1791
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Fri, 26 Jul 2019 21:20:08 +0000 (17:20 -0400)]
cli: use argpar for parsing run command's arguments
A non-obvious change is the handling of the --retry-duration value.
Before this patch, it was automatically converted to a long by popt. We
now call g_ascii_strtoll and validate that it's parsable as an integer.
Also, introduce the help_option_is_specified function (which will be
used for other sub-commands as well). The goal is to check for --help
before everything else, such that doing:
$ babeltrace2 run hello --help
prints the help, rather than saying "Unexpected parameter `hello`".
Change-Id: Ib6873a2e2183e73f340cf25026924df298c8c1e9
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1790
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Wed, 24 Jul 2019 15:56:43 +0000 (11:56 -0400)]
cli: use argpar for top-level args
This patch replaces the hand-made parsing of the top-level arguments
with something based on the internal argpar library.
No functional changes expected from the point of view of the user.
One internal difference is that previously, the argc/argv passed to
sub-commands included an argv[0] that was ignored by popt. In the
subsequent patches, we replace popt with argpar, and argpar doesn't want
an argv[0] containing the name of the program. So instead of passing an
unnecessary argv[0], I have made it such that the argc/argv the
top-level passes to subcommands only represent the actual arguments. So
for example, with:
babeltrace2 convert -c src.ctf.fs
the bt_config_convert_from_args function used to receive this as argv:
["convert", "-c", "src.ctf.fs"]
With
babeltrace2 --debug -c src.ctf.fs
it used to receive
["--debug", "-c", "src.ctf.fs"]
With this patch, it will only receive `["-c", "src.ctf.fs"]`. So
functions bt_config_*_from_args are updated to cope with this
change. In particular, I passed the POPT_CONTEXT_KEEP_FIRST flag to
poptGetContext. But in practice, they will all disappear anyway, in
favor of argpar.
Change-Id: I9f1210f1c338c7eb39e228e20218c83e46961ee4
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1789
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Sun, 7 Jul 2019 03:15:58 +0000 (23:15 -0400)]
Add internal command-line argument parser API
This patch adds an internal command-line argument parser API.
The exact API is well documented in `src/argpar/argpar.h`.
The features that I'm looking for are:
* C API.
* Support for ordered arguments in the results, even between options and
non-option arguments (sometimes called positional arguments).
* Support for GNU-style arguments, including "glued" short options and
long options with `=` to set the option's argument.
* Portable.
* Compatible license.
* No global variables (we're trying to avoid that programming style
throughout the project).
* Easy to use.
None of the popular libraries I looked at, including popt of course, met
all those specifications.
The goal of this is:
1. To simplify the parsing of general options (before the command name)
in bt_config_cli_args_create(). It is currently hard-coded.
This is not possible with popt because it would fail with an unknown
option, and you would not know the position of the command name's
argument within the array.
This is possible with g_option_context_parse().
2. To make it possible, in the `convert` command, to assign parameters
and other position-specific options to a non-option argument, for
example:
babeltrace2 /path/to/trace --params=some=param \
mein-other-trace --name=travel
This is not possible with popt as it collects all the non-option
arguments as an array of leftover arguments.
This is also not possible with g_option_context_parse() for the same
reasons as with popt.
getopt_long() could satisfy both 1. and 2., but it's somewhat a pain to
use and to maintain the available options, as you need to specify the
long options in a table and the equivalent short options as a single
string with a special encoding (the `convert` command's option string
would be particularily ugly). Also: getopt_long() plays a lot with
global variables; it's not thread-safe: the parser's state is global.
Also: the upstream getopt_long()'s (glibc) license is LGPL, which is
more restrictive than our MIT license, so I think we want to avoid that.
I believe having our own (tested) CLI argument parser is beneficial,
especially in the long term: we can drop a direct dependency (and popt
is getting old), not introduce a new one, it's about 450 lines of
documented C code, and if we ever need something very specific in the
argument parsing strategy in the future, we can add it directly. As a
reference, FFmpeg, a project which has complex argument parsing, similar
to Babeltrace's `convert` and `run` commands, has its own argument
parser (see parse_options() in [1]).
Only the CLI will use the bt_argpar_*() API, but to make the unit tests
simpler, I put it in its own convenience library.
[1]: https://github.com/FFmpeg/FFmpeg/blob/
b7b6ddd59693008c35b3247496ecc946331d0856/fftools/cmdutils.c
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Iff4fc305b9e9171c694e1e79428bd3838ddd989d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1646
CI-Build: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Francis Deslauriers [Wed, 31 Jul 2019 13:30:15 +0000 (09:30 -0400)]
bt2: stream.py: add `trace` property to _Stream
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I434ebf2dd9c5a29bb8ef15ef7058479eba5ff9f6
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1807
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Jonathan Rajotte [Tue, 30 Jul 2019 17:16:31 +0000 (13:16 -0400)]
`ctf` plugins: port: cast to intmax_t all off_t variables when printing
This patch fixes new sites since the following commit:
commit
1974687e6b7a08d8383a4a5c75265d0ed3b8c5c9
Author: Michael Jeanson <mjeanson@efficios.com>
Date: Mon Jul 10 12:39:38 2017 -0400
Port: handle 'size_t' / 'off_t' on Solaris
The size of 'size_t' / 'off_t' vary on some platforms, use C99 portable
macros and print formats to handle them.
Print 'size_t' with '%zu' and use SIZE_MAX as max value.
Print 'off_t' casted to 'intmax_t' with '%jd'.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Change-Id: I8c0677f5ff590c25df6257c2d3f5142de93d01b0
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1803
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Mon, 29 Jul 2019 19:32:08 +0000 (15:32 -0400)]
configure.ac: refine Python 3 interpreter and dev. libraries detection
This patch refines how the `configure` script detects the Python 3
interpreter and development libraries.
The goal is to clean up this part of `configure.ac` and to make the
Python 3 interpreter available as a general tool for testing, even
without building the Python bindings or the Python plugin provider.
Before this patch, `configure` ensures that the Python interpreter and
the development libraries exist and are valid if you use the
`--enable-python-bindings` or `--enable-python-plugins` option.
With this patch, `configure` always tries to find the Python
interpreter. If it finds it, it sets the local `have_python` shell
variable and defines the `HAVE_PYTHON` Automake definition.
If `configure` finds the Python interpreter, it always tries to find
`python-config`. It if finds it, it sets the local `have_python_dev`
shell variable and defines the `HAVE_PYTHON_DEV` Automake definition.
You can still override the include directories and linker flags with the
`PYTHON_INCLUDE` and `PYTHON_LDFLAGS` environment variables.
Now, if you specify the `--enable-python-bindings` or
`--enable-python-plugins` option, `$have_python_dev` must be `yes`.
What used to be `HAVE_PYTHON` (Automake definition) is renamed to
`ENABLE_PYTHON_COMMON_DEPS`: this is defined if we need to build any
common Python dependency. As of this patch, the only one is the internal
`py-common` convenience library.
In the final `configure` report, I added:
* Whether or not the Python interpreter was found and its path.
* Whether or not the `python-config` program (called "development
libraries") was found and its path.
This patch was only tested manually, locally removing the Python
interpreter, the `python-config` program, and passing the
`--enable-python-bindings` or `--enable-python-plugins` option to test
the different `configure` paths.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Id0ac60ebc97a9ce2951aee713eccdc24c19d3791
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1799
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Philippe Proulx [Mon, 29 Jul 2019 16:25:42 +0000 (12:25 -0400)]
src.ctf.lttng-live: live_..._list_sessions(): remove handshake
The only user of live_viewer_connection_list_sessions() is
lttng_live_query_list_sessions() which calls
live_viewer_connection_create() before. live_viewer_connection_create()
calls lttng_live_connect_viewer() which calls lttng_live_handshake().
There is no need to call lttng_live_handshake() twice. This makes the
relay daemon assign two viewer IDs to the CLI, as seen in Wireshark.
Babeltrace 1.5 does not send the "connect" command twice either.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I640b9397c21ca37ad5c841415b5ed0c12193825a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1798
Philippe Proulx [Mon, 29 Jul 2019 14:06:58 +0000 (10:06 -0400)]
Fix: live_viewer_connection_create(): `lttng_live_msg_iter` can be `NULL`
Because `lttng_live_msg_iter` can be `NULL` in
live_viewer_connection_create() when it's used from a query operation,
pass the log level to use directly and only read `self_comp` if
`lttng_live_msg_iter` exists.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ied5ec621bcd066df0bb312a6f81899616f9d1a9f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1797
Philippe Proulx [Mon, 29 Jul 2019 19:52:48 +0000 (15:52 -0400)]
Fix: tests/utils/utils.sh: export `BT_TESTS_PYTHON_CONFIG_BIN`
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I7e07fb028df066b130b1932c46d0c91d474172bf
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1800
CI-Build: Michael Jeanson <mjeanson@efficios.com>
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Jonathan Rajotte [Tue, 23 Jul 2019 21:58:29 +0000 (17:58 -0400)]
`ctf` plugin: Use CTF_BYTE_ORDER_UNKNOWN in place of -1
Seen on clang-3.9, 4.0 and clang-1001.0.46.4.
Comparison of constant -1 with expression of type 'enum *****'
is always false [-Wtautological-constant-out-of-range-compare]
Note that the enum underlying type is implementation defined and left to
the choice of the compiler by the standard [1] (6.7.2.2 4). Most
compiler default to unsigned int. The use of -1 is not a problem per see
since wrap around of unsigned int behaviour is not undefined. Using -1
is the equivalent of assigning UINT_MAX here. This warning was removed
for later clang for these specific cases since the effect of always being
false is erroneous.
Still, it make more sense to use a specific enum for such case instead
of relying on the compiler type.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I5dcbaa2066cbab8c429ff54e8d8b8b295fc8baf1
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1764
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
This page took 0.053243 seconds and 4 git commands to generate.
projects / babeltrace.git / log
