Simon Marchi [Wed, 19 Jun 2019 20:22:54 +0000 (16:22 -0400)]
lib: remove "accept port" concept
This patch removes the concept of a component needing to accept a port
connection before a connection is made. Since the component connections
are supposed to be transparent (a component behavior shouldn't change
based on which component is on the other side), there is no point in
letting a component refuse a connection. Refusing a port connection
based on the component class, for example, could lead to tightly coupled
components, which we want to avoid.
Change-Id: Ia2d0e09caaee069e9be22b8089ef5ccbae392bb2 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1522 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Thu, 20 Jun 2019 19:54:48 +0000 (15:54 -0400)]
Fix: `ctf` plugin: do not have an `mdec` variable where not strictly needed
Issue
=====
The `metadata-info` query needs to call
ctf_metadata_decoder_packetized_file_stream_to_buf() with a custom log
level and no self component (`NULL`). Eventually, this reaches
decode_packet() which is in the `decoder.c` file and thus uses
`mdec->self_comp` to find its self component in the BT_COMP_LOG*()
macros. But in this case, `mdec` is `NULL`, so there's an invalid
memory access.
Solution
========
Instead of using BT_COMP_LOG_CUR_LVL() everywhere in this function, and
because decode_packet() and
ctf_metadata_decoder_packetized_file_stream_to_buf_with_mdec() do not
strictly need `mdec` except for logging its address at some places, move
the functions to the new `decoder-packetized-file-stream-to-buf.c` file
and make them not accept an `mdec` argument.
This new file's logging uses the local `log_level` and `self_comp`
variables, so the logging statement are not modified.
Michael Jeanson [Thu, 13 Jun 2019 20:18:40 +0000 (16:18 -0400)]
tests: Move data files to a common directory
* Reduce errors when adding new data files, a single EXTRA_DIST
directive in tests/Makefile.am will cover all data files
* Standardise variables used to refer to the data files path and add
them to 'utils/common.sh'.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I5fd06e5280e5864593aca8e964eeda9121b8a48e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1428 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Simon Marchi [Fri, 21 Jun 2019 15:29:19 +0000 (11:29 -0400)]
bt2: add missing parameter to _UserComponentType._query
While running the Python test suite, I saw this unexpected stack trace:
test_query_not_implemented (test_component_class.UserComponentClassTestCase) ...
06-21 11:22:56.091 24065 24065 E BT2-PY bt_py3_component_class_query@native_bt_wrap.c:4442 Failed to call Python class's _query_from_native() method: py-cls-addr=0x6190001ce898
06-21 11:22:56.092 24065 24065 E BT2-PY bt2_py_loge_exception@native_bt_wrap.c:3838 Traceback (most recent call last):
File "/home/smarchi/build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/component.py", line 539, in _query_from_native
results = cls._query(query_exec, obj, params, log_level)
TypeError: _query() takes 4 positional arguments but 5 were given
This is because method _UserComponentType._query is missing the
log_level argument that was added in a recent commit. The reason the
test still passes is that it expects bt2.Error to be raised (coming from
the raise NotImplementedError). The failure to call _query also ends up
raising a bt2.Error. So it succeeds to fail, but it doesn't fail the
right way, and the test fails to properly catch that failure.
Change-Id: If13bfdecc2876610fe8568a12043be60762d1596 Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1530 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Thu, 20 Jun 2019 19:50:20 +0000 (15:50 -0400)]
Fix: bt_config_convert_from_args(): set default log level at the end too
Issue
=====
There are code paths which make bt_config_convert_from_args() return a
configuration while not having set the default log level.
For example, with `-o ctf-metadata`, the function calls
bt_config_print_ctf_metadata_create() to create such a configuration,
and then goes to the `end` label).
Solution
========
Make sure it is set to the CLI's default log level value when reaching
the end of the function.
Philippe Proulx [Thu, 20 Jun 2019 18:23:39 +0000 (14:23 -0400)]
Rename VERBOSE log level to TRACE
Some people reported that they have a hard time guessing whether the
VERBOSE log level is more or less verbose than the DEBUG log level.
The existing logging frameworks I looked at (Apache Log4j 2, SLF4J,
`java.util.logging`, syslog, Python, Ruby, Boost.Test, ASP.NET Core, to
name a few popular ones) do not have the VERBOSE level; they almost
always have the DEBUG level, and sometimes the TRACE level.
To conform with the common practice, let's use TRACE instead of VERBOSE.
It is my wish that DEBUG vs. TRACE will speak to more developers than
DEBUG vs. VERBOSE.
This means that BT_LOGV() and its variants become BT_LOGT(). Also,
the log level environment variables and the CLI's `--log-level` parameter
accept `T` and `TRACE` instead of `V` and `VERBOSE`. In the log output,
the messages contain `T` instead of `V`:
06-20 14:31:23.356 3612 3612 T PLUGIN/CTF/BFCR
read_unsigned_bitfield@bfcr.c:483 [N/A] Read unsigned bit array:
cur=32, size=8, bo=1, val=33
`CONTRIBUTING.adoc` is updated to document the TRACE level instead of
the VERBOSE level.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I20460c4569f8b755440511c6621a68a1ef6424fc
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1524
CI-Build: Simon Marchi <simon.marchi@efficios.com>
CI-Build: Jonathan Rajotte Julien <jonathan.rajotte-julien@efficios.com> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com> Tested-by: jenkins <jenkins@lttng.org>
Jonathan Rajotte [Wed, 19 Jun 2019 17:28:14 +0000 (13:28 -0400)]
Fix: Unmap before truncating
The mmap/mman compat layer for mingw uses the CreateFileMapping
and UnmapViewOfFile thus implicitly imposes an order for truncation and
ummapping [1].
For truncation calls, the underlying windows API call is SetEndOfFile.
This function requires that the file be unmapped before being called [1].
[1]
If CreateFileMapping is called to create a file mapping object for
hFile, UnmapViewOfFile must be called first to unmap all views and
call CloseHandle to close the file mapping object before you can call
SetEndOfFile.
Michael Jeanson [Mon, 17 Jun 2019 16:10:34 +0000 (12:10 -0400)]
fix: test_trimmer on macOs and Solaris
The 'wc' tool on BSD based system adds leading spaces to it's output,
which breaks the quoted compare. Use bash parameter expansion to trim all
spaces from the 'num_events' variable.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I15aef8c6987e40bcb018f32c7c094eabbc3d83c3
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1476 Reviewed-by: Simon Marchi <simon.marchi@efficios.com> Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com> Tested-by: jenkins <jenkins@lttng.org>
Philippe Proulx [Wed, 19 Jun 2019 07:03:38 +0000 (03:03 -0400)]
lib: use `LIBBABELTRACE2_INIT_LOG_LEVEL` env. var. to initialize log level
Use the `LIBBABELTRACE2_INIT_LOG_LEVEL` environment variable instead of
`BABELTRACE_LOGGING_GLOBAL_LEVEL` to make it very clear that it's about
the library, and not related to the individual component log levels, for
example.
`INIT` is in the name to indicate that it's only the initial log level;
the library user can still change it at run time with
bt_logging_set_global_level(), while the other log level environment
variables, `BABELTRACE_PLUGIN_CTF_METADATA_LOG_LEVEL` and
`BABELTRACE_PYTHON_BT2_LOG_LEVEL`, are used to set the definitive log
levels of those modules.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ibf580bed6c30d88b4fbe3aed20f38cc969bc9295
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1511 Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Philippe Proulx [Tue, 18 Jun 2019 18:38:53 +0000 (14:38 -0400)]
`ctf` plugin: metadata: use BT_COMP_LOG*() instead of BT_LOG*()
Because we need to pass around the log level and the self component
pointer at several places within the files of this directory, this patch
adds a new internal `struct meta_log_config` which contains both and can
be passed as a single parameter and contained as a single member. It is
not meant to be known by anything calling the functions from files
outside this directory.
The generated parser and lexer can't use the BT_COMP_LOG*() macros
because they lack contextual data to access the self component pointer,
so they are left as is.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ifa95256e7931416b890e8da0bb7c2552f396a8d2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1499
Philippe Proulx [Tue, 18 Jun 2019 17:35:19 +0000 (13:35 -0400)]
`ctf` plugin: `bt_bfcr`: use BT_COMP_LOG*() instead of BT_LOG*()
As of this patch, the only user of `bt_bfcr` is `bt_msg_iter`, which
sets the new self component parameter to `NULL` because it's not
available at this point (work reserved for a following patch).
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Idcdbc21f0d6ab3062e4c9350228456bdfdeb23cf
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1497
Philippe Proulx [Tue, 18 Jun 2019 05:22:38 +0000 (01:22 -0400)]
src.ctf.lttng-live: honor component's initial log level and query LL
Except for `viewer-connection.c` which is pretty consistent in having a
named "viewer connection" instance in each function, the other files
deal with too many objects/concepts to have a systematic expression to
identify the log level, so they use local `log_level` variables.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I102408375d4d3d2df21c756b6581c642ece77ff1
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1494 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Philippe Proulx [Tue, 18 Jun 2019 04:01:25 +0000 (00:01 -0400)]
lib/object.h: log conditionally to `BT_OBJECT_DONT_LOG` undefined
There are a few places within the project, but outside the library, that
for historic reasons use the library's internal `bt_object` API to have
reference counting. The `bt_object` API belongs to the library for the
moment and should not be included as such.
There's an issue when you need logging in your module and the functions
which log within `object.h` cannot make sense of the
`BT_LOG_OUTPUT_LEVEL` expression. To avoid this and save short-term
development time, make them not log if `BT_OBJECT_DONT_LOG` is defined.
Then, outside the library, you can do something like:
Philippe Proulx [Mon, 17 Jun 2019 23:20:08 +0000 (19:20 -0400)]
`ctf` plugin: metadata: use local log level
As of this patch, the users of the common metadata functions and objects
set the new log level parameters to their own (global) log level.
The goal is to eventually have a per-component log level for the `ctf`
plugin's component classes.
Unfortunately, `YYFPRINTF` and `YY_FATAL_ERROR` are pretty limited in
terms of available context. I'm leaving the
`BABELTRACE_PLUGIN_CTF_METADATA_LOG_LEVEL` environment variable to set
the logging level of the parser and lexer modules.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I34651e6d0c14b5acc9b4198681c2aea4c003dc7a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1490 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Philippe Proulx [Mon, 17 Jun 2019 17:18:11 +0000 (13:18 -0400)]
flt.lttng-utils.debug-info: use BT_COMP_LOG*() instead of BT_LOG*()
Note that the API of `bin-info.h` is tested in
`tests/plugins/flt.lttng-utils.debug-info/test_bin_info.c`, and since
the BT_COMP_LOG*() macros can call bt_component_get_name(), which is
defined in the Babeltrace library, `test_bin_info` needs to be linked
with the Babeltrace library now.
Philippe Proulx [Sat, 15 Jun 2019 16:58:38 +0000 (12:58 -0400)]
Logging: define `BT_LOG*_SUPPORTED` when logging is supported
Rely on those new `BT_LOG_SUPPORTED`, `BT_LIB_LOG_SUPPORTED`,
`BT_COMP_LOG_SUPPORTED`, definitions instead of the header include
guards to know whether or not logging is supported at a given point.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I4d076e30b528882d036f7cfc367b5dbb39401dda
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1472 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
This new header is intended to be included by the different component
class files so that their logging statements can automatically include
the component's name. The BT_COMP_LOG*() macros prepend a prefix similar
to this to the logging message:
[mein-sink]
where `mein-sink` is a component's name.
Having the component name within each logging message of a given
component class adds valuable context and can help resolve issues
faster.
I did not include the plugin name and the component class's type and
name within the prefix because we already have them thanks to the
logging tag, for example `PLUGIN/SRC.CTF.FS`. This would be redundant,
make the logging message longer, and make `comp-logging.h` more complex.
The BT_COMP_LOG*() macros rely on the `BT_COMP_LOG_SELF_COMP`
definition. This is an expression which is, in the logging statement
context, the `bt_self_component *` address. Because we have the self
component there, we are free to prepend more contextual information to
the logging message in the future, should we need it.
Typically, when a component class source file defines
`BT_LOG_OUTPUT_LEVEL` to `(my_comp->log_level)`, where `my_comp` points
to the component's private structure, it will also define
`BT_COMP_LOG_SELF_COMP` to `(my_comp->self_comp)`.
If `BT_COMP_LOG_SELF_COMP` evaluates to `NULL`, the macros use the
prefix
[N/A]
This could be the case when instrumenting code that is shared for both
graph and non-graph contexts, for example the metadata and stream
decoders of the `ctf` plugin which can also be used within a query
operation.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I62c63ed78d234922f556631726fb7bb852924f0a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1468 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Philippe Proulx [Sat, 15 Jun 2019 03:59:04 +0000 (23:59 -0400)]
Logging: pass dynamic log level to common functions and subsystems
This patch makes the project's common functions and subsystems get a
dynamic log level instead of using a global one set from an environment
variable. The goal is to achieve per-component log levels, so the common
functions and subsystems must receive, eventually, the component's
specific log level.
"Naked" functions (without a context object of some sort) simply get
their log level from a new parameter. In that case `BT_LOG_OUTPUT_LEVEL`
is simply defined to `log_level`, so a local `log_level` variable must
exist when using logging statements. Subsystems, such as an FD cache or
a CTF serializer, get their log level from a new parameter in their
initialization function and then use it for their other operations.
It was somehow hard to do this for `src/common` alone without also doing
it for the rest so I'm doing all of them at once here.
For those subsystems, the local `logging.c` and `logging.h` disappear.
`tests/plugins/flt.lttng-utils.debug-info/test_bin_info.c` needs to pass
a log level to the FD cache subsystem; I used the WARN level instead of
setting the log level from an environment variable for this test alone.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I35979a8fe4c74afe5921c3b6651015cd7ee2760a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1460 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
This new option makes it possible to specify a log level for a specific
component instance. The main program's `--log-level` option becomes the
default log level which the specific `--log-level` options override.
makes the log level of the source be INFO and the log level of the
sink be DEBUG. The log level of the other components (implicit muxer),
of the CLI, and of the library is the default WARN.
makes the log level of the source be INFO and the log level of the sink,
of the other components (implicit muxer), of the CLI, and of the library
be ERROR.
The CLI still sets the known log level environment variables until
subsequent patches make each component honor its initial log level, that
is, until each component uses the log level for its logging statements
as returned by bt_component_get_logging_level().
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I8a3b05b7147fdb0149ecff670c542c9a229beddb
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1458 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
GraphTestCase.test_add_component_logging_level() is pretty much the same
test as GenericComponentTestCase.test_logging_level(), but the former's
purpose is to test the bt2.Graph.add_component() part of this, while the
latter tests the component's public `logging_level` property.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I9c82e8a7ab6a733810eabfee5ccef66cd619a610
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1456 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Philippe Proulx [Sat, 15 Jun 2019 01:16:07 +0000 (21:16 -0400)]
lib: set component's initial log level when adding it to the graph
This patch changes the bt_graph_add_*_component*() functions so that
they accept a log level parameter. This is now considered the
component's initial log level, so that we can achieve per-component log
levels.
The component initialization method types are not changed: because the
log level is part of a component's state, a component initialization
method can retrieve its own, current log level with the new
bt_component_get_logging_level() function. When you call
bt_graph_add_*_component*(), the function creates an initial component
object, sets its initial log level, and then calls its initialization
method.
Eventually, we can add a new, optional "logging level changed" component
method to notify it that its log level changed (caused by a call to
bt_component_set_logging_level(), for example).
The project's components do not honor their initial log level within the
scope of this patch: this is postponed to another patch. In other words,
they still use shared object constructors and environment variables to
set their log level at the component class level.
The CLI passes the configured log level (`--log-level`, `-v`, or `-d`
options) when it calls the bt_graph_add_*_component() functions. As of
this patch, it also still sets the known log level environment
variables.
Python bindings are adjusted accordingly. Their tests are not changed
because the log level parameter is optional (defaults to
`bt2.LoggingLevel.NONE`) and it's the last one of
bt2.Graph.add_component(). The constructor of `bt2.ComponentSpec` also
accepts a new, optional log level parameter to configure the component
specifier as such. You can access the log level of a component with the
new `logging_level` property.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: If258335f46ffabb587dd777b5109fc66d8a59f54
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1454 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Philippe Proulx [Sat, 15 Jun 2019 01:09:51 +0000 (21:09 -0400)]
cli: use log level integer instead of letter in configuration
This patch makes `struct bt_config` have an integer log level (to be set
to one of the `BT_LOG_*` values) instead of a letter. The
bt_log_get_letter_from_level() and bt_log_get_level_from_letter() can be
used to translate from/to a log level letter/integer, so we use that.
This is just cleaner and makes more sense than carrying a letter in the
configuration which is just one encoding of a log level value.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I5cbc3f2cb9ac868bc99e9458468a09c58e716884
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1452 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Philippe Proulx [Fri, 14 Jun 2019 21:15:27 +0000 (17:15 -0400)]
configure.ac: change default minimal log level from VERBOSE to DEBUG
The goal here is that Babeltrace users have enough logging by default
without impacting the performance too much. From what I measured once
this patch is applied, the performance is very similar to an INFO build.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I20346f0f3eecf725e730a3e5add94804637c7193
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1445 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Philippe Proulx [Fri, 14 Jun 2019 21:12:36 +0000 (17:12 -0400)]
lib: plugin.h: force header user to include "lib/logging.h"
This pattern is found in other headers where, instead of including the
logging header for the user, we ask her to set up the logging and then
include the header.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I922b09f63eff41b9c3ba30df7ace0b8b30b5b5b2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1443 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Philippe Proulx [Fri, 14 Jun 2019 20:58:49 +0000 (16:58 -0400)]
Standardize log levels used by logging statements across the project
Let's use this approach now:
INFO level:
Any useful information which a non-developer user would possibly
understand.
Anything logged with this level must _not_ happen repetitively on
the fast path, that is, nothing related to each message, for
example. This level is used for sporadic and one-shot events.
* CLI or component configuration report.
* Successful plugin, component, or message iterator initialization.
* In the library: anything related to plugins, graphs, component
classes, components, message iterators, connections, and ports
which is not on the fast path.
* Successful connection to or disconnection from another system.
* An _optional_ subsystem cannot be loaded.
* An _optional_ field/datum cannot be found.
DEBUG level:
Something that only Babeltrace developers would be interested into,
which can occur on the fast path, but not more often than once per
message.
* Object construction and destruction.
* Object recycling (except fields).
* Object copying (except fields and values).
* Object freezing (whatever the type, as freezing only occurs in
developer mode).
* Object cancellation.
* Calling user methods and logging the result.
* Setting object properties (except fields and values).
VERBOSE level:
Low-level debugging context information (anything that does not fit
the other log levels). More appropriate for tracing in general.
* Reference count change.
* Fast path, low level state machine's state change.
* Get or set an object's property.
* Object comparison's intermediate results.
This will also be written in `CONTRIBUTING.adoc` by another patch.
The goal here is to make the DEBUG level the default minimal, build-time
log level, so that Babeltrace users have enough logging by default
without impacting the performance too much. From what I measured once
this patch is applied, the performance is very similar to an INFO build.
Many DEBUG logging statements are upgraded to the INFO level because
they only occur once, or very rarely, and consist of interesting
information for someone who does not have deep knowledge of the
Babeltrace source code.
Many VERBOSE logging statements are upgraded to the DEBUG level so that,
by default, we have more information in the log file of someone who
needs support. Those statements are executed seldom enough (at most once
per message) to be promoted as such.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Icdd1ec8f1f54396eea5588968d85e446030f282d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1439 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com> Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Philippe Proulx [Fri, 14 Jun 2019 02:22:26 +0000 (22:22 -0400)]
bt2: stream activity messages: create with unknown/infinite default CS
This patch makes it possible to create stream activity beginning and end
messages with unknown or infinite default clock snapshots. It was
already possible to create them with known default clock snapshots, that
is, with a value.
With this patch, you pass either `self._unknown_clock_snapshot` or
`self._infinite_clock_snapshot`, when implementing a
`bt2._UserMessageIterator`, as the default clock snapshot when you call
self._create_stream_activity_beginning_message() or
self._create_stream_activity_end_message(). They are instances of
internal types which are only used for this. The creation methods use
`_unknown_clock_snapshot` by default. This is similar to how
bt_message_stream_activity_beginning_create() and
bt_message_stream_activity_end_create() create messages with unknown
default clock snapshots.
`test_message.py` tests the new feature.
Also in this patch: `_BaseClockSnapshot` does not inherit
`object._UniqueObject` anymore so that `_UnknownClockSnapshot` and
`_InfiniteClockSnapshot` do not either. This is so that the default
clock snapshot properties of the stream activity beginning/end messages
can return instances of those types without having actual native
pointers, because there are none.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I56d5cd9fffb77d505f61522073aaf80ccd58eb8e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1430 Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
This script was the runner for the `flt.utils.muxer` test running in C,
which does not exist anymore. We'll write `flt.utils.muxer` tests either
in Python or with `sink.text.details` and expectation files.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I28df6d20c3bd3ee0b056790046d041787e702430
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1422 Tested-by: jenkins Reviewed-by: Michael Jeanson <mjeanson@efficios.com>
Philippe Proulx [Tue, 11 Jun 2019 23:08:46 +0000 (19:08 -0400)]
Add facilities to test CLI and plugin regressions with expectation files
This patch adds `tests/utils/diff.sh.in` (which becomes
`tests/utils/diff.sh`) which contains shell functions to use the CLI and
a `sink.text.details` component to easily test the CLI itself and
plugins for regressions with expectation files.
There are three functions in `tests/utils/diff.sh.in`:
bt_diff_cli():
Compares the output of the CLI with specific arguments to the
content of a file.
bt_diff_details_ctf_single():
Compares the output of the CLI using an `src.ctf.fs` component to
open a specific CTF trace and a `sink.text.details` component to the
content of a file.
bt_diff_details_ctf_gen_single():
Like bt_diff_details_ctf_single(), but runs a given CTF trace
generating program instead of using an existing CTF trace.
When there's any difference, the functions write the full diff to the
standard error. This makes it possible to look at what's wrong in the
CI.
`tests/plugins/ctf` is moved to `tests/plugins/src.ctf.fs` to indicate
that we're testing this specific component class. The specific query
tests are moved to `tests/plugins/src.ctf.fs/query`.
`tests/plugins/src.ctf.fs/diff/test_diff.in` uses `tests/utils/diff.sh`
to show how to use bt_diff_details_ctf_single() and
bt_diff_details_ctf_gen_single().
The functions test_ctf_single() and test_ctf_gen_single() use resp.
bt_diff_details_ctf_single() and bt_diff_details_ctf_gen_single() to
compare the given CTF traces using a `sink.text.details` component to
expectation files ending with `.expect` in the same directory.
In both test_ctf_single() and test_ctf_gen_single(), we make the
`sink.text.details` component hide the trace and stream names because
`src.ctf.fs` puts absolute paths in them. For test_ctf_gen_single(), we
also make the `sink.text.details` component hide UUIDs because CTF
writer creates random ones by default.
More specifically, this line:
test_ctf_gen_single simple
means:
1. Run the program `gen-trace-simple` in the same directory to generate
a CTF trace.
2. Compare the content of 1. to the file `trace-smalltrace.expect` in
the same directory.
Eventually, we can move the test_ctf_gen_single() and test_ctf_single()
functions outside the `src.ctf.fs` tests to make them available to test
other plugins and parts of the project. They are placed there for the
moment because only the `src.ctf.fs` tests need them.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I53660111c23117926e3ee114b5d0cd060fff87d3
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1414 Tested-by: jenkins
Philippe Proulx [Sat, 8 Jun 2019 00:26:34 +0000 (20:26 -0400)]
Add `sink.text.details`, a testing textual sink which prints details
This patch adds a new sink component class, `sink.text.details`. Such
a sink component prints all the messages and their details, including
the full metadata objects, in a deterministic way.
You can test it easily like this:
babeltrace2 /path/to/trace -c sink.text.details
The purpose of this sink is to use it for regression testing using the
following approach:
1. Run the `babeltrace2` program with a `sink.text.details` component,
providing a known input trace to the graph, for example a CTF trace
with `src.ctf.fs`. The graph can include one or more filters with
specific parameters to test them specifically.
2. Inspect the output with your human eyes and brain. Verify that
everything is as expected. It is possible that, for a given test,
some elements of the output are less important (for example, the
metadata objects or the message times): there are component
parameters to remove parts of the output to reduce the noise.
3. Save the output of 2. to a file.
4. Create a test which runs 1. with the same input and compares its
output to the file saved in 3. Any regression will show up as a
difference between both inputs.
Step 2. is particularly important because this is where you validate
that the output shows what you expect from the graph. To make this
easier, I tried to make the output format of `sink.text.details` as
human-readable as possible:
* The component optionally colorizes the output. This is the default
when the terminal supports it. There's no color codes by default when
you redirect the output.
* The component honors the preferred display base of integer fields and
prepends the typical `0x` and `0` prefixes.
* The component inserts digit separators to make it easier to read large
numbers, for example `102,354,895,784,619` and `0x7f4d:d04f:c636`.
* The component does not print the metadata objects interlaced with the
data and message objects. I found that this was noisy, for example
showing all the integer field class details for each integer field
when what you really want is to inspect the event payloads from one
message to the other.
Instead, I chose to print trace classes, stream classes, and event
classes only once, and before the message which is associated to them.
This typically occurs before a stream beginning message, but for event
classes, it can also occur before an event message, as a source can
add event classes to an existing stream class.
* The component assigns a unique ID to each trace object to follow the
messages associated to a specific stream more easily, as many streams
can share the same ID, across difference traces or even with the same
trace, when they have different stream classes.
The component prints this unique trace ID, as well as the stream
class and stream IDs, before each message type as such
(`{Trace 1, Stream class ID 0, Stream ID 2}` in the following
example):
[102,354,895,784,619 cycles, 1,441,852,773,172,610,636 ns from origin]
{Trace 1, Stream class ID 0, Stream ID 2}
Event `lttng_ust_statedump:build_id` (Class ID 56):
Common context:
vpid: 17,868
ip: 0x7f4d:d04f:c636
Payload:
baddr: 0x7f4d:d02d:1000
...
A unique trace ID is never reused for two different traces within the
lifetime of a `sink.text.details` component.
You can track all the messages of a given stream with this unique
tuple. For example, you can find the last packet beginning message for
the packet of a given event message having unique trace ID 0, stream
class ID 2, and stream ID 5 by finding the last
{Trace 0, Stream class ID 2, Stream ID 5}
Packet beginning
string.
This makes it possible to avoid redundancy and reduce textual noise in
the printed messages. For example, the component does not print the
packet context field for each event message, but only for the packet
beginning message.
In order to make step 4. continue to work in the future and really test
regressions, the output of step 1. must always be the same
(deterministic). For `sink.text.details`, this means:
* The component does not print run-time addresses.
* The component sorts mappings of enumeration field classes (by label),
and ranges within such mappings, before it prints them.
* The component sorts event classes of stream classes (by ID) before
it prints them.
* The component sorts stream classes of trace classes (by ID) before
it prints them.
* The component sorts streams of traces (by ID and class ID) before
it prints them.
* The component sorts environment entries of trace classes (by name)
before it prints them.
To make step 3. easier, the component does not print trailing spaces
or trailing empty lines.
A `sink.text.details` component can also be used for support if we want
all the details of a given graph's output.
The available component parameters are:
`color`:
`never`, `auto`, or `always` (just like `sink.text.pretty`).
Default: `auto`
`with-metadata`:
True to print metadata objects (only the first time they are
encountered).
Default: true
`with-time`:
True to print message times.
Default: true
`with-trace-class-name`:
True to print trace class names.
Default: true
`with-trace-name`:
True to print trace names.
Default: true
`with-stream-class-name`:
True to print stream class names.
Default: true
`with-stream-name`:
True to print stream names.
Default: true
`with-uuid`:
True to print UUIDs.
Default: true
`compact`:
True to make the output compact (one message per line, no property
names).
Default: false
This patch does not add the facilities to create tests easily using the
CLI and a `sink.text.details` component: this is reserved for a future
patch.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I9cc39cdfe44436974600388342029a2e4e2acb76
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1412 Tested-by: jenkins Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Michael Jeanson [Wed, 12 Jun 2019 22:09:37 +0000 (18:09 -0400)]
Cleanup: remove private babeltrace.h
* Remove unused macros
* Replace duplicated macros with their glib equivalent
* Move the *safe* static inlines to their own header
* Namespace the remaining macros and move them to macros.h
* Explicitly include 'common/macros.h' in all private headers that use
'BT_HIDDEN'.
* Remove BT_UNUSED, the attribute is a hard requirement anyway
* Remove zmalloc and replace the only callsite with calloc
Change-Id: I1bd765ad27c808c8bbe5ef232062267cb4a230d9 Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1424 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Thu, 13 Jun 2019 14:50:42 +0000 (10:50 -0400)]
fix: Static build fails with ctfser missing symbols
Spliting the ctf-writer from the main library had the side effect of not
including the ctfser convience library in the main lib anymore. Thus it
should now be linked with the binairies to satisfy the dependencies when
we statically link the plugins in.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I4626dcabce1914f0af2287afe8c3fcf7b7db9946
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1421
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com> Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Mon, 10 Jun 2019 21:44:09 +0000 (17:44 -0400)]
Re-organize sources
* Create a top-level 'src' directory.
* Move all private headers from the public include directory to the
'src' directory.
* Rename all those private headers to remove the 'internal' notation.
* Use double quotes in `#include` directives when including a private
header so that we can easily know.
Change-Id: I05fbb81d969b3735aaf303ec2c222be7142c19ab Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1413 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Fix: src.ctf.fs: increment packet offset before init of index entry
Issue
=====
The `current_packet_offset_bytes` variable is incremented before the
initialization of the current `index_entry` by `init_index_entry()`.
This lead to each entry having the offset of the next entry and the last
entry having a erroneous offset.
Solution
========
Increment the `current_packet_offset_bytes` variable after the
initialization of each entry.
Drawbacks
=========
None.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I7592b2d70f402aa413de97a687bfe4fe1bf1b825
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1409 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Fri, 7 Jun 2019 16:25:10 +0000 (12:25 -0400)]
ctf-writer: externalize libbabeltrace2-ctf-writer
Completely split the ctf writer part from the main babeltrace2 library.
* Remove the obsolete libbabeltrace2-ctf library
* Add a new libbabeltrace2-ctf-writer library
* Remove the ctf writer includes from the main include file
* Fork an internal copy of assert-pre.h that doesn't use the library
structures.
* Add a logging context specific to ctf writer
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Iff9ab62ee9eeb8abd290fb1d758a73050e21c99b
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1410 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Philippe Proulx [Fri, 7 Jun 2019 15:13:35 +0000 (11:13 -0400)]
Fix: doc/bindings/python/Makefile.am: run the `sphinx` Python 3 package
In `sphinx-build.py` (removed), importing `main` and `make_main` from
the `sphinx` package is not considered stable: it used to work for me,
but now it doesn't (cannot import). Instead, run the `sphinx` package
directly with the Python interpreter's `-m` option.
We do this instead of running `sphinx-build` directly because we want to
pick Sphinx for Python 3 (because we're importing the `bt2` package for
autodoc), not for Python 2, and the installed `sphinx-build` program can
be the one for Python 2 on some distributions.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I158dbb96a8b3a26db67dd6e16254a64ac9d24446
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1401
Philippe Proulx [Thu, 6 Jun 2019 22:17:14 +0000 (18:17 -0400)]
Remove everything related to the `bt2.ctf_writer` Python module
This module wraps CTF writer, but is currently incomplete and would
require much work to finish, as CTF writer is completely decoupled from
the Babeltrace library.
Since we plan to make Babeltrace 1 and Babeltrace 2 coinstallable, and
that CTF writer 2 does not add many important features to CTF writer 1,
we decided to postpone the development of its Python bindings to when it
supports CTF 2.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ie8ceeac8bfcce86dec3cb426d7030ad445274cd2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1392
Philippe Proulx [Fri, 7 Jun 2019 02:47:32 +0000 (22:47 -0400)]
Remove everything related to the `babeltrace` Python package
This package is the Babeltrace 1 package, and since we plan to make
Babeltrace 1 and Babeltrace 2 coinstallable, we decided to drop the
support for this one.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I37f2de0129df00ae5d78fc3f81ad6c57763809cb
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1391 Tested-by: jenkins
Simon Marchi [Fri, 7 Jun 2019 18:28:53 +0000 (14:28 -0400)]
tests: clean up test_trimmer
This patch changes test_trimmer to reduce duplication of the
boilerplate. I intend to add some test cases to this test in a
following patch, so this change allows me to understand better what the
test does currently and will help adding test cases.
Note that there are pairs of tests that are pretty much redundant, such
as:
- "Read a trace with the trimmer enabled (GMT relative timestamps)"
- "Ran successfully with --begin and --end (GMT relative timestamps)"
So I have only kept the second test of such pairs.
Change-Id: I075df21ed9c03495ab516d3870ccf7b63276d3ce Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1407 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
src.ctf.fs: trace-info: omit stream `range-ns` field when no TS
Don't print -1 timestamps when packet contexts don't have
`timestamp_begin` and `timestamp_end` fields. Simply omit this part of
the query reply.
Remove FIXME comment as the only user of the index now handles absent
timestamps.
Tests
=====
Adapt `test_ctf_plugin` stream sorting function to fall back on using
the `paths` field if `range-ns` field is absent and rename it to
abstract implementation details.
Add test case to test that `range-ns` fields are absent on traces
without `timestamp_begin` and `timestamp_end` fields in their packet
context.
Example
=======
Here is an example of the output of the `trace-info` query on a trace
without packet contexts before this commit:
-
streams:
-
range-ns:
begin: -1
end: -1
paths:
- /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context/stream
class-id: 0
port-name: /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context | 0 | /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context/stream
range-ns:
begin: -1
end: 0
name: no-packet-context
path: /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context
Here is an example of the output of the `trace-info` query on a trace
without packet contexts after this commit:
-
streams:
-
class-id: 0
port-name: /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context | 0 | /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context/stream
paths:
- /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context/stream
name: no-packet-context
path: /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I419f57960a52d8ed2823fdb06982e5c20710b6da
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1387
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com> Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Michael Jeanson [Mon, 3 Jun 2019 21:07:05 +0000 (17:07 -0400)]
lib: Reset libbabeltrace2 to SONANE 0
Following the rename of the library, reset the SONAME to zero and
decouple this from the project versioning which is not tied directly to
the library ABI.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: If3a3c890674d6236cfe33b5f3f1e3fe4229c5691
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1404 Tested-by: jenkins Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
projects / babeltrace.git / log
