src/plugins/ctf/common: restructure subtree This patch restructures the `src/plugins/ctf/common` subtree as such: `src`: Everything common to source component classes. `src/metadata/tsdl`: TSDL-specific code. As of this patch, everything which was part of `src/plugins/ctf/common/metadata` is moved here, but some code will be shared with CTF 2 sources eventually and move back to `src/plugins/ctf/common/src/metadata`. `print.hpp` isn't needed anymore. This new structure will make it easier to add common `ctf` plugin code for both source and sink component classes, as well as to add code specific to CTF 2 and common to both CTF 1 and CTF 2. Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: I4246d6ba37ff26efa5d4b7b4be2ec8600d3207e4 Reviewed-on: https://review.lttng.org/c/babeltrace/+/7839 Reviewed-on: https://review.lttng.org/c/babeltrace/+/12190 CI-Build: Simon Marchi <simon.marchi@efficios.com> Tested-by: jenkins <jenkins@lttng.org>
tests: normalize names of files and more The current naming situation is somewhat of a mess in the `tests` directory, making it difficult to deduce (or write) a convention. Use hyphens to separate words as much as possible, following what we usually like for source file names (see `tree src`). Pretty much the only things remaining with underscores are Python file names because of the `import` constraints and because it's Python's own convention. Also make all shell scripts have the `.sh` extension. This makes any worthy editor highlight the shell syntax correctly and tells you immediately it's not a compiled ("binary") executable. I'd prefer `.bash` for Bash code, but I don't think we need to go as far. Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: I43cd279d78c240a7e3686ad8afd6759bad9f575f Reviewed-on: https://review.lttng.org/c/babeltrace/+/11170 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Simon Marchi <simon.marchi@efficios.com> CI-Build: Simon Marchi <simon.marchi@efficios.com>
Reduce the number of Makefiles in 'src/plugins/utils' Continue the effort started by Simon to reduce the overall number of Makefiles to allow for faster parallel builds. Also reduce the number of convenience libraries for faster linking. Change-Id: I772572c3dc1d2481ec94526a4301d4bb79f9d855 Signed-off-by: Michael Jeanson <mjeanson@efficios.com> Reviewed-on: https://review.lttng.org/c/babeltrace/+/11151 Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
fix: python bindings: use stdlib distutils when setuptools is installed When the setuptools package is installed, it monkey patches the standard library distutils even if the user code doesn't import setuptools. This results in a failure to install the Python agent in a directory which isn't in the current PYTHONPATH. To allow this, setuptools requires the '--single-version-externally-managed' options which is not implemented in distutils. To resolve this, force the use of distutils for Python < 3.12 even when setuptools is installed with the 'SETUPTOOLS_USE_DISTUTILS' environment variable and use the previously mentionned setuptools option with Python >= 3.12 which doesn't include distutils anymore. Change-Id: I9e8412021c6ec79b0a9ea38759c475113e4ea018 Signed-off-by: Michael Jeanson <mjeanson@efficios.com> Reviewed-on: https://review.lttng.org/c/babeltrace/+/10371 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Add a single generated env file to the test suite Regroup all the configure detected values relevant to the test suite in a single generated file. This file will be automatically sourced by the test suite in most scenarios but can also be sourced in the shell of a user. * All user overridable variables start with 'BT_TESTS_'. * The priority for variables is : Environment -> env.sh -> utils.sh (defaults). * A user can source 'env.sh', override some of the values and manually run test scripts. * The test suite can run without an 'env.sh' file present. Scenarios: * Manually running tests in an in-tree build: The test suite sources 'env.sh' relative to itself, no environment variables required. * Manually running tests in an out-of-tree build: The user sets BT_TESTS_BUILDDIR in its environment or sources 'env.sh'. * Running the test suite with 'make check' in either builds: The Makefile sets BT_TESTS_BUILDDIR in the environment which allows the test suite to source env.sh in either in-tree or out-of-tree builds. Change-Id: I1ba3775ea3d724345cab769af9b2efc22ee53585 Signed-off-by: Michael Jeanson <mjeanson@efficios.com> Reviewed-on: https://review.lttng.org/c/babeltrace/+/6314 Reviewed-by: Simon Marchi <simon.marchi@efficios.com> CI-Build: Simon Marchi <simon.marchi@efficios.com> Tested-by: jenkins <jenkins@lttng.org>
tests: add libbabeltrace2 pre/postcondition testing infrastructure This patch adds a basic libbabeltrace2 pre/postcondition testing infrastructure to the project. When a libbabeltrace2 public function precondition or postcondition is not satisfied, the function logs a FATAL-level message which indicates what's wrong and then aborts. Slow path functions check such conditions unconditionally, while fast path functions check them only in developer mode. Testing that a libbabeltrace2 function catches an unsatisfied pre/postcondition and reports it is not straightforward: the library aborts the program, so we can't use the usual approach because the test program itself would abort. The solution brought by this patch is the following process in `tests/lib/conds`: 1. The `conds-triggers.c` program contains all the pre/postcondition failure triggering instructions to test, one condition per function. Each triggering function triggers a single pre/postcondition check. It is expected that the function actually aborts. main() calls ppc_main(), an internal utility, with its command-line arguments and an array of condition trigger descriptors. Each condition trigger descriptor has: * A condition type: precondition or postcondition. * The ID of the condition to trigger. * A name suffix. * A condition triggering function. Each condition trigger descriptor has a unique name: its condition ID and an optional name suffix. As of this patch, the PPC_TRIGGER_*_RUN_IN_COMP_CLS_INIT() macros create condition trigger descriptors of which the function runs within a component class initialization function (accepts a `bt_self_component *` parameter). This is often needed as many libbabeltrace2 functions are only accessible through a self component (all the trace IR API, for example). ppc_main() handles two command-line subcommands: `list`: Prints a JSON array of objects which represent the triggering descriptors, for example (output for this patch): [ { "cond-id": "pre:field-class-integer-set-field-value-range:valid-n", "name": "pre:field-class-integer-set-field-value-range:valid-n-0" }, { "cond-id": "pre:field-class-integer-set-field-value-range:valid-n", "name": "pre:field-class-integer-set-field-value-range:valid-n-gt-64" }, { "cond-id": "pre:field-class-integer-set-field-value-range:not-null:field-class", "name": "pre:field-class-integer-set-field-value-range:not-null:field-class" } ] `run INDEX`: Runs the condition triggering function for the descriptor at index `INDEX` in the `list` array. It is expected that this command aborts. 2. `test_conds` is a Bash script which only does this: reldir=lib/conds export BT_TESTS_LIB_CONDS_TRIGGER_BIN="$BT_TESTS_BUILDDIR/$reldir/conds-triggers" if [ "$BT_OS_TYPE" = "mingw" ]; then BT_TESTS_LIB_CONDS_TRIGGER_BIN="$BT_TESTS_LIB_CONDS_TRIGGER_BIN.exe" fi run_python_bt2_test "$BT_TESTS_SRCDIR/$reldir" test.py In other words, it runs the Python TAP test runner to discover and run tests in `test.py`. This script is part of the `make check` test set. 3. In `test.py`, a function: a) Runs `conds-triggers list` and decodes the output to get the list of available condition trigger descriptors. This step also validates the regular expressions and checks that there are no duplicate descriptor names. b) For each condition trigger descriptor, creates and adds a test to its single test case class which: I. Executes `conds-triggers run INDEX` as a subprocess, where `INDEX` is the descriptor's index. II. Reads the complete process's standard error. III. Asserts that the process aborts (`SIGABRT` signal). This seems to be only possible on a POSIX system with the `subprocess.Popen` API. IV. Asserts that the standard error (II) contains the descriptor's condition ID. All this is only enabled if, at configuration time: * Python 3 is available. * The Babeltrace 2 developer mode is enabled (`BABELTRACE_DEV_MODE=1`). This patch's `conds-triggers.c` includes two precondition triggering functions to confirm that everything works as expected: trigger_fc_int_set_field_value_range_n_0(): When calling bt_field_class_integer_set_field_value_range(), the parameter `N` cannot be greater than 64. trigger_fc_int_set_field_value_range_n_gt_64(): When calling bt_field_class_integer_set_field_value_range(), the parameter `N` cannot be 0. trigger_fc_int_set_field_value_range_null(): When calling bt_field_class_integer_set_field_value_range(), the field class cannot be `NULL`. Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: I70713d690f7dbfeac5804e6cfcec989242823611 Reviewed-on: https://review.lttng.org/c/babeltrace/+/3401
common: support custom, extra information for build's version This patch adds a system of extra version information also found in LTTng-tools. `src/common/Makefile` generates `src/common/version.i` at every build. This file contains: * The current Git revision description. * Extra name of the version (found in `version/extra_version_name`). * Extra description of the version (found in `version/extra_version_description`). * A list of patch file names found in `version/extra_patches`. All definitions can be empty strings. See `version/README.adoc` to learn more. As of this patch, libbabeltrace2 does not offer getters for this data and the CLI does not print it with the `--version` option. This is reserved for subsequent patches. Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: Ife50e5bcaa6b3bdeda6ee4e7c1fdeb2fb1f63887 Reviewed-on: https://review.lttng.org/c/babeltrace/+/2836 Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com> Reviewed-by: Michael Jeanson <mjeanson@efficios.com> Tested-by: jenkins <jenkins@lttng.org>
Document libbabeltrace2's C API This patch adds initial documentation for the Babeltrace 2 library's C API using Doxygen. The Doxygen project is located in `doc/api/libbabeltrace2`, as we can eventually add `doc/api/libbabeltrace2-ctf-writer`. To be able to use Doxygen's member grouping [1], I had to join some header files (`const` and non `const` headers, for example), because otherwise I could not get some functions in separate files to be in the same member group in the order I want. In the end, the library user includes `<babeltrace2/babeltrace.h>`, so how we organize the headers exactly is not so crucial. [1]: http://www.doxygen.nl/manual/grouping.html#memgroup Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: I6d1dc2e7c5ee63fcd4220d0fd9f0931d361d2f31 Reviewed-on: https://review.lttng.org/c/babeltrace/+/2807 Tested-by: jenkins <jenkins@lttng.org>
.gitignore: add missing `/tests/lib/test_remove_..._destruction_listener` Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: I20a520668a9f2bff7c47891072f907fd2b53d9e6 Reviewed-on: https://review.lttng.org/c/babeltrace/+/2635 Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
.gitignore: add missing `/tests/param-validation/test_param_validation` Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: Iba60cf173cdbcbf9986758a548ed732cc3008454 Reviewed-on: https://review.lttng.org/c/babeltrace/+/2316 Reviewed-by: Simon Marchi <simon.marchi@efficios.com> Tested-by: jenkins <jenkins@lttng.org>
Add bt_graph_add_simple_sink_component() tests As of this patch, the tests are basic and only check that the statuses returned by the user functions are correctly translated and returned by bt_graph_run_once(). The user initialization function status is not returned by bt_graph_add_simple_sink_component() because the initialization function is called within the sink component's "graph is configured" method. Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: Id8ccfd7df4e4a8f0a39af386ed4ad13fe657f86a Reviewed-on: https://review.lttng.org/c/babeltrace/+/1830 Reviewed-by: Simon Marchi <simon.marchi@efficios.com> Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Add internal command-line argument parser API This patch adds an internal command-line argument parser API. The exact API is well documented in `src/argpar/argpar.h`. The features that I'm looking for are: * C API. * Support for ordered arguments in the results, even between options and non-option arguments (sometimes called positional arguments). * Support for GNU-style arguments, including "glued" short options and long options with `=` to set the option's argument. * Portable. * Compatible license. * No global variables (we're trying to avoid that programming style throughout the project). * Easy to use. None of the popular libraries I looked at, including popt of course, met all those specifications. The goal of this is: 1. To simplify the parsing of general options (before the command name) in bt_config_cli_args_create(). It is currently hard-coded. This is not possible with popt because it would fail with an unknown option, and you would not know the position of the command name's argument within the array. This is possible with g_option_context_parse(). 2. To make it possible, in the `convert` command, to assign parameters and other position-specific options to a non-option argument, for example: babeltrace2 /path/to/trace --params=some=param \ mein-other-trace --name=travel This is not possible with popt as it collects all the non-option arguments as an array of leftover arguments. This is also not possible with g_option_context_parse() for the same reasons as with popt. getopt_long() could satisfy both 1. and 2., but it's somewhat a pain to use and to maintain the available options, as you need to specify the long options in a table and the equivalent short options as a single string with a special encoding (the `convert` command's option string would be particularily ugly). Also: getopt_long() plays a lot with global variables; it's not thread-safe: the parser's state is global. Also: the upstream getopt_long()'s (glibc) license is LGPL, which is more restrictive than our MIT license, so I think we want to avoid that. I believe having our own (tested) CLI argument parser is beneficial, especially in the long term: we can drop a direct dependency (and popt is getting old), not introduce a new one, it's about 450 lines of documented C code, and if we ever need something very specific in the argument parsing strategy in the future, we can add it directly. As a reference, FFmpeg, a project which has complex argument parsing, similar to Babeltrace's `convert` and `run` commands, has its own argument parser (see parse_options() in [1]). Only the CLI will use the bt_argpar_*() API, but to make the unit tests simpler, I put it in its own convenience library. [1]: https://github.com/FFmpeg/FFmpeg/blob/b7b6ddd59693008c35b3247496ecc946331d0856/fftools/cmdutils.c Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: Iff4fc305b9e9171c694e1e79428bd3838ddd989d Reviewed-on: https://review.lttng.org/c/babeltrace/+/1646 CI-Build: Simon Marchi <simon.marchi@efficios.com> Tested-by: jenkins <jenkins@lttng.org>
Tests: move `test_bitfield` outside lib tests, add `check-no-bitfield` target While the bitfield test (`test_bitfield`) is very valuable, it is also very slow with a typical Babeltrace developer configuration, that is: BABELTRACE_DEV_MODE=1 BABELTRACE_DEBUG_MODE=1 \ BABELTRACE_MINIMAL_LOG_LEVEL=TRACE \ CFLAGS='-O0 -g3 -Werror -Wall -Wno-error=unused-function' On my system and with the build configuration above, `test_bitfield` takes 3:10 to run, while all the other tests, including the Python bindings tests, but excluding `test_bitfield`, take 0:16. Because `test_bitfield` exclusively tests the functions in the `bitfield.h` header, and because they almost never change, I never run the `test_bitfield` test, removing the line in `tests/Makefile.am` every time. The CI runs all the tests anyway, so for a first pass, I believe it's not worth the wait. This patch moves `test_bitfield` into its own `tests/bitfield` directory: `bitfield.h` is not part of the library anyway, so I don't know why it's in `tests/lib` in the first place. In `tests/Makefile.am`, a new `check-bitfield` target is added to run just this test, as well as a new `check-no-bitfield` target which runs all the tests but `test_bitfield`. Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: Ib9f6a8e7cb9d01e7ef117b1a4d9c84788a4a68d2 Reviewed-on: https://review.lttng.org/c/babeltrace/+/1655 Tested-by: jenkins <jenkins@lttng.org>
Replace libuuid with internal implementation We use a very small subset of libuuid features to transform UUIDs between their string and binary representation. Plus we have a lot of compat code for different platforms with some unspecified default behavior regarding the use of upper/lower case. Drop the dependency on libuuid and replace it with a minimal internal implementation that respects RFC4122. Change-Id: Ic170ce26ade23d177195cad117bd0fab590b328e Signed-off-by: Michael Jeanson <mjeanson@efficios.com> Reviewed-on: https://review.lttng.org/c/babeltrace/+/1572 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Fix: bt_ctfser_write_float64(): use `double` in union, not `float` This bug made both CTF writer and `sink.ctf.fs` write wrong 64-bit floating point fields. This patch also adds two `sink.ctf.fs` tests to catch this. Both tests run a CTF generator which uses CTF writer, then run babeltrace2 /path/to/generated/trace -o ctf -w /path/to/converted/trace and then pass the converted trace into a `sink.text.details` sink to verify that the content is expected. Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: If69ee9a0e1d4038dd28a72860ecc11f18bb6b50e Reviewed-on: https://review.lttng.org/c/babeltrace/+/1645 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
tests: Rework test suite * Test script are not generated by autoconf anymore, instead the parameters are passed trought a set of environment variable prefixed with BT_TESTS_. This will enable the test suite to be used against a system installed version of babeltrace with minimal configuration. * Move all duplicated shell script code to 'utils.sh'. * Add 'set -u' to common.sh to catch undefined variables in test scripts. Variables that are expected to be undefined must be used as '${var:-}'. * All shell script is shellcheck tested. * Removed unused test 'test_trace_ir' * The python test runner now returns an error when no python test scripts are found. Signed-off-by: Michael Jeanson <mjeanson@efficios.com> Change-Id: I15dcda1d147c8c72c6e4ee00052591eb6f9e30bd Reviewed-on: https://review.lttng.org/c/babeltrace/+/1429 Tested-by: jenkins <jenkins@lttng.org> Reviewed-by: Simon Marchi <simon.marchi@efficios.com> Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Put `flt.lttng-utils.debug-info` tests into their own directory The goal is to have each component class tests in their own directory under `tests/plugins`. Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com> Change-Id: I3f68d2af9087690c6a34d238197c9bb9e96b021c Reviewed-on: https://review.lttng.org/c/babeltrace/+/1423 Tested-by: jenkins Reviewed-by: Michael Jeanson <mjeanson@efficios.com>