git.efficios.com Git - babeltrace.git/log

Remove everything related to the `babeltrace` Python package

This package is the Babeltrace 1 package, and since we plan to make
Babeltrace 1 and Babeltrace 2 coinstallable, we decided to drop the
support for this one.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I37f2de0129df00ae5d78fc3f81ad6c57763809cb
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1391
Tested-by: jenkins

tests: clean up test_trimmer

This patch changes test_trimmer to reduce duplication of the
boilerplate. I intend to add some test cases to this test in a
following patch, so this change allows me to understand better what the
test does currently and will help adding test cases.

Note that there are pairs of tests that are pretty much redundant, such
as:

- "Read a trace with the trimmer enabled (GMT relative timestamps)"
- "Ran successfully with --begin and --end (GMT relative timestamps)"

So I have only kept the second test of such pairs.

Change-Id: I075df21ed9c03495ab516d3870ccf7b63276d3ce
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1407
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

lib: make precondition failure messages more clear on message creation

Suggested-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: If1486d9bbb3fad02235ce14f53a974e57283fc1b
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1390
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>

src.ctf.fs: trace-info: omit stream `range-ns` field when no TS

Don't print -1 timestamps when packet contexts don't have
`timestamp_begin` and `timestamp_end` fields. Simply omit this part of
the query reply.

Remove FIXME comment as the only user of the index now handles absent
timestamps.

Tests
=====
Adapt `test_ctf_plugin` stream sorting function to fall back on using
the `paths` field if `range-ns` field is absent and rename it to
abstract implementation details.

Add test case to test that `range-ns` fields are absent on traces
without `timestamp_begin` and `timestamp_end` fields in their packet
context.

Example
=======
Here is an example of the output of the `trace-info` query on a trace
without packet contexts before this commit:
  -
    streams:
      -
       range-ns:
          begin: -1
          end: -1
       paths:
          - /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context/stream
       class-id: 0
       port-name: /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context | 0 | /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context/stream
    range-ns:
      begin: -1
      end: 0
    name: no-packet-context
    path: /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context

Here is an example of the output of the `trace-info` query on a trace
without packet contexts after this commit:
  -
    streams:
      -
       class-id: 0
       port-name: /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context | 0 | /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context/stream
       paths:
          - /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context/stream
    name: no-packet-context
    path: /home/frdeso/projets/babeltrace/tests/ctf-traces/succeed/no-packet-context

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I419f57960a52d8ed2823fdb06982e5c20710b6da
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1387
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

doc: Rename to babeltrace2

Adjust the documentation and manpages following the rename of the
executables and library.

Change-Id: I42cc662a94be6ece489af85d0415440bf1c1f36e
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1406
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

lib: rename include dir to babeltrace2

Rename the include dir to be co-installable with bt1.

Change-Id: Icb9a048988544eaca7054648b9c7a9e76d70b9db
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1405
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

lib: Reset libbabeltrace2 to SONANE 0

Following the rename of the library, reset the SONAME to zero and
decouple this from the project versioning which is not tied directly to
the library ABI.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: If3a3c890674d6236cfe33b5f3f1e3fe4229c5691
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1404
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

lib: Rename to libbabeltrace2

We decided to make babeltrace2 co-installable with babeltrace1 since it
offers backward compatibility but is not a full drop in replacement.

Change-Id: I6da0230858601dd35ae35424540c73ba7b143858
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1403
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

cli: Rename to babeltrace2

We decided to make babeltrace2 co-installable with babeltrace1 since it
offers backward compatibility but is not a full drop in replacement.

This first change renames the command line executables.

Change-Id: I52db05629b6349a5221f6b684cb15b289db8c6b3
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1402
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Fix: lib: expose bt_lib_log_level symbol

The Python plugin provider does not work.  It fails to be loaded:

    $ ./cli/babeltrace -v list-plugins --plugin-path /tmp/python
    ...
    06-06 15:03:51.883 13378 13378 I PLUGIN init_python_plugin_provider@plugin.c:80 Cannot open `libbabeltrace-python-plugin-provider.so`: /home/smarchi/build/babeltrace/python-plugin-provider/.libs/libbabeltrace-python-plugin-provider.so: undefined symbol: bt_lib_log_level: continuing without Python plugin support.
    ...

The Python plugin provider is built as a .so separate from the main
libbabeltrace.so lib.  However, it uses the logging system of the lib,
including the bt_lib_log_level symbol.  This symbol is currently not
exposed to other shared objects because it is marked as BT_HIDDEN.
Remove BT_HIDDEN to make the symbol visible to the python plugin
provider shared object.

Now that the Python bindings are merged, re-enable the Python plugin
provider test, which exposes the problem.

Note that the Python plugin provider is built as a separate .so for
packaging purposes (so it can be distributed separately), but is still
considered part of the lib.  This is why it is allowed to include and
use internal lib things.

Reported-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I716ae2eb6e2457c7a4f130a394f97391d56454e4
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1388
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Add missing test_output_ctf_metadata to gitignore

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Ia73e2a4510e2085c54794caa179a13cbac89fa77
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1384
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Cleanup: use detected asciidoc to build CONTRIBUTING.html

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I3ed041bf02711fe5f3267865410756b8df8bfb3f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1352
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Fix: src.ctf.fs: trace-info: fields no appended if range not set

Issue
=====
If the stream range is not set, the entire `if` clause is skipped and
the `paths`, `stream-id`, and `port-name` are not returned by the query
for that stream.

Solution
========
Append the value of this file group info even if the stream range is not
set.

Drawbacks
=========
None.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I7e185bcf3b640a684fc7944423eeb8d337764b19
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1386
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins

src.ctf.fs: merge all indexes to the fs_ds_group level

Background
==========
The index contains information on the packets of the underlying CTF
trace (e.g. timestamp begin and end). It is currently used to compute
the time boundaries of the all streams during the `trace-info` query. It
will probably be used for seeking in the future. Currently, each
datastream file has its own index.

To find the time boundaries of a given stream, we need to get the first
entry of the first datastream file and the last entry of the last
datastream file.

Why merge all the indexes ?
===========================
There are two reasons why we want to merge the file indexes into a
single stream index:
* It's slightly simpler to extract the time boundaries of the stream.

* Changes to overcome various packet timestamps tracer bugs are about to
be introduced and will need to iterate over all sorted index entries of
each stream.

Approach
========
Move the index from `struct ctf_fs_ds_file_info` to
`struct ctf_fs_ds_file_group` and merge the indexes of a given stream
when merging the different chunks of the same trace.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Iabee86c7c76c69b56cbf86449f8930e6d35969e2
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1362
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

src.ctf.fs: bubble up `ctf_fs_ds_file_group_create()` error

This will be used by a future commit to bubble up other allocation
errors.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I3a847d8e2e2db15f661360305b4818afba1666ab
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1372
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

sink.ctf.fs: add more comments in code where it's not straightforward

This patch adds a few comments in the code of `sink.ctf.fs` to help
future contributors without deep knowledge of CTF.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ibaa9ae27adb42f76329d0eab5a321368f452fc56
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1376
Tested-by: jenkins
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>

Rename "default beginning/end CS" -> "beginning/end default CS"

Because the target is the "default clock", it makes more sense to keep
those words together and have the beginning/end adjective outside.

Suggested-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I9fdd32c6acefc51b63b322dc41470c50b9226dfa
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1369
Tested-by: jenkins
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>

src.ctf.fs: support no packet beg/end CS and no discarded events/packets

This patch makes `src.ctf.fs` support CTF traces where a packet context
field class has or does not have the `timestamp_begin`, `timestamp_end`,
`events_discarded`, and `packet_seq_num` members. The corresponding
trace IR stream class's properties are set accordingly.

This fixes a bug having those base conditions:

1. Stream class has a default clock class.

2. Packet context's `timestamp_begin` and `timestamp_end` members
   are missing.

Then:

a) Because of 1., it was mandatory to set the default clock snapshot of
   packet beginning and end messages. It was also mandatory to set the
   default clock snapshots of discarded events/packets messages.

b) Because we didn't have the packet beginning and end times (because of
   2.), we used 0 for the first packet's beginning default clock
   snapshot.

c) Because CTF uses the packet beginning/end times to set the discarded
   events ranges, and because of b), we also used 0 as a discarded
   events message's beginning default clock snapshot if the events were
   discarded during the first packet.

0 is wrong for both b) and c): it's not 0, we just don't know the value.
It is wrong to tell a downstream component that events were possibly
discarded from 0 to another time because we just don't know those times.

I'm adding two test traces to make this code covered:

`no-packet-context`:
    No packet context at all, which means no packet beginning/end times,
    no discarded events, and no discarded packets.

`ev-disc-no-ts-begin-end`:
    A packet context with an `events_discarded` member, but without the
    `timestamp_begin` and `timestamp_end` members.

    In this case, when there's a discarded events message, events are
    known to have been discarded contextually, between two other
    messages with a default clock snapshot, but not at a specific time.

    `sink.text.pretty` prints:

        WARNING: Tracer discarded 17 events (unknown time range) in
        trace "ev-disc-no-ts-begin-end" (no UUID) within stream
        "/path/to/babeltrace/tests/ctf-traces/succeed/ev-disc-no-ts-begin-end/stream"
        (stream class ID: 0, stream ID: 0).

    while Babeltrace 1.5 prints:

        [warning] Tracer discarded 17 events between
        [19:00:00.000000000] and [19:00:00.000000000] in trace UUID
        0000000000000000, at path:
        "tests/ctf-traces/succeed/ev-disc-no-ts-begin-end", within
        stream id 0, at relative path: "stream". You should consider
        recording a new trace with larger buffers or with fewer events
        enabled.

    which is not exact.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ieebf679a67bd0089e48708372caf934cc27573dc
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1368
Tested-by: jenkins
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>

Fix: sink.text.pretty: print_discarded_elements_msg(): add missing space

Before:

    WARNING: Tracer discarded 17 events(unknown time range) in trace
    "ev-disc-no-ts-begin-end" (no UUID) within stream
    "/path/to/ev-disc-no-ts-begin-end/stream" (stream class ID: 0,
    stream ID: 0).

After:

    WARNING: Tracer discarded 17 events (unknown time range) in trace
    "ev-disc-no-ts-begin-end" (no UUID) within stream
    "/path/to/ev-disc-no-ts-begin-end/stream" (stream class ID: 0,
    stream ID: 0).

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I3f93f96479b2249c3e16f68e53533c48cf02a679
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1367
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins

sink.ctf.fs: support packets, DE, and DP w/o beginning/end clock snapshots

This patch makes `sink.ctf.fs` support more stream class configurations.
The table below shows the valid configurations, where:

DE:
    Discarded events.

DP:
    Discarded packets.

CS:
    Clock snapshot.

Each table cell indicates whether or not the given message has a default
clock snapshot (or two default clock snapshots for discarded events
packets messages). Each row is a configuration supported by
`sink.ctf.fs`.

    +------------+------------+-------+-------+
    | Pkt beg CS | Pkt end CS | DE CS | DP CS |
    +------------+------------+-------+-------+
    | No         | No         | No    | No    |
    | No         | Yes        | No    | No    |
    | Yes        | No         | No    | No    |
    | Yes        | Yes        | No    | No    |
    | Yes        | Yes        | No    | Yes   |
    | Yes        | Yes        | Yes   | No    |
    | Yes        | Yes        | Yes   | Yes   |
    +------------+------------+-------+-------+

All other configurations are not supported (the component logs an error
and fails).

As shown by the table above, discarded events and packets messages
without default clock snapshots are always supported:

* Any discarded events message occuring between two packet end messages
  increments the discarded event counter for the packet targeted by the
  second packet end message.

* Any discarded packets message occuring between a packet end message
  and a packet beginning message increments the packet sequence number
  for the packet targeted by the packet beginning message.

When discarded events/packets messages have default clock snapshots,
however, packet beginning and end messages must also have default clock
snapshots. This is to ensure that the discarded item time range matches
what is specified by CTF, which relies on the packet time ranges.
`src.ctf.fs` never produces (and will never produce) such a
configuration, so the CTF-to-CTF use case remains completely compatible.

When packet beginning messages have no default clock snapshot, the
generated packet context has no `timestamp_begin` member.

When packet end messages have no default clock snapshot, the generated
packet context has no `timestamp_end` member.

When a stream class does not support discarded event messages, the
generated packet context has no `events_discarded` member.

NOTE: As of this patch, `src.ctf.fs` cannot produce all the
configurations above. This work is reserved for a future patch.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I4e4fcb5c6f4e363d5c467427fb151c725d100cfb
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1366
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins

lib: make discarded events/packets support and clock snapshots optional

This patch adds functions to the stream class API to make it explicit
that discarded events and/or discarded packets are supported for stream
instances. The same functions are used to specify whether or not
discarded events and discarded packets have time ranges, that is,
default beginning and end clock snapshots.

The basic use cases to support are:

1. CTF 1.8: packet context has `timestamp_begin`, `timestamp_end` packet
   context members, as well as `events_discarded` and/or
   `packet_seq_num` members. This is legit: LTTng produces this.

2. CTF 1.8: packet context has `events_discarded` and/or
   `packet_seq_num` packet context members, but no `timestamp_begin` and
   `timestamp_end` members. This is possible (barectf, other tracers).

3. Source has no concept of discarded events and/or packets.

My initial approach was to have only the "discarded events have default
clock snapshots" and "discarded packets have default clock snapshots"
stream class properties, but they would need to be false by default
(because there's no default clock class by default), and then
`sink.ctf.fs` would, at least temporarily, require those default clock
snapshots to exist for those messages when there's a default clock
class, so simple sources without the discarded events/packets concept
would always need to set both stream class properties. I found this
weird, so the default is:

* No discarded events support.
* No discarded packets support.

When you set that the streams of a given stream class support discarded
events and packets, you specify at the same time if they have default
clock snapshots:

    void bt_stream_class_set_supports_discarded_events(
        bt_stream_class *stream_class,
        bt_bool supports_discarded_events,
        bt_bool with_default_clock_snapshots);

    void bt_stream_class_set_supports_discarded_packets(
        bt_stream_class *stream_class,
        bt_bool supports_discarded_packets,
        bt_bool with_default_clock_snapshots);

This means that a simple source can keep the default properties as is
and not create any discarded events/packets messages; `sink.ctf.fs` will
work fine.

It is a precondition that the stream class has a default clock class
when you call bt_stream_class_set_supports_discarded_events() or
bt_stream_class_set_supports_discarded_packets() with the
`with_default_clock_snapshots` parameter set.

This patch satisfies the three use cases above with those
configurations:

1. Discarded events and/or packets supported with default clock
   snapshots.

2. Discarded events and/or packets supported without default clock
   snapshots.

3. Discarded events and/or packets are not supported.

The component classes are not modified to support custom configurations
yet:

* `src.ctf.fs` always supports discarded events/packets, and always with
  default clock snapshots when the stream class has a default clock
  class.

* `sink.ctf.fs` and `flt.utils.trimmer` expect that, if discarded
  events/packets are supported (and those messages are not intentionally
  ignored for `sink.ctf.fs`) and the stream class has a default clock
  class, discarded events/packets have default clock snapshots.

Support for other configurations will be added in future patches.

This change will also make it possible for `sink.ctf.fs` to not write
any `events_discarded` packet context member when discarded events are
known to be not supported.

Python bindings are updated to support this feature. The four new flags
are added as parameters to the TraceClass.create_stream_class(). The
corresponding (private) property setters in `StreamClass` check that you
cannot make the stream class _not_ support discarded events/packets, but
have discarded events/packets have default clock snapshots. The new
tests in `test_stream_class.py` verify this too.

In `message.py`, I changed how we check, for each type of message, if
the message object has a default clock snapshot or not. It used to rely
on its stream class having a default clock class or not, but it's not
the only condition for packet beginning, packet end, and now discarded
events and discarded packets messages. Now, in any default clock
snapshot property, the function checks its own, custom condition.
Because having the stream class have a default clock class is still a
typical condition, there's the helper
_Message._check_has_default_clock_class(). I renamed
`NoDefaultClockClass` to `NonexistentClockSnapshot` to make the
exception more generic, considering what's written in this paragraph.

I also removed the `_DiscardedMessage.default_clock_class` property
because it could not be found in other types of messages, which is weird,
and you can always access it with

    mein_msg.stream.stream_class.default_clock_class

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I18b0bc65ef61e8bbd01521fb20c223d401d2adc9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1365
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins

lib: create_packet_message(): make assertion message less convoluted

Instead of having a message which enumerated the possible precondition
breaks, have it say that the stream class has an unexpected configuration,
and add the "packets have default beginning clock snapshot" and
"packets have default end clock snapshot" properties to the extended
logging of a stream class object in `lib-logging.c`.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ic7b706b9d6ef3afb799072a8891a0ba24370f317
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1364
Tested-by: jenkins
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>

lib: create_packet_message(): use pool to determine beginning or end msg

The `is_beginning` parameter in create_packet_message() is redundant as
the function can use `pool` and `msg_iter` to determine if it's creating
a packet beginning or end message.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I7f9b4183d004947895fba16b2e8396b8cb5be33a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1363
Tested-by: jenkins
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>

tests: build system spring cleanup

* Remove AC_CONFIG_FILES generated files from check_SCRIPTS and
EXTRA_DIST since they are already included.
* Remove useless conditionnals on AC_CONFIG_FILES generated files
since they are always included.
* Reduce the number of mostly unused Makefiles
* Remove execute permission on data files

Change-Id: I4e82bac01bf2033409c67508a4d60127a0e77f14
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1350
CI-Build: Francis Deslauriers <francis.deslauriers@efficios.com>
Tested-by: jenkins
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

tests: replace BT_ASSERT with exit in test_bin_info

Since this is a test executable and not a library, it's cleaner to exit
with an error code than use an assertion which could be disabled.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I1684a21d510ea444106b84f5e06ece4a7ff96ef4
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1382
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>

tests: Add expected failure test to test_dwarf

Add a test to make sure we fail when opening an ELF file with no DWARF
information.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I59419a94fa0bca5589600e7b0691d291531cfa51
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1349
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>

tests: Add powerpc64le-linux-gnu debug-info artifacts

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Ic86e49344dccf3d547bc441c4a16a4dca8553e1f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1348
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>

tests: Add powerpc-linux-gnu debug-info artifacts

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I2b470db8427e20685a744edd6c9cec29fc70c54d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1347
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>

tests: Add i386-linux-gnu debug-info artifacts

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Ia8455b987cc2d4c82b1e119c8d12c2708d3d0ded
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1346
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>

tests: Parametrize test_bin_info to handle multiarch artifacts

The values that change on each build of the test artifacts were
hardcoded in the test source. Parametrize them so the test can be used
on differents artifacts.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I003733f54bdf1f6cee0aca134f31dec79d75b35d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1345
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>

tests: Build new x86_64-linux-gnu debug-info artifacts

Build new debug info tests artifacts using the Makefile and namespace
them and the related tests with 'x86_64-linux-gnu'. This will allow
adding artifacts from different architectures to improve the test
coverage.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I97bd5052e9c3537f625581d21723e848c2a373b7
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1344
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>

tests: Add Makefile to build debug-info test artifacts

The current documentation on building the debug info test artifacts is
missing some important details, add a Makefile to make the process more
repoducible.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I188ce56bbe4d446e8959e18d2f4c50b8a82a4cb0
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1343
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>

Cleanup: test_bitfield: nr_bits should be unsigned

Should not have any effect on the code behavior, but removes
useless type conversions between unsigned and signed types.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I39096ab89936875009a68d3ab8a4546c7f1eb8ae
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1375
CI-Build: Michael Jeanson <mjeanson@efficios.com>
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Fix: lib: set iterator state even in non-dev mode

The following test fails currently when building with
BABELTRACE_DEV_MODE off:

    FAIL: test_finalize (test_message_iterator.UserMessageIteratorTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File ".../babeltrace/tests/bindings/python/bt2/test_message_iterator.py", line 67, in test_finalize
        self.assertTrue(finalized)
    AssertionError: False is not true

The test verifies that the _finalize method (in Python) of an input port
message iterator is called when the iterator is destroyed.  The failure
shows that the method is not getting called.

The issue is that
bt_self_component_port_input_message_iterator_try_finalize doesn't call
the finalize method if the iterator state is
BT_SELF_COMPONENT_PORT_INPUT_MESSAGE_ITERATOR_STATE_NON_INITIALIZED.
However, the iterator state is only set when in dev mode.  In non-dev
mode, the state is left to its initial value of
BT_SELF_COMPONENT_PORT_INPUT_MESSAGE_ITERATOR_STATE_NON_INITIALIZED.

Fix this by setting the state of the iterator regardless of whether we
are in dev mode or not.

Change-Id: I158c9d913777eba117ed0d32e82db5a1b29f50ab
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1385
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins

bt2: change some bt2.CreationError usages to ValueError

The bt2.CreationError exception type is reserved for cases where the
creation functions return None/NULL, which is usually because of
exhausted memory.  Other exception types should be used for other
errors.

This patch fixes a few instances where bt2.CreationError is wrongly
used.  Change them for ValueError, since they are cases of wrong
parameter value passed by the user.  We have already used ValueError for
cases like these, for example in
_UserMessageIterator._create_packet_beginning_message.

Change-Id: Ib66943f8dc1200f7b589764c171bf4764741a6bd
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1383
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: remove unused exception types

There are several exception types defined in __init__.py.in that are not
used anywhere, remove them.

Change-Id: I6ba2c62689a4dc72f4b5325715962a7721169e07
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1381
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

src.ctf.fs: add stream port name to trace-info query, use it for stream intersection

For creating stream intersection trimmers, we currently rely on the fact
that the CLI and the Python bindings know that src.ctf.fs component port
names are the path to the data stream files (or the first data stream
file for the corresponding stream, if there are multiple files).  Using
this information, they can deduce which trace (from the trace-info
query) this port is from, by checking for a trace whose path is a prefix
of the port name.

This patch changes how this works to avoid the CLI and Python bindings
having internal knowledge of how the src.ctf.fs component class names
the ports.  The trace-info query now include a "port-name" property for
each stream.  This property contains the name that a src.ctf.fs
component would give the port associated to this stream.  This allows
the CLI and the Python bindings to match a port with a trace (and
therefore an intersection range) just by comparing the port name in the
actual component instance with the port name in the trace-info query
result.

Here's an excerpt (just two streams) from a trace-info result with this patch
applied:

    -
      range-ns:
        begin: 1556911621056878212
        end: 1556911631077588247
      class-id: 0
      paths:
        - /home/smarchi/src/lttng-envs/augmented-metadata/home/lttng-traces/perpid-20190503-152658/archives/20190503T152700-0400-20190503T152703-0400-1/ust/pid/hello-ust-2335-20190503-152701/chan_2
        - /home/smarchi/src/lttng-envs/augmented-metadata/home/lttng-traces/perpid-20190503-152658/20190503T152705-0400-3/ust/pid/hello-ust-2335-20190503-152701/chan_2
      port-name: 1838b922-0089-497c-96e7-2c7ad63e901e | 0 | 2
      id: 2
    -
      range-ns:
        begin: 1556911621053751789
        end: 1556911631077569864
      class-id: 0
      paths:
        - /home/smarchi/src/lttng-envs/augmented-metadata/home/lttng-traces/perpid-20190503-152658/archives/20190503T152700-0400-20190503T152703-0400-1/ust/pid/hello-ust-2335-20190503-152701/chan_1
        - /home/smarchi/src/lttng-envs/augmented-metadata/home/lttng-traces/perpid-20190503-152658/20190503T152705-0400-3/ust/pid/hello-ust-2335-20190503-152701/chan_1
      port-name: 1838b922-0089-497c-96e7-2c7ad63e901e | 0 | 1
      id: 1

I added some simple tests, although they don't cover all the cases.  We
currently don't have a trace without uuid, or a trace with
stream_instance_id fields, to test the remaining possibilities.  I plan
on adding some soon, when adding tests for the trace merging feature, at
which point we can also enhance
test_query_trace_info.QueryTraceInfoPortNameTestCase.

Change-Id: Ib3afc19f2c0e09a7235475b53f8023c8a2f40ebf
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1339
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Fix: sink.text.pretty: comma in enum fields

The condition for printing commas that separate labels of enum fields is
reversed.  It causes a comma tu be spuriously printed before the first
label:

  enumfield = ( , "zero" : container = 0 )

instead of:

  enumfield = ( "zero" : container = 0 )

With multiple matching labels, before:

  enumfield = ( , "zero""one" : container = 0 )

and after:

  enumfield = ( "zero", "one" : container = 0

Change-Id: I03452b8ece4f97bf0c819f6885ef512f7ddc31e6
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1334
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

cli: Fix exit code of ctf-metadata output

When the command

    babeltrace -o ctf-metadata <trace>

is successful, it exits with exit code 1.  On success, we would expect
it to return 0.

The issue is that when cmd_print_ctf_metadata exits, ret is filled with
the number of bytes returned with fprintf, which is non-zero.  This is then translated to a 1 in main.

If the fprintf is successful, we should set ret to 0 to indicate that
the function has completed successfully.

If the fprintf fails, we should go the the end and avoid setting ret to
0 (the fprintf return value will be passed to the caller).

Add a test to verify the exit code and output of babeltrace when using
-o ctf-metadata.

Change-Id: I64372bce66822ba570a85cd69a32d582ebd3c727
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1332
CI-Build: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_trace_collection_message_iterator.py and make it pass

Update test_trace_collection_message_iterator.py and
trace_collection_message_iterator.py to work with the current Babeltrace
API.

It is no longer possible to pass a "message type" filter to a message
iterator.  This means that the TraceCollectionMessageIterator objects in
the test now return all messages, where we previously filtered to only
have those of type EventMessage.  The new count_msg_by_type function
helps to get around this problem.

In trace_collection_message_iterator.py, there are just a few changes to
account for the API changes.  The src.ctf.fs component now takes a
"paths" parameter instead of "path".  It is no longer possible to get a
component from a port, instead we use the component passed as a
parameter in the callbacks.

The change in the way we handle the timestamps in _create_trimmer is due
to the fact that if we pass a single integer, it is interpreted as
seconds.  Since we have a number of nanoseconds, it is misinterpreted.
Instead, pass a string in the seconds.nanoseconds form.

Change-Id: Iddd1666008d06e49843932dc74ee51ac772d001b
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1328
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: let components attach "user data" to ports

The C API allows components to attach "user data" to ports when creating
them:

    extern bt_self_component_status
    bt_self_component_source_add_output_port(
                    bt_self_component_source *self_component,
                    const char *name, void *user_data,
                    bt_self_component_port_output **self_component_port);

This is useful to identify the purpose of a given port.  This data can later be
fetched when an iterator is created for that port.

This patch makes the Python API offer a similar facility.  When adding a port,
the user can optionally pass an arbitrary Python object as user data:

    self_port = self._add_output_port('port name', user_data={'foo': 23})

which they can then access using the user_data property of
_UserComponentPort:

    print(self_port.user_data)  # {'foo': 23}

We confine the user data under the user_data property to avoid
clashes with future methods and properties that we might add to
_UserComponentPort or its subclasses.

A new "in" typemap is created for the "add port" functions to pass the
PyObject* representing the user data Python object to the creation
function.  Without the typemap, SWIG complains that the passed value (a
PyObject *) is not of the right type (it expects a void *):

    TypeError: in method 'self_component_source_add_output_port', argument 3 of type 'void *'

If the port is created successfully, it now owns a reference to this
Python object.  We must reflect that in the Python object's refcount,
this is done through an "argout" typemap.  In this typemap, we need to
check the return value of the function we called.

When fetching the user data of a port, we use an "out" typemap to
increment the refcount of the returned value.  This is because a Python
method that returns an object must return a new reference that is
transferred to the caller.

Change-Id: I0b83454a81e71bd7c2fe9449c7fc65c09f18fcf4
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1359
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_message_iterator.py and make it pass

Update test_message_iterator.py to work with the current Babeltrace API.
Most changes are related to the fact that it is not possible anymore to
create most objects in isolation, everything is derived from a trace
class.  Message iterators are created in the _graph_is_configured
callback rather than _port_connected. Because of this, we now need to
call graph.run() in the tests to get _graph_is_configured called.

I noticed that the component method of _SelfPortInputMessageIterator was
a bit problematic, as we don't know the type of the component at that
point.  Returning the right type of component (the right Python class,
wrapping the right type of Swig pointer) would require a bit of dirty
work.  I am not sure this is even useful, so I've just removed it and
marked the tests as skipped / commented them out (they can be removed if
we rule this is indeed unnecessary).

An important change is the addition the self_output_port parameter in
the constructor of _UserMessageIterator (which the user is expected to
override).  This parameter is how the iterator is supposed to know what
it's supposed to iterate on (if the component has multiple ports).

There are no other changes in message_iterator.py, since everything had
to be done already for test_message.py to pass.

Change-Id: I574d812ed81d9628a8e4a2a2dd5c593c4d730c95
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1326
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_message.py and make it pass

Update test_message.py to match the current Babeltrace API.  Update
message.py to also match the current API and make the test pass.
message_iterator.py is also updated, as it is required by the test, but
just the essential changes are done.

The test changes substantially, since it is no longer possible to create
a message by itself, they are created from a component message iterator,
which requires quite a bit of setup.  Also, it would be invalid to emit
an "event message" without having first emitted a "stream beginning
message", "packet beginning message", and so on.  So testing the message
types in isolation would required quite a lot of boilerplate.  This is
why all messages types are now tested in the same test.  We have a
source that emits all message types at least once.  The sink on the
other side verifies that what it receives from the source is what we
expect.

Everything related to equality, copy and deep copy is removed.

Change-Id: Ibe6146049cc1065df82dde821a42f10d4e5bba27
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1324
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins

bt2: Adapt test_field.py and make it pass

Update test_field.py and field.py to match the current API of
Babeltrace.

In the test, most changes are related to the fact that field classes and
fields are created ultimately from a trace class.  A bunch of helpers
are added at the top, which allow to easily create a field of a certain
type without repeating the boilerplate.

Anything related to copy and deepcopy is removed.  However, the test
_TestNumericField._test_binop_lhs_value_same relies on copy support, so
I am not too sure what to do with it.  I have currently marked it as
skipped, but we should either adapt it or remove it.

A concept that disappeared (or rather, is not exposed by the public API)
is whether a field is set or not, and the ability to reset a field's
value.  All tests and code related to that are removed.

Change-Id: I27f1ee6a3a2152232556d9d9c301de8411189a2c
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1323
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_field_class.py and make it pass

Update the test_field_class.py test to match the current API of
Babeltrace.  Update field_class.py and some others to make that test
pass.

The main change in test_field_class.py is that field classes are now
created from an existing trace class (tc.create_foo_field_class())
rather than by constructor (FooFieldClass()).  Everything related to
copy, deep copy and equality is removed.  Everything related to
structure alignment, byte order and encoding is removed, as those were
ctf concepts that were removed from Babeltrace's trace-ir API.

All specific field class tests try to follow the same pattern.  The
test_create_default test method verifies the properties of a specific
field class create with passing as few parameters as possible.  Then,
for each possible creation parameter, we verify a case that works and
some problematic cases such as invalid value or invalid type.  More
tests are done for specific field classes that support additional
features, such as container field classes.

The support for modifying field class properties after they have been
created is removed.  All characteristics of a field class must passed
during construction (this doesn't apply to fields of composite field
classes).

A new concept of "field path" was introduced in the library, which is
useful in the context of dynamically-sized arrays and variants.  Support
is therefore added to obtain the "field path" to a variant's selector
field or dynamic array length field.

Change-Id: Ia998119fcf7c61ef5904fbe72baa36a2838d5780
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1319
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_graph.py and make it pass

This patch updates test_graph.py and graph.py to work with the current
BT API.

An important change is that now, the port added and ports connected
listeners can report their failures up the stack.  Therefore, if a
Python listener raises an exception, we now report it as an error.  Some
tests are added for this.

The tests:

- test_connect_ports_canceled
- test_connect_ports_cannot_consume_accept
- test_connect_ports_cannot_consume_connected

were removed, since the library no longer returns an error in these
situations (they are just verified by precondition check).  However, a
test was added to check that when the graph gets cancelled during
execution, a bt2.GraphCanceled gets raised.

Some changes in message.py, message_iterator.py and port.py are
necessary to support the test.  They are not complete, but they are
representative of what will come after.

Any reference to ports being removed or disconnected were removed, since
it's not longer possible to remove or disconnect a port.

Change-Id: Ie8f11553f34208bb58242d9108efc361acba3c18
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1317
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins

lib: Make graph listeners return an error status

Currently, if a graph listener fails for some reason (for example, an
uncaught exception in Python), no error is reported to the initial
caller, so it has no way to handle the failure properly.  If a listener
fails, we can't really trust that the execution will proceed as
intended, so it is likely that the caller wants to exit with an error
too.

All the listener typedefs are updated to return a new type,
bt_graph_listener_status.  If any listener fails, the graph is put in
"faulty" state and we return an error up the stack.

If a listener fails, the action is not rolled back.  This means that if
a port added listener fails, the port that was added is not removed.  If
a ports connected listener fails, the connection is not undone.

A few call sites needed to be updated.  An interesting one is in
cli/babeltrace.c.  Where we previously aborted, we can now return an
error and exit cleanly.

The Python bindings is another user of the listener API.  I have
modified the functions in native_bt_graph.i to keep the code building,
but the errors are currently not propagated from the Python callback.
This will be done in a subsequent patch.

Change-Id: I115773c405162f7b1c617cf4a8302b980315e14d
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1316
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins

lib: remove unused _NO_SINK graph status

Also:

- Add an assert to verify that there is at least a sink component in
  the graph when it's configured.
- Remove the equivalent in the Python bindings, including the test in
  test_graph.py, even though test_graph.py is not functional at the
  moment.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I8d6ce79a2c77527908f500017bb346b2c2d94c63
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1315
CI-Build: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_packet.py and make it pass

This patch updates test_packet.py and packet.py to work with the current
API.

In both files, everything about equality, copy, deep copy and header
fields is removed.  The setter for context_field is also removed.

One side-effect change (required for the test) is in the
Field.field_class property.  Previously, when creating a Field object
wrapping a bt_field pointer, we would create the corresponding
FieldClass object and store it in Field._field_class.  Instead, we can
compute it as needed in the Field.field_class property, I don't really
see any problem with that.

Change-Id: I509b3bd272fb323ed6df7de47d21c31b8aedc72f
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1301
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_clock_class.py and make it pass

This patch adapts test_clock_class to the current BT API and changes
what's needed to make it pass.

One change in test_clock_class is that clock classes need to be created
from self components now, so it requires a bit more boilerplate.

Everthing related to equality, copy and deepcopy is removed from
ClockClass.  However, it remains possible to test a _ClockSnapshot for
equality against an integer.  I thoough it would be useful to support
other relational operations (<, <=, >, >=) for clock snapshots, so I
added support for them using functools.total_ordering.

The constructors for both _ClockSnapshot and ClockClass are removed, as
the user never directly creates those objects anymore.  Clock snapshots
are obtained from messages, while clock classes are created using the
_create_clock_class method of a component.

As in previous patches, the public setters are removed, as we only
support setting properies when creating an object.

Change-Id: I7228b32530f98811cb512243469ae7d0d61a9da1
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1299
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins

bt2: Adapt test_event.py and make it pass

This patch changes test_event.py to work with the current BT API and
adapts event.py accordingly to make it pass.  Since this is a central
piece of the API (it has many related concepts), a few other files are
modified as well, just enough to support the test.  For example, events
are no longer created directly, instead we need to instantiate an
EventMessage.  This requires to get the foundation for messages
working.  Even though the changes in message_iterator (the "create"
methods to create various messages) are not complete, it gives an idea
of what is to come, so it would be good to get some comments on it right
now.

In event.py, things related to clock snapshots, header fields, equality,
copy and deep copy are removed.

In test_event.py, we now need a bigger setup, since events are created
from event messages, which are created from output iterators, which are
created from components.

In object.py, UniqueObject now needs to record its _owner_get_ref
callback, so that if we get a second UniqueObject from an existing
UniqueObject (such as a sub-field from a struct field), the callback can
be passed to the new object.

Change-Id: I72309826a61245b0fe4fdd9a638ddee3689c5921
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1298
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins

Fix: lib: usage of output port message iterator

Fix some issues relate to output port message iterator usage.

The first encountered issue when trying to create one such iterator is:

    05-13 13:51:20.845 14995 14995 F MSG-ITER bt_self_component_port_input_message_iterator_create@iterator.c:444 Graph is not configured: addr=0x612000000640, is-canceled=0, can-consume=0, config-state=BT_GRAPH_CONFIGURATION_STATE_CONFIGURING, comp-count=2, conn-count=1, en-pool-size=0, en-pool-cap=0, pbn-pool-size=0, pbn-pool-cap=0, pen-pool-size=0, pen-pool-cap=0

The problem is that the colander is trying to create an input iterator on its
input port in the "port connected" callback, which is before the graph is
configured.  This is not a valid thing to do anymore, so change it to create
the input iterator during the "graph is configured" step.

The next issue we face is then:

    05-13 14:00:01.903 21344 21344 W COLANDER colander_consume@component-class-sink-colander.c:142 Trying to consume without an upstream message iterator: comp-addr=0x60c0000028c0, c
omp-name="colander-36ac3409-b1a8-4d60-ab1f-4fdf341a8fb1", comp-class-type=BT_COMPONENT_CLASS_TYPE_SINK, comp-class-name="colander", comp-class-partial-descr="", comp-class-is-frozen=1, comp-input-port-count=1, comp-output-port-count=0

This is because the "graph is configured" callback is never invoked,
the reason being that bt_port_output_message_iterator_create just sets
the graph's config state field directly, rather than calling
bt_graph_configure.

Change-Id: Idbf873912ea5d116f0b011639177e57aa78e6759
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1297
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_event_class.py and make it pass

This patch updates event_class.py to be more in line with the current
API and updates test_event_class.py accordingly.

Everything related to equality, copy and deepcopy is removed. An event
class is always created from an existing stream class, so it is not
longer possible to create an event class out of thin air. We only
support passing parameters to the event class when creating it, not
assigning them afterwards.

Change-Id: I2f1ad9f98f25e3e2dfdea511a1410529d014745b
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1296
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_stream.py and make it pass

This patch updates test_stream.py and stream.py. It remove everything
related to equality, copy and deepcopy, as in the previous patches.

Since we don't want to share code between the CTF writer and the
trace-ir objects, the _StreamBase class is removed, and ctf_writer.py is
updated just so import still works.

Change-Id: I72e80694e0c8b401a86ce23d94b6e064afb08ac2
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1292
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_stream_class.py and make it pass

This patch updates test_stream_class.py quite a bit.  In particular, it
removes everything related to equality, copy and deep copy.  Since we
don't support (at least, for the moment) assigning stream class
properties after it has been created, most "assign" tests have become
"create" tests.

Stream class objects in Python are created using
TraceClass.create_stream_class.  This method has been introduced
previously to support TraceClass tests, but this patch now adds the
missing options.  Some field class creation methods are added to
TraceClass, just enough to support test_stream_class.

Changes in stream_class.py are mostly to adjust to past API changes.
The assigns_automatic_event_class_id property is new, everything related
to "header" field classes is removed.  Everything related to equality,
copy and deep copy is also removed.  The __call__ function is removed,
as it's not how create streams anymore (we use Trace.create_stream).

A new concept in the lib is that packet beginning/end messages may or
may not have a default clock snapshot, even if the stream has a default
clock class.  An option to TraceClass.create_stream_class is added for
this.

Changes to other files are there to support the tests, and not meant to
be complete.

Change-Id: Idd182d3f512f4f5772816fee0c04e2b39fd163f2
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1291
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_trace.py and make it pass

test_trace.py is heavily modified, because most of the responsibilities
of Trace have been moved to TraceClass. A number of concepts, such as
packet headers, also disappeared.

Similarly, trace.py loses a lot of weight. Note that copy and equality
operations are removed, since we ruled that they were complex but not
particularly useful.

A trace implements abc.Mapping, and maps stream ids to stream objects.

A trace is created by "calling" a trace class, hence the addition of the
__call__ method to TraceClass.

The test_trace test creates some streams and requires to control the ids
of those streams, so the create_stream_class method and StreamClass type
are enhanced to support that.

Change-Id: If23089abfbcbe5f0064069f93bd91126f1549b62
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1290
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Add bindings for trace classes

This patch adds a Python wrapper for the bt_trace_class concept.

Trace classes in Python are created using the _create_trace_class on a
_UserComponent.

A TraceClass is seen as a mapping of stream class ids to stream classes.
This means that with trace class `tc`, it's possible to access its
stream classes by id using `tc[id]`. It's also possible to iterate on
`tc` to iterate on stream class ids.

A brand new test is also added, along with some test utils that should
be useful in future tests as well.

It is not feature-complete yet, for example there is no way to
instantiate a trace using this trace class, this will come in a
subsequent patch (that will handle test_trace.py). It is possible to
create stream classes in a trace class, but not with every possible
option, just enough to support the trace class tests.

Change-Id: Ib9f29d7ebc21b6ade19e9ffbad3f4b85e790321d
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1283
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Fix: lib: prevent infinite recursion when destroying trace classes and traces

When implementing support for trace and trace class destruction
listeners in Python, we hit a problem where we enter infinite recursion
(until the process crashes).

When the refcount of a trace class reaches 0, the destruction listeners
for that trace class are called. The Python listener creates a
TraceClass object, which acquires a new reference to the trace class
being destroyed. When the listener is done, it drops that reference.
This causes the refcount to reach 0 again, and the lib to call all the
destruction listeners again.

The same happens for traces.

To prevent it, this patch makes the lib increment the refcount by one
before calling all destruction listeners.

Change-Id: Ib7e7bd0428c2a505fb25ad0aa80150b518da4c4e
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1282
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

bt2: Adapt test_connection.py and make it pass

The test_connection test is adapted to account for:

- Connections can't be in an ended state anymore.
- We don't need to define equality between connections.
- Private connections don't exist anymore.

The bindings are also simplified by the fact that private connections
don't exist anymore, and connections can't end. Otherwise, changes are
straightforward to adapt to the existing API.

Change-Id: I52cb89512df1e8cb0cd88670c6d63e68287556c3
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1280
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins

bt2: Adapt test_port.py and make it pass

This patch does the essential to make test_port.py pass.

Everything related to equality between ports was removed from
test_port.py, as we decided that it was not particularly useful.
Everything related to disconnection of ports and removal of ports from
components was also removed, since these concepts don't exist in
Babeltrace anymore.

I have also removed the possibility of getting a component from a port.
This was the only place where an explicit downcast from bt_component to
the specific component type would have been necessary.  Instead, we have
concluded that every time you have access to the port, you already have
access to its component, already with the right type.

The possibility to create an output message iterator from an output port
is also removed, that operation will be done from a graph instead.

The other notable changes in the bindings are:

- Graph: Change add_sink_component to add_component, and handle the
  various component types.
- Port: Simple adaptations to the current API, using the now famous pattern
  of _as_*_ptr static methods.
- Component: Adjust how ports are listed and iterated on to account for
  changes in the API.

Change-Id: Iee42bb4f8508e08d7f2b2cca451cae05d52a0ed0
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1279
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins

Add configuration for python code formatter black

Black will look for pyproject.toml automatically.
pyproject.toml is defined by PEP518 [1]

Use skip-string-normalization for now [2].

Exclude results from swig, build directory of the python bindings and
results of python input file (py.in).

Include py.in files.

[1] https://www.python.org/dev/peps/pep-0518/
[2] https://black.readthedocs.io/en/stable/the_black_code_style.html#strings

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Change-Id: I57adffcba9517dea05f10f9571671e75ac1a860f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1355
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

lib: make packet beginning/end default CS optional

This patch makes it possible to create packet beginning and end messages
using bt_message_packet_beginning_create() and
bt_message_packet_end_create(), that is, without default clock
snapshots, even if the corresponding stream class has a default clock
snapshot. This was considered like a precondition break before this
patch.

Rationale
=========
The main use case for this is to support sources which do not have a
concept of beginning and end times for their packets, but which have
times for their events. Currently, such sources need to make up default
clock snapshot values for those packet messages.

This can be the case of `src.ctf.fs` which, when a given stream class's
packet context does not contain the `timestamp_begin` and
`timestamp_end` members, uses the current default clock's value during
the stream decoding process, except for the beginning time of the first
packet which is set to 0. This is problematic because it means the
stream is active from clock value 0 to the first event, which is
probably not true. A solution would be to use the first event's time in
this scenario, but:

* Several packets can be empty at the beginning, making the task
  difficult because we need to queue pseudo-messages until we have the
  first event's time and then create the real messages with this value.

* All packets can be empty, so no specific time value is available for
  the whole stream.

Although they are weird scenarios, they can still happen. Using 0 for
the first packet's beginning time is just wrong in those cases.

Other sources which do not need the concept of a packet can use this
new feature too: `src.text.dmesg` is one of them.

Library changes
===============
As with other features of stream-related objects, the default clock
snapshot property is not optional _per_ packet beginning/end message:
for the streams of a given stream class, either all packet beginning
and/or end messages have a default clock snapshot, or they don't.

This is controlled by a new stream class flag of which the API is:

    void bt_stream_class_set_packets_have_default_beginning_clock_snapshot(
        bt_stream_class *stream_class, bt_bool value);

    void bt_stream_class_set_packets_have_default_end_clock_snapshot(
        bt_stream_class *stream_class, bt_bool value);

    bt_bool bt_stream_class_packets_have_default_beginning_clock_snapshot(
        const bt_stream_class *stream_class);

    bt_bool bt_stream_class_packets_have_default_end_clock_snapshot(
        const bt_stream_class *stream_class);

By default, a stream class has no default clock class, so its packets
are also known to have no beginning/end default clock snapshots. When
you call
bt_stream_class_set_packets_have_default_beginning_clock_snapshot() or
bt_stream_class_set_packets_have_default_end_clock_snapshot(), you must
have called bt_stream_class_set_default_clock_class() first. This means
you can check if a packet beginning message, for example, has a default
clock snapshot with
bt_stream_class_set_packets_have_default_beginning_clock_snapshot()
without also checking
bt_stream_class_borrow_default_clock_class_const().

It is required that you have called
bt_stream_class_set_packets_have_default_beginning_clock_snapshot() in
order to use
bt_message_packet_beginning_create_with_default_clock_snapshot() for the
corresponding stream class. Same thing for
bt_stream_class_set_packets_have_default_end_clock_snapshot() and
bt_message_packet_end_create_with_default_clock_snapshot().

This is all validated in developer mode.

Plugin changes
==============
Component classes are changed as such:

`src.ctf.fs`:
`src.ctf.lttng-live`:
    In order to separate concerns into different patches, the procedure
    is not changed in this component class: all packet beginning/end
    messages have a default clock snapshot when the corresponding stream
    class has a default clock class, even if the stream class's packet
    context does not contain the `timestamp_begin` and `timestamp_end`
    members.

    Another patch will fix this issue.

`src.text.dmesg`:
    Like `src.ctf.fs`, the procedure is not changed: all packet
    beginning/end messages have a default clock snapshot when the
    corresponding stream class has a default clock class.

    The simplification change will be done in another patch.

`flt.utils.trimmer`:
    Stream classes without a default clock class are already not
    supported. With this patch, packet beginning/end messages without a
    default clock snapshot are not supported.

    The support will be added by another patch.

`flt.utils.muxer`:
    When a packet beginning/end message does not have a default clock
    snapshot, the iterator's last time is used to sort the messages.
    This is similar to other messages without a default clock snapshot.

`flt.lttng-utils.debug-info`:
    Stream class properties and packet beginning/end messages are copied
    considering the new feature.

`sink.ctf.fs`:
    Packet beginning/end without a default clock snapshot, but with a
    stream class which has a default clock class, are not supported.

    Support will be added by another patch.

All the tests still pass because, even though some filter/sink
components do not fully support the new feature, the existing sources do
not use it yet.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I7183bfe8f954235c7f54195b101f781b176ab733
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1361
Tested-by: jenkins

Fix: flt.lttng-utils.debug-info: memory leak bt_dwarf_die struct

Reported-by: Address Sanitizer - Memory Leak
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I4591884681920fc6068572dcdb1923f82c9f81d6
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1351
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>
Tested-by: jenkins

lib: remove "unknown clock snapshot" concept

The team decided that, because this "unknown clock snapshot" feature has
no current use case and that CTF 1.8 cannot use it, we can wait until we
really need it to implement it.

The feature was only partially implemented: you could know that a given
clock snapshot is unknown, but you could not make one unknown. This
asymmetry was so that current consumers (filters and sinks) could
support unknown clock snapshots without taking the development time to
make the producers (sources and filters) create them. This is another
proof that the feature is not needed now.

In the library, all bt_*_borrow_*_clock_snapshot() functions, except for
bt_message_stream_activity_beginning_borrow_default_clock_snapshot_const()
and
bt_message_stream_activity_end_borrow_default_clock_snapshot_const(),
now return the borrowed clock snapshot object instead of returning a
clock snapshot status (known or unknown). Python bindings and plugins
are adapted accordingly.

The `BT_MESSAGE_STREAM_ACTIVITY_CLOCK_SNAPSHOT_STATE_UNKNOWN` state
still exists for stream activity messages because it can be useful to
set that the first stream activity beginning or the last stream activity
end is unknown. `src.ctf.fs` is one user of this state.

Also, the bt_stream_class_default_clock_is_always_known() function is
removed because the default clock is always known. This function
unconditionally returned `BT_TRUE` anyway. Python bindings and plugins
are adapted accordingly.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: I2665f713917b14efdecb181bef16c199ddd9eb81
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1358
Tested-by: jenkins
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>

bt2: always use staticmethod() with native function class attributes

This is a recurrent pattern in the Python bindings:

    class Something(_SomeBase):
        _set_value = native_bt.something_set_value

Then template methods in `_SomeBase` can use this class attribute by
name within the template.

This function needs to be a static method, because template methods do:

    self._set_value(...)

and native_bt.something_set_value() must not receive `self` as its first
argument. For some reason, when you set a class attribute to a native
function (defined within an extension module), it is automatically a
static method, so you don't need to specify it manually. It is not the
case for any other Python function.

With SWIG 3, the generated wrapper code in `native_bt.py` looks like:

    def value_unsigned_integer_set(integer_obj, val):
        return _native_bt.value_unsigned_integer_set(integer_obj, val)
    value_unsigned_integer_set = _native_bt.value_unsigned_integer_set

There's a defined Python function, but then the same name is reassigned
to the native function itself (`_native_bt.value_unsigned_integer_set`).
I didn't investigate why SWIG does that; it could be an optimization,
but then I don't get why the Python wrapper is created in the first
place. This means that, with:

    class UnsignedIntegerValue(_IntegerValue):
        ...
        _set_value = native_bt.value_unsigned_integer_set
        ...

UnsignedIntegerValue._set_value() is a static method. However, with
SWIG 4, the generated wrapper looks like this:

    def value_unsigned_integer_set(integer_obj, val):
        return _native_bt.value_unsigned_integer_set(integer_obj, val)

For some reason, the

    value_unsigned_integer_set = _native_bt.value_unsigned_integer_set

part is gone.

Now, UnsignedIntegerValue._set_value() is not a static method anymore,
so calling it with `self._set_value()` gives the expected:

    TypeError: value_unsigned_integer_set() takes 2 positional arguments
    but 3 were given

Instead of trying to make SWIG 4 generate this native function
assignment again, I prefer to be more explicit/clean and use
staticmethod() to force the function into a static method.

From https://docs.python.org/3/library/functions.html#staticmethod,
the documentation also suggests this:

> Like all decorators, it is also possible to call staticmethod as a
> regular function and do something with its result. This is needed in
> some cases where you need a reference to a function from a class body
> and you want to avoid the automatic transformation to instance method.
> For these cases, use this idiom:
>
>     class C:
>         builtin_open = staticmethod(open)

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ic51397e34132adc78fddc3105878080a8bf74ff9
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1357
Tested-by: jenkins
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>

bindings/python/bt2/setup.py.in: put native module in `bt2` package

In SWIG 3, the native module (`_native_bt.*.so`, which they call the
"C/C++ module") can be found in two locations by the Python module
(`native_bt.py`). From
http://www.swig.org/Doc3.0/SWIGDocumentation.html#Python_package_search:

> The pure Python module needs to load the C/C++ module in order to link
> to the wrapped C/C++ methods. To do this it must make some assumptions
> about what package the C/C++ module may be located in. The approach
> the pure Python module uses to find the C/C++ module is as follows:
>
> 1. The pure Python module, foo.py, tries to load the C/C++ module,
>    _foo, from the same package foo.py is located in. The package name
>    is determined from the __name__ attribute given to foo.py by the
>    Python loader that imported foo.py. If foo.py is not in a package
>    then _foo is loaded as a global module.
>
> 2. If the above import of _foo results in an ImportError being thrown,
>    then foo.py makes a final attempt to load _foo as a global module.

In other words, `native_bt.py` tries:

    from . import _native_bt

If this fails:

    import _native_bt

Currently, we're using the second location (global module).

In SWIG 4.0.0, released 27 april 2019, things changed. From
http://www.swig.org/Doc4.0/SWIGDocumentation.html#Python_package_search:

> The pure Python module needs to load the C/C++ module in order to call
> the wrapped C/C++ methods. To do this it must make some assumptions
> about the location of the C/C++ module. There are two configurations
> that are supported by default.
>
> 1. Both modules in the same package
>
> 2. Both modules are global

and:

> Compatibility Note: Versions of SWIG prior to SWIG-4.0.0 supported
> split modules without the above customization. However, this had to be
> removed as the default import code often led to confusion due to
> obfuscation of genuine Python ImportError problems. Using one of the
> two default configurations is the recommended approach now.

In other words, `native_bt.py` does not try

    import _native_bt

because `native_bt.py` is part of a package; it only tries:

    from . import _native_bt

The result is that, when the bindings are generated by SWIG 4,
`native_bt.py` cannot find `_native_bt`, which makes the bindings not
work at all.

The common method for SWIG 3 and 4 is importing from the same package.
For this to work, `setup.py` needs to install the built extension into
the `bt2` package. This patch changes `bindings/python/bt2/setup.py.in`
to do so.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Icf3622953280466e8d458caee79a28ed4a01fa2b
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1356
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
Tested-by: jenkins

flt.lttng-utils.debug-info: Replace hardcoded '/' in paths

Replace hardcoded '/' directory separators with glib functions that will
automatically select the proper separator per platform.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I53195657f6514930998979612c6b04de5e3c1dd8
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1336
Reviewed-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Fix: test_bitfield: extend coverage by removing off-by-one in bound check

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I4fbf927d4b2ff1c8b68b05c7843298c64b68f5a4
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1342
Tested-by: jenkins

Fix: test_bitfield: extend coverage: 0-len signed write/read

By convention, a 0-len bitfield write is a no-op, and a 0-len read
sets the value of the output to 0.

So we can "encode" the value 0 over a length of 0 bit. Cover this
in the test-cases for signed types. It is already covered for the
unsigned test-cases.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ib6038b98167ef2cfa8c504d7ac29498e71827b38
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1341
Tested-by: jenkins

Fix: flt.lttng-utils.debug-info: build id comparison

Issue
=====
Commit af54bd1801 fixed a endianness bug by coincidence.

The real bug is the following line:
  name_sz = (uint32_t) *buf;

The `buf` variable is a `uint8_t *` that points at a 32bits word storing
the name size field.

On that line we dereference `buf` and cast the result to a uint32_t.

Dereferencing `buf` gives a `uint8_t` containing the first byte of the
32bits word which is then cast to 32bits and stored in `name_sz`.
However, the intent was to read the full 32bits word in `name_sz`.

It was working properly on little-endian architectures because the first
byte of the 32bit word was probably always enough to contain the size of
the name so casting the `uint8_t` to a `uint32_t` did not lead to
truncating.

But on big-endian architectures, that same first byte of 32bits word
containing the name size is probably always zero which lead to a failure
to recognize the note as a build id note.

Wrong Solution
==============
Commit af54bd1801 assumed that the `elf_getdata()` returned the note
section in the file byte order and used `gelf_xlatetom()` to swap the by
to the machine byte order. This assumption is false, the `elf_getdata()`
man page says:
  The returned data descriptor will be setup to contain translated
  data[...]

So no need to translate from the file byte order to the machine byte
order after the extracting the data using `elf_getdata()`.

The mis-guided use of `gelf_xlatetom()` made the testcase pass on
PowerPC because it swapped the bytes of the 32bits word making the cast
described above work. But running that same test on a PowerPC on a
PowerPC binary would not swap the byte order (as machine and file byte
order are the same) and would trigger the casting bug described above.

Right Solution
==============
The right solution is to cast the `buf` pointer to a `uint32_t *` before
dereferencing it. Such as:
  name_sz = (uint32_t) *(uint32_t *)buf;

But, during this investigation, we discovered that the libelf library
offers the `gelf_getnote()` function that does all the work to extract
the note fields. Using this function makes it clearer and safer.

This commit uses this approach as well as reverting all the changes
added by commit af54bd1801.

Drawbacks
=========
None.

Reported-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I952aad69d225c202b3aa9c41724b6645550a2d68
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1340
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Fix: src.ctf.lttng-live: possible memory leak on error path

Free `struct lttng_live_component` if the graph is found to be already
canceled and return the appropriate _END status.

Reported-by: scan-build - Potential leak of memory.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I975e458ed4ce5dc3e50c9bf330a2f6953e5be535
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1308
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Fix: lib-logging: possible buffer not null terminated

Issue
=====
While fixing Coverity warnings, commit 879e14e8 introduced other
warnings. Indeed, the destination string after the `strncpy()` call
would be left not null terminated if the size of the source buffer is
equal or larger than the `TMP_PREFIX_LEN`. Also, the `strncat()` does
not consider the characters added by the previous `strncpy()` call when
supplying the size of the destination buffer. This could result in
buffer overflows.

Solution
========
Use `snprintf()` to concatenate the two strings and set the last
character to `\0`. It's worth it to make the code easier to understand
even though the `snprintf()` is probably more work because of the format
string.

Drawbacks
=========
None.

Notes
=====
List of coverity defect reports:
CID 1401510 (#5 of 5): Buffer not null terminated
(BUFFER_SIZE_WARNING) 19. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401511 (#2 of 2): Buffer not null terminated
(BUFFER_SIZE_WARNING) 4. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401513 (#1 of 1): Buffer not null terminated
(BUFFER_SIZE_WARNING) 19. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401517 (#5 of 5): Buffer not null terminated
(BUFFER_SIZE_WARNING) 9. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

ID 1401519 (#4 of 4): Buffer not null terminated
(BUFFER_SIZE_WARNING) 15. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401523 (#3 of 3): Buffer not null terminated
(BUFFER_SIZE_WARNING) 16. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401525 (#1 of 1): Buffer not null terminated
(BUFFER_SIZE_WARNING) 10. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401526 (#2 of 2): Buffer not null terminated
(BUFFER_SIZE_WARNING) 11. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401528 (#1 of 1): Buffer not null terminated
(BUFFER_SIZE_WARNING) 15. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401532 (#1 of 1): Buffer not null terminated
(BUFFER_SIZE_WARNING) 5. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401533 (#1 of 1): Buffer not null terminated
(BUFFER_SIZE_WARNING) 6. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401534 (#1 of 1): Buffer not null terminated
(BUFFER_SIZE_WARNING) 12. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401535 (#1 of 1): Buffer not null terminated
(BUFFER_SIZE_WARNING) 13. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401537 (#2 of 2): Buffer not null terminated
(BUFFER_SIZE_WARNING) 23. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401539 (#10 of 10): Buffer not null terminated
(BUFFER_SIZE_WARNING) 10. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401540 (#1 of 1): Buffer not null terminated
(BUFFER_SIZE_WARNING) 28. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

CID 1401541 (#1 of 1): Buffer not null terminated
(BUFFER_SIZE_WARNING) 9. buffer_size_warning: Calling strncpy
with a maximum size argument of 64 bytes on destination array
tmp_prefix of size 64 bytes might leave the destination string
unterminated.

Reported-by: Coverity - Buffer not null terminated
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ie10d4d6c8b1a0caff2fe70bbb1046673bbb1a999
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1333
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Update renamed file in gitignore

In f651e9843f0c6131ace24c33ef9cedc6926d7c51, test-utils-muxer-complete
was renamed to test_utils_muxer_complete without updating the gitignore.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Ieae29bf2f02172a0f99e91d3d12d01ff6cd73699
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1338
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Fix: flt.lttng-utils.debug-info: cannot find addr past the first CU

Issue
=====
The `bin_info_lookup_source_location()` function doesn't look past the
first Compile Unit(CU) in the Dwarf information to find the filename and
line number of an address. This function iterates over all CUs of the
binary (or shared object) and tries to translate the address with each
of them. It ends up calling two other functions:
`bin_info_lookup_cu_src_loc_no_inl()`and `bin_info_lookup_cu_src_loc_inl()`
in a loop to cover all the CUs.

The `bin_info_lookup_cu_src_loc_no_inl()` function returns an error
(returning -1) if it can't find the line of the give address making the
calling function exit the loop, and thus abort the resolving.

The information that the resolving was unsuccessful is transmit back to
the caller by omitting to set the `src_loc` output parameter. The return
value of this function is used to notify the caller of an error. Not
resolving the source line should not make the function return -1.

This has the effect that if an address is contained in a CU after the
first one this function will not be able to resolve it.

Solution
========
Return 0 if the line is not found so that the search continues with
other CUs.

Drawbacks
=========
None.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I4d85bc75d32a9a25a7044f3686c6d5940311fa68
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1337
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Fix: flt.lttng-utils.debug-info: note name memcmp() overflow

Issue
=====
If the note section that we are currently parsing has a name longer
than the "GNU" string, the `memcmp()` call will read garbage after the
"GNU" string.

I witnessed this when the component was parsing a note section named
"stapsdt".

Solution
========
Make the section name length comparison explicit.

Drawbacks
=========
None.

Reported-by: Address Sanitizer - Global buffer overflow
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I515f8c883ddbc1884045e86aecef700ee2111959
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1322
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Cleanup: src.ctf.lttng-live: rename _is_canceled function

Rename the `lttng_live_is_canceled()` function to
`lttng_live_graph_is_canceled()` to make it clear it's the graph that
may be canceled and not the lttng_live component.

No behaviour change intended.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I918c5262e4b833eb8ed302eab42df19ceac0cbbc
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1309
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Fix: flt.utils.trimmer: no error when upstream can not seek

Issue
=====
According to commit 7de0e49, every upstream message iterators of a
`flt.utils.trimmer` must support seeking. If it's not the case, the
graph should fail gracefully.

I encountered a situation where this invariant is not held. I can
reproduce this issue by commenting out the `_SEEK_BEGINNING_METHOD()`
and `_CAN_SEEK_BEGINNING_METHOD()` of the `flt.utils.muxer` and ran the
following graph:
  babeltrace run \
  --component=src:src.ctf.fs \
  --params='paths=["/home/frdeso/lttng-traces/auto-20181210-133612"]' \
  --component=mux:filter.utils.muxer  \
  --component=trim:filter.utils.trimmer \
  --params='begin="13:36:12.871289834",end="13:36:12.871303475"' \
  --component=sink:sink.text.pretty \
  \
  --connect=src.*:mux.* \
  --connect=mux.*:trim.* \
  --connect=trim.*:sink.*
Note: The `begin` and `end` parameters are event timestamps from the trace.

When the trimmer component sees that seeking is not supported, it simply
skip the seeking and continue. This triggers a `BT_ASSERT()` later on.

See commit message 7474e7d3f0d005280c6614be5c818735e65d8b3b for more details on
seeking.
See commit message 7de0e49a7c4bb3c04075703e6767d15b7beb277f for more details on
trimmer.

Solution
========
Error out if seeking is disabled on upstream message iterators.

Drawbacks
=========
None.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I35cdec3b8eac54f2f596d0f50661608b624d3e2e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1325
Tested-by: jenkins
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>

Add .gitignore entry for .theia

This directory is used for project-specific settings for the Theia IDE.
Since I use that, I would like if the directory was ignored. Even
though it doesn't apply to everybody, it shouldn't bother anybody to
have this entry. Also, I wouldn't mind if we added similar entries for
other IDEs, if it helped other people.

Change-Id: Ic6f42b9d3f1d855e3d6e9e0dee1492aa484010ac
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1327
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: logging: possible buffer overflows

Issue
=====
Multiple possible buffer overflows on string operations using the
`SET_TMP_PREFIX()` macro that uses `strcpy()` on parameter char array.

Solution
========
Use a #define to set the length of the destination array and use it as
the size parameter of the `strncpy()` and `strncat()` calls.

Drawbacks
=========
None.

Notes
=====

Coverity reported defects:
  CID 1401179 (#5 of 5): Copy into fixed size buffer (STRING_OVERFLOW) 9.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401181 (#1 of 1): Copy into fixed size buffer (STRING_OVERFLOW) 5.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length

  CID 1401186 (#1 of 1): Copy into fixed size buffer (STRING_OVERFLOW) 15.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401192 (#2 of 2): Copy into fixed size buffer (STRING_OVERFLOW) 4.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401197 (#1 of 1): Copy into fixed size buffer (STRING_OVERFLOW) 6.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401198 (#4 of 4): Copy into fixed size buffer (STRING_OVERFLOW) 15.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401203 (#1 of 1): Copy into fixed size buffer (STRING_OVERFLOW) 28.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401212 (#1 of 1): Copy into fixed size buffer (STRING_OVERFLOW) 12.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401215 (#4 of 4): Copy into fixed size buffer (STRING_OVERFLOW) 16.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401221 (#2 of 2): Copy into fixed size buffer (STRING_OVERFLOW) 19.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401227 (#1 of 1): Copy into fixed size buffer (STRING_OVERFLOW) 13.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401230 (#3 of 3): Copy into fixed size buffer (STRING_OVERFLOW) 10.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401232 (#3 of 3): Copy into fixed size buffer (STRING_OVERFLOW) 23.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401234 (#5 of 5): Copy into fixed size buffer (STRING_OVERFLOW) 19.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401254 (#10 of 10): Copy into fixed size buffer (STRING_OVERFLOW) 10.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401257 (#1 of 1): Copy into fixed size buffer (STRING_OVERFLOW) 9.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

  CID 1401261 (#2 of 2): Copy into fixed size buffer (STRING_OVERFLOW) 11.
  fixed_size_dest: You might overrun the 64-character fixed-size string
  tmp_prefix by copying prefix without checking the length.

Reported-by: Coverity - Copy into fixed size buffer (STRING_OVERFLOW)
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I922f1fb82a95e06b0c42627a2e57ba94debe1c5a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1329
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Tested-by: jenkins

Add vim .swp files to gitignore

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Ibfe600455d80a8542a8f7ebcda44f3682b4b0534
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1330
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Replace g_build_path with g_build_filename for portability

g_build_filename will use the correct separator depending on the
platform.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: Iaeeb1a4c30aaea7690ba479ba05108c6fc57dd4e
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1331
Tested-by: jenkins
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: test_bitfield: buffer overrun accesses

This issue is due to a type mismatch between union member vs type passed
as parameter to the macros as found by Coverity:

** CID 1401409: Memory - corruptions (OVERRUN)
/tests/lib/test_bitfield.c: 450 in run_test_signed_write()

Issue introduced by commit 0675eb8c07 "Extend test_bitfield coverage"

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I14c9d69011b369e5dc8be4305ac8bf32cc86d3dd
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1321
CI-Build: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Tested-by: jenkins
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Cleanup: tests: use underscores in filename

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I02a8b61e7c4a234aa017a941f4609f4276f387ee
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1320
Tested-by: jenkins
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Cleanup: flt.utils.muxer: avoid returning a pointer needlessly

Reshuffle `muxer_msg_iter_add_upstream_msg_iter()` function to return an
int rather than a pointer.

Returning a pointer in this case has no value since the newly allocated
structure is stored in a GQueue directly. Returning an int to express
success or error seems more appropriate and less error-prone.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I111cd06e1f667c98aa3bf650375c6e49fcd4c5dd
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1313
Tested-by: jenkins
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: flt.utils.muxer: Potential memory leak

scan-build reports a potential memory leak of the
muxer_upstream_msg_iter pointer allocated in the
`muxer_msg_iter_add_upstream_msg_iter()` function if the `g_queue_new()`
fails.

So, free the muxer_upstream_msg_iter on the error path.

Reported-by: scan-build - Potential leak of memory
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ia37a1be9525b357d063d5c5bd5623ddccbd4e29d
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1312
Tested-by: jenkins
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Cleanup: bitfields: streamline use of underscores

Do not prefix macro arguments with underscores. Use one leading
underscore as prefix for local variables defined within macros.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ie1c2f7f59f605ac62d483aba67b3f70cef27bf21
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1318
Tested-by: jenkins

Silence compiler "always false comparison" warning

Compiling the bitfield test with gcc -Wextra generates those warnings:

../../include/babeltrace/bitfield-internal.h:38:45: warning: comparison of unsigned expression < 0 is always false [-Wtype-limits]
#define _bt_is_signed_type(type) ((type) -1 < (type) 0)

This is the intent of the macro. Disable compiler warnings around use of
that macro.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I2dd980e11ebd6cd37ae71b013925a6fca5d7b483
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1307
Tested-by: jenkins

Extend test_bitfield coverage

test_bitfield was mainly testing various write unit size. Add
variations of read unit size as well.

Previously, the test was only covering input from a 32-bit integer.
Additionally test source and destination of 64-bit.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ic2b7849140e04fe7cca3de576e31bfef8f0a03ae
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1305
Tested-by: jenkins

Fix: bitfield: shift undefined/implementation defined behaviors

bitfield.h uses the left shift operator with a left operand which
may be negative. The C99 standard states that shifting a negative
value is undefined.

When building with -Wshift-negative-value, we get this gcc warning:

In file included from /home/smarchi/src/babeltrace/include/babeltrace/ctfser-internal.h:44:0,
                 from /home/smarchi/src/babeltrace/ctfser/ctfser.c:42:
/home/smarchi/src/babeltrace/include/babeltrace/ctfser-internal.h: In function ‘bt_ctfser_write_unsigned_int’:
/home/smarchi/src/babeltrace/include/babeltrace/bitfield-internal.h:116:24: error: left shift of negative value [-Werror=shift-negative-value]
   mask = ~((~(type) 0) << (__start % ts));  \
                        ^
/home/smarchi/src/babeltrace/include/babeltrace/bitfield-internal.h:222:2: note: in expansion of macro ‘_bt_bitfield_write_le’
  _bt_bitfield_write_le(ptr, type, _start, _length, _v)
  ^~~~~~~~~~~~~~~~~~~~~
/home/smarchi/src/babeltrace/include/babeltrace/ctfser-internal.h:418:3: note: in expansion of macro ‘bt_bitfield_write_le’
   bt_bitfield_write_le(mmap_align_addr(ctfser->base_mma) +
   ^~~~~~~~~~~~~~~~~~~~

This boils down to the fact that the expression ~((uint8_t)0) has type
"signed int", which is used as an operand of the left shift.  This is due
to the integer promotion rules of C99 (6.3.3.1):

    If an int can represent all values of the original type, the value is
    converted to an int; otherwise, it is converted to an unsigned int.
    These are called the integer promotions. All other types are unchanged
    by the integer promotions.

We also need to cast the result explicitly into the left hand
side type to deal with:

warning: large integer implicitly truncated to unsigned type [-Woverflow]

The C99 standard states that a right shift has implementation-defined
behavior when shifting a signed negative value. Add a preprocessor check
that the compiler provides the expected behavior, else provide an
alternative implementation which guarantees the intended behavior.

A preprocessor check is also added to ensure that the compiler
representation for signed values is two's complement, which is expected
by this header.

Document that this header strictly respects the C99 standard, with
the exception of its use of __typeof__.

Adapt use of _bt_piecewise_lshift in plugins/text/pretty/print.c
to the new API.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Acked-by: Simon Marchi <simon.marchi@efficios.com>
Change-Id: I47d2655cfe671baf0fc18813883adf7ce11dfd6a
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1311
Tested-by: jenkins

Fix: flt.utils.trimmer: bt_message_put_ref() on freed message

Issue
=====
In this function, we convert a timestamp in second to a clock value in
order to create a stream activity beginning message. We create a new
message and proceed to call `clock_raw_value_from_ns_from_origin()` to
convert the timestamp to a clock value.

If this call fails, we put the reference on this message using
`bt_message_put_ref()`. This frees the message as the refcount was at 1
because it was just created.

The problem here is that we don't reset the `msg` pointer and return it
to the call via the output parameter. This pointer points to freed data.
The caller then figures that the function failed by looking at the
return value and proceed to call `bt_message_put_ref()` again resulting
and assertion failure.

This can be easily triggered by passing a `begin` parameter of value 0
to a trimmer component.

Solution
========
Use a local variable to keep the pointer to the newly created message
and move that reference to the output parameter only on success.

Drawbacks
=========
None.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Id589f25d9ec6f555abd44ccb71e72af8e1684272
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1314
Tested-by: jenkins
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: flt.utils.muxer: Undefined or garbage value returned

Reported-by: scan-build - Undefined or garbage value returned to caller
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I46c6e431ae4dcb68367dea84971779af017ecded
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1310
Tested-by: jenkins
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

fix: Add PROT_READ to ctfser mmap flags

On Linux PROT_WRITE implies PROT_READ but on other Unices like Solaris,
it's not the case. This resulted in a segfault when reading from the
mapping.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: If11bbcf32e33bf5e682afaade2d3769a141c6583
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1302
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Tested-by: jenkins

Fix: src.ctf.fs: possible use-after-free in the error path

Reported-by: scan-build - Use of memory after it is freed
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I5c19a45b0ddd11eed3872295efeb7b5201967419
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1303
Tested-by: jenkins
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix type-limits warnings, uint can't be < 0

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I14ba4492bd2ea9803daf8985dd8fe21107942df6
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1295
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix old-style-declaration warnings

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I997662eaaabc217b22a0010dccd292e4e946933f
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1294
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Annotate implicit fall-through

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I21c5641869afca1fcac301f612d02d2cb10583db
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1293
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Cleanup: src.ctf.fs: index no longer used in loop

This removal should have been done in an earlier commit modifying this
loop:
  Author: Francis Deslauriers <francis.deslauriers@efficios.com>
  Date:   Tue Apr 30 19:01:17 2019 -0400

      src.ctf.fs: compute stream range using entire file info group

      Given that index entries are sorted within their file info structure and
      that file info structures are sorted by time within the file info group,
      we can simply compute the range of the stream by using the absolute
      first index entry and the absolute last index entry of the entire file
      info group.

Add `BT_ASSERT()` to enforce invariants in developer mode.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I4e0b53cbb808835886b97b6d6001ecd7bcc2abb4
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1288
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Fix: sink.ctf.fs: Dereference before null check

No need to check if `trace->metadata_path` is null at the point because
it has been used by `fopen()` already and `abort()` is called if
`fopen()` fails.

  CID 1401238 (#1 of 1): Dereference before null check (REVERSE_INULL)
  check_after_deref: Null-checking trace->metadata_path suggests that it
  may be null, but it has already been dereferenced on all paths leading
  to the check.

Reported-by: Coverity (1401238) Dereference before null check
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I90590323764810da8a6b4cfb37445a588befd242
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1284
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Tested-by: jenkins

Fix: sink.ctf.fs: possible uses after free because of unchecked return values

Uses after free are possible because we ignore the return value of the
`bt_ctfser_write_*()` function calls. To avoid this, check the return
values and log an error message.

  CID 1401217 (#4 of 4): Use after free (USE_AFTER_FREE)
  11. deref_arg: Calling bt_ctfser_write_byte_aligned_unsigned_int
  dereferences freed pointer stream->ctfser.base_mma

  CID 1401193 (#2 of 2): Unchecked return value (CHECKED_RETURN)
  8. check_return: Calling bt_ctfser_write_byte_aligned_unsigned_int
  without checking return value (as is done elsewhere 8 out of 10
  times).

Reported-by: Coverity (1401217) Use after free
Reported-by: Coverity (1401193) Unchecked return value
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: I7581c0e549d7bf916f42afe306976baa9822897a
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1286
Tested-by: jenkins
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>

Cleanup: common/assert.c: remove useless calls

  CID 1401190 (#1 of 1): Useless call (USELESS_CALL)
  side_effect_free: Calling bt_common_color_bold() is only useful for
  its return value, which is ignored.

  side_effect_free: Calling bt_common_color_fg_red() is only useful for
  its return value, which is ignored.

Reported-by: Coverity (1401190) Useless call
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Change-Id: Ia25823f44c24997782ccc209c48d5b28a45a98f4
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1287
Reviewed-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>