2 years agoFix: sessiond cmd.h: include missing lttng-sessiond.h
Mathieu Desnoyers [Thu, 12 Dec 2019 16:51:40 +0000 (11:51 -0500)] 
Fix: sessiond cmd.h: include missing lttng-sessiond.h

lttng-sessiond.h declares struct command_ctx which is used in cmd.h

It typically does not generate a compilation error because all
compile units including cmd.h also include lttng-sessiond.h, but
it is cleaner to include dependencies directly in cmd.h.

Signed-off-by: Mathieu Desnoyers <>
Change-Id: If3106291309cd543a886b2aecda7d4f9d4b290b3
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: sessiond: ust-app: per-pid buffers: Create empty ust/ directory
Mathieu Desnoyers [Mon, 9 Dec 2019 15:41:43 +0000 (10:41 -0500)] 
Fix: sessiond: ust-app: per-pid buffers: Create empty ust/ directory

When no applications are running, the toplevel ust/ directory is not
created in a chunk on rotation. Make sure this top level directory
is created.

Signed-off-by: Mathieu Desnoyers <>
Change-Id: Ia272f3182df298302e02b3e096785cb51687b8c9
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: utils: utils_stream_file_path separator
Mathieu Desnoyers [Mon, 9 Dec 2019 14:28:13 +0000 (09:28 -0500)] 
Fix: utils: utils_stream_file_path separator

Do not add a separator for empty paths.

Signed-off-by: Mathieu Desnoyers <>
Change-Id: Icec7b07d7abab4c010706c8cb4f199229fa04c6f
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: consumerd: use packet sequence number for rotation position
Mathieu Desnoyers [Tue, 5 Nov 2019 18:07:44 +0000 (13:07 -0500)] 
Fix: consumerd: use packet sequence number for rotation position

Refer to "Fix: relayd: use packet sequence number for rotation position"
for context of this change.

This commit introduces the changes required in the consumerd.

Some notable points related to this commit:

- Internally, the rotate_position (per-stream) is now a 64-bit
  value rather than an unsigned long.
- The scheme to rotate a stream is changed to allow using the
  backward-compatible lttng_consumer_take_snapshot() rather than
  the newer lttng_consumer_get_produced_snapshot(), thus allowing
  backward compatibility of the implicit rotation on destroy with
  pre-2.10 lttng-modules.
- The rotate position used as pivot point for the rotation is
  based on the packet_seq_num of the last packet that has been
  send over the network by consumerd, incremented by the number of
  packets between the sampled produced_pos and the consumed_pos.
  In the worse case scenario where an overwrite mode ring buffer
  overwrites its contents enough to trigger a 4GB overflow on a
  32-bit producer since the last packet was sent (e.g. due to a
  slow network), the difference between produced_pos and
  consumed_pos will be lower that what would have been expected.
  However, because this pivot position is used as a lower bound,
  being smaller than the real value is fine: the data that would
  have been misplaced in the wrong trace chunk were actually
  overwritten, and will therefore never be consumed.
- When interacting with pre-2.8 lttng-modules, the packet sequence
  number is not available. The current approach is to disallow
  rotations performed on sessions which have kernel tracing active
  with a pre-2.8 lttng-modules.

Signed-off-by: Mathieu Desnoyers <>
Change-Id: I8600cb5e2e9c05f3dfba0499a5fc4a3bb85dec24
Signed-off-by: Jérémie Galarneau <>
2 years agoalign.h: Implement ALIGN_FLOOR macro
Mathieu Desnoyers [Tue, 5 Nov 2019 23:56:34 +0000 (18:56 -0500)] 
align.h: Implement ALIGN_FLOOR macro

Implement the ALIGN_FLOOR macro which aligns the given value to the
previous alignment boundary, or keeps the value as-is if it is already

Signed-off-by: Mathieu Desnoyers <>
Change-Id: I36d981e8fb705fafa3ff23ba2d82ec1babe73e45
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: per-pid live: no new metadata vs close
Mathieu Desnoyers [Tue, 3 Dec 2019 09:08:36 +0000 (04:08 -0500)] 
Fix: relayd: per-pid live: no new metadata vs close

When using lttng-live on a per-pid UST trace, the metadata stream is
closed after an application exits. The live viewer observes that the
stream has no more data when getting the metadata returns 0 bytes.

Internally in the relay daemon, it is assumed that a metadata stream
can be put (and thus removed) as soon as all the metadata has been sent
to the viewer, but this is not quite right. The viewer actually needs to
observe a 0-byte reply after receiving all the metadata in order to
gracefully perceive the metadata stream hang up.

Therefore, add a no_new_metadata_notified flag to the metadata stream
to track whether that 0-byte message has been sent to the viewer since
the last metadata content was sent, and postpone the reclamation of the
metadata stream until it is closed _and_ that 0-byte reply was sent to
the live viewer.

Signed-off-by: Mathieu Desnoyers <>
Change-Id: I0a05332284299d62b832046e4f9d22b6029c3a3e
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: use packet sequence number for rotation position
Mathieu Desnoyers [Tue, 5 Nov 2019 15:40:58 +0000 (10:40 -0500)] 
Fix: relayd: use packet sequence number for rotation position

The "network" sequence number (net_seq_num) is a 64-bit sequence number
tagging each packet sent over the network. The net_seq_num increments
monotonically (+1) for each packet sent from consumer daemon to relay
daemon, on a per-stream basis. It is tagged by the consumer daemon when
sending a trace packet to the relay daemon.

The LTTng kernel and user-space ring buffer "consumed position"
(consumed_pos) and "produced position" (produced_pos) are free-running
counters counting the number of bytes consumed and produced so far by
each stream. Because those counters are updated atomically, they are
limited to a size of 32-bit on 32-bit architectures.

The "packet" sequence number (packet_seq_num) is a sequence number
found in the packet header starting from LTTng 2.8. It is a 64-bit
sequence number assigned by the lttng-modules and lttng-ust ring
buffers. It increments monotonically (+1) for each packet produced
within a given ring buffer (stream).

Using produced_pos as rotation position and comparing it to the
net_seq_num has a few issues:

1) It breaks on 32-bit producers after generating more than 4GB of
   data per stream, due to overflow. The net_seq_num is a 64-bit
   counter, which does not overflow, but the produced_pos overflows
   after 4GB on 32-bit architectures. This can lead to never-completing

2) It breaks scenarios where ring buffers are configured in
   overwrite mode, and streaming to a relay daemon. Indeed, when
   the ring buffer moves the consumed_pos ahead, actually overwriting
   data within the ring buffer, it introduces an offset between the
   produced_pos and the net_seq_num. Therefore, if producers are
   generating a low- (or no-) throughput in some streams, the
   rotation may never complete, even on 64-bit architectures.

The solution proposed for this issue is to use the packet_seq_num as
rotation position rather than the net_seq_num. It takes care of
the two problematic scenarios, since the counter is always 64-bit
(even on 32-bit architectures), and because the counter is managed
by the producer, which therefore tracks progress of the ring buffer

This commit introduces changes required at the relayd side. A
separate commit introduces the changes required in the consumerd.

In relayd, one major restriction is the fact that the packet_seq_num
is not sent over the data socket, only through the control socket
receiving the indexes.

Therefore, in order to figure out the pivot position for the data
socket for a given stream, the associated index first needs to be
received. At that point, the corresponding net_seq_num is known,
which provides the pivot position for the data stream. Given that
the data and index sockets provide no ordering guarantees with
respect to their arrival, we handle the fact that data might have
been saved to disk in the wrong (previous) trace chunk by moving
it to the next trace chunk when the pivot position is known.

In order to allow "jumps" in the sequence numbers produced by
overwrite mode buffers, try_rotate_stream_index(), which previously
asserted that each sequence number was received in sequence, now
uses the packet_seq_num pivot position as a lower (inclusive) bound.

Signed-off-by: Mathieu Desnoyers <>
Change-Id: I755329e313f0980655a164b7bdb57e4f3d8e944a
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd stream.c: LTTNG_OPTIONAL_GET address confusion
Mathieu Desnoyers [Wed, 18 Dec 2019 15:56:29 +0000 (10:56 -0500)] 
Fix: relayd stream.c: LTTNG_OPTIONAL_GET address confusion

The lack of proper care around use of macro arguments in LTTNG_OPTIONAL_GET
allowed this code to compile, but caused the following issue (manually
replacing "optional" argument with the a

Using LTTNG_OPTIONAL_GET(&stream->ongoing_rotation)->next_trace_chunk


 #define LTTNG_OPTIONAL_GET(optional)                   \
         ({                                             \
               assert(optional.is_set);                 \
               optional.value;                          \

translates to:


The issue here is the assert(), which just checks that the address
is not NULL, when it should actually check that the value is set.

The prior commit fixing optional.h to add proper parentheses to the
macro rightfully fails to compile this code. Fix the erroneous user.

Signed-off-by: Mathieu Desnoyers <>
Change-Id: I0dd45fb60573cae6ae3e831e24266aff4406f57f
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: optional.h macro missing parentheses and guards
Mathieu Desnoyers [Wed, 18 Dec 2019 15:50:00 +0000 (10:50 -0500)] 
Fix: optional.h macro missing parentheses and guards

The are a few coding style issues with optional.h which leads to
unexpected effects when the macros are used in the caller code.

All macro parameters need to be surrounded by () (except when used near
commas, which is the C operator with least precedence).

All macros that emit code need to be surrounded by do { } while (0) so
not to emit extra ; or omit ;, which can cause subtle issues when used
with if/else statements.

Signed-off-by: Mathieu Desnoyers <>
Change-Id: Iba6fde7c267f4d8c9ec1a89147045f0bcda3a67a
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: use of strnlen breaks build on Solaris 10
Jérémie Galarneau [Thu, 19 Dec 2019 22:00:49 +0000 (17:00 -0500)] 
Fix: use of strnlen breaks build on Solaris 10

The new uuid helper library makes use of strnlen directly, which
doesn't exist on Solaris 10. Using the compat version solves this.

Signed-off-by: Jérémie Galarneau <>
Change-Id: Ie491da37f968fac417e265b5641fdf0dd3298813

2 years agodynamic-array: fix documentation of lttng_dynamic_pointer_array_get_pointer
Simon Marchi [Fri, 13 Dec 2019 21:47:57 +0000 (16:47 -0500)] 
dynamic-array: fix documentation of lttng_dynamic_pointer_array_get_pointer

The documentation of this function says that mutating the array
invalidates the return pointer.  This is not true, since the returned
pointer is the element itself, not a pointer to the array.

Change-Id: I8b8978cddb2d1d6ce0b42ed313e9843ca418c96c
Signed-off-by: Simon Marchi <>
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: make dist fails on sdt-probe test with autotools v1.16+
Jérémie Galarneau [Fri, 13 Dec 2019 20:25:28 +0000 (15:25 -0500)] 
Fix: make dist fails on sdt-probe test with autotools v1.16+

The 'dist' target fails when building with autotools 1.16.1 as the
behaviour of BUILT_SOURCES has changed in 1.16 [1].

BUILT_SOURCES are now built as part of the 'dist' target which causes
it to fail on machines that don't have dtrace installed.

The following error is produced: DTRACE foobar_provider.h /bin/sh: -s:
  command not found

The BUILT_SOURCES is skipped for build configurations where the build
of the test helper 'userspace-probe-sdt-binary' is disabled.

The resulting artifact, foobar_provider.h, is not part of the
distributed tarball (marked as nodist), so this doesn't have any
adverse effect.


Signed-off-by: Michael Jeanson <>
Signed-off-by: Jérémie Galarneau <>
Change-Id: Ibba4689775a8b002e8a959079b1f3c8db9917c5f

2 years agoFix: tests: metadata presence on relayd is not deterministic
Jonathan Rajotte [Wed, 11 Dec 2019 20:58:58 +0000 (15:58 -0500)] 
Fix: tests: metadata presence on relayd is not deterministic

There is no synchronization point guaranteeing the presence of metadata on
lttng-relayd side when the live metadata request is done. It must be
present at some point in the future but might not be at the moment we
perform the request.

Add a retry phase to this step. It must succeed in the future.

Not sure how this test was never marked as flaky.

Signed-off-by: Jonathan Rajotte <>
Change-Id: I033005c6a228013e699c74d8e8faafcb3272dd98
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: move testpoint after state update
Jonathan Rajotte [Tue, 10 Dec 2019 16:15:39 +0000 (11:15 -0500)] 
Fix: move testpoint after state update

This prevent failure and hang for the long_regression test suite.

Otherwise the sessiond error out before the test completion due to an
invalid thread state.

Signed-off-by: Jonathan Rajotte <>
Change-Id: Ie4bf3bea6f84f49f5eceb413ae75a7e6fff08a8f
Signed-off-by: Jérémie Galarneau <>
2 years agoMake lttng_directory_handle reference countable
Jérémie Galarneau [Fri, 29 Nov 2019 20:42:24 +0000 (15:42 -0500)] 
Make lttng_directory_handle reference countable

Currently, the lttng_directory_handle API assumes that the caller has
exclusive ownership of the lttng_directory_handle. Directory handles
are passed by reference and populated by an "init" method and
finalized using the "fini" method.

In order to allow multiple references to a path or directory file
descriptor in follow-up patches, the API is moved to a model where
directory handles are dynamically allocated and released using the
"put" method.

No change in behaviour is intended by this change beyond adapting the

The objective of the change is to make it easier to implement copy
operations of trace chunks that are configured to use an fd-tracker
and reduce the number of open file descriptors when
lttng_directory_handles are copied.

Signed-off-by: Jérémie Galarneau <>
Change-Id: Ibc8e97ea9e949adafef44533c30b61e3a9fa1f7d

2 years agoClean-up: apply clang-format to the newly added fd-tracker
Jérémie Galarneau [Tue, 26 Nov 2019 21:33:22 +0000 (16:33 -0500)] 
Clean-up: apply clang-format to the newly added fd-tracker

The fd-tracker code was just merged but clang-format found some
changes to apply. Since there is no need to backport to this code
yet, clean it up as much as possible now.

Signed-off-by: Jérémie Galarneau <>
Change-Id: Ia0bec41e0531c9f5d611d742e96a984de3f06784

2 years agoFix build: fd-tracker is not part of librelayd
Jérémie Galarneau [Wed, 27 Nov 2019 19:23:56 +0000 (14:23 -0500)] 
Fix build: fd-tracker is not part of librelayd

The fd-tracker library is not part of the internal librelayd; it is
its own stand-alone library.

Add an autoconf conditional to build libfd-tracker by default
and require it when building the relay daemon.

Signed-off-by: Jérémie Galarneau <>
Change-Id: I40934eeaa62e61f5da4508c1f2c91daaa9e57829

2 years agoAdd git-review config
Jérémie Galarneau [Tue, 26 Nov 2019 20:27:59 +0000 (15:27 -0500)] 
Add git-review config

Add .gitreview for contributors wishing to use gerrit for patch

Signed-off-by: Jérémie Galarneau <>
Change-Id: I74c017eeac05c9092be145265e6ce060dddcd791

2 years agodoc: fix typo in lttng-enable-event man page
Simon Marchi [Fri, 6 Dec 2019 21:07:03 +0000 (16:07 -0500)] 
doc: fix typo in lttng-enable-event man page

Change-Id: I81ce8f5d4eb3857ace5b1bc3a3ed3c62751dc503
Signed-off-by: Simon Marchi <>
Signed-off-by: Jérémie Galarneau <>
2 years agotests: make functions static in test_notification.c
Simon Marchi [Mon, 25 Nov 2019 21:15:35 +0000 (16:15 -0500)] 
tests: make functions static in test_notification.c


      CC       test_notification.o
    /home/smarchi/src/lttng-tools/tests/unit/test_notification.c:50:6: error: no previous declaration for ‘test_condition_buffer_usage’ [-Werror=missing-declarations]
     void test_condition_buffer_usage(struct lttng_condition *buffer_usage_condition)
    /home/smarchi/src/lttng-tools/tests/unit/test_notification.c:265:6: error: no previous declaration for ‘test_condition_buffer_usage_low’ [-Werror=missing-declarations]
     void test_condition_buffer_usage_low(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_notification.c:280:6: error: no previous declaration for ‘test_condition_buffer_usage_high’ [-Werror=missing-declarations]
     void test_condition_buffer_usage_high(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_notification.c:295:6: error: no previous declaration for ‘test_action’ [-Werror=missing-declarations]
     void test_action(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_notification.c:305:6: error: no previous declaration for ‘test_trigger’ [-Werror=missing-declarations]
     void test_trigger(void)

Signed-off-by: Simon Marchi <>
Change-Id: I4df29ec42436798005e740d5b6fb34973ea22350
Signed-off-by: Jérémie Galarneau <>
2 years agotests: make functions static in test_utils_compat_poll.c
Simon Marchi [Mon, 25 Nov 2019 21:14:34 +0000 (16:14 -0500)] 
tests: make functions static in test_utils_compat_poll.c


      CC       test_utils_compat_poll.o
    /home/smarchi/src/lttng-tools/tests/unit/test_utils_compat_poll.c:74:6: error: no previous declaration for ‘test_alloc’ [-Werror=missing-declarations]
     void test_alloc(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_utils_compat_poll.c:95:6: error: no previous declaration for ‘test_add_del’ [-Werror=missing-declarations]
     void test_add_del(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_utils_compat_poll.c:133:6: error: no previous declaration for ‘test_mod_wait’ [-Werror=missing-declarations]
     void test_mod_wait(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_utils_compat_poll.c:189:6: error: no previous declaration for ‘test_func_def’ [-Werror=missing-declarations]
     void test_func_def(void)

Signed-off-by: Simon Marchi <>
Change-Id: I1feeed8ee3f76783e22ffedd5292a06deee3e233
Signed-off-by: Jérémie Galarneau <>
2 years agotests: make functions static in test_utils_expand_path.c
Simon Marchi [Mon, 25 Nov 2019 21:13:30 +0000 (16:13 -0500)] 
tests: make functions static in test_utils_expand_path.c


      CC       test_utils_expand_path.o
    /home/smarchi/src/lttng-tools/tests/unit/test_utils_expand_path.c:126:5: error: no previous declaration for ‘prepare_valid_results’ [-Werror=missing-declarations]
     int prepare_valid_results(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_utils_expand_path.c:182:5: error: no previous declaration for ‘free_valid_results’ [-Werror=missing-declarations]
     int free_valid_results(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_utils_expand_path.c:195:5: error: no previous declaration for ‘prepare_symlink_tree’ [-Werror=missing-declarations]
     int prepare_symlink_tree(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_utils_expand_path.c:235:5: error: no previous declaration for ‘free_symlink_tree’ [-Werror=missing-declarations]
     int free_symlink_tree(void)

Signed-off-by: Simon Marchi <>
Change-Id: I220a2f8dadb355adb4fb974a2f3a0e161b0618e6
Signed-off-by: Jérémie Galarneau <>
2 years agotests: make functions static in test_session.c
Simon Marchi [Mon, 25 Nov 2019 21:12:21 +0000 (16:12 -0500)] 
tests: make functions static in test_session.c


      CC       test_session.o
    /home/smarchi/src/lttng-tools/tests/unit/test_session.c:219:6: error: no previous declaration for ‘test_session_list’ [-Werror=missing-declarations]
     void test_session_list(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_session.c:225:6: error: no previous declaration for ‘test_create_one_session’ [-Werror=missing-declarations]
     void test_create_one_session(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_session.c:232:6: error: no previous declaration for ‘test_validate_session’ [-Werror=missing-declarations]
     void test_validate_session(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_session.c:258:6: error: no previous declaration for ‘test_destroy_session’ [-Werror=missing-declarations]
     void test_destroy_session(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_session.c:278:6: error: no previous declaration for ‘test_duplicate_session’ [-Werror=missing-declarations]
     void test_duplicate_session(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_session.c:284:6: error: no previous declaration for ‘test_session_name_generation’ [-Werror=missing-declarations]
     void test_session_name_generation(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_session.c:309:6: error: no previous declaration for ‘test_large_session_number’ [-Werror=missing-declarations]
     void test_large_session_number(void)

Signed-off-by: Simon Marchi <>
Change-Id: Ib1f6a004e6feb4693c4331a8b0ce67ee1697703c
Signed-off-by: Jérémie Galarneau <>
2 years agotests: make functions static in test_uri.c
Simon Marchi [Mon, 25 Nov 2019 21:11:26 +0000 (16:11 -0500)] 
tests: make functions static in test_uri.c


      CC       test_uri.o
    /home/smarchi/src/lttng-tools/tests/unit/test_uri.c:33:6: error: no previous declaration for ‘test_uri_parsing’ [-Werror=missing-declarations]
     void test_uri_parsing(void)
    /home/smarchi/src/lttng-tools/tests/unit/test_uri.c:201:6: error: no previous declaration for ‘test_uri_cmp’ [-Werror=missing-declarations]
     void test_uri_cmp()

Signed-off-by: Simon Marchi <>
Change-Id: Ia0d7b08a10ec9f8d3f6f2d5b4fa9c4f417827024
Signed-off-by: Jérémie Galarneau <>
2 years agotests: include foo.h in foo.c
Simon Marchi [Mon, 25 Nov 2019 21:02:06 +0000 (16:02 -0500)] 
tests: include foo.h in foo.c


      CC       foo.lo
    /home/smarchi/src/lttng-tools/tests/utils/testapp/userspace-probe-elf-binary/foo.c:19:5: error: no previous declaration for ‘dynamic_symbol’ [-Werror=missing-declarations]
     int dynamic_symbol(int a)

Change-Id: I58888da2908bcf4c38d59ba9bacf3e96adc575b6
Signed-off-by: Simon Marchi <>
Signed-off-by: Jérémie Galarneau <>
2 years agocompat: remove always true assertion in compat-poll.c
Simon Marchi [Mon, 25 Nov 2019 20:14:47 +0000 (15:14 -0500)] 
compat: remove always true assertion in compat-poll.c


      CC       compat-poll.lo
    In file included from /home/smarchi/src/lttng-tools/src/common/compat/poll.h:21:0,
                     from /home/smarchi/src/lttng-tools/src/common/compat/compat-poll.c:31:
    /home/smarchi/src/lttng-tools/src/common/compat/compat-poll.c: In function ‘compat_poll_wait’:
    /home/smarchi/src/lttng-tools/src/common/compat/compat-poll.c:317:31: error: comparison of unsigned expression >= 0 is always true [-Werror=type-limits]
      assert(events->current.nb_fd >= 0);

Change-Id: Ie2b43e64d293d4eadb82db9755dfb64751a1d935
Signed-off-by: Simon Marchi <>
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: sessiond: RCU read lock imbalance on get trace chunk id error
Jérémie Galarneau [Mon, 9 Dec 2019 19:31:33 +0000 (14:31 -0500)] 
Fix: sessiond: RCU read lock imbalance on get trace chunk id error

The error path when cmd_setup_relayd() fails to get the ID of a trace
chunk causes the RCU read lock to be released even though it was never

Signed-off-by: Jérémie Galarneau <>
Change-Id: I86aeb99e6cd6ba11634283ab7456a3760a0a6b0d

2 years agotests: add declaration for test_function in userspace-probe-elf-binary.c
Simon Marchi [Mon, 25 Nov 2019 21:04:50 +0000 (16:04 -0500)] 
tests: add declaration for test_function in userspace-probe-elf-binary.c


      CC       userspace-probe-elf-binary.o
    /home/smarchi/src/lttng-tools/tests/utils/testapp/userspace-probe-elf-binary/userspace-probe-elf-binary.c:21:34: error: no previous declaration for ‘test_function’ [-Werror=missing-declarations]
     void __attribute__ ((noinline))  test_function()

I did not make test_function static, because from what I understand, it
is important that test_function remains an exported symbol, for example
for test regression/kernel/test_userspace_probe.

Signed-off-by: Simon Marchi <>
Change-Id: I52019f69de023b24b46cc701e2e0df919e654a76
Signed-off-by: Jérémie Galarneau <>
2 years agotests: add declarations for functions in gen-syscall-events-callstack.c
Simon Marchi [Mon, 25 Nov 2019 21:08:20 +0000 (16:08 -0500)] 
tests: add declarations for functions in gen-syscall-events-callstack.c


      CC       gen-syscall-events-callstack.o
    /home/smarchi/src/lttng-tools/tests/utils/testapp/gen-syscall-events-callstack/gen-syscall-events-callstack.c:46:1: error: no previous declaration for ‘my_gettid’ [-Werror=missing-declarations]
    /home/smarchi/src/lttng-tools/tests/utils/testapp/gen-syscall-events-callstack/gen-syscall-events-callstack.c:72:1: error: no previous declaration for ‘fct_c’ [-Werror=missing-declarations]
    /home/smarchi/src/lttng-tools/tests/utils/testapp/gen-syscall-events-callstack/gen-syscall-events-callstack.c:78:1: error: no previous declaration for ‘fct_b’ [-Werror=missing-declarations]
    /home/smarchi/src/lttng-tools/tests/utils/testapp/gen-syscall-events-callstack/gen-syscall-events-callstack.c:85:1: error: no previous declaration for ‘fct_a’ [-Werror=missing-declarations]

From what I understand, it is important that these symbols stay exported
for test regression/kernel/test_callstack.

Change-Id: I656a4e77bb2653510ba971e94ccc0d8bb65be698
Signed-off-by: Simon Marchi <>
Signed-off-by: Jérémie Galarneau <>
2 years agotests: make entry_handler static in ini_config test
Simon Marchi [Mon, 25 Nov 2019 21:10:17 +0000 (16:10 -0500)] 
tests: make entry_handler static in ini_config test


      CC       ini_config.o
    /home/smarchi/src/lttng-tools/tests/unit/ini_config/ini_config.c:37:5: error: no previous declaration for ‘entry_handler’ [-Werror=missing-declarations]
     int entry_handler(const struct config_entry *entry,

Change-Id: I937fda691d5ab81bf7567ab50fd90903911dfc57
Signed-off-by: Simon Marchi <>
Signed-off-by: Jérémie Galarneau <>
2 years agoFormat lists in src/common/
Simon Marchi [Mon, 25 Nov 2019 19:17:40 +0000 (14:17 -0500)] 
Format lists in src/common/

I'm currently dealing with src/common/, and would find it
more pleasant to modify if the lists were formatted consistently and
sorted (except DIST_SUBDIRS, for which the order is important).

I have taken the liberty to put the corresponding .c and .h on the same
line, since they will always be next to each other.  It reduces a bit
the number of lines.  If you don't like that, I would also be fine with
a completely flat list.

Change-Id: I8a65fe5e0fea37230f1376c51c7d4a547f651c7e
Signed-off-by: Simon Marchi <>
Signed-off-by: Jérémie Galarneau <>
2 years agosessiond: make functions static in ust-registry.c
Simon Marchi [Mon, 25 Nov 2019 20:46:21 +0000 (15:46 -0500)] 
sessiond: make functions static in ust-registry.c


      CC       ust-registry.o
    /home/smarchi/src/lttng-tools/src/bin/lttng-sessiond/ust-registry.c:553:2: error: no previous declaration for ‘ust_registry_lookup_enum’ [-Werror=missing-declarations]
      ust_registry_lookup_enum(struct ust_registry_session *session,
    /home/smarchi/src/lttng-tools/src/bin/lttng-sessiond/ust-registry.c:684:6: error: no previous declaration for ‘ust_registry_destroy_enum’ [-Werror=missing-declarations]
     void ust_registry_destroy_enum(struct ust_registry_session *reg_session,

Signed-off-by: Simon Marchi <>
Change-Id: Ibaabd16a75f0b7b13a641a571d26ca64aec5317b
Signed-off-by: Jérémie Galarneau <>
2 years agosessiond: make thread_consumer_management function static
Simon Marchi [Mon, 25 Nov 2019 20:45:12 +0000 (15:45 -0500)] 
sessiond: make thread_consumer_management function static


      CC       manage-consumer.o
    /home/smarchi/src/lttng-tools/src/bin/lttng-sessiond/manage-consumer.c:64:7: error: no previous declaration for ‘thread_consumer_management’ [-Werror=missing-declarations]
     void *thread_consumer_management(void *data)

Signed-off-by: Simon Marchi <>
Change-Id: Id28f40244e12a618749cc618466fc31d6d773378
Signed-off-by: Jérémie Galarneau <>
2 years agosessiond: make thread_rotation function static
Simon Marchi [Mon, 25 Nov 2019 20:44:15 +0000 (15:44 -0500)] 
sessiond: make thread_rotation function static


      CC       rotation-thread.o
    /home/smarchi/src/lttng-tools/src/bin/lttng-sessiond/rotation-thread.c:799:7: error: no previous declaration for ‘thread_rotation’ [-Werror=missing-declarations]
     void *thread_rotation(void *data)

Signed-off-by: Simon Marchi <>
Change-Id: Ia95212a53c658bce5dc6df155eb43ee29b7980c6
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: build: ust -> kernel mix-up in noinst_SCRIPTS
Jonathan Rajotte [Tue, 3 Dec 2019 16:46:24 +0000 (11:46 -0500)] 
Fix: build: ust -> kernel mix-up in noinst_SCRIPTS

This is simply a copy paste error as test_notification_ust appears

Signed-off-by: Jonathan Rajotte <>
Change-Id: I4e170a092338f5a95479a58d48247088d28d3e31
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: missing metadata stream causes all traces to be skipped
Jérémie Galarneau [Tue, 3 Dec 2019 09:59:10 +0000 (04:59 -0500)] 
Fix: relayd: missing metadata stream causes all traces to be skipped

Commit 123ed7c22 intends for a trace that doesn't have a metadata
stream to be skipped when creating viewer streams. However, the loop
over ctf_traces should be "continued" rather then "broken" from when
this situation arises. Otherwise, all ctf_traces of the session are
skipped, which is not the intention here.

Moreover, a reference to the current ctf_trace is leaked when the
break (now continue) occurs.

Signed-off-by: Jérémie Galarneau <>
Change-Id: Ic630521c6f050c77a78f2c1e20c5741a6b3a66a1

2 years agolttng-ctl: add local declarations for obsolete functions in lttng-ctl.c
Simon Marchi [Mon, 25 Nov 2019 20:34:21 +0000 (15:34 -0500)] 
lttng-ctl: add local declarations for obsolete functions in lttng-ctl.c

The functions concerned by this change are obsolete, but kept to avoid
breaking the ABI.  This patch adds local declaration for these functions
to silence the following errors, when building with

      CC       lttng-ctl.lo
    /home/smarchi/src/lttng-tools/src/lib/lttng-ctl/lttng-ctl.c:2815:5: error: no previous declaration for ‘lttng_enable_consumer’ [-Werror=missing-declarations]
     int lttng_enable_consumer(struct lttng_handle *handle)
    /home/smarchi/src/lttng-tools/src/lib/lttng-ctl/lttng-ctl.c:2823:5: error: no previous declaration for ‘lttng_disable_consumer’ [-Werror=missing-declarations]
     int lttng_disable_consumer(struct lttng_handle *handle)
    /home/smarchi/src/lttng-tools/src/lib/lttng-ctl/lttng-ctl.c:2831:5: error: no previous declaration for ‘_lttng_create_session_ext’ [-Werror=missing-declarations]
     int _lttng_create_session_ext(const char *name, const char *url,

Signed-off-by: Simon Marchi <>
Change-Id: I49750d7646d5823461933c13a9ec4ccef905921c
Signed-off-by: Jérémie Galarneau <>
2 years agosessiond: remove unused log_job_destruction function
Simon Marchi [Mon, 25 Nov 2019 20:43:02 +0000 (15:43 -0500)] 
sessiond: remove unused log_job_destruction function


      CC       rotation-thread.o
    /home/smarchi/src/lttng-tools/src/bin/lttng-sessiond/rotation-thread.c:114:6: error: no previous declaration for ‘log_job_destruction’ [-Werror=missing-declarations]
     void log_job_destruction(const struct rotation_thread_job *job)

Signed-off-by: Simon Marchi <>
Change-Id: I3fce2aceb047a2a251d61e84bff5862be49fe593
Signed-off-by: Jérémie Galarneau <>
2 years agosessiond: make disable_context static
Simon Marchi [Mon, 25 Nov 2019 20:42:04 +0000 (15:42 -0500)] 
sessiond: make disable_context static


      CC       agent.o
    /home/smarchi/src/lttng-tools/src/bin/lttng-sessiond/agent.c:833:5: error: no previous declaration for ‘disable_context’ [-Werror=missing-declarations]
     int disable_context(struct agent_app_ctx *ctx, enum lttng_domain_type domain)

Signed-off-by: Simon Marchi <>
Change-Id: I3f8eb7e658e558f7082781c95b4200607e5dad6f
Signed-off-by: Jérémie Galarneau <>
2 years agosessiond: make ltt_sessions_ht_alloc static
Simon Marchi [Mon, 25 Nov 2019 20:40:35 +0000 (15:40 -0500)] 
sessiond: make ltt_sessions_ht_alloc static


      CC       session.o
    /home/smarchi/src/lttng-tools/src/bin/lttng-sessiond/session.c:305:5: error: no previous declaration for ‘ltt_sessions_ht_alloc’ [-Werror=missing-declarations]
     int ltt_sessions_ht_alloc(void)

Signed-off-by: Simon Marchi <>
Change-Id: Iaf4ca32c1f6ca01fa7ad4a1bfc7edd81a90e6adc
Signed-off-by: Jérémie Galarneau <>
2 years agorelayd: make functions static in ctf-trace.c
Simon Marchi [Mon, 25 Nov 2019 20:38:19 +0000 (15:38 -0500)] 
relayd: make functions static in ctf-trace.c


      CC       ctf-trace.o
    /home/smarchi/src/lttng-tools/src/bin/lttng-relayd/ctf-trace.c:47:6: error: no previous declaration for ‘ctf_trace_destroy’ [-Werror=missing-declarations]
     void ctf_trace_destroy(struct ctf_trace *trace)
    /home/smarchi/src/lttng-tools/src/bin/lttng-relayd/ctf-trace.c:62:6: error: no previous declaration for ‘ctf_trace_release’ [-Werror=missing-declarations]
     void ctf_trace_release(struct urcu_ref *ref)

Signed-off-by: Simon Marchi <>
Change-Id: I4c7ff97b3350d2e50530fb0666c5a7d02599f99f
Signed-off-by: Jérémie Galarneau <>
2 years agoMake validate_url_option_combination static in create.c
Simon Marchi [Mon, 25 Nov 2019 20:35:54 +0000 (15:35 -0500)] 
Make validate_url_option_combination static in create.c


      CC       commands/lttng-create.o
    /home/smarchi/src/lttng-tools/src/bin/lttng/commands/create.c:640:5: error: no previous declaration for ‘validate_url_option_combination’ [-Werror=missing-declarations]
     int validate_url_option_combination(void)

Signed-off-by: Simon Marchi <>
Change-Id: I091b875fb93d4214c47a1c229e6360ec02a0a2f8
Signed-off-by: Jérémie Galarneau <>
2 years agoMake _utils_expand_path static
Simon Marchi [Mon, 25 Nov 2019 20:29:29 +0000 (15:29 -0500)] 
Make _utils_expand_path static

It's only used in its own file, so make it static.


      CC       utils.lo
    /home/smarchi/src/lttng-tools/src/common/utils.c:328:7: error: no previous declaration for ‘_utils_expand_path’ [-Werror=missing-declarations]
     char *_utils_expand_path(const char *path, bool keep_symlink)

Signed-off-by: Simon Marchi <>
Change-Id: Ibab3bf9994f29b1a841e9d3ddc9bfce1f8e73032
Signed-off-by: Jérémie Galarneau <>
2 years agoRemove unused function uri_create
Simon Marchi [Mon, 25 Nov 2019 20:28:33 +0000 (15:28 -0500)] 
Remove unused function uri_create


      CC       uri.lo
    /home/smarchi/src/lttng-tools/src/common/uri.c:273:19: error: no previous declaration for ‘uri_create’ [-Werror=missing-declarations]
     struct lttng_uri *uri_create(void)

Signed-off-by: Simon Marchi <>
Change-Id: I18a58391842c3fba1fa3dd60cf5fe3215280f95a
Signed-off-by: Jérémie Galarneau <>
2 years agouri: put `inline` at beginning of strpbrk_or_eos declaration
Simon Marchi [Mon, 25 Nov 2019 20:26:44 +0000 (15:26 -0500)] 
uri: put `inline` at beginning of strpbrk_or_eos declaration


      CC       uri.lo
    /home/smarchi/src/lttng-tools/src/common/uri.c:64:1: error: ‘inline’ is not at beginning of declaration [-Werror=old-style-declaration]
     static const inline char *strpbrk_or_eos(const char *s, const char *accept)

Signed-off-by: Simon Marchi <>
Change-Id: I2f753143e65ac4301abc01fba3e9ef122b2d38b7
Signed-off-by: Jérémie Galarneau <>
2 years agotests: put `static` keyword at beginning of variable declaration
Simon Marchi [Thu, 28 Nov 2019 21:32:16 +0000 (16:32 -0500)] 
tests: put `static` keyword at beginning of variable declaration


  CC       select_poll_epoll-select_poll_epoll.o
/home/smarchi/src/lttng-tools/tests/regression/kernel/select_poll_epoll.c:34:1: error: ‘static’ is not at beginning of declaration [-Werror=old-style-declaration]
 volatile static int stop_thread;

Change-Id: I2cf1a7d50a01ca0e9226ea527f2e8af70a292f94
Signed-off-by: Simon Marchi <>
Signed-off-by: Jérémie Galarneau <>
2 years agocommon: make some functions static in mi-lttng.c
Simon Marchi [Mon, 25 Nov 2019 20:25:39 +0000 (15:25 -0500)] 
common: make some functions static in mi-lttng.c

These two functions are only used in the file, so make them static.


      CC       mi-lttng.lo
    /home/smarchi/src/lttng-tools/src/common/mi-lttng.c:367:13: error: no previous declaration for ‘mi_lttng_eventtype_string’ [-Werror=missing-declarations]
     const char *mi_lttng_eventtype_string(enum lttng_event_type value)
    /home/smarchi/src/lttng-tools/src/common/mi-lttng.c:1317:5: error: no previous declaration for ‘mi_lttng_event_userspace_probe’ [-Werror=missing-declarations]
     int mi_lttng_event_userspace_probe(struct mi_writer *writer,

There are a bunch of other stuff in mi-lttng.c that could be made
static, but those were the only two problematic with
-Wmissing-declarations, because there have no declaration in the .h.

Signed-off-by: Simon Marchi <>
Change-Id: I9f3a6b0f71e86fac7af5d40e553a6944eb768fbe
Signed-off-by: Jérémie Galarneau <>
2 years agoconsumerd: make lttng_consumer_close_all_metadata static
Simon Marchi [Mon, 25 Nov 2019 20:17:19 +0000 (15:17 -0500)] 
consumerd: make lttng_consumer_close_all_metadata static

It is only used in its file, so make it static.  Fixes:

      CC       consumer.lo
    /home/smarchi/src/lttng-tools/src/common/consumer/consumer.c:2177:6: error: no previous declaration for ‘lttng_consumer_close_all_metadata’ [-Werror=missing-declarations]
     void lttng_consumer_close_all_metadata(void)

Signed-off-by: Simon Marchi <>
Change-Id: I211a31a7fb5edfb76e90f1209e69bbed5dd5ec92
Signed-off-by: Jérémie Galarneau <>
2 years libuuid is no longer a dependency
Jérémie Galarneau [Wed, 27 Nov 2019 20:30:45 +0000 (15:30 -0500)] libuuid is no longer a dependency

Signed-off-by: Jérémie Galarneau <>
Change-Id: I68ce6b7d210a5f645892267968cd01c8347b2370

2 years agoReplace libuuid with internal implementation
Michael Jeanson [Wed, 23 Oct 2019 20:07:37 +0000 (16:07 -0400)] 
Replace libuuid with internal implementation

We use a very small subset of libuuid features to transform UUIDs
between their string and binary representation. Plus we have a lot of
compat code for different platforms with some unspecified default
behavior regarding the use of upper/lower case.

Drop the dependency on libuuid and replace it with a minimal internal
implementation that respects RFC4122.

Signed-off-by: Michael Jeanson <>
Change-Id: I744e3cf65d6a22d0acf7a9943c10943ba64e8468
Signed-off-by: Jérémie Galarneau <>
2 years agoconsumerd: remove unused function consumer_del_data_stream
Simon Marchi [Mon, 25 Nov 2019 20:15:46 +0000 (15:15 -0500)] 
consumerd: remove unused function consumer_del_data_stream


      CC       consumer.lo
    /home/smarchi/src/lttng-tools/src/common/consumer/consumer.c:712:6: error: no previous declaration for ‘consumer_del_data_stream’ [-Werror=missing-declarations]
     void consumer_del_data_stream(struct lttng_consumer_stream *stream)

Signed-off-by: Simon Marchi <>
Change-Id: I92a47e069055713eaa4fdbc7f034a8f8e439f283
Signed-off-by: Jérémie Galarneau <>
2 years agom4: sync ax_pthread.m4 with autoconf archive
Simon Marchi [Fri, 4 Oct 2019 16:18:05 +0000 (12:18 -0400)] 
m4: sync ax_pthread.m4 with autoconf archive

Sync with [1].

In particular, this lets us configure with -Wunused-but-set-parameter.


Signed-off-by: Simon Marchi <>
Change-Id: Id9a6eb7cd0b8c7597ae6abb5587078afdaa420f6
Signed-off-by: Jérémie Galarneau <>
2 years Remove duplicated CMD_DESCR_ROTATE definition
Simon Marchi [Mon, 27 May 2019 15:52:46 +0000 (11:52 -0400)] Remove duplicated CMD_DESCR_ROTATE definition

CMD_DESCR_ROTATE is defined twice by configure.  This breaks configuring
with "-Werror -Wall", as some test programs don't compile due to:

  configure:16990: gcc -o conftest -Wall -Werror   conftest.c  >&5
  conftest.c:163: error: "CONFIG_CMD_DESCR_ROTATE" redefined [-Werror]
   #define CONFIG_CMD_DESCR_ROTATE "Archive a tracing session’s current trace chunk"

  conftest.c:154: note: this is the location of the previous definition
   #define CONFIG_CMD_DESCR_ROTATE "Archive a tracing session's current trace chunk"

  cc1: all warnings being treated as errors

Signed-off-by: Simon Marchi <>
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: tests: missing argument in test output print statement
Jérémie Galarneau [Mon, 25 Nov 2019 23:58:56 +0000 (18:58 -0500)] 
Fix: tests: missing argument in test output print statement

The format string of this print statement uses a string and none
is provided. This results in a crash when running on ARM64
test boards.

Signed-off-by: Jérémie Galarneau <>
2 years agoSilence bogus warning when building with old GCC versions
Jérémie Galarneau [Mon, 25 Nov 2019 22:54:07 +0000 (17:54 -0500)] 
Silence bogus warning when building with old GCC versions

gcc version 7.4.0 (and possibly others) warn that `new_handle`
may be used uninitialized. It is not clear what code path would
trigger this uninitialized use.

Moreover, the warning is not present when building with recent
versions of gcc and clang, nor is it flagged by Coverity. This points
to the warning being bogus.

Still, setting `new_handle` to NULL silences the warning on this
gcc version.

Signed-off-by: Jérémie Galarneau <>
2 years agoClean-up: open_pipe_cloexec() has useless boilerplate
Jérémie Galarneau [Mon, 25 Nov 2019 22:03:29 +0000 (17:03 -0500)] 
Clean-up: open_pipe_cloexec() has useless boilerplate

The open_pipe_cloexec() wrapper can be simplified to simply call
utils_create_pipe_cloexec(). This eliminates a warning of Coverity.

1407700 Identical code for different branches
The condition is redundant
In open_pipe_cloexec: The same code is executed regardless of the condition (CWE-398)

Reported-by: Coverity Scan
Signed-off-by: Jérémie Galarneau <>
2 years agoClean-up: tests: unchecked closedir value in test_fd_tracker
Jérémie Galarneau [Mon, 25 Nov 2019 22:01:19 +0000 (17:01 -0500)] 
Clean-up: tests: unchecked closedir value in test_fd_tracker

closedir's return value is not checked in a utility function
of the test_fd_tracker unit test.

1407702 Unchecked return value
If the function returns an error value, the error value may be mistaken for a normal value.
In fd_count: Value returned from a function is not checked for errors before being used (CWE-252)

Reported-by: Coverity Scan
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: fd-tracker: dereference of lttng_inode after NULL check
Jérémie Galarneau [Mon, 25 Nov 2019 21:40:10 +0000 (16:40 -0500)] 
Fix: fd-tracker: dereference of lttng_inode after NULL check

handle->inode is checked for NULL but used unconditionaly
on line 873 (lttng_inode_put).

1407756 Dereference after null check
Either the check against null is unnecessary, or there may be a null pointer dereference.
In fs_handle_close: Pointer is checked against null but then dereferenced anyway (CWE-476)

Reported-by: Coverity Scan
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: fd-tracker: error path lead to null dereference of handle
Jérémie Galarneau [Mon, 25 Nov 2019 19:37:54 +0000 (14:37 -0500)] 
Fix: fd-tracker: error path lead to null dereference of handle

A number of fd_tracker_open_fs_handle() error paths can lead to a NULL
pointer dereference. The error paths are separated to cover the
various initialization stages of an fs_handle.

Signed-off-by: Jonathan Rajotte <>
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: fd-tracker: crash on close of partially initialized handle
Jérémie Galarneau [Mon, 25 Nov 2019 19:37:38 +0000 (14:37 -0500)] 
Fix: fd-tracker: crash on close of partially initialized handle

A number of internal functions are used in the error path of the
creation of an fs_handle. Since the fs_handle is partially
initialized, those functions need to be adapted to handle cases where
internal members are NULL.

Signed-off-by: Jérémie Galarneau <>
2 years agoTests: add fd-tracker tests for the unlink operation
Jérémie Galarneau [Sat, 23 Nov 2019 00:30:39 +0000 (19:30 -0500)] 
Tests: add fd-tracker tests for the unlink operation

Signed-off-by: Jérémie Galarneau <>
2 years agofd-tracker: use lttng_inode to store fs_handle's path
Jérémie Galarneau [Sat, 23 Nov 2019 00:25:36 +0000 (19:25 -0500)] 
fd-tracker: use lttng_inode to store fs_handle's path

Storing paths as part of the lttng_inode object, which may be shared
by multiple fs_handles reduces the number of copies of the path.

Signed-off-by: Jérémie Galarneau <>
2 years agofd-tracker: add the lttng-inode interface
Jérémie Galarneau [Sat, 23 Nov 2019 00:12:27 +0000 (19:12 -0500)] 
fd-tracker: add the lttng-inode interface

The lttng-inode interface is internal to the fd-tracker. It allows
unlink operations on an fs_handle to determine if another fs_handle
refers to the same file.

When this is deemed to be the case, the unlink can be delayed by
renaming the file to a temporary name and, finally, unlinking it
when all references have been released.

Signed-off-by: Jérémie Galarneau <>
2 years agofd-tracker: add the unlink operation to fs handles
Jérémie Galarneau [Sat, 23 Nov 2019 00:08:18 +0000 (19:08 -0500)] 
fd-tracker: add the unlink operation to fs handles

The unlink method allows user to unlink the file refered-to by
an fs_handle. As indicated in the comments, the unlink operation
leverages the lttng_inode to ensure the actual unlink system call
is only performed when the last reference to an lttng_inode is
released by an fs_handle.

Signed-off-by: Jérémie Galarneau <>
2 years agoTest: add a unit test for the fd tracker
Jérémie Galarneau [Fri, 22 Nov 2019 23:54:58 +0000 (18:54 -0500)] 
Test: add a unit test for the fd tracker

Signed-off-by: Jérémie Galarneau <>
2 years agofd-tracker: add pipe management wrappers to fd-tracker
Jérémie Galarneau [Fri, 22 Nov 2019 23:52:03 +0000 (18:52 -0500)] 
fd-tracker: add pipe management wrappers to fd-tracker

Add wrappers to allow the creation of a pipe the two ends of
which are tracked by an fd-tracker.

Signed-off-by: Jérémie Galarneau <>
2 years agofd-tracker: add epoll/poll management wrappers to fd-tracker
Jérémie Galarneau [Fri, 22 Nov 2019 23:46:47 +0000 (18:46 -0500)] 
fd-tracker: add epoll/poll management wrappers to fd-tracker

Add wrappers which create an lttng_poll_event object, tracking
any file descriptor created in the process of initializing
the object.

When the build is configured to use the epoll interface, the
underlying epoll fd is tracked by the fd-tracker.

If the build is configured to use the poll interace, the fd tracker is
not involved in the process and the utility will simply defer the call
to lttng_poll_create() directly.

Signed-off-by: Jérémie Galarneau <>
2 years agofd-tracker: add an fd-tracker util to common
Jérémie Galarneau [Fri, 22 Nov 2019 23:41:09 +0000 (18:41 -0500)] 
fd-tracker: add an fd-tracker util to common

This commit adds an fd-tracker utility to the common libs.
This interface allows a process to keep track of its open
file descriptors and enforce a limit to the number of file
descriptors that may be simultaneously opened.

The intent is to use this interface as part of the relay daemon
to mitigate file descriptors exhaustion problems that are
encountered when the relay has to handle a large number of streams.

The fd-tracker defines two classes of file descriptors: suspendable
and unsuspendable file descriptors.

Suspendable file descriptors are handles to filesystem objects
(e.g. regular files) that may be closed and re-opened later without
affecting the application.

A suspendable file descriptor can be opened by creating a filesystem
handle (fs_handle) using the fd-tracker. The raw file descritptor
must then be obtained and released using that handle. Closing the
handle will effectively ensure that the file descritptor is closed.

Unsuspendable file descriptors are file descriptors that cannot
be closed without affecting the application's state. For instance,
it is not possible to close and re-open a pipe, a TCP socket, or
an epoll fd without involving some app-specific logic. Thus, the
fd-tracker considers those file descriptors as unsuspendable.

Opening an unsuspendable file descritptor will return a raw file
decriptor to the application. It is its responsability to notify the
fd-tracker of the file descriptor's closing to ensure the number
of active file descriptors can be tracked accurately.

If a request to open a new file descriptors is made to the fd-tracker
and the process has already reached its maximal count of
simultaneously opened file descriptors, an attempt will be made to
suspend a suspendable file descriptor to release a slot.

Suspending a file descriptor involves:
  - verifying that the file is still available on the FS (restorable),
  - sampling its current position,
  - closing the file descriptor.

Note that suspending a file descriptor eliminates the POSIX guarantee
that a file may be unlinked at any time without affecting the
application (provided that it holds an open FD to that
file). Applications using the fd-tracker that need to maintain this
guarantee should open those files as unsuspendable file descriptors.

To protect against unlinking and file replacement scenarios, the
fd-tracker samples the files' inode number when a fs_handle is
created. This inode number will then be checked anytime the handle
is suspended or restored to ensure that the application is made
aware of the file's unavailability. This is preferable to
inadvertently opening another file of the same name if the original
file was unlinked and/or replaced between a fs_handle's suspension
and restoration.

Signed-off-by: Jérémie Galarneau <>
2 years agorelayd: close stdin on launch
Jérémie Galarneau [Sat, 23 Nov 2019 00:20:22 +0000 (19:20 -0500)] 
relayd: close stdin on launch

The relay daemon has no use for the standard input; it can be safely
closed early on launch.

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: fully initialize viewer stream before publishing it
Jérémie Galarneau [Sat, 23 Nov 2019 00:04:16 +0000 (19:04 -0500)] 
Fix: relayd: fully initialize viewer stream before publishing it

Signed-off-by: Jérémie Galarneau <>
2 years agoClean-up relayd: session_release can be marked as static
Jérémie Galarneau [Fri, 22 Nov 2019 23:37:28 +0000 (18:37 -0500)] 
Clean-up relayd: session_release can be marked as static

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: don't send streams if there is no metadata
Francis Deslauriers [Tue, 12 Nov 2019 22:55:25 +0000 (17:55 -0500)] 
Fix: relayd: don't send streams if there is no metadata

When tracing short-live UST apps in per-pid mode, it happens that traces
are closed before we can send all the streams to the viewer. This can
place the viewer in an uncomfortable position where it is aware of data
streams of a trace but can't get the metadata stream to decode the

Only send the data streams if we have the metadata stream or if the
metadata stream was already sent. This ensures that the viewer will
either have all the {data,metadata} streams or none of them.

Signed-off-by: Francis Deslauriers <>
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: update apps on untrack only when session is active
Jonathan Rajotte [Mon, 18 Nov 2019 20:12:20 +0000 (15:12 -0500)] 
Fix: update apps on untrack only when session is active

This mimics what is done on the track side.

Fixes #1210

Signed-off-by: Jonathan Rajotte <>
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: consumerd: assert on null trace chunk on session restart
Jérémie Galarneau [Fri, 22 Nov 2019 20:08:15 +0000 (15:08 -0500)] 
Fix: consumerd: assert on null trace chunk on session restart

The consumer daemon asserts on
`stream->net_seq_idx != (uint64_t) 1ULL || stream>trace_chunk'
when a session is re-started following a rotation.

Steps to reproduce, while an instrumented application is running:
$ lttng create
$ lttng enable-event -u -a
$ lttng start
$ lttng stop
$ lttng rotate
$ lttng start
-> assertion fails

The trace chunk of a stream can be null in this scenario as the
"create trace chunk" consumer command immediately sets the streams'
current trace chunk when they are transitioning from a "null" trace

Since 1f4962443, a session restart will cause a rotation to occur from
"null" to the streams' new trace chunk. When this rotation occurs, the
channel and their streams are seen to be in the same trace chunk.
This is unexpected as this should only occur when transitioning from a
trace chunk to the "null" trace chunk (no output). In that case, the
streams are considered to have reached the point where they should
discard their current trace chunk to enter the "null trace chunk"
state (having no current output). This is exactly the opposite of
the situation here as streams are transitioning from a null to a
non-null trace chunk.

Hence, the immediate assignation of the trace chunk to the streams
should no longer be performed on creation of a trace chunk on the
consumer daemon. The streams transition from the "null" trace chunk to
a new trace chunk through the "rotate channel" command as is done for
all other rotation operations. Removing this bypass also ensures that
the rotation behaviour of both the consumer and relay daemons match.

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: sessiond: don't wait for a rotation from a null chunk to finish
Jérémie Galarneau [Wed, 20 Nov 2019 21:09:50 +0000 (16:09 -0500)] 
Fix: sessiond: don't wait for a rotation from a null chunk to finish

The rotation completion checking does not handle waiting for the
completion of a rotation _from_ a NULL trace chunk. This is correct as
there is essentially nothing to check. Streams always rotate out of a
null trace chunk immediately as it means the stream had no output and
could not receive data.

Concretely, this happens when stopping a session, rotating, and
re-starting a session.

The fix consists in simply re-working the logic of the "rotate"
command to not launch a rotation completion check job and not put the
session in the "rotation ongoing" state.

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: sessiond: duplicated rotation notification sent
Jérémie Galarneau [Wed, 20 Nov 2019 21:06:30 +0000 (16:06 -0500)] 
Fix: sessiond: duplicated rotation notification sent

A duplicate notification that a rotation was launched and completed
is sent by the rotation thread when a rotation is performed after
a session was stopped.

This code is removed at it is a left-over from the time when the
rotation thread had to explicitly rename trace output folders
when a rotation had completed.

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: remove assert of non-null stream trace chunk on rotate
Jérémie Galarneau [Tue, 19 Nov 2019 18:41:51 +0000 (13:41 -0500)] 
Fix: relayd: remove assert of non-null stream trace chunk on rotate

A stream can rotate from a "NULL" trace chunk to a new trace chunk
and the relay daemon should not assert on this condition.

This happens when a session is stopped, rotated, and started again
later on.

Signed-off-by: Jérémie Galarneau <>
2 years agoDocs: verb/noun confusion in comment
Jérémie Galarneau [Mon, 18 Nov 2019 21:09:21 +0000 (16:09 -0500)] 
Docs: verb/noun confusion in comment

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: sessiond: no rotation performed from null chunk to new chunk
Jérémie Galarneau [Mon, 18 Nov 2019 21:01:44 +0000 (16:01 -0500)] 
Fix: sessiond: no rotation performed from null chunk to new chunk

The session daemon must rotate from a null trace chunk to a new trace
chunk when a session's current trace chunk is null. This situation
happens when a session is stopped, rotated, and started again.

This bug leaves existing streams in the "null" trace chunk and causes
the relay daemon to report a protocol error on the reception of the
first data packet following such a start.

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: invalid use of destructor in dynamic pointer array
Jérémie Galarneau [Mon, 18 Nov 2019 19:43:06 +0000 (14:43 -0500)] 
Fix: invalid use of destructor in dynamic pointer array

A dynamic pointer array is built on top of a dynamic array and uses
the dynamic array's internal "destructor" field to store the
user-specified destructor.

lttng_dynamic_pointer_array_remove_pointer currently uses
the dynamic array's remove_element directly which causes the
user destructor to be called with the underlying storage of the
pointer rather than with the pointer itself.

This change re-uses the same pattern as
lttng_dynamic_pointer_array_reset(), namely using the destructor
explicitly and setting it to NULL for the duration of the call to
the dynamic array API.

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: check for a trace chunk before writing a packet
Jérémie Galarneau [Fri, 15 Nov 2019 22:16:32 +0000 (17:16 -0500)] 
Fix: relayd: check for a trace chunk before writing a packet

Protocol errors can cause a packet to be written to a stream that
doesn't have an active trace chunk. Validate this condition for the
init and write packet operations on a stream.

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: viewer session trace chunk not released on detach
Jérémie Galarneau [Thu, 14 Nov 2019 22:12:08 +0000 (17:12 -0500)] 
Fix: relayd: viewer session trace chunk not released on detach

The 'attach' command on a viewer session expects (asserts) the trace
chunk of the viewer session to be NULL. This is reasonable as there is
no reason to hold a reference to a trace chunk while no clients are

Release the reference to the trace chunk on detach. The relay
session's trace chunk will be re-sampled (copied) when the next client
attaches to the viewer session.

Signed-off-by: Jérémie Galarneau <>
2 years agoRequire automake >= 1.12
Michael Jeanson [Thu, 7 Nov 2019 19:02:55 +0000 (14:02 -0500)] 
Require automake >= 1.12

The test suite LOG_DRIVER statement requires that automake >= 1.12 be used
during bootstrap.

Signed-off-by: Michael Jeanson <>
Signed-off-by: Jérémie Galarneau <>
2 years agorelayd: rename viewer_session_set_trace_chunk and hide it
Jérémie Galarneau [Tue, 12 Nov 2019 21:28:50 +0000 (16:28 -0500)] 
relayd: rename viewer_session_set_trace_chunk and hide it

viewer_session_set_trace_chunk() is renamed to
viewer_session_set_trace_chunk_copy() as it creates a copy of the
source trace chunk before setting it on a viewer_session.

The function is also marked static as it is only needed within
the viewer-session.c TU.

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: session trace chunk is copied too late
Jérémie Galarneau [Tue, 12 Nov 2019 04:38:25 +0000 (23:38 -0500)] 
Fix: relayd: session trace chunk is copied too late

In per-pid buffering mode, a viewer can attach to a session while it
is active and find it has been closed by the time it requests new
streams. The viewer session's trace chunk is created as a side-effect
of the "get_new_streams" command and can find that the relay_session's
trace chunk has now been closed, causing the creation of the viewer
session trace chunk to fail.

This results in an unexpected error being reported to the live client.
This fix moves the creation of the viewer session's trace chunk to the
"attach" command. If the creation fails, the session is reported as
being "unknown".

Reported-by: Francis Deslauriers <>
Signed-off-by: Jérémie Galarneau <>
Tested-by: Francis Deslauriers <>
2 years agoFix: overly restrictive datetime regexp rejects valid dates
Jérémie Galarneau [Mon, 11 Nov 2019 23:53:56 +0000 (18:53 -0500)] 
Fix: overly restrictive datetime regexp rejects valid dates

The patch introducting backward_compat_group_by_session() was
edited (by me) before being merged and the regular expression was
made stricter. However, one of the changes was erroneous and
restricted the range of the second digit of the 'month' field
to [0-1].

Change it back to [0-9] to accept all months from 1-12.

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: disallow 0-length session names for 2.4+ peers
Jérémie Galarneau [Fri, 8 Nov 2019 22:45:20 +0000 (17:45 -0500)] 
Fix: relayd: disallow 0-length session names for 2.4+ peers

The group-by-session-name feature assumes sessions are named for peers
more recent than 2.3. Enforce this assumption at session creation

Signed-off-by: Jérémie Galarneau <>
2 years agoDocs: RELAYD(8): document the --group-output-by-session/host options
Jonathan Rajotte [Wed, 27 Jun 2018 00:48:24 +0000 (20:48 -0400)] 
Docs: RELAYD(8): document the --group-output-by-session/host options

Add the description of the newly-introduced --group-output-by-session
and --group-output-by-host options.

Signed-off-by: Jonathan Rajotte <>
Signed-off-by: Jérémie Galarneau <>
2 years agoTests: unit testing for backward compatibility of group-output-by-session
Jonathan Rajotte [Thu, 19 Sep 2019 21:26:06 +0000 (17:26 -0400)] 
Tests: unit testing for backward compatibility of group-output-by-session

These unit tests aim to reproduce the stream path received by
lttng-relayd when dealing with a producer using a protocol version < 2.11.

We then pass them through the transform function for
--group-output-by-session used when producer are using protocol version
greater or equal to 2.4 and small than 2.11.

Signed-off-by: Jonathan Rajotte <>
Signed-off-by: Jérémie Galarneau <>
2 years agorelayd: Add backward compatibility for --group-output-by-session
Jonathan Rajotte [Thu, 19 Sep 2019 20:56:53 +0000 (16:56 -0400)] 
relayd: Add backward compatibility for --group-output-by-session

The current implementation works for producer (consumerd/lttng-sessiond)
using protocol version >= 2.11.

For producer using protocol version >= 2.4 and < 2.11, we can use the
session name passed on the RELAYD_CREATE_SESSION command to infer
information (creation datetime, base path) from the stream path passed on
the RELAYD_ADD_STREAM command. That information is then used to craft a
stream path, in a best effort manner, that comply with the
--group-output-by-session requirements:


Prior to protocol version 2.4, the session name is unknown on lttng-relayd
side. In those cases, we do not perform any transformation and leave the
provided stream path as is. A warning (lttng-relayd) is emitted on such

Signed-off-by: Jonathan Rajotte <>
Signed-off-by: Jérémie Galarneau <>
2 years agoTests: regression testing for lttng-relayd --group-output-by-*
Jonathan Rajotte [Thu, 19 Sep 2019 21:00:51 +0000 (17:00 -0400)] 
Tests: regression testing for lttng-relayd --group-output-by-*

Each test is run for --group-output-by-host and
--group-output-by-session and also with automatic session name and
specific session name.

No need to test against kernel. Tracing domain does not have any influence
at lttng-relayd level.

Signed-off-by: Jonathan Rajotte <>
Signed-off-by: Jérémie Galarneau <>
2 years agoTests: lttng_snapshot_add_output_ok: allow specifying output type
Jonathan Rajotte [Fri, 22 Jun 2018 17:51:58 +0000 (13:51 -0400)] 
Tests: lttng_snapshot_add_output_ok: allow specifying output type

Also add support for additional options.

Signed-off-by: Jonathan Rajotte <>
Signed-off-by: Jérémie Galarneau <>
2 years agorelayd: introduce --group-output-by-session
Jonathan Rajotte [Thu, 19 Sep 2019 20:56:11 +0000 (16:56 -0400)] 
relayd: introduce --group-output-by-session

LTTng-relayd now support the grouping of traces per session.
This mode can be used via the "--group-output-per-session" switch.

The default, and current way, of grouping is done around the host
(hostname) of the traced system.

When grouped by host the following folder hierarchy is mostly found on the filesystem:


When using "--group-output-per-session", the following hierarchy is
found on the filesystem:


We also need to support base path information that come from the URIs
set at the client level:

    lttng create --set-url=net://localhost/extra/path/information

When grouping by host (current behaviour), it result in the following



We want to part from this way of handling the base path since it can lead
to unexpected conflict between session.

When grouping by session using a base path, the following hierarchy is



We encourage user to move away from using base path entirely as it
bypasses introduces potential name clashes and completely bypasses the
relay daemon's storage policy.

Signed-off-by: Jonathan Rajotte <>
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: sessiond: ust: deadlock with per-pid buffers
Mathieu Desnoyers [Fri, 1 Nov 2019 20:23:05 +0000 (16:23 -0400)] 
Fix: sessiond: ust: deadlock with per-pid buffers

Do not hold the registry lock while communicating with the consumerd,
because doing so causes inter-process deadlocks between consumerd and
sessiond with the metadata request notification.

The deadlock involves both sessiond and consumerd:

* lttng-sessiond:

thread 11 - thread_application_management


thread 15 - thread_consumer_management


thread 8 - thread_manage_clients


* lttng-consumerd:

thread 4 - consumer_timer_thread


thread 8 - consumer_thread_sessiond_poll

thread 7 - consumer_thread_data_poll


Signed-off-by: Mathieu Desnoyers <>
Signed-off-by: Jérémie Galarneau <>
2 years agorelayd: close viewer stream trace chunk earlier on release
Jérémie Galarneau [Wed, 6 Nov 2019 19:43:18 +0000 (14:43 -0500)] 
relayd: close viewer stream trace chunk earlier on release

A viewer stream puts its references to its stream and index files
within its "release" method (called when its reference count reaches

However, the reference to its trace chunk is only released during the
RCU reclamation of the viewer stream. This unnecessarily delays the
clean-up of the viewer trace chunk.

For cleanliness' sake, move the release of the viewer stream's trace
chunk to the release method, just after the release of the various
file handles of that stream.

Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: put chunk reference when closing stream
Mathieu Desnoyers [Fri, 1 Nov 2019 20:23:04 +0000 (16:23 -0400)] 
Fix: relayd: put chunk reference when closing stream

If a stream is closed by an application exiting (per-pid buffers), it
needs to put its reference on the stream trace chunk right away, because
otherwise still holding the reference on the trace chunk could allow a
viewer stream (which holds a reference to the stream) to postpone
destroy waiting for the chunk to cease to exist endlessly until the
viewer is detached.

Signed-off-by: Mathieu Desnoyers <>
Signed-off-by: Jérémie Galarneau <>
2 years agoFix: relayd: tracefile rotation: viewer opening missing index file
Mathieu Desnoyers [Fri, 1 Nov 2019 20:23:03 +0000 (16:23 -0400)] 
Fix: relayd: tracefile rotation: viewer opening missing index file

Moving the head position of the tracefile array when the data is
received opens a window where a viewer attaching to the session could
try to open a missing index file (which has not been received yet).

However, we want to bump the tail position as soon as we receive
data, because the prior tail is not valid anymore.

Solve this by introducing two head positions: the "read" head
and the "write" head. The "write" head is the position of the
newest data file (equivalent to the prior "head" position). We
also introduce a "read" head position, which is only moved
forward when the index is received.

The viewer now uses the "read" head position as upper bound, which
ensures it never attempts to open a non-existing index file.

Signed-off-by: Mathieu Desnoyers <>
Signed-off-by: Jérémie Galarneau <>
2 years agoTests: fix shellcheck warning
Jonathan Rajotte [Fri, 25 Oct 2019 22:12:04 +0000 (18:12 -0400)] 
Tests: fix shellcheck warning

No need to use random string for session name here, use the test name.

Signed-off-by: Jonathan Rajotte <>
Signed-off-by: Jérémie Galarneau <>
2 years agoTests: base path: lttng load for session configuration
Jonathan Rajotte [Fri, 25 Oct 2019 22:12:03 +0000 (18:12 -0400)] 
Tests: base path: lttng load for session configuration

Signed-off-by: Jonathan Rajotte <>
Signed-off-by: Jérémie Galarneau <>
2 years agoCleanup: remove unused internal lttng_session_descriptor_get_base_path
Jonathan Rajotte [Fri, 25 Oct 2019 22:12:02 +0000 (18:12 -0400)] 
Cleanup: remove unused internal lttng_session_descriptor_get_base_path

Signed-off-by: Jonathan Rajotte <>
Signed-off-by: Jérémie Galarneau <>
This page took 0.088315 seconds and 5 git commands to generate.