src.ctf.fs: support no packet beg/end CS and no discarded events/packets
authorPhilippe Proulx <eeppeliteloop@gmail.com>
Mon, 3 Jun 2019 19:48:06 +0000 (15:48 -0400)
committerPhilippe Proulx <eeppeliteloop@gmail.com>
Thu, 6 Jun 2019 21:19:12 +0000 (17:19 -0400)
This patch makes `src.ctf.fs` support CTF traces where a packet context
field class has or does not have the `timestamp_begin`, `timestamp_end`,
`events_discarded`, and `packet_seq_num` members. The corresponding
trace IR stream class's properties are set accordingly.

This fixes a bug having those base conditions:

1. Stream class has a default clock class.

2. Packet context's `timestamp_begin` and `timestamp_end` members
   are missing.

Then:

a) Because of 1., it was mandatory to set the default clock snapshot of
   packet beginning and end messages. It was also mandatory to set the
   default clock snapshots of discarded events/packets messages.

b) Because we didn't have the packet beginning and end times (because of
   2.), we used 0 for the first packet's beginning default clock
   snapshot.

c) Because CTF uses the packet beginning/end times to set the discarded
   events ranges, and because of b), we also used 0 as a discarded
   events message's beginning default clock snapshot if the events were
   discarded during the first packet.

0 is wrong for both b) and c): it's not 0, we just don't know the value.
It is wrong to tell a downstream component that events were possibly
discarded from 0 to another time because we just don't know those times.

I'm adding two test traces to make this code covered:

`no-packet-context`:
    No packet context at all, which means no packet beginning/end times,
    no discarded events, and no discarded packets.

`ev-disc-no-ts-begin-end`:
    A packet context with an `events_discarded` member, but without the
    `timestamp_begin` and `timestamp_end` members.

    In this case, when there's a discarded events message, events are
    known to have been discarded contextually, between two other
    messages with a default clock snapshot, but not at a specific time.

    `sink.text.pretty` prints:

        WARNING: Tracer discarded 17 events (unknown time range) in
        trace "ev-disc-no-ts-begin-end" (no UUID) within stream
        "/path/to/babeltrace/tests/ctf-traces/succeed/ev-disc-no-ts-begin-end/stream"
        (stream class ID: 0, stream ID: 0).

    while Babeltrace 1.5 prints:

        [warning] Tracer discarded 17 events between
        [19:00:00.000000000] and [19:00:00.000000000] in trace UUID
        0000000000000000, at path:
        "tests/ctf-traces/succeed/ev-disc-no-ts-begin-end", within
        stream id 0, at relative path: "stream". You should consider
        recording a new trace with larger buffers or with fewer events
        enabled.

    which is not exact.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Change-Id: Ieebf679a67bd0089e48708372caf934cc27573dc
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1368
Tested-by: jenkins
Reviewed-by: Simon Marchi <simon.marchi@efficios.com>
plugins/ctf/common/metadata/Makefile.am
plugins/ctf/common/metadata/ctf-meta-translate.c
plugins/ctf/common/metadata/ctf-meta-update-stream-class-config.c [new file with mode: 0644]
plugins/ctf/common/metadata/ctf-meta-visitors.h
plugins/ctf/common/metadata/ctf-meta.h
plugins/ctf/common/metadata/visitor-generate-ir.c
plugins/ctf/common/msg-iter/msg-iter.c
tests/ctf-traces/succeed/ev-disc-no-ts-begin-end/metadata [new file with mode: 0644]
tests/ctf-traces/succeed/ev-disc-no-ts-begin-end/stream [new file with mode: 0644]
tests/ctf-traces/succeed/no-packet-context/metadata [new file with mode: 0644]
tests/ctf-traces/succeed/no-packet-context/stream [new file with mode: 0644]

index 4dfc22b05514898a3f88495563434dbc4156ccf0..fa5b6f8ef6e757ff4eddc2bdd4f9346061572299 100644 (file)
@@ -33,6 +33,7 @@ libctf_ast_la_SOURCES = \
        ctf-meta-update-default-clock-classes.c \
        ctf-meta-update-text-array-sequence.c \
        ctf-meta-update-value-storing-indexes.c \
+       ctf-meta-update-stream-class-config.c \
        ctf-meta-warn-meaningless-header-fields.c \
        ctf-meta-translate.c \
        ctf-meta-resolve.c
index f65ab64ab05df4a4362f6744a3ec30ca07d97442..2bf1d0b7e70beb8f75cb7842a726cb72f4e0598b 100644 (file)
@@ -485,16 +485,18 @@ void ctf_stream_class_to_ir(struct ctx *ctx)
                ret = bt_stream_class_set_default_clock_class(ctx->ir_sc,
                        ctx->sc->default_clock_class->ir_cc);
                BT_ASSERT(ret == 0);
-               bt_stream_class_set_packets_have_default_beginning_clock_snapshot(
-                       ctx->ir_sc, BT_TRUE);
-               bt_stream_class_set_packets_have_default_end_clock_snapshot(
-                       ctx->ir_sc, BT_TRUE);
        }
 
-       bt_stream_class_set_supports_discarded_events(ctx->ir_sc, BT_TRUE,
-               ctx->sc->default_clock_class != NULL);
-       bt_stream_class_set_supports_discarded_packets(ctx->ir_sc, BT_TRUE,
-               ctx->sc->default_clock_class != NULL);
+       bt_stream_class_set_packets_have_default_beginning_clock_snapshot(
+               ctx->ir_sc, ctx->sc->packets_have_ts_begin);
+       bt_stream_class_set_packets_have_default_end_clock_snapshot(
+               ctx->ir_sc, ctx->sc->packets_have_ts_end);
+       bt_stream_class_set_supports_discarded_events(ctx->ir_sc,
+               ctx->sc->has_discarded_events,
+               ctx->sc->discarded_events_have_default_cs);
+       bt_stream_class_set_supports_discarded_packets(ctx->ir_sc,
+               ctx->sc->has_discarded_packets,
+               ctx->sc->discarded_packets_have_default_cs);
        ctx->sc->is_translated = true;
        ctx->sc->ir_sc = ctx->ir_sc;
 
diff --git a/plugins/ctf/common/metadata/ctf-meta-update-stream-class-config.c b/plugins/ctf/common/metadata/ctf-meta-update-stream-class-config.c
new file mode 100644 (file)
index 0000000..4988d0e
--- /dev/null
@@ -0,0 +1,83 @@
+/*
+ * Copyright 2019 - Philippe Proulx <pproulx@efficios.com>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ */
+
+#define BT_LOG_TAG "PLUGIN-CTF-METADATA-META-UPDATE-SC-CONFIG"
+#include "logging.h"
+
+#include <babeltrace/babeltrace.h>
+#include <babeltrace/babeltrace-internal.h>
+#include <babeltrace/assert-internal.h>
+#include <glib.h>
+#include <stdint.h>
+#include <string.h>
+#include <inttypes.h>
+
+#include "ctf-meta-visitors.h"
+
+BT_HIDDEN
+int ctf_trace_class_update_stream_class_config(struct ctf_trace_class *ctf_tc)
+{
+       struct ctf_field_class_int *int_fc;
+       uint64_t i;
+
+       for (i = 0; i < ctf_tc->stream_classes->len; i++) {
+               struct ctf_stream_class *sc =
+                       ctf_tc->stream_classes->pdata[i];
+
+               if (sc->is_translated) {
+                       continue;
+               }
+
+               if (!sc->packet_context_fc) {
+                       continue;
+               }
+
+               int_fc = ctf_field_class_struct_borrow_member_int_field_class_by_name(
+                       (void *) sc->packet_context_fc, "timestamp_begin");
+               if (int_fc && int_fc->meaning ==
+                               CTF_FIELD_CLASS_MEANING_PACKET_BEGINNING_TIME) {
+                       sc->packets_have_ts_begin = true;
+               }
+
+               int_fc = ctf_field_class_struct_borrow_member_int_field_class_by_name(
+                       (void *) sc->packet_context_fc, "timestamp_end");
+               if (int_fc && int_fc->meaning ==
+                               CTF_FIELD_CLASS_MEANING_PACKET_END_TIME) {
+                       sc->packets_have_ts_end = true;
+               }
+
+               int_fc = ctf_field_class_struct_borrow_member_int_field_class_by_name(
+                       (void *) sc->packet_context_fc, "events_discarded");
+               if (int_fc && int_fc->meaning ==
+                               CTF_FIELD_CLASS_MEANING_DISC_EV_REC_COUNTER_SNAPSHOT) {
+                       sc->has_discarded_events = true;
+               }
+
+               sc->discarded_events_have_default_cs =
+                       sc->has_discarded_events && sc->packets_have_ts_begin &&
+                       sc->packets_have_ts_end;
+               int_fc = ctf_field_class_struct_borrow_member_int_field_class_by_name(
+                       (void *) sc->packet_context_fc, "packet_seq_num");
+               if (int_fc && int_fc->meaning ==
+                               CTF_FIELD_CLASS_MEANING_PACKET_COUNTER_SNAPSHOT) {
+                       sc->has_discarded_packets = true;
+               }
+
+               sc->discarded_packets_have_default_cs =
+                       sc->has_discarded_packets &&
+                       sc->packets_have_ts_begin && sc->packets_have_ts_end;
+       }
+
+       return 0;
+}
index 9e835158e9636301055cb2a950b07aaa4be03788..2856e67124754942bf82a48e4be610786cf87c5c 100644 (file)
@@ -43,6 +43,9 @@ int ctf_trace_class_update_text_array_sequence(struct ctf_trace_class *ctf_tc);
 BT_HIDDEN
 int ctf_trace_class_update_value_storing_indexes(struct ctf_trace_class *ctf_tc);
 
+BT_HIDDEN
+int ctf_trace_class_update_stream_class_config(struct ctf_trace_class *ctf_tc);
+
 BT_HIDDEN
 int ctf_trace_class_validate(struct ctf_trace_class *ctf_tc);
 
index 33e41090724ef761c06d0b352713621ee9245de4..3b338c05c627494825818be880dc313abe4986cd 100644 (file)
@@ -228,6 +228,12 @@ struct ctf_event_class {
 struct ctf_stream_class {
        uint64_t id;
        bool is_translated;
+       bool packets_have_ts_begin;
+       bool packets_have_ts_end;
+       bool has_discarded_events;
+       bool has_discarded_packets;
+       bool discarded_events_have_default_cs;
+       bool discarded_packets_have_default_cs;
 
        /* Owned by this */
        struct ctf_field_class *packet_context_fc;
index 41a613a8dac9826b3093601feb4609bf402a1a5b..c926ad195850504a470cc2cfabcba1f105f1260b 100644 (file)
@@ -5018,6 +5018,13 @@ int ctf_visitor_generate_ir_visit_node(struct ctf_visitor_generate_ir *visitor,
                goto end;
        }
 
+       /* Update stream class configuration */
+       ret = ctf_trace_class_update_stream_class_config(ctx->ctf_tc);
+       if (ret) {
+               ret = -EINVAL;
+               goto end;
+       }
+
        /* Update text arrays and sequences */
        ret = ctf_trace_class_update_text_array_sequence(ctx->ctf_tc);
        if (ret) {
index 22ebc55f090e80bd1cbb3290fadc24f276a5f4e0..8a75d5d5eb6becfe38f30e1e26e1214aaa7bd60b 100644 (file)
@@ -1470,6 +1470,11 @@ enum bt_msg_iter_status check_emit_msg_discarded_events(
 {
        notit->state = STATE_EMIT_MSG_DISCARDED_EVENTS;
 
+       if (!notit->meta.sc->has_discarded_events) {
+               notit->state = STATE_CHECK_EMIT_MSG_DISCARDED_PACKETS;
+               goto end;
+       }
+
        if (notit->prev_packet_snapshots.discarded_events == UINT64_C(-1)) {
                if (notit->snapshots.discarded_events == 0 ||
                                notit->snapshots.discarded_events == UINT64_C(-1)) {
@@ -1497,6 +1502,7 @@ enum bt_msg_iter_status check_emit_msg_discarded_events(
                }
        }
 
+end:
        return BT_MSG_ITER_STATUS_OK;
 }
 
@@ -1506,6 +1512,11 @@ enum bt_msg_iter_status check_emit_msg_discarded_packets(
 {
        notit->state = STATE_EMIT_MSG_DISCARDED_PACKETS;
 
+       if (!notit->meta.sc->has_discarded_packets) {
+               notit->state = STATE_EMIT_MSG_PACKET_BEGINNING;
+               goto end;
+       }
+
        if (notit->prev_packet_snapshots.packets == UINT64_C(-1)) {
                /*
                 * Stream's first packet or no information about
@@ -1533,6 +1544,7 @@ enum bt_msg_iter_status check_emit_msg_discarded_packets(
                }
        }
 
+end:
        return BT_MSG_ITER_STATUS_OK;
 }
 
@@ -2518,20 +2530,11 @@ void create_msg_packet_beginning(struct bt_msg_iter *notit,
 
        BT_ASSERT(notit->msg_iter);
 
-       if (bt_stream_class_borrow_default_clock_class(notit->meta.sc->ir_sc)) {
-               uint64_t value = 0;
-
-               if (notit->snapshots.beginning_clock == UINT64_C(-1)) {
-                       if (notit->prev_packet_snapshots.end_clock != UINT64_C(-1)) {
-                               value = notit->prev_packet_snapshots.end_clock;
-                       }
-               } else {
-                       value = notit->snapshots.beginning_clock;
-               }
-
-
+       if (notit->meta.sc->packets_have_ts_begin) {
+               BT_ASSERT(notit->snapshots.beginning_clock != UINT64_C(-1));
                msg = bt_message_packet_beginning_create_with_default_clock_snapshot(
-                       notit->msg_iter, notit->packet, value);
+                       notit->msg_iter, notit->packet,
+                       notit->snapshots.beginning_clock);
        } else {
                msg = bt_message_packet_beginning_create(notit->msg_iter,
                        notit->packet);
@@ -2566,12 +2569,7 @@ void create_msg_packet_end(struct bt_msg_iter *notit, bt_message **message)
 
        BT_ASSERT(notit->msg_iter);
 
-       if (bt_stream_class_borrow_default_clock_class(notit->meta.sc->ir_sc)) {
-               if (notit->snapshots.end_clock == UINT64_C(-1)) {
-                       notit->snapshots.end_clock =
-                               notit->default_clock_snapshot;
-               }
-
+       if (notit->meta.sc->packets_have_ts_end) {
                BT_ASSERT(notit->snapshots.end_clock != UINT64_C(-1));
                msg = bt_message_packet_end_create_with_default_clock_snapshot(
                        notit->msg_iter, notit->packet,
@@ -2600,29 +2598,27 @@ void create_msg_discarded_events(struct bt_msg_iter *notit,
        bt_message *msg;
        uint64_t beginning_raw_value = UINT64_C(-1);
        uint64_t end_raw_value = UINT64_C(-1);
-       uint64_t count = UINT64_C(-1);
 
        BT_ASSERT(notit->msg_iter);
        BT_ASSERT(notit->stream);
+       BT_ASSERT(notit->meta.sc->has_discarded_events);
 
-       if (notit->prev_packet_snapshots.discarded_events == UINT64_C(-1)) {
-               /*
-                * We discarded events, but before (and possibly
-                * including) the current packet: use this packet's time
-                * range, and do not have a specific count.
-                */
-               beginning_raw_value = notit->snapshots.beginning_clock;
-               end_raw_value = notit->snapshots.end_clock;
-       } else {
-               count = notit->snapshots.discarded_events -
-                       notit->prev_packet_snapshots.discarded_events;
-               BT_ASSERT(count > 0);
-               beginning_raw_value = notit->prev_packet_snapshots.end_clock;
-               end_raw_value = notit->snapshots.end_clock;
-       }
+       if (notit->meta.sc->discarded_events_have_default_cs) {
+               if (notit->prev_packet_snapshots.discarded_events == UINT64_C(-1)) {
+                       /*
+                        * We discarded events, but before (and possibly
+                        * including) the current packet: use this packet's time
+                        * range, and do not have a specific count.
+                        */
+                       beginning_raw_value = notit->snapshots.beginning_clock;
+                       end_raw_value = notit->snapshots.end_clock;
+               } else {
+                       beginning_raw_value = notit->prev_packet_snapshots.end_clock;
+                       end_raw_value = notit->snapshots.end_clock;
+               }
 
-       if (beginning_raw_value != UINT64_C(-1) &&
-                       end_raw_value != UINT64_C(-1)) {
+               BT_ASSERT(beginning_raw_value != UINT64_C(-1));
+               BT_ASSERT(end_raw_value != UINT64_C(-1));
                msg = bt_message_discarded_events_create_with_default_clock_snapshots(
                        notit->msg_iter, notit->stream, beginning_raw_value,
                        end_raw_value);
@@ -2638,8 +2634,10 @@ void create_msg_discarded_events(struct bt_msg_iter *notit,
                return;
        }
 
-       if (count != UINT64_C(-1)) {
-               bt_message_discarded_events_set_count(msg, count);
+       if (notit->prev_packet_snapshots.discarded_events != UINT64_C(-1)) {
+               bt_message_discarded_events_set_count(msg,
+                       notit->snapshots.discarded_events -
+                       notit->prev_packet_snapshots.discarded_events);
        }
 
        *message = msg;
@@ -2653,11 +2651,13 @@ void create_msg_discarded_packets(struct bt_msg_iter *notit,
 
        BT_ASSERT(notit->msg_iter);
        BT_ASSERT(notit->stream);
+       BT_ASSERT(notit->meta.sc->has_discarded_packets);
        BT_ASSERT(notit->prev_packet_snapshots.packets !=
                UINT64_C(-1));
 
-       if (notit->prev_packet_snapshots.end_clock != UINT64_C(-1) &&
-                       notit->snapshots.beginning_clock != UINT64_C(-1)) {
+       if (notit->meta.sc->discarded_packets_have_default_cs) {
+               BT_ASSERT(notit->prev_packet_snapshots.end_clock != UINT64_C(-1));
+               BT_ASSERT(notit->snapshots.beginning_clock != UINT64_C(-1));
                msg = bt_message_discarded_packets_create_with_default_clock_snapshots(
                        notit->msg_iter, notit->stream,
                        notit->prev_packet_snapshots.end_clock,
diff --git a/tests/ctf-traces/succeed/ev-disc-no-ts-begin-end/metadata b/tests/ctf-traces/succeed/ev-disc-no-ts-begin-end/metadata
new file mode 100644 (file)
index 0000000..1b4803b
--- /dev/null
@@ -0,0 +1,21 @@
+/* CTF 1.8 */
+
+trace {
+       major = 1;
+       minor = 8;
+       byte_order = be;
+};
+
+stream {
+       packet.context := struct {
+               integer { size = 8; } packet_size;
+               integer { size = 8; } events_discarded;
+       };
+};
+
+event {
+       name = "ev";
+       fields := struct {
+               string s;
+       };
+};
diff --git a/tests/ctf-traces/succeed/ev-disc-no-ts-begin-end/stream b/tests/ctf-traces/succeed/ev-disc-no-ts-begin-end/stream
new file mode 100644 (file)
index 0000000..b42129a
Binary files /dev/null and b/tests/ctf-traces/succeed/ev-disc-no-ts-begin-end/stream differ
diff --git a/tests/ctf-traces/succeed/no-packet-context/metadata b/tests/ctf-traces/succeed/no-packet-context/metadata
new file mode 100644 (file)
index 0000000..c811432
--- /dev/null
@@ -0,0 +1,14 @@
+/* CTF 1.8 */
+
+trace {
+       major = 1;
+       minor = 8;
+       byte_order = be;
+};
+
+event {
+       name = "ev";
+       fields := struct {
+               string s;
+       };
+};
diff --git a/tests/ctf-traces/succeed/no-packet-context/stream b/tests/ctf-traces/succeed/no-packet-context/stream
new file mode 100644 (file)
index 0000000..0db2632
Binary files /dev/null and b/tests/ctf-traces/succeed/no-packet-context/stream differ
This page took 0.032303 seconds and 4 git commands to generate.