From: Philippe Proulx Date: Fri, 31 May 2019 19:08:37 +0000 (-0400) Subject: lib: make packet beginning/end default CS optional X-Git-Url: http://git.efficios.com/?p=babeltrace.git;a=commitdiff_plain;h=649934d212dbc58e6d11dbd9eae6bdb931e65522 lib: make packet beginning/end default CS optional This patch makes it possible to create packet beginning and end messages using bt_message_packet_beginning_create() and bt_message_packet_end_create(), that is, without default clock snapshots, even if the corresponding stream class has a default clock snapshot. This was considered like a precondition break before this patch. Rationale ========= The main use case for this is to support sources which do not have a concept of beginning and end times for their packets, but which have times for their events. Currently, such sources need to make up default clock snapshot values for those packet messages. This can be the case of `src.ctf.fs` which, when a given stream class's packet context does not contain the `timestamp_begin` and `timestamp_end` members, uses the current default clock's value during the stream decoding process, except for the beginning time of the first packet which is set to 0. This is problematic because it means the stream is active from clock value 0 to the first event, which is probably not true. A solution would be to use the first event's time in this scenario, but: * Several packets can be empty at the beginning, making the task difficult because we need to queue pseudo-messages until we have the first event's time and then create the real messages with this value. * All packets can be empty, so no specific time value is available for the whole stream. Although they are weird scenarios, they can still happen. Using 0 for the first packet's beginning time is just wrong in those cases. Other sources which do not need the concept of a packet can use this new feature too: `src.text.dmesg` is one of them. Library changes =============== As with other features of stream-related objects, the default clock snapshot property is not optional _per_ packet beginning/end message: for the streams of a given stream class, either all packet beginning and/or end messages have a default clock snapshot, or they don't. This is controlled by a new stream class flag of which the API is: void bt_stream_class_set_packets_have_default_beginning_clock_snapshot( bt_stream_class *stream_class, bt_bool value); void bt_stream_class_set_packets_have_default_end_clock_snapshot( bt_stream_class *stream_class, bt_bool value); bt_bool bt_stream_class_packets_have_default_beginning_clock_snapshot( const bt_stream_class *stream_class); bt_bool bt_stream_class_packets_have_default_end_clock_snapshot( const bt_stream_class *stream_class); By default, a stream class has no default clock class, so its packets are also known to have no beginning/end default clock snapshots. When you call bt_stream_class_set_packets_have_default_beginning_clock_snapshot() or bt_stream_class_set_packets_have_default_end_clock_snapshot(), you must have called bt_stream_class_set_default_clock_class() first. This means you can check if a packet beginning message, for example, has a default clock snapshot with bt_stream_class_set_packets_have_default_beginning_clock_snapshot() without also checking bt_stream_class_borrow_default_clock_class_const(). It is required that you have called bt_stream_class_set_packets_have_default_beginning_clock_snapshot() in order to use bt_message_packet_beginning_create_with_default_clock_snapshot() for the corresponding stream class. Same thing for bt_stream_class_set_packets_have_default_end_clock_snapshot() and bt_message_packet_end_create_with_default_clock_snapshot(). This is all validated in developer mode. Plugin changes ============== Component classes are changed as such: `src.ctf.fs`: `src.ctf.lttng-live`: In order to separate concerns into different patches, the procedure is not changed in this component class: all packet beginning/end messages have a default clock snapshot when the corresponding stream class has a default clock class, even if the stream class's packet context does not contain the `timestamp_begin` and `timestamp_end` members. Another patch will fix this issue. `src.text.dmesg`: Like `src.ctf.fs`, the procedure is not changed: all packet beginning/end messages have a default clock snapshot when the corresponding stream class has a default clock class. The simplification change will be done in another patch. `flt.utils.trimmer`: Stream classes without a default clock class are already not supported. With this patch, packet beginning/end messages without a default clock snapshot are not supported. The support will be added by another patch. `flt.utils.muxer`: When a packet beginning/end message does not have a default clock snapshot, the iterator's last time is used to sort the messages. This is similar to other messages without a default clock snapshot. `flt.lttng-utils.debug-info`: Stream class properties and packet beginning/end messages are copied considering the new feature. `sink.ctf.fs`: Packet beginning/end without a default clock snapshot, but with a stream class which has a default clock class, are not supported. Support will be added by another patch. All the tests still pass because, even though some filter/sink components do not fully support the new feature, the existing sources do not use it yet. Signed-off-by: Philippe Proulx Change-Id: I7183bfe8f954235c7f54195b101f781b176ab733 Reviewed-on: https://review.lttng.org/c/babeltrace/+/1361 Tested-by: jenkins --- diff --git a/include/babeltrace/trace-ir/stream-class-const.h b/include/babeltrace/trace-ir/stream-class-const.h index c8fe18f9..2b3da093 100644 --- a/include/babeltrace/trace-ir/stream-class-const.h +++ b/include/babeltrace/trace-ir/stream-class-const.h @@ -56,6 +56,12 @@ extern bt_bool bt_stream_class_assigns_automatic_event_class_id( extern bt_bool bt_stream_class_assigns_automatic_stream_id( const bt_stream_class *stream_class); +extern bt_bool bt_stream_class_packets_have_default_beginning_clock_snapshot( + const bt_stream_class *stream_class); + +extern bt_bool bt_stream_class_packets_have_default_end_clock_snapshot( + const bt_stream_class *stream_class); + extern uint64_t bt_stream_class_get_id( const bt_stream_class *stream_class); diff --git a/include/babeltrace/trace-ir/stream-class-internal.h b/include/babeltrace/trace-ir/stream-class-internal.h index 72493bc2..d24b5d6a 100644 --- a/include/babeltrace/trace-ir/stream-class-internal.h +++ b/include/babeltrace/trace-ir/stream-class-internal.h @@ -48,6 +48,8 @@ struct bt_stream_class { uint64_t id; bool assigns_automatic_event_class_id; bool assigns_automatic_stream_id; + bool packets_have_default_beginning_clock_snapshot; + bool packets_have_default_end_clock_snapshot; struct bt_field_class *packet_context_fc; struct bt_field_class *event_common_context_fc; struct bt_clock_class *default_clock_class; diff --git a/include/babeltrace/trace-ir/stream-class.h b/include/babeltrace/trace-ir/stream-class.h index 863f2936..ac7ca664 100644 --- a/include/babeltrace/trace-ir/stream-class.h +++ b/include/babeltrace/trace-ir/stream-class.h @@ -60,6 +60,12 @@ extern void bt_stream_class_set_assigns_automatic_event_class_id( extern void bt_stream_class_set_assigns_automatic_stream_id( bt_stream_class *stream_class, bt_bool value); +extern void bt_stream_class_set_packets_have_default_beginning_clock_snapshot( + bt_stream_class *stream_class, bt_bool value); + +extern void bt_stream_class_set_packets_have_default_end_clock_snapshot( + bt_stream_class *stream_class, bt_bool value); + extern bt_stream_class_status bt_stream_class_set_packet_context_field_class( bt_stream_class *stream_class, diff --git a/lib/graph/message/packet.c b/lib/graph/message/packet.c index ee378e58..df3e0e99 100644 --- a/lib/graph/message/packet.c +++ b/lib/graph/message/packet.c @@ -82,11 +82,12 @@ static inline struct bt_message *create_packet_message( struct bt_self_component_port_input_message_iterator *msg_iter, struct bt_packet *packet, struct bt_object_pool *pool, - bool with_cs, uint64_t raw_value) + bool with_cs, bool is_beginning, uint64_t raw_value) { struct bt_message_packet *message = NULL; struct bt_stream *stream; struct bt_stream_class *stream_class; + bool packet_has_default_clock_snapshot; BT_ASSERT(msg_iter); BT_ASSERT_PRE_NON_NULL(packet, "Packet"); @@ -94,8 +95,21 @@ struct bt_message *create_packet_message( BT_ASSERT(stream); stream_class = bt_stream_borrow_class(stream); BT_ASSERT(stream_class); - BT_ASSERT_PRE((with_cs && stream_class->default_clock_class) || - (!with_cs && !stream_class->default_clock_class), + + if (is_beginning) { + packet_has_default_clock_snapshot = + stream_class->packets_have_default_beginning_clock_snapshot; + } else { + packet_has_default_clock_snapshot = + stream_class->packets_have_default_end_clock_snapshot; + } + + /* + * `packet_has_default_clock_snapshot` implies that the stream + * class has a default clock class (precondition). + */ + BT_ASSERT_PRE((with_cs && packet_has_default_clock_snapshot) || + (!with_cs && !packet_has_default_clock_snapshot), "Creating a packet message with a default clock snapshot, but without " "a default clock class, or without a default clock snapshot, " "but with a default clock class: ", @@ -147,7 +161,7 @@ struct bt_message *bt_message_packet_beginning_create( BT_ASSERT_PRE_NON_NULL(msg_iter, "Message iterator"); return create_packet_message(msg_iter, (void *) packet, - &msg_iter->graph->packet_begin_msg_pool, false, 0); + &msg_iter->graph->packet_begin_msg_pool, false, true, 0); } struct bt_message *bt_message_packet_beginning_create_with_default_clock_snapshot( @@ -159,7 +173,7 @@ struct bt_message *bt_message_packet_beginning_create_with_default_clock_snapsho BT_ASSERT_PRE_NON_NULL(msg_iter, "Message iterator"); return create_packet_message(msg_iter, (void *) packet, - &msg_iter->graph->packet_begin_msg_pool, true, raw_value); + &msg_iter->graph->packet_begin_msg_pool, true, true, raw_value); } struct bt_message *bt_message_packet_end_create( @@ -171,7 +185,7 @@ struct bt_message *bt_message_packet_end_create( BT_ASSERT_PRE_NON_NULL(msg_iter, "Message iterator"); return create_packet_message(msg_iter, (void *) packet, - &msg_iter->graph->packet_end_msg_pool, false, 0); + &msg_iter->graph->packet_end_msg_pool, false, false, 0); } struct bt_message *bt_message_packet_end_create_with_default_clock_snapshot( @@ -183,7 +197,7 @@ struct bt_message *bt_message_packet_end_create_with_default_clock_snapshot( BT_ASSERT_PRE_NON_NULL(msg_iter, "Message iterator"); return create_packet_message(msg_iter, (void *) packet, - &msg_iter->graph->packet_end_msg_pool, true, raw_value); + &msg_iter->graph->packet_end_msg_pool, true, false, raw_value); } BT_HIDDEN diff --git a/lib/trace-ir/stream-class.c b/lib/trace-ir/stream-class.c index e4f48f4e..89470b5b 100644 --- a/lib/trace-ir/stream-class.c +++ b/lib/trace-ir/stream-class.c @@ -451,6 +451,46 @@ void bt_stream_class_set_assigns_automatic_event_class_id( "assignment property: %!+S", stream_class); } +bt_bool bt_stream_class_packets_have_default_beginning_clock_snapshot( + const struct bt_stream_class *stream_class) +{ + BT_ASSERT_PRE_NON_NULL(stream_class, "Stream class"); + return (bt_bool) stream_class->packets_have_default_beginning_clock_snapshot; +} + +void bt_stream_class_set_packets_have_default_beginning_clock_snapshot( + struct bt_stream_class *stream_class, bt_bool value) +{ + BT_ASSERT_PRE_NON_NULL(stream_class, "Stream class"); + BT_ASSERT_PRE_STREAM_CLASS_HOT(stream_class); + BT_ASSERT_PRE(!value || stream_class->default_clock_class, + "Stream class has no default clock class: %!+S", stream_class); + stream_class->packets_have_default_beginning_clock_snapshot = + (bool) value; + BT_LIB_LOGV("Set stream class's \"packets have default beginning " + "clock snapshot\" property: %!+S", stream_class); +} + +bt_bool bt_stream_class_packets_have_default_end_clock_snapshot( + const struct bt_stream_class *stream_class) +{ + BT_ASSERT_PRE_NON_NULL(stream_class, "Stream class"); + return (bt_bool) stream_class->packets_have_default_end_clock_snapshot; +} + +void bt_stream_class_set_packets_have_default_end_clock_snapshot( + struct bt_stream_class *stream_class, bt_bool value) +{ + BT_ASSERT_PRE_NON_NULL(stream_class, "Stream class"); + BT_ASSERT_PRE_STREAM_CLASS_HOT(stream_class); + BT_ASSERT_PRE(!value || stream_class->default_clock_class, + "Stream class has no default clock class: %!+S", stream_class); + stream_class->packets_have_default_end_clock_snapshot = + (bool) value; + BT_LIB_LOGV("Set stream class's \"packets have default end " + "clock snapshot\" property: %!+S", stream_class); +} + bt_bool bt_stream_class_assigns_automatic_stream_id( const struct bt_stream_class *stream_class) { diff --git a/plugins/ctf/common/metadata/ctf-meta-translate.c b/plugins/ctf/common/metadata/ctf-meta-translate.c index b116c053..e8486f96 100644 --- a/plugins/ctf/common/metadata/ctf-meta-translate.c +++ b/plugins/ctf/common/metadata/ctf-meta-translate.c @@ -485,6 +485,10 @@ void ctf_stream_class_to_ir(struct ctx *ctx) ret = bt_stream_class_set_default_clock_class(ctx->ir_sc, ctx->sc->default_clock_class->ir_cc); BT_ASSERT(ret == 0); + bt_stream_class_set_packets_have_default_beginning_clock_snapshot( + ctx->ir_sc, BT_TRUE); + bt_stream_class_set_packets_have_default_end_clock_snapshot( + ctx->ir_sc, BT_TRUE); } ctx->sc->is_translated = true; diff --git a/plugins/ctf/fs-sink/fs-sink.c b/plugins/ctf/fs-sink/fs-sink.c index 83742ed1..da3307fe 100644 --- a/plugins/ctf/fs-sink/fs-sink.c +++ b/plugins/ctf/fs-sink/fs-sink.c @@ -516,8 +516,44 @@ bt_self_component_status handle_stream_beginning_msg( bt_self_component_status status = BT_SELF_COMPONENT_STATUS_OK; const bt_stream *ir_stream = bt_message_stream_beginning_borrow_stream_const(msg); + const bt_stream_class *ir_sc = + bt_stream_borrow_class_const(ir_stream); struct fs_sink_stream *stream; + /* + * Temporary: if the stream's class has a default clock class, + * make sure packet beginning and end messages have default + * clock snapshots until the support for not having them is + * implemented. + */ + if (bt_stream_class_borrow_default_clock_class_const(ir_sc)) { + if (!bt_stream_class_packets_have_default_beginning_clock_snapshot( + ir_sc)) { + BT_LOGE("Unsupported stream: packets have " + "no beginning clock snapshot: " + "stream-addr=%p, " + "stream-id=%" PRIu64 ", " + "stream-name=\"%s\"", + ir_stream, bt_stream_get_id(ir_stream), + bt_stream_get_name(ir_stream)); + status = BT_SELF_MESSAGE_ITERATOR_STATUS_ERROR; + goto end; + } + + if (!bt_stream_class_packets_have_default_end_clock_snapshot( + ir_sc)) { + BT_LOGE("Unsupported stream: packets have " + "no end clock snapshot: " + "stream-addr=%p, " + "stream-id=%" PRIu64 ", " + "stream-name=\"%s\"", + ir_stream, bt_stream_get_id(ir_stream), + bt_stream_get_name(ir_stream)); + status = BT_SELF_MESSAGE_ITERATOR_STATUS_ERROR; + goto end; + } + } + stream = borrow_stream(fs_sink, ir_stream); if (!stream) { status = BT_SELF_COMPONENT_STATUS_ERROR; diff --git a/plugins/lttng-utils/debug-info/debug-info.c b/plugins/lttng-utils/debug-info/debug-info.c index 4c2b7b22..1859d273 100644 --- a/plugins/lttng-utils/debug-info/debug-info.c +++ b/plugins/lttng-utils/debug-info/debug-info.c @@ -1334,7 +1334,7 @@ static bt_message *handle_packet_begin_message(struct debug_info_msg_iter *debug_it, const bt_message *in_message) { - const bt_clock_class *default_cc; + bool has_default_clock_snapshot; const bt_clock_snapshot *cs; bt_message *out_message = NULL; bt_packet *out_packet; @@ -1352,10 +1352,11 @@ bt_message *handle_packet_begin_message(struct debug_info_msg_iter *debug_it, BT_ASSERT(out_packet); - default_cc = bt_stream_class_borrow_default_clock_class_const( + has_default_clock_snapshot = + bt_stream_class_packets_have_default_beginning_clock_snapshot( bt_stream_borrow_class_const( bt_packet_borrow_stream_const(in_packet))); - if (default_cc) { + if (has_default_clock_snapshot) { /* Borrow clock snapshot. */ cs = bt_message_packet_beginning_borrow_default_clock_snapshot_const( in_message); @@ -1380,9 +1381,9 @@ static bt_message *handle_packet_end_message(struct debug_info_msg_iter *debug_it, const bt_message *in_message) { + bool has_default_clock_snapshot; const bt_clock_snapshot *cs; const bt_packet *in_packet; - const bt_clock_class *default_cc; bt_message *out_message = NULL; bt_packet *out_packet; @@ -1392,10 +1393,11 @@ bt_message *handle_packet_end_message(struct debug_info_msg_iter *debug_it, out_packet = trace_ir_mapping_borrow_mapped_packet(debug_it->ir_maps, in_packet); BT_ASSERT(out_packet); - default_cc = bt_stream_class_borrow_default_clock_class_const( + has_default_clock_snapshot = + bt_stream_class_packets_have_default_end_clock_snapshot( bt_stream_borrow_class_const( bt_packet_borrow_stream_const(in_packet))); - if (default_cc) { + if (has_default_clock_snapshot) { /* Borrow clock snapshot. */ cs = bt_message_packet_end_borrow_default_clock_snapshot_const( in_message); diff --git a/plugins/lttng-utils/debug-info/trace-ir-metadata-copy.c b/plugins/lttng-utils/debug-info/trace-ir-metadata-copy.c index af6c8742..c07b7bdd 100644 --- a/plugins/lttng-utils/debug-info/trace-ir-metadata-copy.c +++ b/plugins/lttng-utils/debug-info/trace-ir-metadata-copy.c @@ -255,6 +255,15 @@ int copy_stream_class_content(struct trace_ir_maps *ir_maps, } + bt_stream_class_set_packets_have_default_beginning_clock_snapshot( + out_stream_class, + bt_stream_class_packets_have_default_beginning_clock_snapshot( + in_stream_class)); + bt_stream_class_set_packets_have_default_end_clock_snapshot( + out_stream_class, + bt_stream_class_packets_have_default_end_clock_snapshot( + in_stream_class)); + in_name = bt_stream_class_get_name(in_stream_class); if (in_name) { status = bt_stream_class_set_name(out_stream_class, in_name); diff --git a/plugins/text/dmesg/dmesg.c b/plugins/text/dmesg/dmesg.c index c93fa78e..4e0bb16a 100644 --- a/plugins/text/dmesg/dmesg.c +++ b/plugins/text/dmesg/dmesg.c @@ -161,6 +161,11 @@ int create_meta(struct dmesg_component *dmesg_comp, bool has_ts) BT_LOGE_STR("Cannot set stream class's default clock class."); goto error; } + + bt_stream_class_set_packets_have_default_beginning_clock_snapshot( + dmesg_comp->stream_class, BT_TRUE); + bt_stream_class_set_packets_have_default_end_clock_snapshot( + dmesg_comp->stream_class, BT_TRUE); } dmesg_comp->event_class = bt_event_class_create( diff --git a/plugins/utils/muxer/muxer.c b/plugins/utils/muxer/muxer.c index b0df3bae..de917806 100644 --- a/plugins/utils/muxer/muxer.c +++ b/plugins/utils/muxer/muxer.c @@ -506,6 +506,8 @@ int get_msg_ts_ns(struct muxer_comp *muxer_comp, const bt_clock_snapshot *clock_snapshot = NULL; int ret = 0; bt_message_stream_activity_clock_snapshot_state sa_cs_state; + const bt_stream_class *stream_class = NULL; + bt_message_type msg_type; BT_ASSERT(msg); BT_ASSERT(ts_ns); @@ -520,7 +522,21 @@ int get_msg_ts_ns(struct muxer_comp *muxer_comp, goto end; } - switch (bt_message_get_type(msg)) { + msg_type = bt_message_get_type(msg); + + if (unlikely(msg_type == BT_MESSAGE_TYPE_PACKET_BEGINNING)) { + stream_class = bt_stream_borrow_class_const( + bt_packet_borrow_stream_const( + bt_message_packet_beginning_borrow_packet_const( + msg))); + } else if (unlikely(msg_type == BT_MESSAGE_TYPE_PACKET_END)) { + stream_class = bt_stream_borrow_class_const( + bt_packet_borrow_stream_const( + bt_message_packet_end_borrow_packet_const( + msg))); + } + + switch (msg_type) { case BT_MESSAGE_TYPE_EVENT: BT_ASSERT(bt_message_event_borrow_stream_class_default_clock_class_const( msg)); @@ -528,16 +544,24 @@ int get_msg_ts_ns(struct muxer_comp *muxer_comp, msg); break; case BT_MESSAGE_TYPE_PACKET_BEGINNING: - BT_ASSERT(bt_message_packet_beginning_borrow_stream_class_default_clock_class_const( - msg)); - clock_snapshot = bt_message_packet_beginning_borrow_default_clock_snapshot_const( - msg); + if (bt_stream_class_packets_have_default_beginning_clock_snapshot( + stream_class)) { + clock_snapshot = bt_message_packet_beginning_borrow_default_clock_snapshot_const( + msg); + } else { + goto no_clock_snapshot; + } + break; case BT_MESSAGE_TYPE_PACKET_END: - BT_ASSERT(bt_message_packet_end_borrow_stream_class_default_clock_class_const( - msg)); - clock_snapshot = bt_message_packet_end_borrow_default_clock_snapshot_const( - msg); + if (bt_stream_class_packets_have_default_end_clock_snapshot( + stream_class)) { + clock_snapshot = bt_message_packet_end_borrow_default_clock_snapshot_const( + msg); + } else { + goto no_clock_snapshot; + } + break; case BT_MESSAGE_TYPE_DISCARDED_EVENTS: BT_ASSERT(bt_message_discarded_events_borrow_stream_class_default_clock_class_const( diff --git a/plugins/utils/trimmer/trimmer.c b/plugins/utils/trimmer/trimmer.c index 4d52e22e..db5b65b3 100644 --- a/plugins/utils/trimmer/trimmer.c +++ b/plugins/utils/trimmer/trimmer.c @@ -1733,6 +1733,38 @@ bt_self_message_iterator_status handle_message( goto end; } + /* + * Temporary: make sure packet beginning and end + * messages have default clock snapshots until + * the support for not having them is + * implemented. + */ + if (!bt_stream_class_packets_have_default_beginning_clock_snapshot( + sc)) { + BT_LOGE("Unsupported stream: packets have " + "no beginning clock snapshot: " + "stream-addr=%p, " + "stream-id=%" PRIu64 ", " + "stream-name=\"%s\"", + stream, bt_stream_get_id(stream), + bt_stream_get_name(stream)); + status = BT_SELF_MESSAGE_ITERATOR_STATUS_ERROR; + goto end; + } + + if (!bt_stream_class_packets_have_default_end_clock_snapshot( + sc)) { + BT_LOGE("Unsupported stream: packets have " + "no end clock snapshot: " + "stream-addr=%p, " + "stream-id=%" PRIu64 ", " + "stream-name=\"%s\"", + stream, bt_stream_get_id(stream), + bt_stream_get_name(stream)); + status = BT_SELF_MESSAGE_ITERATOR_STATUS_ERROR; + goto end; + } + sstate = g_new0(struct trimmer_iterator_stream_state, 1); if (!sstate) {