tests: add `src.ctf.fs` single field testing framework
authorOlivier Dion <odion@efficios.com>
Wed, 18 Oct 2023 15:02:30 +0000 (11:02 -0400)
committerPhilippe Proulx <eeppeliteloop@gmail.com>
Wed, 7 Feb 2024 20:45:48 +0000 (15:45 -0500)
This patch adds a framework to test the decoding of single CTF 1.8
fields by a `src.ctf.fs` component.

Single CTF 1.8 field tests reside as moultipart documents (see
`tests/utils/python/moultipart.py`) in
`tests/data/plugins/src.ctf.fs/field/ctf-1`.

The new `test-field.sh` script does the following, for each `pass-*.mp`
file found in `tests/data/plugins/src.ctf.fs/field/ctf-1`:

1. Creates a temporary directory TD to hold a CTF 1.8 trace (`TD/trace`)
   and an expectation file (`TD/expect`).

2. Runs `data_from_mp.py` on the moultipart document, also passing TD.

   This script splits the moultipart document to produce `TD/trace` and
   `TD/expect`. The expected parts are, in this order:

   a) A TSDL field class (metadata), without any field name, for
      example:

          integer { size = 32; byte_order = be; }

      If you need to test the decoding of an array field, you may use
      `@` in this part to indicate to `data_from_mp.py` where the
      field name goes, for example:

          string @[23]

   b) A Normand [1] text representing the exact data of an instance
      of a).

   c) What you expect the `sink.test-text.single` component to print
      when it receives an event message containing what a `src.ctf.fs`
      component decoded from `TD/trace`.

      `sink.test-text.single` prints exactly the value of b) with a
      YAML-like format.

3. Runs something like this, keeping the standard output text:

       $ babeltrace2 -c sink.test-text.single "TD/trace"

4. Compares the result of step 3 with `TD/expect` using bt_diff().

Now it becomes easy to test regular and corner cases of our CTF 1.8
decoding by adding new files to
`tests/data/plugins/src.ctf.fs/field/ctf-1`.

As initial examples, `tests/data/plugins/src.ctf.fs/field/ctf-1`
contains a few single field tests already.

Philippe changes:

• Made some style adjustments to the original patch.

• Using `mktemp -d` instead of `mktemp --directory` because macOS
  doesn't seem to know the latter.

• In `utils.sh`, changed bt_diff() to use the new bt_remove_cr_inline()
  to remove CR characters from both files because this is the first time
  the test generates both files.

  Leaving bt_remove_cr() as is because another test uses it directly.

• In the _print_field() function of `bt_plugin_test_text.py`:

  ‣ Changed parts of the strategy to fix some rendering bugs.
  ‣ Using int() for an integer field to avoid printing enum. labels.
  ‣ Added the empty structure field special case.
  ‣ Added comments with output examples.

• In `data_from_mp.py`:

  ‣ Added lots of useful TSDL type aliases available to any CTF 1
    single field test.

  ‣ Made the default Normand byte order little-endian since it's also
    the default TSDL byte order in _make_ctf_1_metadata().

• In `test-field.sh`, using a hard-coded test count.

  See the Gerrit discussion [2] for the rationale.

• Added more initial tests in
  `tests/data/plugins/src.ctf.fs/field/ctf-1`.

[1]: https://github.com/efficios/normand
[2]: https://review.lttng.org/c/babeltrace/+/11149

Change-Id: I7539b46d49200b5e75fe3525401b47b8ff418f6c
Signed-off-by: Olivier Dion <odion@efficios.com>
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/11149
Tested-by: jenkins <jenkins@lttng.org>
12 files changed:
tests/Makefile.am
tests/data/plugins/src.ctf.fs/field/bt_plugin_test_text.py [new file with mode: 0644]
tests/data/plugins/src.ctf.fs/field/ctf-1/pass-fixed-len-uint-32-be.mp [new file with mode: 0644]
tests/data/plugins/src.ctf.fs/field/ctf-1/pass-fixed-len-uint-32-le.mp [new file with mode: 0644]
tests/data/plugins/src.ctf.fs/field/ctf-1/pass-static-len-array-of-struct.mp [new file with mode: 0644]
tests/data/plugins/src.ctf.fs/field/ctf-1/pass-struct-empty.mp [new file with mode: 0644]
tests/data/plugins/src.ctf.fs/field/ctf-1/pass-struct.mp [new file with mode: 0644]
tests/data/plugins/src.ctf.fs/field/ctf-1/pass-variant.mp [new file with mode: 0644]
tests/data/plugins/src.ctf.fs/field/data_from_mp.py [new file with mode: 0644]
tests/plugins/src.ctf.fs/Makefile.am
tests/plugins/src.ctf.fs/field/test-field.sh [new file with mode: 0755]
tests/utils/utils.sh

index a24f32083ae536fe26c904e94e46bc2a79ae5925..a9200a0c687c53b726ec0f73edce8db68555839c 100644 (file)
@@ -162,7 +162,8 @@ TESTS_CLI += \
 
 TESTS_PLUGINS += plugins/flt.utils.trimmer/test-trimming.sh \
        plugins/flt.utils.muxer/succeed/test-succeed.sh \
-       plugins/sink.text.pretty/test-enum.sh
+       plugins/sink.text.pretty/test-enum.sh \
+       plugins/src.ctf.fs/field/test-field.sh
 endif
 endif
 
diff --git a/tests/data/plugins/src.ctf.fs/field/bt_plugin_test_text.py b/tests/data/plugins/src.ctf.fs/field/bt_plugin_test_text.py
new file mode 100644 (file)
index 0000000..e177798
--- /dev/null
@@ -0,0 +1,154 @@
+# SPDX-License-Identifier: MIT
+#
+# Copyright (c) 2023 Olivier Dion <odion@efficios.com>
+# Copyright (c) 2024 Philippe Proulx <pproulx@efficios.com>
+
+import bt2
+
+_array_elem = object()
+
+
+# Recursively prints the contents of `field` with the indentation level
+# `indent` and the introduction `intro`.
+#
+# `intro` is one of:
+#
+# `None`:
+#     No introduction (root field).
+#
+# A string:
+#     Structure field member name.
+#
+# `_array_elem`:
+#     `field` is an array field element.
+def _print_field(intro, field, indent=0):
+    indent_str = " " * indent * 2
+    intro_str = ""
+
+    if intro is _array_elem:
+        intro_str = "- "
+    elif intro is not None:
+        intro_str = "{}: ".format(intro)
+
+    if isinstance(field, bt2._StringFieldConst):
+        print('{}{}"{}"'.format(indent_str, intro_str, field))
+    elif isinstance(field, bt2._StructureFieldConst):
+        print(indent_str + intro_str, end="")
+
+        if len(field) == 0:
+            # Special case for an empty structure field
+            print("{}")
+        else:
+            if intro is _array_elem:
+                # Structure field is an array field element itself:
+                # print the first element first, and then print the
+                # remaining ones indented.
+                #
+                # Example:
+                #
+                #     - meow: "mix"
+                #       montant: -23.599312
+                #       bateau: "jacques"
+                sub_field_names = list(field)
+                _print_field(sub_field_names[0], field[sub_field_names[0]], 0)
+
+                for sub_field_name in sub_field_names[1:]:
+                    _print_field(sub_field_name, field[sub_field_name], indent + 1)
+            else:
+                add_indent = 0
+
+                if intro is not None:
+                    # Structure field has a name (already printed at
+                    # this point): print a newline, and then print all
+                    # the members indented (one more level):
+                    #
+                    # Example:
+                    #
+                    #     struct_name:
+                    #       meow: "mix"
+                    #       montant: -23.599312
+                    #       bateau: "jacques"
+                    add_indent = 1
+                    print()
+
+                for sub_field_name in field:
+                    _print_field(
+                        sub_field_name,
+                        field[sub_field_name],
+                        indent + add_indent,
+                    )
+    elif isinstance(field, bt2._ArrayFieldConst):
+        add_indent = 0
+
+        if intro is not None:
+            # Array field has an intro: print it, then print a newline,
+            # and then print all the elements indented (one more level).
+            #
+            # Example 1 (parent is an structure field):
+            #
+            #     array_name:
+            #       - -17
+            #       - "salut"
+            #       - 23
+            #
+            # Example 2 (parent is an array field):
+            #
+            #     -
+            #       - -17
+            #       - "salut"
+            #       - 23
+            add_indent = 1
+            print(indent_str + intro_str.rstrip())
+
+        for sub_field in field:
+            _print_field(_array_elem, sub_field, indent + add_indent)
+    elif isinstance(field.cls, bt2._IntegerFieldClassConst):
+        # Honor the preferred display base
+        base = field.cls.preferred_display_base
+        print(indent_str + intro_str, end="")
+
+        if base == 10:
+            print(int(field))
+        elif base == 16:
+            print(hex(field))
+        elif base == 8:
+            print(oct(field))
+        elif base == 2:
+            print(bin(field))
+    elif isinstance(field, bt2._BitArrayFieldConst):
+        print(indent_str + intro_str + bin(field))
+    elif isinstance(field, bt2._BoolFieldConst):
+        print(indent_str + intro_str + ("yes" if field else "no"))
+    elif isinstance(field, bt2._RealFieldConst):
+        print("{}{}{:.6f}".format(indent_str, intro_str, float(field)))
+    elif isinstance(field, bt2._OptionFieldConst):
+        if field.has_field:
+            _print_field(intro, field.field, indent)
+        else:
+            # Special case for an option field without a field
+            print("{}{}~".format(indent_str, intro_str))
+    elif isinstance(field, bt2._VariantFieldConst):
+        _print_field(intro, field.selected_option, indent)
+    else:
+        print(indent_str + intro_str + field)
+
+
+@bt2.plugin_component_class
+class _SingleSinkComponent(bt2._UserSinkComponent, name="single"):
+    def __init__(self, config, params, obj):
+        self._input = self._add_input_port("input")
+        self._field_name = str(params.get("field-name", "root"))
+
+    def _user_graph_is_configured(self):
+        self._it = self._create_message_iterator(self._input)
+
+    def _user_consume(self):
+        msg = next(self._it)
+
+        if type(msg) is bt2._EventMessageConst:
+            assert self._field_name in msg.event.payload_field
+            assert len(msg.event.payload_field) == 1
+            _print_field(None, msg.event.payload_field[self._field_name])
+
+
+bt2.register_plugin(__name__, "test-text")
diff --git a/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-fixed-len-uint-32-be.mp b/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-fixed-len-uint-32-be.mp
new file mode 100644 (file)
index 0000000..4e72ded
--- /dev/null
@@ -0,0 +1,8 @@
+---
+u32be
+
+---
+[3187239923:32be]
+
+---
+3187239923
diff --git a/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-fixed-len-uint-32-le.mp b/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-fixed-len-uint-32-le.mp
new file mode 100644 (file)
index 0000000..53364d1
--- /dev/null
@@ -0,0 +1,8 @@
+---
+u32le
+
+---
+[3187239923:32le]
+
+---
+3187239923
diff --git a/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-static-len-array-of-struct.mp b/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-static-len-array-of-struct.mp
new file mode 100644 (file)
index 0000000..4ed8683
--- /dev/null
@@ -0,0 +1,42 @@
+---
+struct {
+  struct {
+    u8 a;
+    nt_str b;
+  } x[3];
+} @[2]
+
+---
+01              # `a`
+"salut\0"       # `b`
+
+02              # `a`
+"patente\0"     # `b`
+
+03              # `a`
+"Quebec\0"      # `b`
+
+04              # `a`
+"chez nous\0"   # `b`
+
+05              # `a`
+"aidez-moi\0"   # `b`
+
+06              # `a`
+"rasseye\0"     # `b`
+
+---
+- x:
+  - a: 1
+    b: "salut"
+  - a: 2
+    b: "patente"
+  - a: 3
+    b: "Quebec"
+- x:
+  - a: 4
+    b: "chez nous"
+  - a: 5
+    b: "aidez-moi"
+  - a: 6
+    b: "rasseye"
diff --git a/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-struct-empty.mp b/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-struct-empty.mp
new file mode 100644 (file)
index 0000000..2665098
--- /dev/null
@@ -0,0 +1,17 @@
+---
+struct {
+  u8 a;
+
+  struct {
+  } b;
+
+  u8 c;
+}
+---
+01  # `a`
+02  # `c`
+
+---
+a: 1
+b: {}
+c: 2
diff --git a/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-struct.mp b/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-struct.mp
new file mode 100644 (file)
index 0000000..07ad223
--- /dev/null
@@ -0,0 +1,16 @@
+---
+struct {
+  i16 meow;
+  nt_str mix;
+  flt32 cat;
+}
+
+---
+[-1717:16]        # `meow`
+"rapidement!\0"   # `mix`
+[2.897771955:32]  # `cat`
+
+---
+meow: -1717
+mix: "rapidement!"
+cat: 2.897772
diff --git a/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-variant.mp b/tests/data/plugins/src.ctf.fs/field/ctf-1/pass-variant.mp
new file mode 100644 (file)
index 0000000..3b01754
--- /dev/null
@@ -0,0 +1,24 @@
+---
+struct {
+  enum : u8 {
+    MEOW,
+    MIX,
+  } tag;
+
+  variant <tag> {
+    u16 MEOW;
+    nt_str MIX;
+  } var;
+} @[2]
+
+---
+00                # `tag`
+[1995:16]         # `var`
+
+01                # `tag`
+"hello there!\0"  # `var`
+---
+- tag: 0
+  var: 1995
+- tag: 1
+  var: "hello there!"
diff --git a/tests/data/plugins/src.ctf.fs/field/data_from_mp.py b/tests/data/plugins/src.ctf.fs/field/data_from_mp.py
new file mode 100644 (file)
index 0000000..2a1a170
--- /dev/null
@@ -0,0 +1,112 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2023 EfficiOS Inc.
+#
+# pyright: strict, reportTypeCommentUsage=false
+
+import os
+import string
+import argparse
+
+import normand
+import moultipart
+
+
+def _make_ctf_1_metadata(payload_fc: str):
+    payload_fc = payload_fc.strip()
+
+    if "@" in payload_fc:
+        payload_fc = payload_fc.replace("@", "root")
+    else:
+        payload_fc += " root"
+
+    return string.Template(
+        """\
+/* CTF 1.8 */
+
+trace {
+    major = 1;
+    minor = 8;
+    byte_order = le;
+};
+
+typealias integer { size = 8; } := u8;
+typealias integer { size = 16; } := u16;
+typealias integer { size = 32; } := u32;
+typealias integer { size = 64; } := u64;
+typealias integer { size = 8; byte_order = le; } := u8le;
+typealias integer { size = 16; byte_order = le; } := u16le;
+typealias integer { size = 32; byte_order = le; } := u32le;
+typealias integer { size = 64; byte_order = le; } := u64le;
+typealias integer { size = 8; byte_order = be; } := u8be;
+typealias integer { size = 16; byte_order = be; } := u16be;
+typealias integer { size = 32; byte_order = be; } := u32be;
+typealias integer { size = 64; byte_order = be; } := u64be;
+typealias integer { signed = true; size = 8; } := i8;
+typealias integer { signed = true; size = 16; } := i16;
+typealias integer { signed = true; size = 32; } := i32;
+typealias integer { signed = true; size = 64; } := i64;
+typealias integer { signed = true; size = 8; byte_order = le; } := i8le;
+typealias integer { signed = true; size = 16; byte_order = le; } := i16le;
+typealias integer { signed = true; size = 32; byte_order = le; } := i32le;
+typealias integer { signed = true; size = 64; byte_order = le; } := i64le;
+typealias integer { signed = true; size = 8; byte_order = be; } := i8be;
+typealias integer { signed = true; size = 16; byte_order = be; } := i16be;
+typealias integer { signed = true; size = 32; byte_order = be; } := i32be;
+typealias integer { signed = true; size = 64; byte_order = be; } := i64be;
+typealias floating_point { exp_dig = 8; mant_dig = 24; } := flt32;
+typealias floating_point { exp_dig = 11; mant_dig = 53; } := flt64;
+typealias floating_point { exp_dig = 8; mant_dig = 24; byte_order = le; } := flt32le;
+typealias floating_point { exp_dig = 11; mant_dig = 53; byte_order = le; } := flt64le;
+typealias floating_point { exp_dig = 8; mant_dig = 24; byte_order = be; } := flt32be;
+typealias floating_point { exp_dig = 11; mant_dig = 53; byte_order = be; } := flt64be;
+typealias string { encoding = UTF8; } := nt_str;
+
+event {
+    name = the_event;
+    fields := struct {
+        ${payload_fc};
+    };
+};
+"""
+    ).substitute(payload_fc=payload_fc)
+
+
+def _make_ctf_1_data(normand_text: str):
+    # Default to little-endian because that's also the TSDL default in
+    # _make_ctf_1_metadata() above.
+    return normand.parse("!le\n" + normand_text).data
+
+
+def _create_files_from_mp(mp_path: str, output_dir: str):
+    trace_dir = os.path.join(output_dir, "trace")
+    expect_path = os.path.join(output_dir, "expect")
+    metadata_path = os.path.join(trace_dir, "metadata")
+    data_path = os.path.join(trace_dir, "data")
+    os.makedirs(trace_dir, exist_ok=True)
+
+    with open(mp_path, "r") as f:
+        parts = moultipart.parse(f)
+
+    with open(metadata_path, "w") as f:
+        f.write(_make_ctf_1_metadata(parts[0].content))
+
+    with open(data_path, "wb") as f:
+        f.write(_make_ctf_1_data(parts[1].content))
+
+    with open(expect_path, "w") as f:
+        f.write(parts[2].content)
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "mp_path", metavar="MP-PATH", help="moultipart document to process"
+    )
+    parser.add_argument(
+        "output_dir",
+        metavar="OUTPUT-DIR",
+        help="output directory for the CTF trace and expectation file",
+    )
+    args = parser.parse_args()
+    _create_files_from_mp(args.mp_path, args.output_dir)
index 02628d3db25955d0930b3ad89b2de7061491cc44..8c3673534a9e782e16b826a713adc8d9a229a130 100644 (file)
@@ -9,4 +9,5 @@ dist_check_SCRIPTS = \
        query/test_query_support_info.py \
        query/test-query-trace-info.sh \
        query/test_query_trace_info.py \
-       test-deterministic-ordering.sh
+       test-deterministic-ordering.sh \
+       field/test-field.sh
diff --git a/tests/plugins/src.ctf.fs/field/test-field.sh b/tests/plugins/src.ctf.fs/field/test-field.sh
new file mode 100755 (executable)
index 0000000..df035d3
--- /dev/null
@@ -0,0 +1,40 @@
+#!/bin/bash
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (C) 2023 Efficios, Inc.
+
+SH_TAP=1
+
+if [[ -n ${BT_TESTS_SRCDIR:-} ]]; then
+        UTILSSH=$BT_TESTS_SRCDIR/utils/utils.sh
+else
+        UTILSSH=$(dirname "$0")/../../../utils/utils.sh
+fi
+
+# shellcheck source=../../../utils/utils.sh
+source "$UTILSSH"
+
+# Directory containing the plugin
+data_dir=$BT_TESTS_DATADIR/plugins/src.ctf.fs/field
+
+test_pass() {
+    local -r mp_path=$1
+    local -r output_dir=$(mktemp -d)
+
+    run_python "$BT_TESTS_PYTHON_BIN" "$data_dir/data_from_mp.py" "$mp_path" "$output_dir"
+
+    local -r res_path=$(mktemp)
+
+    bt_cli "$res_path" /dev/null --plugin-path="$data_dir" \
+        -c sink.test-text.single "$output_dir/trace"
+    bt_diff "$res_path" "$output_dir/expect"
+    ok $? "$mp_path"
+    rm -rf "$output_dir" "$res_path"
+}
+
+plan_tests 6
+
+for mp_path in "$data_dir"/ctf-1/pass-*.mp; do
+    test_pass "$mp_path"
+done
index 18b5482362bac98e07e8bd7465c85e741873ee31..037f5307ae0006a82013df74bcd94f526ef232f4 100644 (file)
@@ -156,6 +156,10 @@ bt_remove_cr() {
        "$BT_TESTS_SED_BIN" -i'' -e 's/\r//g' "$1"
 }
 
+bt_remove_cr_inline() {
+       "$BT_TESTS_SED_BIN" 's/\r//g' "$1"
+}
+
 # Run the Babeltrace CLI, redirecting stdout and stderr to specified files.
 #
 #   $1: file to redirect stdout to
@@ -195,9 +199,7 @@ bt_diff() {
        # Strip any \r present due to Windows (\n -> \r\n).
        # "diff --string-trailing-cr" is not used since it is not present on
        # Solaris.
-       bt_remove_cr "$actual_file"
-
-       diff -u "$expected_file" "$actual_file" 1>&2
+       diff -u <(bt_remove_cr_inline "$expected_file") <(bt_remove_cr_inline "$actual_file") 1>&2
 
        return $?
 }
This page took 0.032729 seconds and 4 git commands to generate.