From 73760435c5a44d48313a85d94af093c293bc17ef Mon Sep 17 00:00:00 2001 From: Simon Marchi Date: Fri, 5 Jul 2019 15:50:58 -0400 Subject: [PATCH] cli: automatically detect sources for leftover arguments This patch adds a source auto-discovery feature. The goal is for the user to be able to just pass some inputs (which can be strings of a form understood by a particular component class, or paths to files/directories) and for babeltrace to figure out which component classes are best suited to handle those inputs. Currently, any leftover argument passed by the user is passed to a src.ctf.fs instance. To use a different source component, the user must use the --component argument. This system will therefore help usability for users of non-src.ctf.fs sources. It will also allow splitting the src.ctf.fs source into a "generic CTF" one and an "LTTng CTF" one, which includes the fixups specific to CTF files produced by various LTTng versions. The big picture is that for each leftover argument (called `input`), we ask all component classes if they can handle it (see `support-info query` below), the component classes reply with a weight, and the input is attributed to the component class that gave the largest weight. More precisely, this is what we do for each leftover argument: 1. Ask all known source component classes if they recognize the argument as an arbitrary string, which would be in a format that makes sense to them (but doesn't point to a file or directory on disk, unless it's a coincidence). The obvious example is src.ctf.lttng-live, which would be apt to handle paths of the form 'net://...'. If some component classes claim to understand the argument, the one with the largest weight is chosen, and a component of that class will be instantiated. 2. If no component class has claimed the argument as an arbitrary string and the argument points to a file or directory on disk, ask them all if they recognize it as a file or directory they can handle. If some component classes do, choose the one with the biggest weight. 3. If no component class has recognized it so far, and the input points to a directory, we start to dig: for each child of that directory, apply the sophisticated algorithm described in step 2. If a leftover argument (including all its children, if it's a directory) is not handled by any source component class, we show a warning. If nothing is discovered and no explicit source is instantiated either, the excecution fails on the "No source component" check that is already there. Component classes have the ability to "group" their inputs as they wish. This means that multiple inputs attributed to a given component class can be passed to a single instance of that class, all to separate instances, or any combination in between. To achieve this, component classes are able to also reply with a group key (a string of their choice). Inputs with the same group key will be passed to the same instance. Implementation details and choices ---------------------------------- * Since leftovers are now passed to the source auto-discovery mechanism as opposed to an implicit src.ctf.fs component previously, this src.ctf.fs implicit component is no longer needed. This changes how we handle some of the legacy (compatibility with babeltrace 1) flags. If the user passes --clock-offset or --clock-offset-ns, we will search in the auto-discovered sources and apply it to any src.ctf.fs instance created. If no src.ctf.fs component would be instantiated, we issue an error. If the user passes --input-format ctf, we don't want other formats possibly being read. We still use the auto-discovery mechanism, but we restrict it to the src.ctf.fs component class (other component classes won't be queried and therefore won't be instantiated). * This also means that to keep the basic use case of "babeltrace2 " working, we need to implement the support-info query for the src.ctf.fs component class in the current patch. Not doing so immediatly would break many tests. The simplest possible implementation was added. It looks for inputs of type "directory", which have a "metadata" file in it. It always reply with the same group value, such that a single instance of the component class is used (keeping the current behavior). * Since the strings we pass to support-info queries (and eventually to components we instantiate) are not necessarily paths to directories or files on disk (they can be URLs, for example), we now use the term "inputs" rather than "paths" for all of them. * A support-info query returning ERROR aborts the auto-discovery process, making it return an ERROR as well. * If we can't open a directory because we don't have permission to read it (EACCES), we log a warning and continue. Other errors abort the auto-discovery process. support-info query ------------------ The support-info query is a contract between the CLI and source component classes. Source component classes that don't support it will simply not be able to automatically discover inputs, and will have to be explicitly instantiated using --component. The parameters of a support-info query are a map containing these keys: - `type` (string): possible values are "string", "directory" or "file", indicating the nature of `input`, described below. The value "string" means that the component class may try to interpret `input` as a string with a format it can recognize. "directory" and "file" respectively mean to interpret `input` as a directory and file. When type is "directory" or "file", the component class can assume that the corresponding directory or file exists on disk. - input (string): input descriptor, to be interpreted according to `type`. A support-info response can be - A real or integer value between 0 and 1, representing the weight - A map value containing these keys: - `weight` (real or integer), mandatory: between 0 and 1 - `group` (string or null), optional: a key by which to group inputs, when instantiating components. A component class that does not support a given must reply with a weight of 0. All inputs attributed to the same component class, sharing the same group key, will be passed to the same component instance. inputs whose group key is missing or null are not grouped with other inputs. Components created by the auto-discovery mechanism are passed the `inputs` parameter, an array of strings containing all inputs attributed to this instance. testing ------- I have a brief catch-all test for this, it just covers a few important cases. The test strategy is the following: - Run babeltrace2 with an arbitrary string and a directory as leftovers. - One source recognizes the arbitrary string. - Various sources recognizes files and directories inside the passed directory. - Each instantiated source outputs one line including its name and the sorted list of its inputs. - We sort the output of babeltrace and compare it with an expected string. Since everything is sorted, the output should be stable. Change-Id: I7f884551d7cb576974ea53420ead9c4a8005e99d Signed-off-by: Simon Marchi Reviewed-on: https://review.lttng.org/c/babeltrace/+/1644 Tested-by: jenkins Reviewed-by: Philippe Proulx --- .../bt2/trace_collection_message_iterator.py | 10 +- src/cli/Makefile.am | 2 + src/cli/babeltrace2-cfg-cli-args.c | 311 ++++++-- src/cli/babeltrace2-cfg-src-auto-disc.c | 688 ++++++++++++++++++ src/cli/babeltrace2-cfg-src-auto-disc.h | 74 ++ src/cli/babeltrace2-plugins.c | 2 +- src/cli/babeltrace2-plugins.h | 2 +- src/cli/babeltrace2.c | 2 +- src/plugins/ctf/fs-src/fs.c | 38 +- src/plugins/ctf/fs-src/query.c | 79 +- src/plugins/ctf/fs-src/query.h | 6 + .../test_trace_collection_message_iterator.py | 4 +- .../test_auto_source_discovery | 57 ++ tests/cli/test_convert_args | 87 +-- .../auto-source-discovery/bt_plugin_test.py | 101 +++ .../cli/auto-source-discovery/traces/aaa1 | 0 .../cli/auto-source-discovery/traces/aaa2 | 0 .../cli/auto-source-discovery/traces/aaa3 | 0 .../cli/auto-source-discovery/traces/bbb1 | 0 .../cli/auto-source-discovery/traces/bbb2 | 0 .../cli/auto-source-discovery/traces/ccc1 | 0 .../cli/auto-source-discovery/traces/ccc2 | 0 .../cli/auto-source-discovery/traces/ccc3 | 0 .../cli/auto-source-discovery/traces/ccc4 | 0 .../traces/some-dir/aaa10 | 0 .../src.ctf.fs/query/test_query_trace_info.py | 24 +- 26 files changed, 1351 insertions(+), 136 deletions(-) create mode 100644 src/cli/babeltrace2-cfg-src-auto-disc.c create mode 100644 src/cli/babeltrace2-cfg-src-auto-disc.h create mode 100755 tests/cli/auto-source-discovery/test_auto_source_discovery create mode 100644 tests/data/cli/auto-source-discovery/bt_plugin_test.py create mode 100644 tests/data/cli/auto-source-discovery/traces/aaa1 create mode 100644 tests/data/cli/auto-source-discovery/traces/aaa2 create mode 100644 tests/data/cli/auto-source-discovery/traces/aaa3 create mode 100644 tests/data/cli/auto-source-discovery/traces/bbb1 create mode 100644 tests/data/cli/auto-source-discovery/traces/bbb2 create mode 100644 tests/data/cli/auto-source-discovery/traces/ccc1 create mode 100644 tests/data/cli/auto-source-discovery/traces/ccc2 create mode 100644 tests/data/cli/auto-source-discovery/traces/ccc3 create mode 100644 tests/data/cli/auto-source-discovery/traces/ccc4 create mode 100644 tests/data/cli/auto-source-discovery/traces/some-dir/aaa10 diff --git a/src/bindings/python/bt2/bt2/trace_collection_message_iterator.py b/src/bindings/python/bt2/bt2/trace_collection_message_iterator.py index 592c7f15..c6d147af 100644 --- a/src/bindings/python/bt2/bt2/trace_collection_message_iterator.py +++ b/src/bindings/python/bt2/bt2/trace_collection_message_iterator.py @@ -49,7 +49,7 @@ class ComponentSpec: self._logging_level = logging_level if type(params) is str: - self._params = bt2.create_value({'paths': [params]}) + self._params = bt2.create_value({'inputs': [params]}) else: self._params = bt2.create_value(params) @@ -141,19 +141,19 @@ class TraceCollectionMessageIterator(bt2.message_iterator._MessageIterator): def _create_stream_intersection_trimmer(self, component, port): # find the original parameters specified by the user to create - # this port's component to get the `path` parameter + # this port's component to get the `inputs` parameter for src_comp_and_spec in self._src_comps_and_specs: if component == src_comp_and_spec.comp: break try: - paths = src_comp_and_spec.spec.params['paths'] + inputs = src_comp_and_spec.spec.params['inputs'] except Exception as e: raise bt2.Error( - 'all source components must be created with a "paths" parameter in stream intersection mode' + 'all source components must be created with an "inputs" parameter in stream intersection mode' ) from e - params = {'paths': paths} + params = {'inputs': inputs} # query the port's component for the `trace-info` object which # contains the stream intersection range for each exposed diff --git a/src/cli/Makefile.am b/src/cli/Makefile.am index ece7146b..8c871757 100644 --- a/src/cli/Makefile.am +++ b/src/cli/Makefile.am @@ -32,6 +32,8 @@ babeltrace2_bin_SOURCES = \ babeltrace2-cfg-cli-args-default.c \ babeltrace2-cfg-cli-params-arg.c \ babeltrace2-cfg-cli-params-arg.h \ + babeltrace2-cfg-src-auto-disc.c \ + babeltrace2-cfg-src-auto-disc.h \ babeltrace2-plugins.c \ babeltrace2-plugins.h \ logging.c \ diff --git a/src/cli/babeltrace2-cfg-cli-args.c b/src/cli/babeltrace2-cfg-cli-args.c index 6214a594..7fc04884 100644 --- a/src/cli/babeltrace2-cfg-cli-args.c +++ b/src/cli/babeltrace2-cfg-cli-args.c @@ -41,6 +41,8 @@ #include "babeltrace2-cfg-cli-args.h" #include "babeltrace2-cfg-cli-args-connect.h" #include "babeltrace2-cfg-cli-params-arg.h" +#include "babeltrace2-plugins.h" +#include "babeltrace2-cfg-src-auto-disc.h" #include "common/version.h" static const int cli_default_log_level = BT_LOG_WARNING; @@ -2517,8 +2519,13 @@ end: struct implicit_component_args { bool exists; + + /* The component class name (e.g. src.ctf.fs). */ GString *comp_arg; + + /* The component instance name. */ GString *name_arg; + GString *params_arg; bt_value *extra_params; }; @@ -2639,6 +2646,8 @@ end: return ret; } +/* Free the fields of a `struct implicit_component_args`. */ + static void finalize_implicit_component_args(struct implicit_component_args *args) { @@ -2659,6 +2668,17 @@ void finalize_implicit_component_args(struct implicit_component_args *args) bt_value_put_ref(args->extra_params); } +/* Destroy a dynamically-allocated `struct implicit_component_args`. */ + +static +void destroy_implicit_component_args(struct implicit_component_args *args) +{ + finalize_implicit_component_args(args); + g_free(args); +} + +/* Initialize the fields of an already allocated `struct implicit_component_args`. */ + static int init_implicit_component_args(struct implicit_component_args *args, const char *comp_arg, bool exists) @@ -2683,6 +2703,31 @@ end: return ret; } +/* Dynamically allocate and initialize a `struct implicit_component_args`. */ + +static +struct implicit_component_args *create_implicit_component_args( + const char *comp_arg) +{ + struct implicit_component_args *args; + int status; + + args = g_new(struct implicit_component_args, 1); + if (!args) { + BT_CLI_LOGE_APPEND_CAUSE_OOM(); + goto end; + } + + status = init_implicit_component_args(args, comp_arg, true); + if (status != 0) { + g_free(args); + args = NULL; + } + +end: + return args; +} + static void append_implicit_component_param(struct implicit_component_args *args, const char *key, const char *value) @@ -2693,6 +2738,33 @@ void append_implicit_component_param(struct implicit_component_args *args, append_param_arg(args->params_arg, key, value); } +/* + * Append the given parameter (`key=value`) to all component specifications + * in `implicit_comp_args` (an array of `struct implicit_component_args *`) + * which match `comp_arg`. + * + * Return the number of matching components. + */ + +static +int append_multiple_implicit_components_param(GPtrArray *implicit_comp_args, + const char *comp_arg, const char *key, const char *value) +{ + int i; + int n = 0; + + for (i = 0; i < implicit_comp_args->len; i++) { + struct implicit_component_args *args = implicit_comp_args->pdata[i]; + + if (strcmp(args->comp_arg->str, comp_arg) == 0) { + append_implicit_component_param(args, key, value); + n++; + } + } + + return n; +} + /* Escape value to make it suitable to use as a string parameter value. */ static gchar *escape_string_value(const char *value) @@ -3208,6 +3280,55 @@ end: return ret; } +/* + * Create `struct implicit_component_args` structures for each of the source + * components we identified. Add them to `component_args`. + */ + +static +void create_implicit_component_args_from_auto_discovered_sources( + const struct auto_source_discovery *auto_disc, GPtrArray *component_args) +{ + gchar *cc_name = NULL; + struct implicit_component_args *comp = NULL; + int status; + guint i, len; + + len = auto_disc->results->len; + + for (i = 0; i < len; i++) { + struct auto_source_discovery_result *res = + g_ptr_array_index(auto_disc->results, i); + + g_free(cc_name); + cc_name = g_strdup_printf("source.%s.%s", res->plugin_name, res->source_cc_name); + if (!cc_name) { + BT_CLI_LOGE_APPEND_CAUSE_OOM(); + goto end; + } + + comp = create_implicit_component_args(cc_name); + if (!comp) { + goto end; + } + + status = append_parameter_to_args(comp->extra_params, "inputs", res->inputs); + if (status != 0) { + goto end; + } + + g_ptr_array_add(component_args, comp); + comp = NULL; + } + +end: + g_free(cc_name); + + if (comp) { + destroy_implicit_component_args(comp); + } +} + /* * Creates a Babeltrace config object from the arguments of a convert * command. @@ -3243,7 +3364,6 @@ struct bt_config *bt_config_convert_from_args(int argc, const char *argv[], GList *filter_names = NULL; GList *sink_names = NULL; bt_value *leftovers = NULL; - struct implicit_component_args implicit_ctf_input_args = { 0 }; struct implicit_component_args implicit_ctf_output_args = { 0 }; struct implicit_component_args implicit_lttng_live_args = { 0 }; struct implicit_component_args implicit_dummy_args = { 0 }; @@ -3256,6 +3376,24 @@ struct bt_config *bt_config_convert_from_args(int argc, const char *argv[], size_t i; struct bt_common_lttng_live_url_parts lttng_live_url_parts = { 0 }; char *output = NULL; + struct auto_source_discovery auto_disc = { NULL }; + GString *auto_disc_comp_name = NULL; + + /* + * Array of `struct implicit_component_args *` created for the sources + * we have auto-discovered. + */ + GPtrArray *discovered_source_args = NULL; + + /* + * If set, restrict automatic source discovery to this component class + * of this plugin. + */ + const char *auto_source_discovery_restrict_plugin_name = NULL; + const char *auto_source_discovery_restrict_component_class_name = NULL; + + gchar *ctf_fs_source_clock_class_offset_arg = NULL; + gchar *ctf_fs_source_clock_class_offset_ns_arg = NULL; (void) bt_value_copy(initial_plugin_paths, &plugin_paths); @@ -3267,11 +3405,6 @@ struct bt_config *bt_config_convert_from_args(int argc, const char *argv[], goto end; } - if (init_implicit_component_args(&implicit_ctf_input_args, - "source.ctf.fs", false)) { - goto error; - } - if (init_implicit_component_args(&implicit_ctf_output_args, "sink.ctf.fs", false)) { goto error; @@ -3342,6 +3475,23 @@ struct bt_config *bt_config_convert_from_args(int argc, const char *argv[], goto error; } + if (auto_source_discovery_init(&auto_disc) != 0) { + goto error; + } + + discovered_source_args = + g_ptr_array_new_with_free_func((GDestroyNotify) destroy_implicit_component_args); + if (!discovered_source_args) { + BT_CLI_LOGE_APPEND_CAUSE_OOM(); + goto error; + } + + auto_disc_comp_name = g_string_new(NULL); + if (!auto_disc_comp_name) { + BT_CLI_LOGE_APPEND_CAUSE_OOM(); + goto error; + } + /* * First pass: collect all arguments which need to be passed * as is to the run command. This pass can also add --name @@ -3727,16 +3877,28 @@ struct bt_config *bt_config_convert_from_args(int argc, const char *argv[], implicit_text_args.exists = true; break; case OPT_CLOCK_OFFSET: - implicit_ctf_input_args.exists = true; - append_implicit_component_param( - &implicit_ctf_input_args, - "clock-class-offset-s", arg); + if (ctf_fs_source_clock_class_offset_arg) { + BT_CLI_LOGE_APPEND_CAUSE("Duplicate --clock-offset option\n"); + goto error; + } + + ctf_fs_source_clock_class_offset_arg = g_strdup(arg); + if (!ctf_fs_source_clock_class_offset_arg) { + BT_CLI_LOGE_APPEND_CAUSE_OOM(); + goto error; + } break; case OPT_CLOCK_OFFSET_NS: - implicit_ctf_input_args.exists = true; - append_implicit_component_param( - &implicit_ctf_input_args, - "clock-class-offset-ns", arg); + if (ctf_fs_source_clock_class_offset_ns_arg) { + BT_CLI_LOGE_APPEND_CAUSE("Duplicate --clock-offset-ns option\n"); + goto error; + } + + ctf_fs_source_clock_class_offset_ns_arg = g_strdup(arg); + if (!ctf_fs_source_clock_class_offset_ns_arg) { + BT_CLI_LOGE_APPEND_CAUSE_OOM(); + goto error; + } break; case OPT_CLOCK_SECONDS: append_implicit_component_param( @@ -3826,8 +3988,11 @@ struct bt_config *bt_config_convert_from_args(int argc, const char *argv[], got_input_format_opt = true; if (strcmp(arg, "ctf") == 0) { - implicit_ctf_input_args.exists = true; + auto_source_discovery_restrict_plugin_name = "ctf"; + auto_source_discovery_restrict_component_class_name = "fs"; } else if (strcmp(arg, "lttng-live") == 0) { + auto_source_discovery_restrict_plugin_name = "ctf"; + auto_source_discovery_restrict_component_class_name = "lttng-live"; implicit_lttng_live_args.exists = true; } else { BT_CLI_LOGE_APPEND_CAUSE("Unknown legacy input format:\n %s", @@ -4086,42 +4251,55 @@ struct bt_config *bt_config_convert_from_args(int argc, const char *argv[], goto error; } } else { - /* - * Create one source.ctf.fs component, pass it an array - * with the leftovers. - * Note that it still has to be named later. - */ - implicit_ctf_input_args.exists = true; - ret = append_parameter_to_args(implicit_ctf_input_args.extra_params, - "paths", leftovers); - if (ret) { + int status; + + status = auto_discover_source_components(plugin_paths, leftovers, + auto_source_discovery_restrict_plugin_name, + auto_source_discovery_restrict_component_class_name, + *default_log_level >= 0 ? *default_log_level : cli_default_log_level, + &auto_disc); + + if (status != 0) { goto error; } + + create_implicit_component_args_from_auto_discovered_sources( + &auto_disc, discovered_source_args); } } - /* - * Ensure mutual exclusion between implicit `source.ctf.fs` and - * `source.ctf.lttng-live` components. - */ - if (implicit_ctf_input_args.exists && implicit_lttng_live_args.exists) { - BT_CLI_LOGE_APPEND_CAUSE("Cannot create both implicit `%s` and `%s` components.", - implicit_ctf_input_args.comp_arg->str, - implicit_lttng_live_args.comp_arg->str); - goto error; + /* If --clock-offset was given, apply it to any src.ctf.fs component. */ + if (ctf_fs_source_clock_class_offset_arg) { + int n; + + n = append_multiple_implicit_components_param( + discovered_source_args, "source.ctf.fs", "clock-class-offset-s", + ctf_fs_source_clock_class_offset_arg); + + if (n == 0) { + BT_CLI_LOGE_APPEND_CAUSE("--clock-offset specified, but no source.ctf.fs component instantiated."); + goto error; + } } - /* - * If the implicit `source.ctf.fs` or `source.ctf.lttng-live` - * components exists, make sure there's at least one leftover - * (which is the path or URL). - */ - if (implicit_ctf_input_args.exists && bt_value_array_is_empty(leftovers)) { - BT_CLI_LOGE_APPEND_CAUSE("Missing path for implicit `%s` component.", - implicit_ctf_input_args.comp_arg->str); - goto error; + /* If --clock-offset-ns was given, apply it to any src.ctf.fs component. */ + if (ctf_fs_source_clock_class_offset_ns_arg) { + int n; + + n = append_multiple_implicit_components_param( + discovered_source_args, "source.ctf.fs", "clock-class-offset-ns", + ctf_fs_source_clock_class_offset_ns_arg); + + if (n == 0) { + BT_CLI_LOGE_APPEND_CAUSE("--clock-offset-ns specified, but no source.ctf.fs component instantiated."); + goto error; + } } + /* + * If the implicit `source.ctf.lttng-live` component exists, make sure + * there's at least one leftover (which is the URL). + */ if (implicit_lttng_live_args.exists && bt_value_array_is_empty(leftovers)) { BT_CLI_LOGE_APPEND_CAUSE("Missing URL for implicit `%s` component.", implicit_lttng_live_args.comp_arg->str); @@ -4129,10 +4307,26 @@ struct bt_config *bt_config_convert_from_args(int argc, const char *argv[], } /* Assign names to implicit components */ - ret = assign_name_to_implicit_component(&implicit_ctf_input_args, - "source-ctf-fs", all_names, &source_names, true); - if (ret) { - goto error; + for (i = 0; i < discovered_source_args->len; i++) { + struct implicit_component_args *args; + int j; + + args = discovered_source_args->pdata[i]; + + g_string_printf(auto_disc_comp_name, "auto-disc-%s", args->comp_arg->str); + + /* Give it a name like `auto-disc-src-ctf-fs`. */ + for (j = 0; j < auto_disc_comp_name->len; j++) { + if (auto_disc_comp_name->str[j] == '.') { + auto_disc_comp_name->str[j] = '-'; + } + } + + ret = assign_name_to_implicit_component(args, + auto_disc_comp_name->str, all_names, &source_names, true); + if (ret) { + goto error; + } } ret = assign_name_to_implicit_component(&implicit_lttng_live_args, @@ -4218,9 +4412,14 @@ struct bt_config *bt_config_convert_from_args(int argc, const char *argv[], * Append the equivalent run arguments for the implicit * components. */ - ret = append_run_args_for_implicit_component(&implicit_ctf_input_args, run_args); - if (ret) { - goto error; + for (i = 0; i < discovered_source_args->len; i++) { + struct implicit_component_args *args = + discovered_source_args->pdata[i]; + + ret = append_run_args_for_implicit_component(args, run_args); + if (ret) { + goto error; + } } ret = append_run_args_for_implicit_component(&implicit_lttng_live_args, @@ -4379,7 +4578,6 @@ end: destroy_glist_of_gstring(filter_names); destroy_glist_of_gstring(sink_names); bt_value_put_ref(leftovers); - finalize_implicit_component_args(&implicit_ctf_input_args); finalize_implicit_component_args(&implicit_ctf_output_args); finalize_implicit_component_args(&implicit_lttng_live_args); finalize_implicit_component_args(&implicit_dummy_args); @@ -4389,6 +4587,19 @@ end: finalize_implicit_component_args(&implicit_trimmer_args); bt_value_put_ref(plugin_paths); bt_common_destroy_lttng_live_url_parts(<tng_live_url_parts); + auto_source_discovery_fini(&auto_disc); + + if (discovered_source_args) { + g_ptr_array_free(discovered_source_args, TRUE); + } + + g_free(ctf_fs_source_clock_class_offset_arg); + g_free(ctf_fs_source_clock_class_offset_ns_arg); + + if (auto_disc_comp_name) { + g_string_free(auto_disc_comp_name, TRUE); + } + return cfg; } diff --git a/src/cli/babeltrace2-cfg-src-auto-disc.c b/src/cli/babeltrace2-cfg-src-auto-disc.c new file mode 100644 index 00000000..505a265f --- /dev/null +++ b/src/cli/babeltrace2-cfg-src-auto-disc.c @@ -0,0 +1,688 @@ +/* + * Copyright (c) 2019 EfficiOS Inc. and Linux Foundation + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#define BT_LOG_TAG "CLI-CFG-SRC-AUTO-DISC" +#include "logging.h" + +#include "babeltrace2-cfg-src-auto-disc.h" +#include "babeltrace2-plugins.h" +#include "common/common.h" + +/* Finalize and free a `struct auto_source_discovery_result`. */ + +static +void auto_source_discovery_result_destroy(struct auto_source_discovery_result *res) +{ + if (res) { + g_free(res->group); + bt_value_put_ref(res->inputs); + g_free(res); + } +} + +/* Allocate and initialize a `struct auto_source_discovery_result`. */ + +static +struct auto_source_discovery_result *auto_source_discovery_result_create( + const char *plugin_name, const char *source_cc_name, + const char *group) +{ + struct auto_source_discovery_result *res; + + res = g_new0(struct auto_source_discovery_result, 1); + if (!res) { + BT_CLI_LOGE_APPEND_CAUSE( + "Failed to allocate a auto_source_discovery_result structure."); + goto error; + } + + res->plugin_name = plugin_name; + res->source_cc_name = source_cc_name; + res->group = g_strdup(group); + if (group && !res->group) { + BT_CLI_LOGE_APPEND_CAUSE("Failed to allocate a string."); + goto error; + } + + res->inputs = bt_value_array_create(); + if (!res->inputs) { + BT_CLI_LOGE_APPEND_CAUSE("Failed to allocate an array value."); + goto error; + } + + goto end; +error: + auto_source_discovery_result_destroy(res); + +end: + return res; +} + +/* Finalize a `struct auto_source_discovery`. */ + +void auto_source_discovery_fini(struct auto_source_discovery *auto_disc) +{ + if (auto_disc->results) { + g_ptr_array_free(auto_disc->results, TRUE); + } +} + +/* Initialize an already allocated `struct auto_source_discovery`. */ + +int auto_source_discovery_init(struct auto_source_discovery *auto_disc) +{ + int status; + + auto_disc->results = g_ptr_array_new_with_free_func( + (GDestroyNotify) auto_source_discovery_result_destroy); + + if (!auto_disc->results) { + goto error; + } + + status = 0; + goto end; + +error: + auto_source_discovery_fini(auto_disc); + status = -1; + +end: + + return status; +} + +/* + * Assign `input` to source component class `source_cc_name` of plugin + * `plugin_name`, in the group with key `group`. + */ + +static +int auto_source_discovery_add(struct auto_source_discovery *auto_disc, + const char *plugin_name, + const char *source_cc_name, + const char *group, + const char *input) +{ + int status; + bt_value_array_append_element_status append_status; + guint len; + guint i; + struct auto_source_discovery_result *res = NULL; + + len = auto_disc->results->len; + i = len; + + if (group) { + for (i = 0; i < len; i++) { + res = g_ptr_array_index(auto_disc->results, i); + + if (strcmp(res->plugin_name, plugin_name) != 0) { + continue; + } + + if (strcmp(res->source_cc_name, source_cc_name) != 0) { + continue; + } + + if (g_strcmp0(res->group, group) != 0) { + continue; + } + + break; + } + } + + if (i == len) { + /* Add a new result entry. */ + res = auto_source_discovery_result_create(plugin_name, + source_cc_name, group); + if (!res) { + goto error; + } + + g_ptr_array_add(auto_disc->results, res); + } + + append_status = bt_value_array_append_string_element(res->inputs, input); + if (append_status != BT_VALUE_ARRAY_APPEND_ELEMENT_STATUS_OK) { + BT_CLI_LOGE_APPEND_CAUSE("Failed to append a string value."); + goto error; + } + + + status = 0; + goto end; + +error: + status = -1; + +end: + return status; +} + +static +int convert_weight_value(const bt_value *weight_value, double *weight, + const char *plugin_name, const char *source_cc_name, + const char *input, const char *input_type) +{ + enum bt_value_type weight_value_type; + int status; + + weight_value_type = bt_value_get_type(weight_value); + + if (weight_value_type == BT_VALUE_TYPE_REAL) { + *weight = bt_value_real_get(weight_value); + } else if (weight_value_type == BT_VALUE_TYPE_SIGNED_INTEGER) { + /* Accept signed integer as a convenience for "return 0" or "return 1" in Python. */ + *weight = bt_value_integer_signed_get(weight_value); + } else { + BT_LOGW("support-info query: unexpected type for weight: " + "component-class-name=source.%s.%s, input=%s, input-type=%s, " + "expected-entry-type=%s, actual-entry-type=%s", + plugin_name, source_cc_name, input, input_type, + bt_common_value_type_string(BT_VALUE_TYPE_REAL), + bt_common_value_type_string(bt_value_get_type(weight_value))); + goto error; + } + + if (*weight < 0.0 || *weight > 1.0) { + BT_LOGW("support-info query: weight value is out of range [0.0, 1.0]: " + "component-class-name=source.%s.%s, input=%s, input-type=%s, " + "weight=%f", + plugin_name, source_cc_name, input, input_type, *weight); + goto error; + } + + status = 0; + goto end; + +error: + status = -1; + +end: + return status; +} + +/* + * Query all known source components to see if any of them can handle `input` + * as the given `type`(arbitrary string, directory or file). + * + * If `plugin_restrict` is non-NULL, only query source component classes provided + * by the plugin with that name. + * + * If `component_class_restrict` is non-NULL, only query source component classes + * with that name. + * + * Return: + * + * - > 0 on success, if no source component class has reported that it handles `input` + * - 0 on success, if a source component class has reported that it handles `input` + * - < 0 on failure (e.g. memory error) + */ +static +int support_info_query_all_sources(const char *input, + const char *input_type, + bt_query_executor *query_executor, size_t plugin_count, + const char *plugin_restrict, + const char *component_class_restrict, + enum bt_logging_level log_level, + struct auto_source_discovery *auto_disc) +{ + bt_value_map_insert_entry_status insert_status; + bt_value *query_params = NULL; + int status; + size_t i_plugins; + const struct bt_value *query_result = NULL; + struct { + const bt_component_class_source *source; + const bt_plugin *plugin; + const bt_value *group; + double weigth; + } winner = { NULL, NULL, NULL, 0 }; + + query_params = bt_value_map_create(); + if (!query_params) { + BT_CLI_LOGE_APPEND_CAUSE("Failed to allocate a map value."); + goto error; + } + + insert_status = bt_value_map_insert_string_entry(query_params, "input", input); + if (insert_status != BT_VALUE_MAP_INSERT_ENTRY_STATUS_OK) { + BT_CLI_LOGE_APPEND_CAUSE("Failed to insert a map entry."); + goto error; + } + + insert_status = bt_value_map_insert_string_entry(query_params, "type", input_type); + if (insert_status != BT_VALUE_MAP_INSERT_ENTRY_STATUS_OK) { + BT_CLI_LOGE_APPEND_CAUSE("Failed to insert a map entry."); + goto error; + } + + for (i_plugins = 0; i_plugins < plugin_count; i_plugins++) { + const bt_plugin *plugin; + const char *plugin_name; + uint64_t source_count; + uint64_t i_sources; + + plugin = borrow_loaded_plugin(i_plugins); + plugin_name = bt_plugin_get_name(plugin); + + /* + * If the search is restricted to a specific plugin, only consider + * the plugin with that name. + */ + if (plugin_restrict && strcmp(plugin_restrict, plugin_name) != 0) { + continue; + } + + source_count = bt_plugin_get_source_component_class_count(plugin); + + for (i_sources = 0; i_sources < source_count; i_sources++) { + const bt_component_class_source *source_cc; + const bt_component_class *cc; + const char *source_cc_name; + bt_query_executor_query_status query_status; + + source_cc = bt_plugin_borrow_source_component_class_by_index_const(plugin, i_sources); + cc = bt_component_class_source_as_component_class_const(source_cc); + source_cc_name = bt_component_class_get_name(cc); + + /* + * If the search is restricted to a specific component class, only consider the + * component classes with that name. + */ + if (component_class_restrict && strcmp(component_class_restrict, source_cc_name) != 0) { + continue; + } + + BT_LOGD("support-info query: before: component-class-name=source.%s.%s, input=%s, " + "type=%s", plugin_name, source_cc_name, input, input_type); + + BT_VALUE_PUT_REF_AND_RESET(query_result); + query_status = bt_query_executor_query(query_executor, cc, "support-info", + query_params, log_level, &query_result); + + if (query_status == BT_QUERY_EXECUTOR_QUERY_STATUS_OK) { + double weight; + const bt_value *group_value = NULL; + enum bt_value_type query_result_type; + + BT_ASSERT(query_result); + + query_result_type = bt_value_get_type(query_result); + + if (query_result_type == BT_VALUE_TYPE_REAL || query_result_type == BT_VALUE_TYPE_SIGNED_INTEGER) { + if (convert_weight_value(query_result, &weight, plugin_name, source_cc_name, input, input_type) != 0) { + /* convert_weight_value has already warned. */ + continue; + } + } else if (query_result_type == BT_VALUE_TYPE_MAP) { + const bt_value *weight_value; + + if (!bt_value_map_has_entry(query_result, "weight")) { + BT_LOGW("support-info query: result is missing `weight` entry: " + "component-class-name=source.%s.%s, input=%s, input-type=%s", + bt_plugin_get_name(plugin), + bt_component_class_get_name(cc), input, + input_type); + continue; + } + + weight_value = bt_value_map_borrow_entry_value_const(query_result, "weight"); + BT_ASSERT(weight_value); + + if (convert_weight_value(weight_value, &weight, plugin_name, source_cc_name, input, input_type) != 0) { + /* convert_weight_value has already warned. */ + continue; + } + + if (bt_value_map_has_entry(query_result, "group")) { + enum bt_value_type group_value_type; + + group_value = bt_value_map_borrow_entry_value_const(query_result, "group"); + BT_ASSERT(group_value); + + group_value_type = bt_value_get_type(group_value); + + if (group_value_type == BT_VALUE_TYPE_NULL) { + /* Do as if no value was passed. */ + group_value = NULL; + } else if (bt_value_get_type(group_value) != BT_VALUE_TYPE_STRING) { + BT_LOGW("support-info query: unexpected type for entry `group`: " + "component-class-name=source.%s.%s, input=%s, input-type=%s, " + "expected-entry-type=%s,%s, actual-entry-type=%s", + bt_plugin_get_name(plugin), + bt_component_class_get_name(cc), input, + input_type, + bt_common_value_type_string(BT_VALUE_TYPE_NULL), + bt_common_value_type_string(BT_VALUE_TYPE_STRING), + bt_common_value_type_string(bt_value_get_type(group_value))); + continue; + } + } + } else { + BT_LOGW("support-info query: unexpected result type: " + "component-class-name=source.%s.%s, input=%s, input-type=%s, " + "expected-types=%s,%s,%s, actual-type=%s", + bt_plugin_get_name(plugin), + bt_component_class_get_name(cc), input, + input_type, + bt_common_value_type_string(BT_VALUE_TYPE_REAL), + bt_common_value_type_string(BT_VALUE_TYPE_MAP), + bt_common_value_type_string(BT_VALUE_TYPE_SIGNED_INTEGER), + bt_common_value_type_string(bt_value_get_type(query_result))); + continue; + } + + BT_LOGD("support-info query: success: component-class-name=source.%s.%s, input=%s, " + "type=%s, weight=%f\n", + bt_plugin_get_name(plugin), bt_component_class_get_name(cc), input, + input_type, weight); + + if (weight > winner.weigth) { + winner.source = source_cc; + winner.plugin = plugin; + + bt_value_put_ref(winner.group); + winner.group = group_value; + bt_value_get_ref(winner.group); + + winner.weigth = weight; + } + } else if (query_status == BT_QUERY_EXECUTOR_QUERY_STATUS_ERROR) { + BT_CLI_LOGE_APPEND_CAUSE("support-info query failed."); + goto error; + } else if (query_status == BT_QUERY_EXECUTOR_QUERY_STATUS_MEMORY_ERROR) { + BT_CLI_LOGE_APPEND_CAUSE("Memory error."); + goto error; + } else { + BT_LOGD("support-info query: failure: component-class-name=source.%s.%s, input=%s, " + "type=%s, status=%s\n", + bt_plugin_get_name(plugin), bt_component_class_get_name(cc), input, + input_type, + bt_common_func_status_string(query_status)); + } + } + } + + if (winner.source) { + const char *source_name; + const char *plugin_name; + const char *group; + + source_name = bt_component_class_get_name( + bt_component_class_source_as_component_class_const(winner.source)); + plugin_name = bt_plugin_get_name(winner.plugin); + group = winner.group ? bt_value_string_get(winner.group) : NULL; + + BT_LOGI("Input %s is awarded to component class source.%s.%s with weight %f", + input, plugin_name, source_name, winner.weigth); + + status = auto_source_discovery_add(auto_disc, plugin_name, source_name, group, input); + if (status != 0) { + goto error; + } + } else { + BT_LOGI("Input %s (%s) was not recognized by any source component class.", + input, input_type); + status = 1; + } + + goto end; + +error: + status = -1; + +end: + bt_value_put_ref(query_result); + bt_value_put_ref(query_params); + bt_value_put_ref(winner.group); + + return status; +} + +/* + * Look for a source component class that recognizes `input` as an arbitrary + * string. + * + * Same return value semantic as `support_info_query_all_sources`. + */ + +static +int auto_discover_source_for_input_as_string(const char *input, + bt_query_executor *query_executor, size_t plugin_count, + const char *plugin_restrict, + const char *component_class_restrict, + enum bt_logging_level log_level, + struct auto_source_discovery *auto_disc) +{ + return support_info_query_all_sources(input, "string", + query_executor, plugin_count, plugin_restrict, + component_class_restrict, log_level, auto_disc); +} + +static +int auto_discover_source_for_input_as_dir_or_file_rec(GString *input, + bt_query_executor *query_executor, size_t plugin_count, + const char *plugin_restrict, + const char *component_class_restrict, + enum bt_logging_level log_level, + struct auto_source_discovery *auto_disc) +{ + int status; + GError *error = NULL; + + if (g_file_test(input->str, G_FILE_TEST_IS_REGULAR)) { + /* It's a file. */ + status = support_info_query_all_sources(input->str, + "file", query_executor, plugin_count, + plugin_restrict, component_class_restrict, log_level, auto_disc); + } else if (g_file_test(input->str, G_FILE_TEST_IS_DIR)) { + GDir *dir; + const gchar *dirent; + gsize saved_input_len; + int dir_status = 1; + + /* It's a directory. */ + status = support_info_query_all_sources(input->str, + "directory", query_executor, plugin_count, + plugin_restrict, component_class_restrict, log_level, + auto_disc); + + if (status < 0) { + /* Fatal error. */ + goto error; + } else if (status == 0) { + /* + * A component class claimed this input as a directory, + * don't recurse. + */ + goto end; + } + + dir = g_dir_open(input->str, 0, &error); + if (!dir) { + const char *fmt = "Failed to open directory %s: %s"; + BT_LOGW(fmt, input->str, error->message); + + if (errno == EACCES) { + /* This is not a fatal error, we just skip it. */ + status = 1; + goto end; + } else { + BT_CLI_LOGE_APPEND_CAUSE(fmt, input->str, + error->message); + goto error; + } + } + + saved_input_len = input->len; + + do { + errno = 0; + dirent = g_dir_read_name(dir); + if (dirent) { + g_string_append_c_inline(input, G_DIR_SEPARATOR); + g_string_append(input, dirent); + + status = auto_discover_source_for_input_as_dir_or_file_rec( + input, query_executor, plugin_count, + plugin_restrict, component_class_restrict, + log_level, auto_disc); + + g_string_truncate(input, saved_input_len); + + if (status < 0) { + /* Fatal error. */ + goto error; + } else if (status == 0) { + dir_status = 0; + } + } else if (errno != 0) { + BT_LOGW_ERRNO("Failed to read directory entry", ": dir=%s", input->str); + goto error; + } + } while (dirent != NULL); + + status = dir_status; + + g_dir_close(dir); + } else { + BT_LOGD("Skipping %s, not a file or directory", input->str); + status = 1; + } + + goto end; + +error: + status = -1; + +end: + + if (error) { + g_error_free(error); + } + + return status; +} + +/* + * Look for a source component class that recognizes `input` as a directory or + * file. If `input` is a directory and is not directly recognized, recurse and + * apply the same logic to children nodes. + * + * Same return value semantic as `support_info_query_all_sources`. + */ + +static +int auto_discover_source_for_input_as_dir_or_file(const char *input, + bt_query_executor *query_executor, size_t plugin_count, + const char *plugin_restrict, + const char *component_class_restrict, + enum bt_logging_level log_level, + struct auto_source_discovery *auto_disc) +{ + GString *mutable_input; + int status; + + mutable_input = g_string_new(input); + if (!mutable_input) { + status = -1; + goto end; + } + + status = auto_discover_source_for_input_as_dir_or_file_rec( + mutable_input, query_executor, plugin_count, plugin_restrict, + component_class_restrict, log_level, auto_disc); + + g_string_free(mutable_input, TRUE); +end: + return status; +} + +int auto_discover_source_components( + const bt_value *plugin_paths, + const bt_value *inputs, + const char *plugin_restrict, + const char *component_class_restrict, + enum bt_logging_level log_level, + struct auto_source_discovery *auto_disc) +{ + uint64_t i_inputs, input_count; + int status; + size_t plugin_count; + bt_query_executor *query_executor = NULL; + + input_count = bt_value_array_get_size(inputs); + + status = require_loaded_plugins(plugin_paths); + if (status != 0) { + goto end; + } + + plugin_count = get_loaded_plugins_count(); + + query_executor = bt_query_executor_create(); + if (!query_executor) { + BT_CLI_LOGE_APPEND_CAUSE("Failed to allocate a query executor."); + goto end; + } + + for (i_inputs = 0; i_inputs < input_count; i_inputs++) { + const bt_value *input_value; + const char *input; + + input_value = bt_value_array_borrow_element_by_index_const(inputs, i_inputs); + input = bt_value_string_get(input_value); + status = auto_discover_source_for_input_as_string(input, query_executor, + plugin_count, plugin_restrict, component_class_restrict, + log_level, auto_disc); + if (status < 0) { + /* Fatal error. */ + goto end; + } else if (status == 0) { + /* A component class has claimed this input as an arbitrary string. */ + continue; + } + + status = auto_discover_source_for_input_as_dir_or_file(input, + query_executor, plugin_count, plugin_restrict, + component_class_restrict, log_level, auto_disc); + if (status < 0) { + /* Fatal error. */ + goto end; + } else if (status == 0) { + /* + * This input (or something under it) was recognized. + */ + continue; + } + + BT_LOGW("No trace was found based on input `%s`.", input); + } + + status = 0; +end: + bt_query_executor_put_ref(query_executor); + return status; +} diff --git a/src/cli/babeltrace2-cfg-src-auto-disc.h b/src/cli/babeltrace2-cfg-src-auto-disc.h new file mode 100644 index 00000000..0d85c05e --- /dev/null +++ b/src/cli/babeltrace2-cfg-src-auto-disc.h @@ -0,0 +1,74 @@ +#ifndef CLI_BABELTRACE_CFG_SRC_AUTO_DISC_H +#define CLI_BABELTRACE_CFG_SRC_AUTO_DISC_H + +/* + * Copyright (c) 2019 EfficiOS Inc. and Linux Foundation + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +#include + +#include + +struct auto_source_discovery { + /* Array of `struct auto_source_discovery_result *`. */ + GPtrArray *results; +}; + +/* Value type of the `auto_source_discovery::results` array. */ + +struct auto_source_discovery_result { + /* + * `plugin_name` and `source_cc_name` are borrowed from the plugin and source + * component class (which outlive this structure). + */ + const char *plugin_name; + const char *source_cc_name; + + /* + * `group` is owned by this structure. + * + * May be NULL, to mean "no group". + */ + gchar *group; + + /* Array of input strings. */ + bt_value *inputs; +}; + +int auto_source_discovery_init(struct auto_source_discovery *auto_disc); +void auto_source_discovery_fini(struct auto_source_discovery *auto_disc); + +/* + * Given `inputs` a list of strings, query source component classes to discover + * which source components should be instantiated to deal with these inputs. + * + * Return 0 if execution completed successfully, < 0 otherwise. + */ + +int auto_discover_source_components( + const bt_value *plugin_paths, + const bt_value *inputs, + const char *plugin_filter, + const char *component_class_filter, + enum bt_logging_level log_level, + struct auto_source_discovery *auto_disc); + +#endif /* CLI_BABELTRACE_CFG_SRC_AUTO_DISC_H */ diff --git a/src/cli/babeltrace2-plugins.c b/src/cli/babeltrace2-plugins.c index 09ca6dc6..cc950ff2 100644 --- a/src/cli/babeltrace2-plugins.c +++ b/src/cli/babeltrace2-plugins.c @@ -205,7 +205,7 @@ end: return ret; } -int load_all_plugins(const bt_value *plugin_paths) +int require_loaded_plugins(const bt_value *plugin_paths) { static bool loaded = false; static int ret = 0; diff --git a/src/cli/babeltrace2-plugins.h b/src/cli/babeltrace2-plugins.h index 5f177c4a..83a67930 100644 --- a/src/cli/babeltrace2-plugins.h +++ b/src/cli/babeltrace2-plugins.h @@ -31,7 +31,7 @@ BT_HIDDEN void init_loaded_plugins(void); BT_HIDDEN void fini_loaded_plugins(void); -BT_HIDDEN int load_all_plugins(const bt_value *plugin_paths); +BT_HIDDEN int require_loaded_plugins(const bt_value *plugin_paths); BT_HIDDEN const bt_plugin *find_loaded_plugin(const char *name); BT_HIDDEN size_t get_loaded_plugins_count(void); diff --git a/src/cli/babeltrace2.c b/src/cli/babeltrace2.c index 6bdad883..2c90fc0e 100644 --- a/src/cli/babeltrace2.c +++ b/src/cli/babeltrace2.c @@ -2699,7 +2699,7 @@ int main(int argc, const char **argv) print_cfg(cfg); if (cfg->command_needs_plugins) { - ret = load_all_plugins(cfg->plugin_paths); + ret = require_loaded_plugins(cfg->plugin_paths); if (ret) { BT_CLI_LOGE_APPEND_CAUSE( "Failed to load plugins: ret=%d", ret); diff --git a/src/plugins/ctf/fs-src/fs.c b/src/plugins/ctf/fs-src/fs.c index 57384014..375ca0f3 100644 --- a/src/plugins/ctf/fs-src/fs.c +++ b/src/plugins/ctf/fs-src/fs.c @@ -1705,10 +1705,10 @@ int ctf_fs_component_create_ctf_fs_traces(bt_self_component_source *self_comp, for (i = 0; i < bt_value_array_get_size(paths_value); i++) { const bt_value *path_value = bt_value_array_borrow_element_by_index_const(paths_value, i); - const char *path = bt_value_string_get(path_value); + const char *input = bt_value_string_get(path_value); ret = ctf_fs_component_create_ctf_fs_traces_one_root(ctf_fs, - path); + input); if (ret) { goto end; } @@ -1826,8 +1826,8 @@ end: */ static -bool validate_paths_parameter(struct ctf_fs_component *ctf_fs, - const bt_value *paths) +bool validate_inputs_parameter(struct ctf_fs_component *ctf_fs, + const bt_value *inputs) { bool ret; bt_value_type type; @@ -1835,25 +1835,25 @@ bool validate_paths_parameter(struct ctf_fs_component *ctf_fs, bt_logging_level log_level = ctf_fs->log_level; bt_self_component *self_comp = ctf_fs->self_comp; - if (!paths) { - BT_COMP_LOGE("missing \"paths\" parameter"); + if (!inputs) { + BT_COMP_LOGE("missing \"inputs\" parameter"); goto error; } - type = bt_value_get_type(paths); + type = bt_value_get_type(inputs); if (type != BT_VALUE_TYPE_ARRAY) { - BT_COMP_LOGE("`paths` parameter: expecting array value: type=%s", + BT_COMP_LOGE("`inputs` parameter: expecting array value: type=%s", bt_common_value_type_string(type)); goto error; } - for (i = 0; i < bt_value_array_get_size(paths); i++) { + for (i = 0; i < bt_value_array_get_size(inputs); i++) { const bt_value *elem; - elem = bt_value_array_borrow_element_by_index_const(paths, i); + elem = bt_value_array_borrow_element_by_index_const(inputs, i); type = bt_value_get_type(elem); if (type != BT_VALUE_TYPE_STRING) { - BT_COMP_LOGE("`paths` parameter: expecting string value: index=%" PRIu64 ", type=%s", + BT_COMP_LOGE("`inputs` parameter: expecting string value: index=%" PRIu64 ", type=%s", i, bt_common_value_type_string(type)); goto error; } @@ -1870,15 +1870,15 @@ end: } bool read_src_fs_parameters(const bt_value *params, - const bt_value **paths, struct ctf_fs_component *ctf_fs) { + const bt_value **inputs, struct ctf_fs_component *ctf_fs) { bool ret; const bt_value *value; bt_logging_level log_level = ctf_fs->log_level; bt_self_component *self_comp = ctf_fs->self_comp; - /* paths parameter */ - *paths = bt_value_map_borrow_entry_value_const(params, "paths"); - if (!validate_paths_parameter(ctf_fs, *paths)) { + /* inputs parameter */ + *inputs = bt_value_map_borrow_entry_value_const(params, "inputs"); + if (!validate_inputs_parameter(ctf_fs, *inputs)) { goto error; } @@ -1924,7 +1924,7 @@ struct ctf_fs_component *ctf_fs_create( { struct ctf_fs_component *ctf_fs = NULL; guint i; - const bt_value *paths_value; + const bt_value *inputs_value; bt_self_component *self_comp = bt_self_component_source_as_self_component(self_comp_src); @@ -1934,7 +1934,7 @@ struct ctf_fs_component *ctf_fs_create( goto error; } - if (!read_src_fs_parameters(params, &paths_value, ctf_fs)) { + if (!read_src_fs_parameters(params, &inputs_value, ctf_fs)) { goto error; } @@ -1942,7 +1942,7 @@ struct ctf_fs_component *ctf_fs_create( ctf_fs->self_comp = self_comp; ctf_fs->self_comp_src = self_comp_src; - if (ctf_fs_component_create_ctf_fs_traces(self_comp_src, ctf_fs, paths_value)) { + if (ctf_fs_component_create_ctf_fs_traces(self_comp_src, ctf_fs, inputs_value)) { goto error; } @@ -2003,6 +2003,8 @@ bt_component_class_query_method_status ctf_fs_query( } else if (strcmp(object, "trace-info") == 0) { status = trace_info_query(comp_class, params, log_level, result); + } else if (!strcmp(object, "support-info")) { + status = support_info_query(comp_class, params, log_level, result); } else { BT_LOGE("Unknown query object `%s`", object); status = BT_COMPONENT_CLASS_QUERY_METHOD_STATUS_INVALID_OBJECT; diff --git a/src/plugins/ctf/fs-src/query.c b/src/plugins/ctf/fs-src/query.c index b9811b19..872caddf 100644 --- a/src/plugins/ctf/fs-src/query.c +++ b/src/plugins/ctf/fs-src/query.c @@ -484,7 +484,7 @@ bt_component_class_query_method_status trace_info_query( bt_component_class_query_method_status status = BT_COMPONENT_CLASS_QUERY_METHOD_STATUS_OK; bt_value *result = NULL; - const bt_value *paths_value = NULL; + const bt_value *inputs_value = NULL; int ret = 0; guint i; @@ -501,12 +501,12 @@ bt_component_class_query_method_status trace_info_query( goto error; } - if (!read_src_fs_parameters(params, &paths_value, ctf_fs)) { + if (!read_src_fs_parameters(params, &inputs_value, ctf_fs)) { status = BT_COMPONENT_CLASS_QUERY_METHOD_STATUS_INVALID_PARAMS; goto error; } - if (ctf_fs_component_create_ctf_fs_traces(NULL, ctf_fs, paths_value)) { + if (ctf_fs_component_create_ctf_fs_traces(NULL, ctf_fs, inputs_value)) { goto error; } @@ -563,3 +563,76 @@ end: *user_result = result; return status; } + +BT_HIDDEN +bt_component_class_query_method_status support_info_query( + bt_self_component_class_source *comp_class, + const bt_value *params, bt_logging_level log_level, + const bt_value **user_result) +{ + const bt_value *input_type_value; + const char *input_type; + bt_component_class_query_method_status status; + double weight = 0; + gchar *metadata_path = NULL; + bt_value *result = NULL; + + input_type_value = bt_value_map_borrow_entry_value_const(params, "type"); + BT_ASSERT(input_type_value); + BT_ASSERT(bt_value_get_type(input_type_value) == BT_VALUE_TYPE_STRING); + input_type = bt_value_string_get(input_type_value); + + result = bt_value_map_create(); + if (!result) { + status = BT_COMPONENT_CLASS_QUERY_METHOD_STATUS_MEMORY_ERROR; + goto end; + } + + if (strcmp(input_type, "directory") == 0) { + const bt_value *input_value; + const char *path; + + input_value = bt_value_map_borrow_entry_value_const(params, "input"); + BT_ASSERT(input_value); + BT_ASSERT(bt_value_get_type(input_value) == BT_VALUE_TYPE_STRING); + path = bt_value_string_get(input_value); + + metadata_path = g_build_filename(path, CTF_FS_METADATA_FILENAME, NULL); + if (!metadata_path) { + status = BT_COMPONENT_CLASS_QUERY_METHOD_STATUS_MEMORY_ERROR; + goto end; + } + + /* + * If the metadata file exists in this directory, consider it to + * be a CTF trace. + */ + if (g_file_test(metadata_path, G_FILE_TEST_EXISTS)) { + weight = 0.5; + } + } + + if (bt_value_map_insert_real_entry(result, "weight", weight) != BT_VALUE_MAP_INSERT_ENTRY_STATUS_OK) { + status = BT_COMPONENT_CLASS_QUERY_METHOD_STATUS_MEMORY_ERROR; + goto end; + } + + /* + * Use the arbitrary constant string "ctf" as the group, such that all + * found ctf traces are passed to the same instance of src.ctf.fs. + */ + if (bt_value_map_insert_string_entry(result, "group", "ctf") != BT_VALUE_MAP_INSERT_ENTRY_STATUS_OK) { + status = BT_COMPONENT_CLASS_QUERY_METHOD_STATUS_MEMORY_ERROR; + goto end; + } + + *user_result = result; + result = NULL; + status = BT_COMPONENT_CLASS_QUERY_METHOD_STATUS_OK; + +end: + g_free(metadata_path); + bt_value_put_ref(result); + + return status; +} diff --git a/src/plugins/ctf/fs-src/query.h b/src/plugins/ctf/fs-src/query.h index 50ad5870..527474fa 100644 --- a/src/plugins/ctf/fs-src/query.h +++ b/src/plugins/ctf/fs-src/query.h @@ -40,4 +40,10 @@ bt_component_class_query_method_status trace_info_query( const bt_value *params, bt_logging_level log_level, const bt_value **result); +BT_HIDDEN +bt_component_class_query_method_status support_info_query( + bt_self_component_class_source *comp_class, + const bt_value *params, bt_logging_level log_level, + const bt_value **result); + #endif /* BABELTRACE_PLUGIN_CTF_FS_QUERY_H */ diff --git a/tests/bindings/python/bt2/test_trace_collection_message_iterator.py b/tests/bindings/python/bt2/test_trace_collection_message_iterator.py index f3b33763..63198c68 100644 --- a/tests/bindings/python/bt2/test_trace_collection_message_iterator.py +++ b/tests/bindings/python/bt2/test_trace_collection_message_iterator.py @@ -38,7 +38,7 @@ class ComponentSpecTestCase(unittest.TestCase): def test_create_good_with_path_params(self): spec = bt2.ComponentSpec('plugin', 'compcls', 'a path') - self.assertEqual(spec.params['paths'], ['a path']) + self.assertEqual(spec.params['inputs'], ['a path']) def test_create_wrong_plugin_name_type(self): with self.assertRaises(TypeError): @@ -142,7 +142,7 @@ class TraceCollectionMessageIteratorTestCase(unittest.TestCase): hist = _count_msgs_by_type(msgs) self.assertEqual(hist[bt2.message._EventMessage], 3) - def test_iter_intersection_no_path_param(self): + def test_iter_intersection_no_inputs_param(self): specs = [bt2.ComponentSpec('text', 'dmesg', {'read-from-stdin': True})] with self.assertRaises(bt2.Error): diff --git a/tests/cli/auto-source-discovery/test_auto_source_discovery b/tests/cli/auto-source-discovery/test_auto_source_discovery new file mode 100755 index 00000000..208928dc --- /dev/null +++ b/tests/cli/auto-source-discovery/test_auto_source_discovery @@ -0,0 +1,57 @@ +#!/bin/bash +# +# Copyright (C) 2019 Simon Marchi +# +# This program is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License, version 2 only, as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +# more details. +# +# You should have received a copy of the GNU General Public License along with +# this program; if not, write to the Free Software Foundation, Inc., 51 +# Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +# Test the auto source disovery mechanism of the CLI. + +if [ "x${BT_TESTS_SRCDIR:-}" != "x" ]; then + UTILSSH="$BT_TESTS_SRCDIR/utils/utils.sh" +else + UTILSSH="$(dirname "$0")/../../utils/utils.sh" +fi + +# shellcheck source=../../utils/utils.sh +SH_TAP=1 source "$UTILSSH" + +NUM_TESTS=2 + +plan_tests $NUM_TESTS + +data_dir=$(readlink -f "${BT_TESTS_DATADIR}/cli/auto-source-discovery") +plugin_dir="${data_dir}" +trace_dir="${data_dir}/traces" + +expected_file=$(mktemp expected.XXXXXX) +actual_file=$(mktemp actual.XXXXXX) + +run_python_bt2 "$BT_TESTS_BT2_BIN" convert --plugin-path "${plugin_dir}" "ABCDE" "${trace_dir}" some_other_leftover | sort > "$actual_file" +ok "${PIPESTATUS[0]}" "successful execution" + +cat > "$expected_file" <