cli: automatically detect sources for leftover arguments

cli: automatically detect sources for leftover arguments

This patch adds a source auto-discovery feature.  The goal is for the
user to be able to just pass some inputs (which can be strings of a form
understood by a particular component class, or paths to
files/directories) and for babeltrace to figure out which component
classes are best suited to handle those inputs.

Currently, any leftover argument passed by the user is passed to a
src.ctf.fs instance.  To use a different source component, the user must
use the --component argument.  This system will therefore help usability
for users of non-src.ctf.fs sources.  It will also allow splitting the
src.ctf.fs source into a "generic CTF" one and an "LTTng CTF" one, which
includes the fixups specific to CTF files produced by various LTTng
versions.

The big picture is that for each leftover argument (called `input`), we
ask all component classes if they can handle it (see `support-info
query` below), the component classes reply with a weight, and the input
is attributed to the component class that gave the largest weight.

More precisely, this is what we do for each leftover argument:

1. Ask all known source component classes if they recognize the
   argument as an arbitrary string, which would be in a format that
   makes sense to them (but doesn't point to a file or directory on
   disk, unless it's a coincidence).  The obvious example is
   src.ctf.lttng-live, which would be apt to handle paths of the form
   'net://...'.  If some component classes claim to understand the
   argument, the one with the largest weight is chosen, and a component
   of that class will be instantiated.

2. If no component class has claimed the argument as an arbitrary
   string and the argument points to a file or directory on disk, ask
   them all if they recognize it as a file or directory they can handle.
   If some component classes do, choose the one with the biggest weight.

3. If no component class has recognized it so far, and the input points
   to a directory, we start to dig: for each child of that directory,
   apply the sophisticated algorithm described in step 2.

If a leftover argument (including all its children, if it's a directory)
is not handled by any source component class, we show a warning.  If
nothing is discovered and no explicit source is instantiated either, the
excecution fails on the "No source component" check that is already
there.

Component classes have the ability to "group" their inputs as they wish.
This means that multiple inputs attributed to a given component class
can be passed to a single instance of that class, all to separate
instances, or any combination in between.  To achieve this, component
classes are able to also reply with a group key (a string of their
choice).  Inputs with the same group key will be passed to the same
instance.

Implementation details and choices
----------------------------------

* Since leftovers are now passed to the source auto-discovery mechanism
  as opposed to an implicit src.ctf.fs component previously, this
  src.ctf.fs implicit component is no longer needed.  This changes how we
  handle some of the legacy (compatibility with babeltrace 1) flags.

  If the user passes --clock-offset or --clock-offset-ns, we will search
  in the auto-discovered sources and apply it to any src.ctf.fs instance
  created.  If no src.ctf.fs component would be instantiated, we issue
  an error.

  If the user passes --input-format ctf, we don't want other formats
  possibly being read.  We still use the auto-discovery mechanism, but
  we restrict it to the src.ctf.fs component class (other component
  classes won't be queried and therefore won't be instantiated).

* This also means that to keep the basic use case of "babeltrace2
  <dir-with-ctf-traces>" working, we need to implement the support-info
  query for the src.ctf.fs component class in the current patch.  Not
  doing so immediatly would break many tests.  The simplest possible
  implementation was added.  It looks for inputs of type "directory",
  which have a "metadata" file in it.  It always reply with the same
  group value, such that a single instance of the component class is
  used (keeping the current behavior).

* Since the strings we pass to support-info queries (and eventually to
  components we instantiate) are not necessarily paths to directories or
  files on disk (they can be URLs, for example), we now use the term
  "inputs" rather than "paths" for all of them.

* A support-info query returning ERROR aborts the auto-discovery
  process, making it return an ERROR as well.

* If we can't open a directory because we don't have permission to read
  it (EACCES), we log a warning and continue.  Other errors abort the
  auto-discovery process.

support-info query
------------------

The support-info query is a contract between the CLI and source
component classes.  Source component classes that don't support it will
simply not be able to automatically discover inputs, and will have to be
explicitly instantiated using --component.

The parameters of a support-info query are a map containing these keys:

- `type` (string): possible values are "string", "directory" or "file",
  indicating the nature of `input`, described below.  The value "string"
  means that the component class may try to interpret `input` as a
  string with a format it can recognize.  "directory" and "file"
  respectively mean to interpret `input` as a directory and file.  When
  type is "directory" or "file", the component class can assume that
  the corresponding directory or file exists on disk.
- input (string): input descriptor, to be interpreted according to
  `type`.

A support-info response can be

- A real or integer value between 0 and 1, representing the weight
- A map value containing these keys:
  - `weight` (real or integer), mandatory: between 0 and 1
  - `group` (string or null), optional: a key by which to group inputs,
    when instantiating components.

A component class that does not support a given must reply with a weight
of 0.

All inputs attributed to the same component class, sharing the same
group key, will be passed to the same component instance.  inputs whose
group key is missing or null are not grouped with other inputs.

Components created by the auto-discovery mechanism are passed the
`inputs` parameter, an array of strings containing all inputs attributed
to this instance.

testing
-------

I have a brief catch-all test for this, it just covers a few important
cases.  The test strategy is the following:

- Run babeltrace2 with an arbitrary string and a directory as
  leftovers.
- One source recognizes the arbitrary string.
- Various sources recognizes files and directories inside the passed
  directory.
- Each instantiated source outputs one line including its name and the
  sorted list of its inputs.
- We sort the output of babeltrace and compare it with an expected
  string.  Since everything is sorted, the output should be stable.

Change-Id: I7f884551d7cb576974ea53420ead9c4a8005e99d
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/1644
Tested-by: jenkins <jenkins@lttng.org>
Reviewed-by: Philippe Proulx <eeppeliteloop@gmail.com>