RFC - LTTng session and daemon configuration save and restore Author: Jérémie Galarneau Version: - v0.2: 04/12/2013 * Reorganized the text following David Goulet's recommendations - v0.1: 03/12/2013 * Initial proposal Introduction ------------- This proposal details the file formats, LTTng APIs and command line interfaces used to save and restore session and daemon configurations. Daemon Configuration File Format -------------------------------- The LTTng session and relay daemons shall be configurable using INI configuration files. The set of configuration options accepted by LTTng daemons corresponds to the command line options available for each daemon. The sections in the files must match the daemons' names: [sessiond] and [relayd]. The entries under each section must match the long option names supported by the daemons. Options specified on the command line shall always have precedence over options set by the configuration files. An example demonstrating the use of such configuration options follows. [sessiond] consumerd32-path=/lib/lttng/libexec/lttng-consumerd consumerd64-path=/lib64/lttng/libexec/lttng-consumerd no-kernel=1 [relayd] data-port=tcp://0.0.0.0:5343 output=/tmp/MyTraces Daemon configuration files are, by default, either stored system-wide under /etc/lttng/*.conf or in the user's home directory under $HOME/.lttng/*.conf. New options may be added to the daemon configuration file as features are added to the relay and session daemons. The daemons shall ignore unrecognized options to ensure forward compatibility. LTTng Command Line - Daemon Configuration ----------------------------------------- On launch, the session, relay and consumer daemons will search for files with the .conf extension in the following folders, in order of precedence: 1) Any path specified using a new --load-config option 2) $HOME/.lttng/*.conf 3) /etc/lttng/*.conf Session Configuration File Format --------------------------------- The LTTng session configuration file format must enable the serialization of the session, channel and event structures of the LTTng API. Since these concepts are eminently hierarchical, it only makes sense to use a file format that can express such relationships. While an INI based file format was the first considered, for its ease of use and human readability, it appears that INI provides no standard way of expressing hierarchies. It is, of course, possible to flatten session/channel hierarchies using prefixed section names, but this is both error prone and unsupported by most INI file reading libraries. This means that we would have to code our own INI handling code and produce a specification to document our additions to the INI format. INI also provides no standard versioning mechanism. In keeping with the desire to use a human-readable format, we have considered rolling our own format. This format could express hierarchy using tabulations to delimit scopes, not unlike the Python language. Such a format would be both relatively easy to parse, but also readable. However, this both requires documenting the format in a specification, rolling our own parsing and validation code, thus losing the benefit of reusing external libraries. This option has the highest maintenance cost considering the amount of code and documentation to be produced while presenting marginal benefits over the INI approach. Finally, it seems that using a standard hierarchical format such as JSON or XML would be the most appropriate choice. Both of these formats have intrinsic support for representation of hierarchical relationships. They also benefit from having a lot of hardened external libraries that will provide parsing, validation and write support. This is a huge advantage from both maintainability and interoperability standpoints. However, both formats present the disadvantage of being harder, although minimally, to edit manually. It remains to be seen if manual editing of session configurations really is an important use-case. Using lttng's save feature would probably prove more convenient than editing the files manually. Furthermore, the addition of a "dry-run" option to the lttng client would significantly alleviate this pain point. XML seems like a better option than JSON since it makes it possible to have robust file validation using a pre-defined schema. The use of version-specific schemas could also be beneficial for backward compatibility as the format moves forward. Finally, character encoding handling is already a solved problem for most XML libraries. LTTng ABI/API ------------- Two new functions are added to the lttng.h API. int lttng_load_sessions(const char *url, const char *session_name, int flags/struct lttng_session_load_attr *attr); Load sessions. If url is a directory, all .lttng files it contains will be loaded. If session_name is NULL, all sessions found in url are restored. If a session_name is provided, only this session will be restored. A supplementary argument, either a "flags" argument or an attribute structure, is used to indicate whether or not the sessions currently known to the session daemon should be replaced by the ones found in the configuration file(s), provided that their names are the same. Even though this is the only current use-case for this argument, a structure with a reasonable amount of padding may be more suitable than a primary type to accommodate new features. Thoughts? int lttng_save_sessions(const char *url, const char *session_name, int flags/struct lttng_session_save_attr *attr); Save sessions. If url is a directory, the session configuration will be saved as session_name.lttng. If a complete file name is provided, the session(s) to be saved will be written to this file. If session_name is NULL, all sessions currently known to the session daemon will be saved. If a name is provided, only that session will be saved. The reasoning for the flags/attr argument is the same as for the lttng_load_sessions() function, but for a configuration file overwrite option. LTTng Command Line - Session Configuration ------------------------------------------ Tracing session configurations can be saved and restored using the lttng command line client. This capability introduces two new command line options. -- Load session configurations -- $ lttng load -s SESSION_NAME [OPTIONS] Loads tracing session configurations. .lttng files will be searched in the following default paths, in order of precedence: 1) $(HOME)/.lttng/sessions 2) /etc/lttng/sessions A session name or the -a option must be passed as the SESSION_NAME argument. Like some other lttng commands, such as enable-event, the -a option stands for "all". If this option is used, every session found in the paths will be loaded. If a session was saved as active, the tracing for that session will be activated automatically on load. The current session configuration of the session daemon is also preserved when a configuration is loaded. The new session configurations are added to the current session set. Tracing sessions saved in an active state will be resumed on load. --input-path, -i Path in which to search for SESSION_NAME's description. If the path is a directory, all .lttng files it contains will be opened and searched for SESSION_NAME. If an input path is provided, the -a option will load all session configurations found in this path. The default paths are ignored when this option is used. --force, -f If a restored session's name conflicts with a currently existing session, the original session configuration is preserved. However, if the --force option is used, the restored session takes precedence. The original session will be destroyed. -- Save session configurations -- $ lttng save -s SESSION_NAME [OPTIONS] Saves tracing configurations. The selected tracing session's configuration will be saved in the $HOME/.lttng/sessions folder. If the -a option is used, every session known to the session daemon will be saved. --output-path, -o If the provided output path is a directory, the session will be saved in a file named "SESSION_NAME.lttng". However, if the path is a file, the session(s) will all be saved in the same file.