--- /dev/null
+
+RFC - libside
+
+[ This document is under heavy construction. Please beware of the
+ potholes as you wander through it. ]
+
+* Introduction
+
+The purpose of the libside API/ABI is to allow a kernel tracer and many
+user-space tracers to attach to static and dynamic instrumentation of
+user-space applications.
+
+The libside library expresses the instrumentation description as data
+(no generated code). Instrumentation arguments are passed on the stack
+as an array of typed items, along with a reference to the
+instrumentation description.
+
+-- TODO API vs ABI
+This library exposes a type system and a set of macros to help
+applications declare their instrumentation and insert instrumentation
+calls. It exposes APIs to kernel and user-space tracers so they can list
+and connect to the instrumentation, and conditionally enables
+instrumentation when at least one tracer is using it.
+
+The type system includes support for statically known types and dynamic
+types. Nested structures, arrays, and variable-length arrays are
+supported.
+
+This library learns from the user feedback about experience with
+LTTng-UST and Linux kernel tracepoints, and therefore it introduces
+significant changes (and vast simplifications) to the way
+instrumentation is done compared to LTTng-UST and Linux kernel
+tracepoints.
+
+
+* Genesis
+
+- Linux kernel User Events ABI
+ - Exposes a stable ABI allowing applications to register their event
+ names/field types to the kernel,
+ - Can be expected to have a large effect on application instrumentation,
+ - My concerns:
+ – Should be co-designed with a userspace instrumentation API/ABI rather than only
+ focusing on the kernel ABI,
+ – Should allow purely userspace tracers to use the same instrumentation as userspace
+ tracers implemented within the Linux kernel,
+ – Tracers can target their specific use-cases, but infrastructure should be shared,
+ – Limit fragmentation of the instrumentation ecosystem.
+
+- Improvements over tracepoints:
+ - Improve compiler error reporting vs tracepoints
+ - API uses standard header inclusion practices
+ - share ABI across runtimes (no need to reimplement tracepoints for
+ each language, or to use string only payloads)
+- Improvements over SDT: allow expressing additional event semantic
+ (e.g. user attributes, versioning, nested and compound data types)
+ - libside has less impact on control flow when disabled (no stack setup)
+ - SDT ABI is focused on architecture calling conventions, libside ABI
+ is easier to use from runtime environments which have an ABI
+ different from the native architecture (golang, rust, python, java).
+ libside instrumentation ABI calls a small fixed set of functions.
+- Comparison with ETW
+ - similar to libside in terms of array of arguments,
+ - does not support pre-registration of events (static typing)
+ - type information received at runtime from the instrumentation
+ callsite.
+
+* Desiderata
+
+- Common instrumentation for kernel and purely userspace tracers,
+ - Instrumentation is self-described,
+ - Support compound and nested types,
+ - Support pre-registration of events,
+ - Do not rely on compiled event-specific code,
+ - Independent from ELF,
+ - Simple ABI for instrumented code, kernel, and user-space tracers,
+ - Support concurrent tracers,
+ - Expose API to allow dynamic instrumentation libraries to register
+ their events/payloads.
+
+- Support statically typed instrumentation
+
+- Support dynamically typed instrumentation
+ - Natively cover dynamically-typed languages
+ - The support for events with dynamic fields allows lessening the number
+ of statically declared events in situation where an application
+ possesses seldom-used events with a large variety of parameter types.
+ - The support for mixed static and dynamic event fields allows
+ implementation of post-processing string formatting along with a
+ variadic payload, while keeping trace data in a structured format.
+
+- Performance considerations for userspace tracers.
+ - Maintain performance characteristics comparable to existing
+ userspace tracers.
+ - Low overhead, good scalability when used by userspace tracers.
+
+- Allows tracing user-space through a kernel tracer. Even through it is
+ an approach that adds more overhead, it has the benefit of not
+ requiring agent threads to be deployed into applications, which is
+ useful to trace locked-down processes.
+
+- Instrumentation registration APIs
+ - Instrumentation can be generated at runtime
+ - dynamic patching,
+ - JIT
+ - Instrumentation can be declared statically (static instrumentation)
+ - Instrumentation can be enabled dynamically.
+ - Very low overhead when not in use.
+
+- libside must be extensible in the future.
+ - Extension scheme should allow adding new types in the future without
+ requiring complex logic to future-proof tracers.
+ - Exposed types are invariant,
+ - libside ABI and API can be extended by adding new types.
+
+- the side ABI should allow multiple instances and versions within
+ a process (e.g. libside for C/C++, Java side ABI, Python side ABI...).
+
+- Both event description and payload are data (no generated text).
+ - It allows tracers to directly interpret the event payload from their
+ description, removing the need for code generation. This lessens the
+ instruction cache pollution compared to code generation approaches.
+ - Tracer interpreter for filtering and field capture can directly use
+ the instrumentation data, without need for setting up a structured
+ argument layout on the stack within the tracer.
+
+- Validation of argument vs event description coherence.
+
+- Passing arguments to events should be:
+ - Conveniently express application data structures to be expected as
+ instrumentation input.
+ - Flexible,
+ - Efficient,
+ - If all are not possible combined, specialize types for each purpose.
+
+- Allow tracers to passively collect application state transitions.
+
+- Allow tracers to actively sample the current state of an application.
+
+- Error messages generated when misusing the API should be easy to
+ comprehend and resolve.
+
+- Allow expressing additional custom semantic augmenting events and
+ types.
+
+
+* Design / Architecture
+
+
+- Compiler error messages are easy to understand because it is a simple
+ header file without any repeated inclusion tricks.
+
+
+- Variadic events.
+
+
+- Instrumentation API/ABI:
+ – Type system,
+ - Type visitor callbacks
+ - (perfetto)
+ - Stack-copy types
+ - Data-gathering types
+ - Dynamic types.
+ – Helper macros for C/C++,
+ – Express instrumentation description as data,
+ – Instrumentation arguments are passed on the stack as a data array
+ (similar to iovec) along with a reference to instrumentation
+ description,
+ – Instrumentation is conditionally enabled when at least one tracer is
+ registered to it.
+
+- Tracer-agnostic API/ABI:
+ – Available events notifications,
+ – Conditionally enabling instrumentation,
+ – Synchronize registered user-space tracer callbacks with RCU,
+ – Co-designed to interact with User Events.
+
+- Application state dump
+ - How are applications/libraries meant to provide state information ?
+ - How are tracers meant to interact with state dump ?
+ - statedump mode polling
+ - statedump mode agent thread
+
+- RCU to synchronize userspace tracers registration vs invocation
+
+- How tracers are meant to interact with libside ?
+
+- How is C/C++ language instrumentation is meant to be used ?
+
+- How are dynamic instrumentation facilities meant to interact with
+ libside ?
+
+- How is a kernel tracer meant to interact with libside ?
+
+- How is gdb (ptrace) meant to interact with libside ?
+
+- Validation that instrumentation arguments match event description
+ fields cannot be done by the compiler, requires either:
+ - run time check,
+ - static checker (only for static instrumentation).
+
+- Event attributes.
+
+- Type attributes.
+
+
+
+
union {
void (*call)(const struct side_event_description *desc,
const struct side_arg_vec *side_arg_vec,
- void *priv);
+ void *priv, void *caller_addr);
void (*call_variadic)(const struct side_event_description *desc,
const struct side_arg_vec *side_arg_vec,
const struct side_arg_dynamic_struct *var_struct,
- void *priv);
+ void *priv, void *caller_addr);
} u;
void *priv;
uint64_t key;
*/
void side_ptrace_hook(const struct side_event_state *event_state __attribute__((unused)),
const struct side_arg_vec *side_arg_vec __attribute__((unused)),
- const struct side_arg_dynamic_struct *var_struct __attribute__((unused)))
+ const struct side_arg_dynamic_struct *var_struct __attribute__((unused)),
+ void *caller_addr __attribute__((unused)))
__attribute__((noinline));
void side_ptrace_hook(const struct side_event_state *event_state __attribute__((unused)),
const struct side_arg_vec *side_arg_vec __attribute__((unused)),
- const struct side_arg_dynamic_struct *var_struct __attribute__((unused)))
+ const struct side_arg_dynamic_struct *var_struct __attribute__((unused)),
+ void *caller_addr __attribute__((unused)))
{
}
-static
+static inline __attribute__((always_inline))
void _side_call(const struct side_event_state *event_state, const struct side_arg_vec *side_arg_vec, uint64_t key)
{
+ void *caller_addr = __builtin_return_address(0);
struct side_rcu_read_state rcu_read_state;
const struct side_event_state_0 *es0;
const struct side_callback *side_cb;
}
if ((enabled & SIDE_EVENT_ENABLED_SHARED_PTRACE_MASK) &&
(key == SIDE_KEY_MATCH_ALL || key == SIDE_KEY_PTRACE))
- side_ptrace_hook(event_state, side_arg_vec, NULL);
+ side_ptrace_hook(event_state, side_arg_vec, NULL, caller_addr);
}
side_rcu_read_begin(&event_rcu_gp, &rcu_read_state);
for (side_cb = side_rcu_dereference(es0->callbacks); side_cb->u.call != NULL; side_cb++) {
if (key != SIDE_KEY_MATCH_ALL && side_cb->key != SIDE_KEY_MATCH_ALL && side_cb->key != key)
continue;
- side_cb->u.call(es0->desc, side_arg_vec, side_cb->priv);
+ side_cb->u.call(es0->desc, side_arg_vec, side_cb->priv, caller_addr);
}
side_rcu_read_end(&event_rcu_gp, &rcu_read_state);
}
_side_call(event_state, side_arg_vec, *(const uint64_t *) statedump_request_key);
}
-static
+static inline __attribute__((always_inline))
void _side_call_variadic(const struct side_event_state *event_state,
const struct side_arg_vec *side_arg_vec,
const struct side_arg_dynamic_struct *var_struct,
uint64_t key)
{
+ void *caller_addr = __builtin_return_address(0);
struct side_rcu_read_state rcu_read_state;
const struct side_event_state_0 *es0;
const struct side_callback *side_cb;
}
if ((enabled & SIDE_EVENT_ENABLED_SHARED_PTRACE_MASK) &&
(key == SIDE_KEY_MATCH_ALL || key == SIDE_KEY_PTRACE))
- side_ptrace_hook(event_state, side_arg_vec, var_struct);
+ side_ptrace_hook(event_state, side_arg_vec, var_struct, caller_addr);
}
side_rcu_read_begin(&event_rcu_gp, &rcu_read_state);
for (side_cb = side_rcu_dereference(es0->callbacks); side_cb->u.call_variadic != NULL; side_cb++) {
if (key != SIDE_KEY_MATCH_ALL && side_cb->key != SIDE_KEY_MATCH_ALL && side_cb->key != key)
continue;
- side_cb->u.call_variadic(es0->desc, side_arg_vec, var_struct, side_cb->priv);
+ side_cb->u.call_variadic(es0->desc, side_arg_vec, var_struct, side_cb->priv, caller_addr);
}
side_rcu_read_end(&event_rcu_gp, &rcu_read_state);
}
static
void tracer_print_static_fields(const struct side_event_description *desc,
const struct side_arg_vec *side_arg_vec,
- uint32_t *nr_items)
+ uint32_t *nr_items, void *caller_addr)
{
const struct side_arg *sav = side_ptr_get(side_arg_vec->sav);
uint32_t i, side_sav_len = side_arg_vec->len;
- printf("provider: %s, event: %s", side_ptr_get(desc->provider_name), side_ptr_get(desc->event_name));
+ printf("caller: [%p], provider: %s, event: %s", caller_addr,
+ side_ptr_get(desc->provider_name),
+ side_ptr_get(desc->event_name));
if (desc->nr_fields != side_sav_len) {
fprintf(stderr, "ERROR: number of fields mismatch between description and arguments\n");
abort();
static
void tracer_call(const struct side_event_description *desc,
const struct side_arg_vec *side_arg_vec,
- void *priv __attribute__((unused)))
+ void *priv __attribute__((unused)),
+ void *caller_addr)
{
uint32_t nr_fields = 0;
- tracer_print_static_fields(desc, side_arg_vec, &nr_fields);
+ tracer_print_static_fields(desc, side_arg_vec, &nr_fields, caller_addr);
printf("\n");
}
void tracer_call_variadic(const struct side_event_description *desc,
const struct side_arg_vec *side_arg_vec,
const struct side_arg_dynamic_struct *var_struct,
- void *priv __attribute__((unused)))
+ void *priv __attribute__((unused)),
+ void *caller_addr)
{
uint32_t nr_fields = 0, i, var_struct_len = var_struct->len;
- tracer_print_static_fields(desc, side_arg_vec, &nr_fields);
+ tracer_print_static_fields(desc, side_arg_vec, &nr_fields, caller_addr);
if (side_unlikely(!(desc->flags & SIDE_EVENT_FLAG_VARIADIC))) {
fprintf(stderr, "ERROR: unexpected non-variadic event description\n");
projects / libside.git / commitdiff
