lttng-enable-event.1.txt: document dynamic user space probes

[lttng-tools.git] / doc / man / lttng-enable-event.1.txt

lttng-enable-event(1)
=====================


NAME
----
lttng-enable-event - Create or enable LTTng event rules


SYNOPSIS
--------
Create or enable Linux kernel event rules:

[verse]
*lttng* ['linkgenoptions:(GENERAL OPTIONS)'] *enable-event* option:--kernel
      [option:--probe='SOURCE' | option:--function='SOURCE' | option:--syscall |
       option:--userspace-probe='SOURCE']
      [option:--filter='EXPR'] [option:--session='SESSION']
      [option:--channel='CHANNEL'] 'EVENT'[,'EVENT']...

Create or enable an "all" Linux kernel event rule:

[verse]
*lttng* ['linkgenoptions:(GENERAL OPTIONS)'] *enable-event* option:--kernel option:--all [option:--syscall]
      [option:--filter='EXPR'] [option:--session='SESSION'] [option:--channel='CHANNEL']

Create or enable application/library event rules:

[verse]
*lttng* ['linkgenoptions:(GENERAL OPTIONS)'] *enable-event*
      (option:--userspace | option:--jul | option:--log4j | option:--python)
      [option:--filter='EXPR'] [option:--exclude='EVENT'[,'EVENT']...]
      [option:--loglevel='LOGLEVEL' | option:--loglevel-only='LOGLEVEL']
      [option:--session='SESSION'] [option:--channel='CHANNEL'] (option:--all | 'EVENT'[,'EVENT']...)


DESCRIPTION
-----------
The `lttng enable-event` command can create a new event rule, or enable
one or more existing and disabled ones.

An event rule created by `lttng enable-event` is a set of conditions
that must be satisfied in order for an actual event to be emitted by an
LTTng tracer when the execution of an application or a library or the
Linux kernel reaches an event source (tracepoint, system call, dynamic
probe). Event sources can be listed with the man:lttng-list(1) command.

The man:lttng-disable-event(1) command can be used to disable
existing event rules.

Event rules are always assigned to a channel when they are created. If
the option:--channel option is omitted, a default channel named
`channel0` is used (and created automatically if it does not exist for
the specified domain in the selected tracing session).

If the option:--session option is omitted, the chosen channel is picked
from the current tracing session.

Events can be enabled while tracing is active
(use man:lttng-start(1) to make a tracing session active).


Event source types
~~~~~~~~~~~~~~~~~~
Five types of event sources are available in the Linux kernel tracing
domain (option:--kernel option):

Tracepoint (option:--tracepoint option; default)::
    A Linux kernel tracepoint, that is, a static instrumentation point
    placed in the kernel source code. Standard tracepoints are designed
    and placed in the source code by developers and record useful
    payload fields.

Dynamic kernel probe (option:--probe option)::
    A Linux kernel kprobe, that is, an instrumentation point placed
    dynamically in the compiled kernel code. Dynamic probe events do not
    record any payload field.

Dynamic user space probe (option:--userspace-probe option)::
    A Linux kernel uprobe, that is, an instrumentation point placed
    dynamically in the compiled user space application/library through
    the kernel. Dynamic user space probe events do not record any
    payload field.
+
See the <<userspace-probe,Dynamic user space probes>> section for more
information.

Function probe (option:--function option)::
    A Linux kernel kretprobe, that is, two instrumentation points placed
    dynamically where a function is entered and where it returns in the
    compiled kernel code. Function probe events do not record any
    payload field.

System call (option:--syscall option)::
    A Linux kernel system call. Two instrumentation points are
    statically placed where a system call function is entered and where
    it returns in the compiled kernel code. System call event sources
    record useful payload fields.

The application tracing domains (option:--userspace, option:--jul,
option:--log4j, or option:--python options) only support tracepoints.
In the cases of the JUL, Apache log4j, and Python domains, the event
names correspond to _logger_ names.


Understanding event rule conditions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
When creating an event rule with `lttng enable-event`, conditions are
specified using options. The logical conjunction (logical AND) of all
those conditions must be true when an event source is reached by an
application or by the Linux kernel in order for an actual event
to be emitted by an LTTng tracer.

Any condition that is not explicitly specified on creation is considered
a _don't care_.

For example, consider the following commands:

[role="term"]
----
$ lttng enable-event --userspace hello:world
$ lttng enable-event --userspace hello:world --loglevel=TRACE_INFO
----

Here, two event rules are created. The first one has a single condition:
the tracepoint name must match `hello:world`. The second one has two
conditions:

* The tracepoint name must match `hello:world`, _and_
* The tracepoint's defined log level must be at least as severe as
  the `TRACE_INFO` level.

In this case, the second event rule is pointless because the first one
is more general: it does not care about the tracepoint's log level.
If an event source matching both event rules is reached by the
application's execution, only one event is emitted.

The available conditions for the Linux kernel domain are:

* Tracepoint/system call name ('EVENT' argument with option:--tracepoint
  or option:--syscall options) or dynamic probe/function name/address
  (option:--probe, option:--userspace-probe, and option:--function
  option's argument) which must match event source's equivalent.
+
You can use `*` characters at any place in the tracepoint or system
call name as wildcards to match zero or more characters. To use a
literal `*` character, use :escwc:.

* Filter expression (option:--filter option) executed against the
  dynamic values of event fields at execution time that must evaluate
  to true. See the <<filter-expr,Filter expression>> section
  below for more information.

The available conditions for the application domains are:

* Tracepoint name ('EVENT' with option:--tracepoint option) which must
  match event source's equivalent.
+
You can use `*` characters at any place in the tracepoint name as
wildcards to match zero or more characters. To use a literal `*`
character, use :escwc:. When you create an event rule with a tracepoint
name containing a wildcard, you can exclude specific tracepoint names
from the match with the option:--exclude option.

* Filter expression (option:--filter option) executed against the
  dynamic values of event fields at execution time that must evaluate
  to true. See the <<filter-expr,Filter expression>> section
  below for more information.
* Event's log level that must be at least as severe as a given
  log level (option:--loglevel option) or match exactly a given log
  level (option:--loglevel-only option).

When using `lttng enable-event` with a set of conditions that does not
currently exist for the chosen tracing session, domain, and channel,
a new event rule is created. Otherwise, the existing event rule is
enabled if it is currently disabled
(see man:lttng-disable-event(1)).

The option:--all option can be used alongside the option:--tracepoint
or option:--syscall options. When this option is used, no 'EVENT'
argument must be specified. This option defines a single event rule
matching _all_ the possible events of a given tracing domain for the
chosen channel and tracing session. It is the equivalent of an 'EVENT'
argument named `*` (wildcard).


[[filter-expr]]
Filter expression
~~~~~~~~~~~~~~~~~
A filter expression can be specified with the option:--filter option
when creating a new event rule. If the filter expression evaluates
to true when executed against the dynamic values of an event's fields
when tracing, the filtering condition passes.

NOTE: Make sure to **single-quote** the filter expression when running
the command from a shell, as filter expressions typically include
characters having a special meaning for most shells.

The filter expression syntax is similar to C language conditional
expressions (expressions that can be evaluated by an `if` statement),
albeit with a few differences:

* C integer and floating point number constants are supported, as well
  as literal strings between double quotes (`"`). You can use `*`
  characters at any place in a literal string as wildcards to match zero
  or more characters. To use a literal `*` character, use :escwc:.
+
Examples: `32`, `-0x17`, `0755`, `12.34`,
+"a :escbs:"literal string:escbs:""+, `"src/*/*.h"`.

* The dynamic value of an event field is read by using its name as a C
  identifier.
+
The dot and square bracket notations are available, like in the C
language, to access nested structure and array/sequence fields.
Only a constant, positive integer number can be used within square
brackets. If the index is out of bounds, the whole filter expression
evaluates to false (the event is discarded).
+
An enumeration field's value is an integer.
+
When the expression's field does not exist, the whole filter expression
evaluates to false.
+
Examples: `my_field`, `target_cpu`, `seq[7]`, `msg.user[1].data[2][17]`.

* The dynamic value of a statically-known context field is read by
  prefixing its name with `$ctx.`. Statically-known context fields are
  context fields added to channels without the `$app.` prefix using the
  man:lttng-add-context(1) command.
+
When the expression's statically-known context field does not exist,
the whole filter expression evaluates to false.
+
Examples: `$ctx.prio`, `$ctx.preemptible`,
`$ctx.perf:cpu:stalled-cycles-frontend`.

* The dynamic value of an application-specific context field is read by
  prefixing its name with `$app.` (follows the format used to add such a
  context field with the man:lttng-add-context(1) command).
+
When the expression's application-specific context field does not exist,
the whole filter expression evaluates to false.
+
Example: `$app.server:cur_user`.

The following precedence table shows the operators which are supported
in a filter expression. In this table, the highest precedence is 1.
Parentheses are supported to bypass the default order.

IMPORTANT: Unlike the C language, the `lttng enable-event` filter
expression syntax's bitwise AND and OR operators (`&` and `|`) take
precedence over relational operators (`<`, `<=`, `>`, `>=`, `==`, and
`!=`). This means the filter expression `2 & 2 == 2` is true while the
equivalent C expression is false.

[options="header"]
|===
|Precedence |Operator |Description |Associativity
|1 |`-` |Unary minus |Right-to-left
|1 |`+` |Unary plus |Right-to-left
|1 |`!` |Logical NOT |Right-to-left
|1 |`~` |Bitwise NOT |Right-to-left
|2 |`<<` |Bitwise left shift |Left-to-right
|2 |`>>` |Bitwise right shift |Left-to-right
|3 |`&` |Bitwise AND |Left-to-right
|4 |`^` |Bitwise XOR |Left-to-right
|5 |`\|` |Bitwise OR |Left-to-right
|6 |`<` |Less than |Left-to-right
|6 |`<=` |Less than or equal to |Left-to-right
|6 |`>` |Greater than |Left-to-right
|6 |`>=` |Greater than or equal to |Left-to-right
|7 |`==` |Equal to |Left-to-right
|7 |`!=` |Not equal to |Left-to-right
|8 |`&&` |Logical AND |Left-to-right
|9 |`\|\|` |Logical OR |Left-to-right
|===

The arithmetic operators are :not: supported.

All integer constants and fields are first casted to signed 64-bit
integers. The representation of negative integers is two's complement.
This means that, for example, the signed 8-bit integer field 0xff (-1)
becomes 0xffffffffffffffff (still -1) once casted.

Before a bitwise operator is applied, all its operands are casted to
unsigned 64-bit integers, and the result is casted back to a signed
64-bit integer. For the bitwise NOT operator, it is the equivalent of
this C expression:

[source,c]
----
(int64_t) ~((uint64_t) val)
----

For the binary bitwise operators, it is the equivalent of those C
expressions:

[source,c]
----
(int64_t) ((uint64_t) lhs >> (uint64_t) rhs)
(int64_t) ((uint64_t) lhs << (uint64_t) rhs)
(int64_t) ((uint64_t) lhs & (uint64_t) rhs)
(int64_t) ((uint64_t) lhs ^ (uint64_t) rhs)
(int64_t) ((uint64_t) lhs | (uint64_t) rhs)
----

If the right-hand side of a bitwise shift operator (`<<` and `>>`) is
not in the [0,{nbsp}63] range, the whole filter expression evaluates to
false.

NOTE: Although it is possible to filter the process ID of an event when
the `pid` context has been added to its channel using, for example,
`$ctx.pid == 2832`, it is recommended to use the PID tracker instead,
which is much more efficient (see man:lttng-track(1)).

Filter expression examples:

----------------------------
msg_id == 23 && size >= 2048
----------------------------

-------------------------------------------------
$ctx.procname == "lttng*" && (!flag || poel < 34)
-------------------------------------------------

---------------------------------------------------------
$app.my_provider:my_context == 17.34e9 || some_enum >= 14
---------------------------------------------------------

---------------------------------------
$ctx.cpu_id == 2 && filename != "*.log"
---------------------------------------

------------------------------------------------
eax_reg & 0xff7 == 0x240 && x[4] >> 12 <= 0x1234
------------------------------------------------


[[log-levels]]
Log levels
~~~~~~~~~~
Tracepoints and log statements in applications have an attached log
level. Application event rules can contain a _log level_ condition.

With the option:--loglevel option, the event source's log level must
be at least as severe as the option's argument. With the
option:--loglevel-only option, the event source's log level must match
the option's argument.

The available log levels are:

User space domain (option:--userspace option)::
    Shortcuts such as `system` are allowed.
+
* `TRACE_EMERG` (0)
* `TRACE_ALERT` (1)
* `TRACE_CRIT` (2)
* `TRACE_ERR` (3)
* `TRACE_WARNING` (4)
* `TRACE_NOTICE` (5)
* `TRACE_INFO` (6)
* `TRACE_DEBUG_SYSTEM` (7)
* `TRACE_DEBUG_PROGRAM` (8)
* `TRACE_DEBUG_PROCESS` (9)
* `TRACE_DEBUG_MODULE` (10)
* `TRACE_DEBUG_UNIT` (11)
* `TRACE_DEBUG_FUNCTION` (12)
* `TRACE_DEBUG_LINE` (13)
* `TRACE_DEBUG` (14)

`java.util.logging` domain (option:--jul option)::
    Shortcuts such as `severe` are allowed.
+
* `JUL_OFF` (`INT32_MAX`)
* `JUL_SEVERE` (1000)
* `JUL_WARNING` (900)
* `JUL_INFO` (800)
* `JUL_CONFIG` (700)
* `JUL_FINE` (500)
* `JUL_FINER` (400)
* `JUL_FINEST` (300)
* `JUL_ALL` (`INT32_MIN`)

Apache log4j domain (option:--log4j option)::
    Shortcuts such as `severe` are allowed.
+
* `LOG4J_OFF` (`INT32_MAX`)
* `LOG4J_FATAL` (50000)
* `LOG4J_ERROR` (40000)
* `LOG4J_WARN` (30000)
* `LOG4J_INFO` (20000)
* `LOG4J_DEBUG` (10000)
* `LOG4J_TRACE` (5000)
* `LOG4J_ALL` (`INT32_MIN`)

Python domain (option:--python option)::
    Shortcuts such as `critical` are allowed.
+
* `PYTHON_CRITICAL` (50)
* `PYTHON_ERROR` (40)
* `PYTHON_WARNING` (30)
* `PYTHON_INFO` (20)
* `PYTHON_DEBUG` (10)
* `PYTHON_NOTSET` (0)


[[userspace-probe]]
Dynamic user space probes
~~~~~~~~~~~~~~~~~~~~~~~~~
With the option:--userspace-probe option, you can instrument function
entries of any user space binary (application or library) using either
an available symbol name or a SystemTap SDT probe's provider and probe
names.

The option:--userspace-probe option must be specified with the
option:--kernel option because it uses Linux's uprobe feature to
dynamically instrument a user space application or library.

As of this version, dynamic probe events do not record any payload
field.

The two available option:--userspace-probe option's argument formats
are:

option:--userspace-probe=`[elf:]PATH:SYMBOL`::
    Dynamically instrument an available symbol within a user space
    executable.
+
--
'PATH'::
    Application or library path.
+
This can be:
+
* An absolute path.
* A relative path.
* An executable's name as found in the directories listed in the
  `PATH` environment variable.

'SYMBOL'::
    Symbol name of the function of which to instrument the entry.
+
This can be any defined code symbol listed by the man:nm(1) command
(including with its nloption:--dynamic option which lists dynamic
symbols).
--
+
As of this version, not specifying `elf:` is equivalent to specifying
it.
+
Examples:
+
* `--userspace-probe=/usr/lib/libc.so.6:malloc`
* `--userspace-probe=./myapp:createUser`
* `--userspace-probe=httpd:ap_run_open_htaccess`

option:--userspace-probe=`sdt:PATH:PROVIDER:NAME`::
    Dynamically instrument an SDT probe within a user space executable.
+
--
'PATH'::
    Application or library path.
+
This can be:
+
* An absolute path.
* A relative path.
* An executable's name as found in the directories listed in the
  `PATH` environment variable.

__PROVIDER__:__NAME__::
    SDT provider and probe names.
+
For example, with the following SDT probe:
+
[source,c]
----
DTRACE_PROBE2("server", "accept_request",
              request_id, ip_addr);
----
+
The provider/probe name pair is `server:accept_request`.
--
+
Example:
+
* `--userspace-probe=sdt:./build/server:server:accept_request`


include::common-cmd-options-head.txt[]


Domain
~~~~~~
One of:

option:-j, option:--jul::
    Create or enable event rules in the `java.util.logging`
    (JUL) domain.

option:-k, option:--kernel::
    Create or enable event rules in the Linux kernel domain.

option:-l, option:--log4j::
    Create or enable event rules in the Apache log4j domain.

option:-p, option:--python::
    Create or enable event rules in the Python domain.

option:-u, option:--userspace::
    Create or enable event rules in the user space domain.


Target
~~~~~~
option:-c 'CHANNEL', option:--channel='CHANNEL'::
    Create or enable event rules in the channel named 'CHANNEL' instead
    of the default channel name `channel0`.

option:-s 'SESSION', option:--session='SESSION'::
    Create or enable event rules in the tracing session named 'SESSION'
    instead of the current tracing session.


Event source type
~~~~~~~~~~~~~~~~~
One of:

option:--function='SOURCE'::
    Linux kernel kretprobe. Only available with the option:--kernel
    domain option. 'SOURCE' is one of:
+
* Function address (`0x` prefix supported)
* Function symbol
* Function symbol and offset (`SYMBOL+OFFSET` format)

option:--probe='SOURCE'::
    Linux kernel kprobe. Only available with the option:--kernel
    domain option. 'SOURCE' is one of:
+
* Address (`0x` prefix supported)
* Symbol
* Symbol and offset (`SYMBOL+OFFSET` format)

option:--userspace-probe='SOURCE'::
    Linux kernel uprobe. Only available with the option:--kernel
    domain option.
+
See the <<userspace-probe,Dynamic user space probes>> section for more
information about the option's argument 'SOURCE'.

option:--syscall::
    Linux kernel system call. Only available with the option:--kernel
    domain option.

option:--tracepoint::
    Linux kernel or application tracepoint (default).


Log level
~~~~~~~~~
One of:

option:--loglevel='LOGLEVEL'::
    Add log level condition to the event rule: the event source's
    defined log level must be at least as severe as 'LOGLEVEL'.
    See the <<log-levels,Log levels>> section above for the available
    log levels. Only available with application domains.

option:--loglevel-only='LOGLEVEL'::
    Add log level condition to the event rule: the event source's
    defined log level must match 'LOGLEVEL'. See the
    <<log-levels,Log levels>> section above for the available log
    levels. Only available with application domains.


Filtering and exclusion
~~~~~~~~~~~~~~~~~~~~~~~
option:-x 'EVENT'[,'EVENT']..., option:--exclude='EVENT'[,'EVENT']...::
    Exclude events named 'EVENT' from the event rule. This option
    can be used when the command's 'EVENT' argument contains at least
    one wildcard star (`*`) to exclude specific names. 'EVENT' can also
    contain wildcard stars. To use a
    literal `,` character, use :esccomma:.
    Only available with the option:--userspace domain.

option:-f 'EXPR', option:--filter='EXPR'::
    Add filter expression condition to the event rule. Expression 'EXPR'
    must evaluate to true when executed against the dynamic values of
    event fields. See the <<filter-expr,Filter expression>>
    section above for more information.


Shortcuts
~~~~~~~~~
option:-a, option:--all::
    Equivalent to an 'EVENT' argument named `*` (wildcard) when also
    using the option:--tracepoint (default) or option:--syscall option.


include::common-cmd-help-options.txt[]


include::common-cmd-footer.txt[]


SEE ALSO
--------
man:lttng-disable-event(1),
man:lttng(1)