1 # LTTng analyses machine interface (LAMI) v0.1
3 This document explains the input and output formats of the LTTng
4 analyses' **machine interface** (LAMI), version 0.1.
6 The LTTng analyses project is a set of scripts which analyze one or more
7 traces and output the results of this analysis. Each script is
8 responsible for one analysis.
13 * **Analysis**: A producer of LAMI. An analysis is a program which
14 receives an input following the LAMI [input format](#input-format),
15 performs an analysis on a set of traces, and outputs the results
16 following the LAMI [output format](#output-format).
17 * **Consumer**: A consumer of LAMI. A consumer, usually some sort of
18 user interface, executes an analysis program following the LAMI
19 input format, and receives its results, following the LAMI output
25 A consumer executes an analysis program with one or more standard
26 command-line arguments, then reads the standard output of the analysis
27 to its metadata or results following the [output
28 format](#output-format).
30 There are two different phases of analysis execution:
32 1. **Metadata phase**: The consumer obtains the static metadata of
34 2. **Analysis phase**: The consumer can perform as many analyses as
35 needed on different time ranges and set of traces, using the
36 metadata of step 1 to interpret the results.
38 Having two phases avoids attaching the same metadata to each result
41 The LAMI input format is a list of standard command-line arguments.
45 | Argument | Description | Required? | Default |
47 | `--metadata` | Output the analysis' metadata instead of analyzing | Yes | N/A |
51 | Argument | Description | Required? | Default |
53 | 1st positional | Path to trace(s) to analyze | Yes | N/A |
54 | `--begin=TS` | Set beginning timestamp of analysis to `TS` ns | No | Absolute beginning of the analyzed traces |
55 | `--end=TS` | Set end timestamp of analysis to `TS` ns | No | Absolute end of the analyzed traces |
56 | `--limit=COUNT` | Set maximum number of output rows per result table to `COUNT` (use `unlimited` for no maximum number of rows) | No | `unlimited` |
57 | `--output-progress` | Output [progress data](#progress) before outputting the results | No | No progress output |
62 The LAMI output format is produced by the analysis and is consumed by
65 An analysis has two output channels:
67 1. Its standard output, which contains progress data, a metadata
68 object, an analysis result object, or an error object.
69 2. Its exit status, which indicates if the analysis was successful
72 If an analysis is successful, its exit status is set to 0. Otherwise,
75 During the [metadata phase](#metadata), the standard output of the
76 analysis provides everything about the analysis which is not result
77 data: analysis title, authors, description, result table column
78 classes/titles/units, etc. This metadata is essential to interpret the
79 result objects of the analysis phase.
81 The output format of the metadata phase is always an
82 UTF-8 [JSON](http://json.org/) object.
84 During the [analysis phase](#analysis-phase), the consumer can perform
85 as many analyses as required by running the analysis with the mandatory
88 The output format of the analysis phase depends on the command-line
89 arguments passed to the analysis program:
91 * If `--output-progress` is passed, then the output format _may_
92 contain [progress indication](#progress) lines, followed by an UTF-8
93 [JSON](http://json.org/) object.
94 * If `--output-progress` is _not_ passed, then the output format is
95 always an UTF-8 [JSON](http://json.org/) object.
97 In all the objects of the output format, an unknown key must be
98 **ignored** by the consumer.
103 The following subsections document objects that can be written during
104 both the [metadata phase](#metadata) and the [analysis
110 An _error object_ indicates that the analysis encountered an error
111 during its execution.
115 | Property | Type | Description | Required? | Default value |
116 |---|---|---|---|---|
117 | `error-code` | String or number | Error code | No | No error code |
118 | `error-message` | String | Error message | Yes | N/A |
125 "error-message": "Cannot open trace \"/root/lttng-traces/my-session\": Permission denied",
133 _Data objects_ contain result data of specific classes.
135 All data objects share a common `class` property which identifies the
136 object's class. The available values are:
138 | Class name (string) | Object |
140 | `unknown` | [Unknown object](#unknown-object) |
141 | `ratio` | [Ratio object](#ratio-object) |
142 | `timestamp` | [Timestamp object](#timestamp-object) |
143 | `time-range` | [Time range object](#time-range-object) |
144 | `duration` | [Duration object](#duration-object) |
145 | `size` | [Size object](#size-object) |
146 | `bitrate` | [Bitrate object](#bitrate-object) |
147 | `syscall` | [Syscall object](#syscall-object) |
148 | `process` | [Process object](#process-object) |
149 | `path` | [Path object](#path-object) |
150 | `fd` | [File descriptor object](#file-descriptor-object) |
151 | `irq` | [IRQ object](#irq-object) |
152 | `cpu` | [CPU object](#cpu-object) |
153 | `disk` | [Disk object](#disk-object) |
154 | `part` | [Disk partition object](#disk-partition-object) |
155 | `netif` | [Network interface object](#network-interface-object) |
157 The following subsections explain each class of data object.
162 The special _unknown object_ represents an unknown value. It is
163 typically used in result table cells where a given computation cannot
164 produce a result for some reason.
168 | Property | Type | Description | Required? | Default value |
169 |---|---|---|---|---|
170 | `class` | String | Set to `unknown` | Yes | N/A |
183 A _ratio object_ describes a simple, dimensionless ratio, that is,
184 a relationship between two quantities having the same unit indicating
185 how many times the first quantity contains the second.
187 It is suggested that the consumer shows a ratio object as a percentage.
191 | Property | Type | Description | Required? | Default value |
192 |---|---|---|---|---|
193 | `class` | String | Set to `ratio` | Yes | N/A |
194 | `value` | Number | Ratio as a decimal fraction | Yes | N/A |
206 ##### Timestamp object
208 A _timestamp object_ describes a specific point in time.
212 | Property | Type | Description | Required? | Default value |
213 |---|---|---|---|---|
214 | `class` | String | Set to `timestamp` | Yes | N/A |
215 | `value` | Number | Number of nanoseconds since Unix epoch | Yes | N/A |
221 "class": "timestamp",
222 "value": 1444334398154194201
227 ##### Time range object
229 A _time range object_ describes an interval bounded by two point in
234 | Property | Type | Description | Required? | Default value |
235 |---|---|---|---|---|
236 | `class` | String | Set to `time-range` | Yes | N/A |
237 | `begin` | Number | Beginning timestamp (number of nanoseconds since Unix epoch) | Yes | N/A |
238 | `end` | Number | End timestamp (number of nanoseconds since Unix epoch) | Yes | N/A |
240 The `end` property must have a value greater or equal to the value of
241 the `begin` property.
247 "class": "time-range",
248 "begin": 1444334398154194201,
249 "end": 1444334425194487548
254 ##### Duration object
256 A _duration object_ describes the difference between two points in time.
260 | Property | Type | Description | Required? | Default value |
261 |---|---|---|---|---|
262 | `class` | String | Set to `duration` | Yes | N/A |
263 | `value` | Number | Time duration in nanoseconds | Yes | N/A |
277 A _size object_ describes the size of a file, of a buffer, of a
282 | Property | Type | Description | Required? | Default value |
283 |---|---|---|---|---|
284 | `class` | String | Set to `size` | Yes | N/A |
285 | `value` | Integer | Size in bytes | Yes | N/A |
299 A _bitrate object_ describes a transfer rate.
303 | Property | Type | Description | Required? | Default value |
304 |---|---|---|---|---|
305 | `class` | String | Set to `bitrate` | Yes | N/A |
306 | `value` | Number | Bitrate in bits/second | Yes | N/A |
320 A _syscall object_ describes the name of a system call.
324 | Property | Type | Description | Required? | Default value |
325 |---|---|---|---|---|
326 | `class` | String | Set to `syscall` | Yes | N/A |
327 | `name` | String | System call name | Yes | N/A |
341 A _process object_ describes a system process.
345 | Property | Type | Description | Required? | Default value |
346 |---|---|---|---|---|
347 | `class` | String | Set to `process` | Yes | N/A |
348 | `name` | String | Process name | No | No process name |
349 | `pid` | Integer | Process ID (PID) | No | No process ID |
350 | `tid` | Integer | Thread ID (TID) | No | No thread ID |
366 A _path object_ describes a relative or absolute file system path.
370 | Property | Type | Description | Required? | Default value |
371 |---|---|---|---|---|
372 | `class` | String | Set to `path` | Yes | N/A |
373 | `path` | String | File system path | Yes | N/A |
380 "path": "/usr/bin/grep"
385 ##### File descriptor object
387 A _file descriptor object_ describes the numeric descriptor of a file.
391 | Property | Type | Description | Required? | Default value |
392 |---|---|---|---|---|
393 | `class` | String | Set to `fd` | Yes | N/A |
394 | `fd` | Integer | File descriptor | Yes | N/A |
408 An _IRQ object_ describes an interrupt source.
412 | Property | Type | Description | Required? | Default value |
413 |---|---|---|---|---|
414 | `class` | String | Set to `irq` | Yes | N/A |
415 | `hard` | Boolean | `true` if this interrupt source generates hardware interrupts, `false` for software interrupts | No | `true` |
416 | `nr` | Integer | Interrupt source number | Yes | N/A |
417 | `name` | String | Interrupt source name | No | No interrupt source name |
433 A _CPU object_ describes a numeric CPU identifier.
437 | Property | Type | Description | Required? | Default value |
438 |---|---|---|---|---|
439 | `class` | String | Set to `cpu` | Yes | N/A |
440 | `id` | Integer | CPU identifier number | Yes | N/A |
454 A _disk object_ describes a disk name.
458 | Property | Type | Description | Required? | Default value |
459 |---|---|---|---|---|
460 | `class` | String | Set to `disk` | Yes | N/A |
461 | `name` | String | Disk name | Yes | N/A |
473 ##### Disk partition object
475 A _disk partition object_ describes a disk partition name.
479 | Property | Type | Description | Required? | Default value |
480 |---|---|---|---|---|
481 | `class` | String | Set to `part` | Yes | N/A |
482 | `name` | String | Disk partition name | Yes | N/A |
494 ##### Network interface object
496 A _network interface object_ describes a network interface name.
500 | Property | Type | Description | Required? | Default value |
501 |---|---|---|---|---|
502 | `class` | String | Set to `netif` | Yes | N/A |
503 | `name` | String | Network interface name | Yes | N/A |
517 The _metadata phase_ explains the analysis. It provides an optional
518 title for the analysis and the format of the result tables (outputted
519 during the [analysis phase](#analysis) by the same analysis).
521 The metadata phase writes either:
523 * A [metadata object](#metadata-object), or
524 * An [error object](#error-object).
526 The following subsections document objects that can only be written
527 during the metadata phase.
530 #### Column description object
532 A _column description object_ describes one table _column_.
536 | Property | Type | Description | Required? | Default value |
537 |---|---|---|---|---|
538 | `title` | String | Column's title | No | No title |
539 | `class` | String | Class of data in column's cells, amongst: <ul><li>`string`: JSON strings</li><li>`int`: JSON numbers limited to integers</li><li>`number`: JSON numbers</li><li>`bool`: JSON booleans</li><li>`ratio`: [ratio objects](#ratio-object)</li><li>`timestamp`: [timestamp objects](#timestamp-object)</li><li>`time-range`: [time range objects](#time-range-object)</li><li>`duration`: [duration objects](#duration-object)</li><li>`size`: [size objects](#size-object)</li><li>`bitrate`: [bitrate objects](#bitrate-object)</li><li>`syscall`: [syscall objects](#syscall-object)</li><li>`process`: [process objects](#process-object)</li><li>`path`: [path objects](#path-object)</li><li>`fd`: [file descriptor objects](#file-descriptor-object)</li><li>`irq`: [IRQ objects](#irq-object)</li><li>`cpu`: [CPU objects](#cpu-object)</li><li>`disk`: [disk objects](#disk-object)</li><li>`part`: [disk partition objects](#disk-partition-object)</li><li>`netif`: [network interface objects](#network-interface-object)</li><li>`mixed`: any object</li></ul> | No | `mixed` |
540 | `unit` | String | Column's unit, if the `class` property is `string`, `int`, `number`, or `bool` | No | No unit |
546 "title": "System call",
560 #### Table class object
562 A _table class object_ describes one class of
563 [result table](#result-table-object).
567 | Property | Type | Description | Required? | Default value |
568 |---|---|---|---|---|
569 | `inherit` | String | Name of inherited table class | No | No inheritance |
570 | `title` | String | Table's title | No | No title |
571 | `column-descriptions` | Array of [column description objects](#column-description-object) | Descriptions of table's columns | No | Zero columns |
573 When inheriting another table class using the `inherit` property,
574 the `title` and `column-descriptions` properties override the
577 **Example** (no inheritance):
581 "title": "Handler duration and raise latency statistics (hard IRQ)",
582 "column-descriptions": [
593 "title": "Minimum duration",
597 "title": "Average duration",
601 "title": "Maximum duration",
605 "title": "Standard deviation",
612 **Example** (with inheritance, redefining the title, keeping the same
613 column descriptions):
617 "inherit": "irq-stats",
618 "title": "IRQ statistics (ehci_hcd:usb1 [16])"
627 | Property | Type | Description | Required? | Default value |
628 |---|---|---|---|---|
629 | `major` | Integer | Major version | Yes, if `minor` exists | No major version |
630 | `minor` | Integer | Minor version | Yes, if `patch` exists | No minor version |
631 | `patch` | Integer | Patch version | No | No patch version |
632 | `extra` | String | Extra version information (e.g., `dev`, `pre`, `rc2`, commit ID) | No | No extra version |
650 | Property | Type | Description | Required? | Default value |
651 |---|---|---|---|---|
652 | `mi-version` | [Version object](#version-object) | Latest version of the LAMI standard supported by this analysis, amongst:<ul><li>`{"major": 0, "minor": 1}`</li></ul> | Yes | N/A |
653 | `version` | [Version object](#version-object) | Version of the analysis | No | No version |
654 | `title` | String | Analysis title | No | No title |
655 | `authors` | Array of strings | Author(s) of the analysis | No | No authors |
656 | `description` | String | Analysis description | No | No description |
657 | `url` | String | URL where to find the analysis | No | No URL |
658 | `tags` | Array of strings | List of tags associated with the analysis | No | No tags |
659 | `table-classes` | Object mapping table class names (strings) to [table class objects](#table-class-object) | Classes of potential result tables | Yes (at least one table class) | N/A |
661 A consumer not implementing the LAMI standard version indicated by
662 the `mi-version` property should not perform the analysis.
664 The `table-classes` property describes all the potential result
665 tables with a static layout that can be generated by the
666 [analysis phase](#analysis). A result table can specify the name
667 of its table class, or define a full table class in place for
668 dynamic result tables.
684 "title": "I/O latency statistics",
689 "description": "Provides statistics about the latency involved in various I/O operations.",
690 "url": "https://github.com/lttng/lttng-analyses",
699 "title": "System calls latency statistics",
700 "column-descriptions": [
701 {"title": "System call", "class": "syscall"},
702 {"title": "Count", "class": "int", "unit": "operations"},
703 {"title": "Minimum duration", "class": "duration"},
704 {"title": "Average duration", "class": "duration"},
705 {"title": "Maximum duration", "class": "duration"},
706 {"title": "Standard deviation", "class": "duration"}
710 "title": "Disk latency statistics",
711 "column-descriptions": [
712 {"title": "Disk name", "class": "disk"},
713 {"title": "Count", "class": "int", "unit": "operations"},
714 {"title": "Minimum duration", "class": "duration"},
715 {"title": "Average duration", "class": "duration"},
716 {"title": "Maximum duration", "class": "duration"},
717 {"title": "Standard deviation", "class": "duration"}
727 The _analysis phase_ outputs the actual data computer by the analysis.
728 The consumer needs the metadata object of the [metadata
729 phase](#metadata) in order to interpret the results of the analysis
732 If the `--output-progress` option is passed to the analysis program,
733 then the analysis _may_ output [progress indication](#progress) lines
734 before writing its main object.
736 Then, the analysis phase writes either:
738 * A [result object](#analysis-results-object), or
739 * An [error object](#error-object).
741 The following subsections document objects that can only be written
742 during the analysis phase.
747 Zero or more _progress lines_ may be written by the analysis during the
748 analysis phase _before_ it writes its main object. Progress lines are
749 only written if the `--output-progress` option is passed to the analysis
752 The format of a progress line is as follows (plain text):
758 * `VALUE`: a floating point number from 0 to 1 indicating the current
759 progress of the analysis, or the string `*` which means that the
760 analysis is not able to estimate when it will finish.
761 * `MESSAGE`: an optional message which accompanies the progress
764 Note that `VALUE` and `MESSAGE` are delimited by a single space
767 The line must be terminated by a Unix newline (ASCII LF).
769 If one progress line has the `*` value, _all_ the progress lines should
772 **Example** (with progress value):
775 0 Starting the analysis
776 0 About to process 1248 events
777 0.17 38/1248 events procesed
778 0.342 142/1248 events processed
779 0.53 203/1248 events processed
780 0.54 Connecting to database
784 0.681 511/1248 events processed
785 0.759 810/1248 events processed
786 0.84 1051/1248 events processed
787 0.932 1194/1248 events processed
788 0.98 1248/1248 events processed
790 { JSON result object }
793 **Example** (without progress value):
796 * Starting the analysis
797 * 124 events procesed
798 * 1150 events processed
799 * 3845 events processed
800 * Connecting to database
802 * 9451 events processed
803 * 11542 events processed
804 * 15464 events processed
805 * 17704 events processed
806 * 21513 events processed
808 { JSON result object }
812 #### Result table object
814 A _result table object_ represents the data of an analysis in rows and
819 | Property | Type | Description | Required? | Default value |
820 |---|---|---|---|---|
821 | `time-range` | [Time range object](#time-range-object) | Time range over which the results contained in this table apply | Yes | N/A |
822 | `class` | String or [table class object](#table-class-object) | Table class name or table class object containing the metadata of this result table | Yes | N/A |
823 | `data` | Array of arrays of data objects/plain JSON values | Result table rows | Yes (at least one row) | N/A |
825 The `class` property indicates either:
827 * The name of the [table class object](#table-class-object),
828 as defined in the [metadata phase](#metadata), describing this
830 * A complete [table class object](#table-class-object). This is
831 useful when the result table's layout is dynamic (dynamic title,
832 dynamic column descriptions).
834 The `data` property is a JSON array of rows. Each row is a JSON array of
835 column cells. Each column cell contains a value (either a plain JSON
836 value, or a [data object](#data-objects)), as described by the `class`
837 property of the associated
838 [column description object](#column-description-object).
840 Any column cell may contain the [unknown object](#unknown-object) when
841 it would be possible to get a result for this cell, but the result is
844 Any column cell may contain `null` when the cell is **empty**.
851 "class": "time-range",
852 "begin": 1444334398154194201,
853 "end": 1444334425194487548
855 "class": "syscall-latency",
858 {"class": "syscall", "name": "open"},
860 {"class": "duration", "value": 5562},
861 {"class": "duration", "value": 13835},
862 {"class": "duration", "value": 77683},
863 {"class": "duration", "value": 15263}
866 {"class": "syscall", "name": "read"},
868 {"class": "duration", "value": 316},
869 {"class": "duration", "value": 5774},
870 {"class": "unknown"},
871 {"class": "duration", "value": 9277}
877 **Example** (dynamic title):
882 "class": "time-range",
883 "begin": 1444334398154194201,
884 "end": 1444334425194487548
887 "inherit": "some-latency",
888 "title": "Latency of my stuff [42, 19, -3]"
892 {"class": "syscall", "name": "open"},
894 {"class": "duration", "value": 5562},
895 {"class": "duration", "value": 13835},
896 {"class": "duration", "value": 77683},
897 {"class": "duration", "value": 15263}
900 {"class": "syscall", "name": "read"},
902 {"class": "duration", "value": 316},
903 {"class": "duration", "value": 5774},
904 {"class": "unknown"},
905 {"class": "duration", "value": 9277}
911 **Example** (dynamic column descriptions):
916 "class": "time-range",
917 "begin": 1444334398154194201,
918 "end": 1444334425194487548
921 "title": "System call stuff for process zsh [4723]",
922 "column-descriptions": [
924 "title": "System call involved",
928 "title": "Count in region AX:23",
932 "title": "Count in region BC:86",
936 "title": "Count in region HE:37",
965 #### Analysis results object
967 An _analysis results object_ contains the actual data outputted by the
972 | Property | Type | Description | Required? | Default value |
973 |---|---|---|---|---|
974 | `results` | Array of [result table objects](#result-table-object) | Analysis results tables | Yes (at least one table) | N/A |
983 "class": "time-range",
984 "begin": 1444334398154194201,
985 "end": 1444334425194487548
987 "class": "syscall-latency",
990 {"class": "syscall", "name": "open"},
992 {"class": "duration", "value": 5562},
993 {"class": "duration", "value": 13835},
994 {"class": "duration", "value": 77683},
995 {"class": "duration", "value": 15263}
998 {"class": "syscall", "name": "read"},
1000 {"class": "duration", "value": 316},
1001 {"class": "duration", "value": 5774},
1002 {"class": "unknown"},
1003 {"class": "duration", "value": 9277}
1009 "class": "time-range",
1010 "begin": 1444334425194487549,
1011 "end": 1444334425254887190
1013 "class": "syscall-latency",
1016 {"class": "syscall", "name": "open"},
1018 {"class": "duration", "value": 1578},
1019 {"class": "duration", "value": 16648},
1020 {"class": "duration", "value": 15444},
1021 {"class": "duration", "value": 68540}
1024 {"class": "syscall", "name": "read"},
1026 {"class": "duration", "value": 78},
1027 {"class": "duration", "value": 1948},
1028 {"class": "duration", "value": 11184},
1029 {"class": "duration", "value": 94670}