doc/mi.md

   1 # LTTng analyses machine interface
   2
   3 This document explains the input and output of the LTTng analyses'
   4 **machine interface**.
   5
   6 The LTTng analyses project is a set of scripts which analyze one or more
   7 traces and output the results of this analysis. Each script is responsible
   8 for one analysis.
   9
  10
  11 ## Input format
  12
  13 There are two different phases. The client first gets the static metadata
  14 of the analysis, then the client can perform as many analyses as needed
  15 on different time ranges using this metadata.
  16
  17 The input format of the LTTng analyses MI scripts is a list of standard
  18 command-line arguments.
  19
  20 **Metadata phase**:
  21
  22 | Argument | Description | Default |
  23 |---|---|---|
  24 | `--metadata` | Output the analysis' metadata instead of analyzing | N/A |
  25
  26 **Analysis phase**:
  27
  28 | Argument | Description | Default |
  29 |---|---|---|
  30 | 1st positional | Path to trace(s) to analyze | N/A |
  31 | `--begin` | Beginning timestamp of analysis (ns) | Absolute beginning of the analyzed traces |
  32 | `--end` | End timestamp of analysis (ns) | Absolute end of the analyzed traces |
  33 | `--limit` | Maximum number of output rows per result table or `unlimited` | `unlimited` |
  34
  35
  36 ## Output format
  37
  38 The output format is always UTF-8 [JSON](http://json.org/).
  39
  40 There are two different output phases. The client should first get the
  41 analysis' metadata by running:
  42
  43     script --metadata
  44
  45 where `script` is the script containing the analysis. This is know as the
  46 [metadata phase](#metadata). This output provides everything about the analysis
  47 which is not result data: analysis title, authors, description, result table
  48 column classes/titles/units, etc.
  49
  50 Then, the client can perform as many analyses as required by running the script
  51 with the mandatory trace path argument. This is known
  52 as the [analysis phase](#analysis).
  53
  54 Note that any [result table](#result-table) can still provide a dynamic table class
  55 object along with the data when, for example, dynamic columns are required.
  56
  57
  58 ### Data objects
  59
  60 _Data objects_ contain data of specific classes.
  61
  62 All data objects share a common `class` property which identifies the
  63 object's class. The available values are:
  64
  65 | Class name (string) | Object |
  66 |---|---|
  67 | `unknown` | [Unknown object](#unknown-object) |
  68 | `ratio` | [Ratio object](#ratio-object) |
  69 | `timestamp` | [Timestamp object](#timestamp-object) |
  70 | `time-range` | [Time range object](#time-range-object) |
  71 | `duration` | [Duration object](#duration-object) |
  72 | `size` | [Size object](#size-object) |
  73 | `bitrate` | [Bitrate object](#bitrate-object) |
  74 | `syscall` | [Syscall object](#syscall-object) |
  75 | `process` | [Process object](#process-object) |
  76 | `path` | [Path object](#path-object) |
  77 | `fd` | [File descriptor object](#file-descriptor-object) |
  78 | `irq` | [IRQ object](#irq-object) |
  79 | `cpu` | [CPU object](#cpu-object) |
  80 | `disk` | [Disk object](#disk-object) |
  81 | `part` | [Disk partition object](#disk-partition-object) |
  82 | `netif` | [Network interface object](#network-interface-object) |
  83
  84 The following subsections explain each class of data object.
  85
  86
  87 #### Unknown object
  88
  89 The special _unknown object_ represents an unknown value. It is
  90 typically used in result table cells where a given computation cannot
  91 produce a result for some reason.
  92
  93 This object has no properties.
  94
  95 **Example**:
  96
  97 ```json
  98 {
  99   "class": "unknown"
 100 }
 101 ```
 102
 103
 104 #### Ratio object
 105
 106 A _ratio object_ describes a simple, dimensionless ratio, that is,
 107 a relationship between two quantities having the same unit indicating
 108 how many times the first quantity contains the second.
 109
 110 It is suggested that the client shows a ratio object as a percentage.
 111
 112 **Properties**:
 113
 114 | Property | Type | Description | Required? | Default value |
 115 |---|---|---|---|---|
 116 | `value` | Number | Ratio as a decimal fraction | Yes | N/A |
 117
 118 **Example**:
 119
 120 ```json
 121 {
 122   "class": "ratio",
 123   "value": 0.57
 124 }
 125 ```
 126
 127
 128 #### Timestamp object
 129
 130 A _timestamp object_ describes a specific point in time.
 131
 132 **Properties**:
 133
 134 | Property | Type | Description | Required? | Default value |
 135 |---|---|---|---|---|
 136 | `value` | Number | Number of nanoseconds since Unix epoch | Yes | N/A |
 137
 138 **Example**:
 139
 140 ```json
 141 {
 142   "class": "timestamp",
 143   "value": 1444334398154194201
 144 }
 145 ```
 146
 147
 148 #### Time range object
 149
 150 A _time range object_ describes an interval bounded by two point in
 151 time.
 152
 153 **Properties**:
 154
 155 | Property | Type | Description | Required? | Default value |
 156 |---|---|---|---|---|
 157 | `begin` | Number | Beginning timestamp (number of nanoseconds since Unix epoch) | Yes | N/A |
 158 | `end` | Number | End timestamp (number of nanoseconds since Unix epoch) | Yes | N/A |
 159
 160 The `end` property must have a value greater or equal to the value of
 161 the `begin` property.
 162
 163 **Examples**:
 164
 165 ```json
 166 {
 167   "class": "time-range",
 168   "begin": 1444334398154194201,
 169   "end": 1444334425194487548
 170 }
 171 ```
 172
 173
 174 #### Duration object
 175
 176 A _duration object_ describes the difference between two points in time.
 177
 178 **Properties**:
 179
 180 | Property | Type | Description | Required? | Default value |
 181 |---|---|---|---|---|
 182 | `value` | Number | Time duration in nanoseconds | Yes | N/A |
 183
 184 **Example**:
 185
 186 ```json
 187 {
 188   "class": "duration",
 189   "value": 917238723
 190 }
 191 ```
 192
 193
 194 #### Size object
 195
 196 A _size object_ describes the size of a file, of a buffer, of a transfer, etc.
 197
 198 **Properties**:
 199
 200 | Property | Type | Description | Required? | Default value |
 201 |---|---|---|---|---|
 202 | `value` | Integer | Size in bytes | Yes | N/A |
 203
 204 **Example**:
 205
 206 ```json
 207 {
 208   "class": "size",
 209   "value": 4994857
 210 }
 211 ```
 212
 213
 214 #### Bitrate object
 215
 216 A _bitrate object_ describes a transfer rate.
 217
 218 **Properties**:
 219
 220 | Property | Type | Description | Required? | Default value |
 221 |---|---|---|---|---|
 222 | `value` | Number | Bitrate in bits/second | Yes | N/A |
 223
 224 **Example**:
 225
 226 ```json
 227 {
 228   "class": "bitrate",
 229   "value": 9845154
 230 }
 231 ```
 232
 233
 234 #### Syscall object
 235
 236 A _syscall object_ describes the name of a system call.
 237
 238 **Properties**:
 239
 240 | Property | Type | Description | Required? | Default value |
 241 |---|---|---|---|---|
 242 | `name` | String | System call name | Yes | N/A |
 243
 244 **Example**:
 245
 246 ```json
 247 {
 248   "class": "syscall",
 249   "name": "write"
 250 }
 251 ```
 252
 253
 254 #### Process object
 255
 256 A _process object_ describes a system process.
 257
 258 **Properties**:
 259
 260 | Property | Type | Description | Required? | Default value |
 261 |---|---|---|---|---|
 262 | `name` | String | Process name | No | No process name |
 263 | `pid` | Integer | Process ID (PID) | No | No process ID |
 264 | `tid` | Integer | Thread ID (TID) | No | No thread ID |
 265
 266 **Example**:
 267
 268 ```json
 269 {
 270   "class": "process",
 271   "name": "python",
 272   "pid": 1548,
 273   "tid": 1549
 274 }
 275 ```
 276
 277
 278 #### Path object
 279
 280 A _path object_ describes a relative or absolute file system path.
 281
 282 **Properties**:
 283
 284 | Property | Type | Description | Required? | Default value |
 285 |---|---|---|---|---|
 286 | `path` | String | File system path | Yes | N/A |
 287
 288 **Example**:
 289
 290 ```json
 291 {
 292   "class": "path",
 293   "path": "/usr/bin/grep"
 294 }
 295 ```
 296
 297
 298 #### File descriptor object
 299
 300 A _file descriptor object_ describes the numeric descriptor of a file.
 301
 302 **Properties**:
 303
 304 | Property | Type | Description | Required? | Default value |
 305 |---|---|---|---|---|
 306 | `fd` | Integer | File descriptor | Yes | N/A |
 307
 308 **Example**:
 309
 310 ```json
 311 {
 312   "class": "fd",
 313   "fd": 8
 314 }
 315 ```
 316
 317
 318 #### IRQ object
 319
 320 An _IRQ object_ describes an interrupt source.
 321
 322 **Properties**:
 323
 324 | Property | Type | Description | Required? | Default value |
 325 |---|---|---|---|---|
 326 | `hard` | Boolean | `true` if this interrupt source generates hardware interrupts, `false` for software interrupts | No | `true` |
 327 | `nr` | Integer | Interrupt source number | Yes | N/A |
 328 | `name` | String | Interrupt source name | No | No interrupt source name |
 329
 330 **Example**:
 331
 332 ```json
 333 {
 334   "class": "irq",
 335   "hard": true,
 336   "nr": 42,
 337   "name": "ahci"
 338 }
 339 ```
 340
 341
 342 #### CPU object
 343
 344 A _CPU object_ describes a numeric CPU identifier.
 345
 346 **Properties**:
 347
 348 | Property | Type | Description | Required? | Default value |
 349 |---|---|---|---|---|
 350 | `id` | Integer | CPU identifier number | Yes | N/A |
 351
 352 **Example**:
 353
 354 ```json
 355 {
 356   "class": "cpu",
 357   "id": 1
 358 }
 359 ```
 360
 361
 362 #### Disk object
 363
 364 A _disk object_ describes a disk name.
 365
 366 **Properties**:
 367
 368 | Property | Type | Description | Required? | Default value |
 369 |---|---|---|---|---|
 370 | `name` | String | Disk name | Yes | N/A |
 371
 372 **Example**:
 373
 374 ```json
 375 {
 376   "class": "disk",
 377   "name": "sda"
 378 }
 379 ```
 380
 381
 382 #### Disk partition object
 383
 384 A _disk partition object_ describes a disk partition name.
 385
 386 **Properties**:
 387
 388 | Property | Type | Description | Required? | Default value |
 389 |---|---|---|---|---|
 390 | `name` | String | Disk partition name | Yes | N/A |
 391
 392 **Example**:
 393
 394 ```json
 395 {
 396   "class": "part",
 397   "name": "sdb2"
 398 }
 399 ```
 400
 401
 402 #### Network interface object
 403
 404 A _network interface object_ describes a network interface name.
 405
 406 **Properties**:
 407
 408 | Property | Type | Description | Required? | Default value |
 409 |---|---|---|---|---|
 410 | `name` | String | Network interface name | Yes | N/A |
 411
 412 **Example**:
 413
 414 ```json
 415 {
 416   "class": "netif",
 417   "name": "eth0"
 418 }
 419 ```
 420
 421
 422 ### Metadata
 423
 424 The _metadata phase_ explains the analysis. It provides an optional title for the
 425 analysis and the format of the result tables (outputted in the
 426 [analysis phase](#analysis) by the same analysis).
 427
 428 The metadata phase writes a [metadata object](#metadata-object).
 429
 430
 431 #### Column description object
 432
 433 A _column description object_ describes one table _column_.
 434
 435 **Properties**:
 436
 437 | Property | Type | Description | Required? | Default value |
 438 |---|---|---|---|---|
 439 | `title` | String | Column's title | No | No title |
 440 | `class` | String | Class of data in column's cells, amongst: <ul><li>`string`: JSON strings</li><li>`int`: JSON numbers limited to integers</li><li>`number`: JSON numbers</li><li>`bool`: JSON booleans</li><li>`ratio`: [ratio objects](#ratio-object)</li><li>`timestamp`: [timestamp objects](#timestamp-object)</li><li>`time-range`: [time range objects](#time-range-object)</li><li>`duration`: [duration objects](#duration-object)</li><li>`size`: [size objects](#size-object)</li><li>`bitrate`: [bitrate objects](#bitrate-object)</li><li>`syscall`: [syscall objects](#syscall-object)</li><li>`process`: [process objects](#process-object)</li><li>`path`: [path objects](#path-object)</li><li>`fd`: [file descriptor objects](#file-descriptor-object)</li><li>`irq`: [IRQ objects](#irq-object)</li><li>`cpu`: [CPU objects](#cpu-object)</li><li>`disk`: [disk objects](#disk-object)</li><li>`part`: [disk partition objects](#disk-partition-object)</li><li>`netif`: [network interface objects](#network-interface-object)</li><li>`mixed`: any object</li></ul> | No | `mixed` |
 441 | `unit` | String | Column's unit, if the `class` property is `string`, `int`, `number`, or `bool` | No | No unit |
 442
 443 **Examples**:
 444
 445 ```json
 446 {
 447   "title": "System call",
 448   "class": "syscall"
 449 }
 450 ```
 451
 452 ```json
 453 {
 454   "title": "Count",
 455   "class": "int",
 456   "unit": "interrupts"
 457 }
 458 ```
 459
 460
 461 #### Table class object
 462
 463 A _table class object_ describes one class of
 464 [result table](#result-table-object).
 465
 466 **Properties**:
 467
 468 | Property | Type | Description | Required? | Default value |
 469 |---|---|---|---|---|
 470 | `inherit` | String | Name of inherited table class | No | No inheritance |
 471 | `title` | String | Table's title | No | No title |
 472 | `column-descriptions` | Array of [column description objects](#column-description-object) | Descriptions of table's columns | No | Zero columns |
 473
 474 When inheriting another table class using the `inherit` property,
 475 the `title` and `column-descriptions` properties override the
 476 inherited values.
 477
 478 **Example** (no inheritance):
 479
 480 ```json
 481 {
 482   "title": "Handler duration and raise latency statistics (hard IRQ)",
 483   "column-descriptions": [
 484     {
 485       "title": "IRQ",
 486       "class": "irq"
 487     },
 488     {
 489       "title": "Count",
 490       "class": "int",
 491       "unit": "interrupts"
 492     },
 493     {
 494       "title": "Minimum duration",
 495       "class": "duration"
 496     },
 497     {
 498       "title": "Average duration",
 499       "class": "duration"
 500     },
 501     {
 502       "title": "Maximum duration",
 503       "class": "duration"
 504     },
 505     {
 506       "title": "Standard deviation",
 507       "class": "duration"
 508     }
 509   ]
 510 }
 511 ```
 512
 513 **Example** (with inheritance, redefining the title, keeping the same
 514 column descriptions):
 515
 516 ```json
 517 {
 518   "inherit": "irq-stats",
 519   "title": "IRQ statistics (ehci_hcd:usb1 [16])"
 520 }
 521 ```
 522
 523
 524 #### Version object
 525
 526 **Properties**:
 527
 528 | Property | Type | Description | Required? | Default value |
 529 |---|---|---|---|---|
 530 | `major` | Integer | Major version | Yes, if `minor` exists | No major version |
 531 | `minor` | Integer | Minor version | Yes, if `patch` exists | No minor version |
 532 | `patch` | Integer | Patch version | No | No patch version |
 533 | `extra` | String | Extra version information (e.g., `dev`, `pre`, `rc2`, commit ID) | No | No extra version |
 534
 535 **Example**:
 536
 537 ```json
 538 {
 539   "major": 1,
 540   "minor": 2,
 541   "patch": 5,
 542   "extra": "dev"
 543 }
 544 ```
 545
 546
 547 #### Metadata object
 548
 549 **Properties**:
 550
 551 | Property | Type | Description | Required? | Default value |
 552 |---|---|---|---|---|
 553 | `version` | [Version object](#version-object) | Version of the analysis | No | No version |
 554 | `title` | String | Analysis title | No | No title |
 555 | `authors` | Array of strings | Author(s) of the analysis | No | No authors |
 556 | `description` | String | Analysis description | No | No description |
 557 | `url` | String | URL where to find the analysis | No | No URL |
 558 | `tags` | Array of strings | List of tags associated with the analysis | No | No tags |
 559 | `table-classes` | Object mapping table class names (strings) to [table class objects](#table-class-object) | Classes of potential result tables | Yes (at least one table class) | N/A |
 560
 561 The `table-classes` property describes all the potential result
 562 tables with a static layout that can be generated by the
 563 [analysis phase](#analysis). A result table can specify the name
 564 of its table class, or define a full table class in place for
 565 dynamic result tables.
 566
 567 **Example**:
 568
 569 ```json
 570 {
 571   "version": {
 572     "major": 1,
 573     "minor": 2,
 574     "patch": 5,
 575     "extra": "dev"
 576   },
 577   "title": "I/O latency statistics",
 578   "authors": [
 579     "Julien Desfossez",
 580     "Antoine Busque"
 581   ],
 582   "description": "Provides statistics about the latency involved in various I/O operations.",
 583   "url": "https://github.com/lttng/lttng-analyses",
 584   "tags": [
 585     "io",
 586     "stats",
 587     "linux-kernel",
 588     "lttng-analyses"
 589   ],
 590   "table-classes": {
 591     "syscall-latency": {
 592       "title": "System calls latency statistics",
 593       "column-descriptions": [
 594         {"title": "System call", "class": "syscall"},
 595         {"title": "Count", "class": "int", "unit": "operations"},
 596         {"title": "Minimum duration", "class": "duration"},
 597         {"title": "Average duration", "class": "duration"},
 598         {"title": "Maximum duration", "class": "duration"},
 599         {"title": "Standard deviation", "class": "duration"}
 600       ]
 601     },
 602     "disk-latency": {
 603       "title": "Disk latency statistics",
 604       "column-descriptions": [
 605         {"title": "Disk name", "class": "disk"},
 606         {"title": "Count", "class": "int", "unit": "operations"},
 607         {"title": "Minimum duration", "class": "duration"},
 608         {"title": "Average duration", "class": "duration"},
 609         {"title": "Maximum duration", "class": "duration"},
 610         {"title": "Standard deviation", "class": "duration"}
 611       ]
 612     }
 613   }
 614 }
 615 ```
 616
 617
 618 ### Analysis
 619
 620 The _analysis phase_ outputs the actual data of the analysis.
 621
 622 The analysis phase writes an [analysis results object](#analysis-results-object).
 623
 624
 625 #### Result table object
 626
 627 A _result table object_ represents the data of an analysis in rows
 628 and columns.
 629
 630 **Properties**:
 631
 632 | Property | Type | Description | Required? | Default value |
 633 |---|---|---|---|---|
 634 | `time-range` | [Time range object](#time-range-object) | Time range over which the results contained in this table apply | Yes | N/A |
 635 | `class` | String or [table class object](#table-class-object) | Table class name or table class object containing the metadata of this result table | Yes | N/A |
 636 | `data` | Array of arrays of data objects/plain JSON values | Result table rows | Yes (at least one row) | N/A |
 637
 638 The `class` property indicates either:
 639
 640   * The name of the [table class object](#table-class-object),
 641     as defined in the [metadata phase](#metadata), describing this
 642     result table.
 643   * A complete [table class object](#table-class-object). This is
 644     useful when the result table's layout is dynamic (dynamic title,
 645     dynamic column descriptions).
 646
 647 The `data` property is a JSON array of rows. Each row is a JSON array of
 648 column cells. Each column cell contains a value (either a plain JSON
 649 value, or a [data object](#data-objects)), as described by the `class`
 650 property of the associated [column description object](#column-description-object).
 651
 652 Any column cell may contain the [unknown object](#unknown-object) when
 653 it would be possible to get a result for this cell, but the result is
 654 unknown.
 655
 656 Any column cell may contain `null` when the cell is **empty**.
 657
 658 **Example**:
 659
 660 ```json
 661 {
 662   "time-range": {
 663     "class": "time-range",
 664     "begin": 1444334398154194201,
 665     "end": 1444334425194487548
 666   },
 667   "class": "syscall-latency",
 668   "data": [
 669     [
 670       {"class": "syscall", "name": "open"},
 671       45,
 672       {"class": "duration", "value": 5562},
 673       {"class": "duration", "value": 13835},
 674       {"class": "duration", "value": 77683},
 675       {"class": "duration", "value": 15263}
 676     ],
 677     [
 678       {"class": "syscall", "name": "read"},
 679       109,
 680       {"class": "duration", "value": 316},
 681       {"class": "duration", "value": 5774},
 682       {"class": "unknown"},
 683       {"class": "duration", "value": 9277}
 684     ]
 685   ]
 686 }
 687 ```
 688
 689 **Example** (dynamic title):
 690
 691 ```json
 692 {
 693   "time-range": {
 694     "class": "time-range",
 695     "begin": 1444334398154194201,
 696     "end": 1444334425194487548
 697   },
 698   "class": {
 699     "inherit": "some-latency",
 700     "title": "Latency of my stuff [42, 19, -3]"
 701   },
 702   "data": [
 703     [
 704       {"class": "syscall", "name": "open"},
 705       45,
 706       {"class": "duration", "value": 5562},
 707       {"class": "duration", "value": 13835},
 708       {"class": "duration", "value": 77683},
 709       {"class": "duration", "value": 15263}
 710     ],
 711     [
 712       {"class": "syscall", "name": "read"},
 713       109,
 714       {"class": "duration", "value": 316},
 715       {"class": "duration", "value": 5774},
 716       {"class": "unknown"},
 717       {"class": "duration", "value": 9277}
 718     ]
 719   ]
 720 }
 721 ```
 722
 723 **Example** (dynamic column descriptions):
 724
 725 ```json
 726 {
 727   "time-range": {
 728     "class": "time-range",
 729     "begin": 1444334398154194201,
 730     "end": 1444334425194487548
 731   },
 732   "class": {
 733     "title": "System call stuff for process zsh [4723]",
 734     "column-descriptions": [
 735       {
 736         "title": "System call involved",
 737         "class": "syscall"
 738       },
 739       {
 740         "title": "Count in region AX:23",
 741         "class": "int"
 742       },
 743       {
 744         "title": "Count in region BC:86",
 745         "class": "int"
 746       },
 747       {
 748         "title": "Count in region HE:37",
 749         "class": "int"
 750       }
 751     ]
 752   },
 753   "data": [
 754     [
 755       {
 756         "class": "syscall",
 757         "name": "read"
 758       },
 759       19,
 760       155,
 761       2
 762     ],
 763     [
 764       {
 765         "class": "syscall",
 766         "name": "write"
 767       },
 768       45,
 769       192,
 770       17
 771     ]
 772   ]
 773 }
 774 ```
 775
 776
 777 #### Analysis results object
 778
 779 An _analysis results object_ contains the actual data outputted by the
 780 analysis.
 781
 782 **Properties**:
 783
 784 | Property | Type | Description | Required? | Default value |
 785 |---|---|---|---|---|
 786 | `results` | Array of [result table objects](#result-table-object) | Analysis results tables | Yes (at least one table) | N/A |
 787
 788 **Example**:
 789
 790 ```json
 791 {
 792   "results": [
 793     {
 794       "time-range": {
 795         "class": "time-range",
 796         "begin": 1444334398154194201,
 797         "end": 1444334425194487548
 798       },
 799       "class": "syscall-latency",
 800       "data": [
 801         [
 802           {"class": "syscall", "name": "open"},
 803           45,
 804           {"class": "duration", "value": 5562},
 805           {"class": "duration", "value": 13835},
 806           {"class": "duration", "value": 77683},
 807           {"class": "duration", "value": 15263}
 808         ],
 809         [
 810           {"class": "syscall", "name": "read"},
 811           109,
 812           {"class": "duration", "value": 316},
 813           {"class": "duration", "value": 5774},
 814           {"class": "unknown"},
 815           {"class": "duration", "value": 9277}
 816         ]
 817       ]
 818     },
 819     {
 820       "time-range": {
 821         "class": "time-range",
 822         "begin": 1444334425194487549,
 823         "end": 1444334425254887190
 824       },
 825       "class": "syscall-latency",
 826       "data": [
 827         [
 828           {"class": "syscall", "name": "open"},
 829           45,
 830           {"class": "duration", "value": 1578},
 831           {"class": "duration", "value": 16648},
 832           {"class": "duration", "value": 15444},
 833           {"class": "duration", "value": 68540}
 834         ],
 835         [
 836           {"class": "syscall", "name": "read"},
 837           109,
 838           {"class": "duration", "value": 78},
 839           {"class": "duration", "value": 1948},
 840           {"class": "duration", "value": 11184},
 841           {"class": "duration", "value": 94670}
 842         ]
 843       ]
 844     }
 845   ]
 846 }
 847 ```