When Finish is clicked, the trace is imported in the target folder. The folder structure from the trace package is restored in the target folder.
+=== Refreshing of Trace and Trace Folder ===
+Traces and trace folders in the workspace might be updated on the media. To refresh the content, right-click the trace or trace folder and select menu item '''Refresh'''. Alternatively, select the trace or trace folder and press key '''F5'''.
+
=== Remote Fetching ===
It is possible to import traces automatically from one or more remote hosts according to a predefined remote profile by using the '''Fetch Remote Traces''' wizard.
''Note'': The columns in the text file are separated by tabs.
+=== Refreshing of Trace ===
+It's possible to refresh the content of the trace and resume indexing in case the current open trace was updated on the media. To refresh the trace, right-click into the table and select menu item '''Refresh'''. Alternatively, press key '''F5'''.
+
=== Collapsing of Repetitive Events ===
The implementation for collapsing of repetitive events is trace type specific and is only available for certain trace types. For example, a trace type could allow collapsing of consecutive events that have the same event content but not the same timestamp. If a trace type supports this feature then it is possible to select the '''Collapse Events''' menu item after pressing the right mouse button in the table.
[[Image:images/LTTng2AssignedEvents.png]]
-=== Configuring Filter Expression On UST Event Fields ===
+=== Configuring Filter Expression When Enabling Events ===
-Since LTTng Tools v2.1.0 it is possible to configure a filter expression on UST event fields. To configure a filter expression on UST event fields, open the enable event dialog as described in chapters [[#Enabling UST Events On Session Level | Enabling UST Events On Session Level]], [[#Enabling Events On Domain Level | Enabling Events On Domain Level]] or [[#Enabling Events On Channel Level | Enabling Events On Channel Level]], select UST if needed, select the relevant '''Tracepoint''' event(s) and enter the filter expression in the '''Filter Expression''' text field.
+It is possible to provide a filter expression when enabling events for UST or Kernel. This feature has been available for UST since LTTng v2.1.0 and for Kernel since v2.7.0. To configure a filter expression, open the enable event dialog as described in previous chapters [[#Enabling UST Events On Session Level | Enabling UST Events On Session Level]], [[#Enabling Kernel Events On Session Level | Enabling Kernel Events On Session Level]], [[#Enabling Events On Domain Level | Enabling Events On Domain Level]] or [[#Enabling Events On Channel Level | Enabling Events On Channel Level]]. Then configure the relevant events and enter the filter expression in the '''Filter Expression''' text field.
-[[Image:images/LTTng2EnableEventWithFilter.png]]
+[[Image:images/LTTng2EnableEventWithFilter.png]] [[Image:images/LTTng2EnableEventWithKernelFilter.png]]
-Alternatively, open the dialog box for assigning events to a session and channel described in [[#Enabling Tracepoint Events From Provider | Enabling Tracepoint Events From Provider]] (for UST providers) and enter the filter expression in the '''Filter Expression''' text field.
+Alternatively, open the dialog box for assigning events to a session and channel described in [[#Enabling Tracepoint Events From Provider | Enabling Tracepoint Events From Provider]] and enter the filter expression in the '''Filter Expression''' text field.
[[Image:images/LTTng2AssignEventDialogWithFilter.png]]
Upon successful operation, the tree in the Control View will be refreshed with the remote host configuration.
-=== Quantifing LTTng overhead (Calibrate) ===
-
-The LTTng calibrate command can be used to find out the combined average overhead of the LTTng tracer and the instrumentation mechanisms used. For now, the only calibration implemented is that of the kernel function
-instrumentation (kretprobes). To run the calibrate command, select the a domain (e.g. '''Kernel'''), click the right mouse button on the domain tree node. A context-sensitive menu will show. Select the '''Calibrate''' menu item.
-
-[[Image:images/LTTng2CalibrateAction.png]]
-
-Upon successful operation, the calibrate command is executed and relevant information is stored in the trace. Note: that the trace has to be active so that to command as any effect.
-
=== Importing Session Traces to a Tracing Project ===
To import traces from a tracing session, select the relevant session and click on the '''Import''' Button. Alternatively, click the right mouse button on the session tree node and select the menu item '''Import...''' from the context-sensitive menu.
If one selects '''Rename All''', '''Overwrite All''' or '''Skip All''' the choice will be applied for all traces with a name conflict.
-
=== Importing Network Traces to a Tracing Project ===
Since LTTng Tools v2.1.0 it is possible to store traces over the network. To import network traces, execute the '''Import''' action as described in chapter [[#Importing Session Traces to a Tracing Project | Importing Session Traces to a Tracing Project]]. For network traces the '''Import Trace Wizard''' will be displayed. Follow the instructions in chapter [[#Importing | Importing]] to import the network traces of the current session.
+=== Saving Tracing Sessions ===
+Since LTTng Tools v2.5.0 it is possible to save tracing sessions. The LTTng Tools command-line tool will save the sessions to XML files located by default in a subdirectory of the user's home directory. The Trace Compass '''Control''' view integration for this feature will also store this session profile file into the user's Trace Compass workspace. This will allow user's to re-use session profiles across remote nodes. To save one or more sessions, select the tree nodes of the relevant sessions and press the right mouse button. Then select the '''Save...''' entry of the context-sensitive menu.
+
+[[Image:images/LTTng2SaveAction.png]]
+
+A new display will open for saving the sessions.
+
+[[Image:images/LTTng2SaveDialog.png]]
+
+By default the '''force''' button is selected that will overwrite any conflicting session profile files on the remote node. Click on '''Ok''' to save the session(s) otherwise click on '''Cancel'''. Upon successful operation, the session profile files will be saved on the remote node and then will be downloaded to the user's Trace Compass workspace. In the case that a session XML file already exists in the workspace the user will be prompted to skip or overwrite the existing profile file.
+
+=== Loading Tracing Sessions ===
+Since LTTng Tools v2.5.0 it is possible to load tracing sessions. The Trace Compass '''Control''' view integrations for this feature will allow to load session profiles that are located in the user's Trace Compass workspace, or alternatively, that are located on the remote node. In the first case the session profiles will be uploaded to the remote node before the load command is executed.
+
+To load one or more sessions, select the tree node '''Sessions''' and press the right mouse button. Then select the '''Load...''' entry of the context-sensitive menu.
+
+[[Image:images/LTTng2LoadAction.png]]
+
+A new display will open for loading session profiles.
+
+[[Image:images/LTTng2LoadDialog.png]]
+
+By default the '''Local''' button and '''force''' buttons are selected and session profile files of the user's workspace will be listed. Select one or more profiles, update the '''force''' button if needed and then click '''Ok'''. This will upload the session profile files to the remote node. If a session profile file with the same name already exist on the remote node, it will be overwritten. If the '''force''' button is selected any existing session with a conflicting name will be destroyed and a new one will be created.
+
+Alternatively, one can select the '''Remote''' button to list all available session profile files on the remote node. To load one of the remote session profiles, select one or more profiles, update the '''force''' button if needed and then click '''Ok'''.
+
+[[Image:images/LTTng2LoadRemoteDialog.png]]
+
+Upon successful operation, the tracing sessions of the selected session profiles are created and added under the tree node '''Sessions''' the '''Control''' view.
+
+=== Managing Tracing Session Profiles ===
+The '''LTTng Remote Profiles''' preference page is used to manage the list of LTTng session profiles that are stored in the user's Trace Compass workspace. To open the preference page, select the '''Manage...''' button of the '''Load Sessions''' dialog described in chapter [[#Loading Tracing Sessions |Loading Tracing Sessions]]. Alternatively, select '''Window -> Preferences''' from the top level menu and go to '''Tracing -> LTTng Remote Profiles'''.
+
+[[Image:images/LTTng2ManageSessionConfig.png]]
+
+The following actions can be performed from this dialog:
+
+* Delete
+
+Select one or more LTTng session profiles from the list and click the '''Delete''' button to remove the profile from the Trace Compass workspace. The user will be prompted to confirm the deletion.
+
+* Import...
+
+Click the '''Import...''' button and select a file from the opened file dialog to import a session profile file. If the file name conflicts with an existing profile file, the user will be prompted to skip or overwrite the existing profile file.
+* Export...
+
+Select one or more session profile files from the list, click the '''Export...''' button and enter or select a directory in the opened directory dialog to export the profile files. If the file name conflicts with an existing profile file in the destination directory, the user will be prompted to skip or overwrite the existing profile file.
+
== Properties View ==
The Control View provides property information of selected tree component. Depending on the selected tree component different properties are displayed in the property view. For example, when selecting the node level the property view will be filled as followed:
[[Image:images/LTTng2PreferencesTimeout.png]]
+
= LTTng Kernel Analysis =
Historically, LTTng was developped to trace the Linux kernel and, over time, a number of kernel-oriented analysis views were developped and organized in a perspective.
The TID column shows the process node's '''thread ID''' and the PTID column shows its '''parent thread ID''' (nothing is shown if the process has no parent).
+It is possible to sort the columns of the tree by clicking on the column header. Subsequent clicking will change the sort order. The hierarchy, i.e. the parent-child relationship is kept. When opening a trace for the first time, the processes are sorted by '''birth time'''. The sort order and column will be preserved when switching between open traces. Note that when opening an experiment the processes will be sorted within each trace.
+
=== Control flow ===
This part of the Control Flow View is probably the most interesting one. Using the mouse, you can navigate through the trace (go left, right) and zoom on a specific region to inspect its details.
* '''right-drag horizontally''': [[#Zoom region|zoom region]]
* '''click on a colored bar''': the associated process node is selected and the current time indicator is moved where the click happened
* '''mouse wheel up/down''': scroll up or down
-* '''Ctrl-mouse wheel up/down''': zoom in or out
+* '''Ctrl-mouse wheel up/down''': zoom in or out horizontally
+* '''Shift-Ctrl-mouse wheel up/down''': zoom in or out vertically
* '''drag the time ruler horizontally''': zoom in or out with fixed start time
* '''double-click the time ruler''': reset zoom to full range
|-
| [[Image:images/filter_items.gif]]
| Show View Filter
-| Opens the process filter dialog.
+| Opens the process filter dialog. Filter settings will be preserved when switching between open traces.
|-
| [[Image:images/show_legend.gif]]
| Show Legend
|-
| [[Image:images/filter_items.gif]]
| Show View Filter
-| Opens the process filter dialog.
+| Opens the resources filter dialog. Filter settings will be preserved when switching between open traces.
|-
| [[Image:images/show_legend.gif]]
| Show Legend
| [[Image:images/zoomout_nav.gif]]
| Zoom Out
| Zooms out on the selection by 50%.
-|-
-| [[Image:images/hide_arrows.gif]]
-| Hide Arrows
-| Toggles the display of arrows on or off.
-|-
-| [[Image:images/follow_arrow_bwd.gif]]
-| Follow CPU Backward
-| Selects the previous state following CPU execution across processes. Pressing the '''Shift''' key at the same time will update the selection end time of the current selection range.
-|-
-| [[Image:images/follow_arrow_fwd.gif]]
-| Follow CPU Forward
-| Selects the next state following CPU execution across processes. Pressing the '''Shift''' key at the same time will update the selection end time of the current selection range.
|}
View Menu
[[Image:images/XML_xy_chart.png| XML XY chart]]
+= Latency Analyses =
+
+Trace Compass offers a feature called Latency analysis. This allows an analysis to return intervals and these intervals will be displayed in four different views. An example analysis is provided with kernel system call latencies being provided. The available views are:
+
+* System Call Latencies
+A '''table''' of the raw latencies. This view is useful to inspect individual latencies.
+
+ [[Image:images/LatenciesTable.png| Latency Table example - System Call Latencies]]
+
+
+* System Call Latency vs Time
+A time aligned '''scatter chart''' of the latencies with respect to the current window range. This view is useful to see the overall form of the latencies as they arrive.
+
+[[Image:images/LatenciesScatter.png| Latency Scatter Chart example - System Call Latency vs Time]]
+
+
+* System Call Latency Statistics
+A view of the total '''statistics''' of the latencies. These show the ''minimum'', ''maximum'', ''average'' and ''standard deviation'' of the latencies when applicable. This tool is useful for finding the outliers on a per-category basis.
+
+[[Image:images/LatenciesStatistics.png| Latency Statistics example - System Call Latency Statistics]]
+
+
+* System Call Density
+A '''density''' view, analyzing the current time range. This is useful to find global outliers.
+
+[[Image:images/LatenciesDensity.png| Latency Densities example - System Call Density]]
+
+= Virtual Machine Analysis =
+
+Virtual environments are usually composed of host machines, who each run an hypervisor program on which one or many guests can be run. Tracing a guest machine alone can often yield some strange results as from its point of view, it has full use of the resources, but in reality, most resources are shared with the host and other guests.
+
+To better understand what is happening in such an environment, it is necessary to trace all the machines involved, guests and hosts, and correlate this information in an experiment that will display a complete view of the virtualized environment.
+
+== Virtual Machine Experiment ==
+
+A trace has to be taken for each machine, guest and host, in the virtualized environment. The host trace is the most important to have, as missing guests will only give an incomplete view of the system, but missing hosts usually won't allow to identify the hypervisor, nor determine when a guest is preempted from the host CPUs. The virtual machine analysis only makes sense if the host trace is available.
+
+Once all the traces are imported in Trace Compass, they can be [[#Creating a Experiment | added to an experiment]]. The type of the experiment should by set to '''Virtual Machine Experiment''' by clicking on the right mouse button over the experiment name, then selecting '''Select Experiment Type...'''.
+
+[[Image:images/vmAnalysis/VM_experiment.png | Virtual Machine Experiment]]
+
+Depending on the hypervisor used, traces might need to be [[#Trace synchronization | synchronized]] so that they have the same time reference and their events can be correctly correlated.
+
+== Virtual CPU View ==
+
+The Virtual CPU view shows the status of CPUs and threads on guests augmented with the preemption and hypervisor data we get from the host.
+
+In the image below, we see for the virtual CPU status that it has a few more states than the CPUs in the [[#Resources View | Resources View]]: in red and purple respectively, when the virtual CPU is running hypervisor code and when the CPU is preempted on the host.
+
+The entries for each thread of the machine corresponds to the one from the [[#Control flow | Control Flow View]], augmented with the data from the Virtual CPU, so that we see that even though it is running from the guest's point of view, it is actually not running when the Virtual CPU it runs on is in preempted or hypervisor mode.
+
+[[Image:images/vmAnalysis/VM_CPU_view.png | Virtual CPU view]]
+
+== Hypervisor-specific Tracing ==
+
+In order to be able to correlate data from the guests and hosts traces, each hypervisor supported by Trace Compass requires some specific events, that are sometimes not available in the default installation of the tracer.
+
+The following sections describe how to obtain traces for each hypervisor.
+
+=== Qemu/KVM ===
+
+The Qemu/KVM hypervisor require extra tracepoints not yet shipped in LTTng for both guests and hosts, as well as compilation with the full kernel source tree on the host, to have access to kvm_entry/kvm_exit events on x86.
+
+Obtain the source code with extra tracepoints, along with lttng-modules
+
+ # git clone https://github.com/giraldeau/lttng-modules.git
+ # cd lttng-modules
+
+Checkout the addons branch, compile and install lttng-modules as per the lttng-modules documentation.
+
+ # git checkout addons
+ # make
+ # sudo make modules_install
+ # sudo depmod -a
+
+On the host, to have complete kvm tracepoints support, the make command has to include the full kernel tree. So first, you'll need to obtain the kernel source tree. See your distribution's documentation on how to get it. This will compile extra modules, including lttng-probe-kvm-x86, which we need.
+
+ # make KERNELDIR=/path/to/kernel/dir
+
+The lttng addons modules must be inserted manually for the virtual machine extra tracepoints to be available:
+
+ # sudo modprobe lttng-addons
+ # sudo modprobe lttng-vmsync-host # on the host
+ # sudo modprobe lttng-vmsync-guest # on the guest
+
+The following tracepoints will be available
+
+ # sudo lttng list -k
+ Kernel events:
+ -------------
+ ...
+ kvm_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)
+ kvm_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
+ vmsync_gh_guest (loglevel: TRACE_EMERG (0)) (type: tracepoint) # on the guest
+ vmsync_hg_guest (loglevel: TRACE_EMERG (0)) (type: tracepoint) # on the guest
+ vmsync_gh_host (loglevel: TRACE_EMERG (0)) (type: tracepoint) # on the host
+ vmsync_hg_host (loglevel: TRACE_EMERG (0)) (type: tracepoint) # on the host
+ ...
+
+Host and guests can now be traced together and their traces added to an experiment. Because each guest has a different clock than the host, it is necessary to synchronize the traces together. Unfortunately, automatic synchronization with the virtual machine events is not completely implemented yet, so another kind of synchronization needs to be done, with TCP packets for instance. See section on [[#Trace synchronization | trace synchronization]] for information on how to obtain synchronizable traces.
+
= Limitations =
* When parsing text traces, the timestamps are assumed to be in the local time zone. This means that when combining it to CTF binary traces, there could be offsets by a few hours depending on where the traces were taken and where they were read.
projects / deliverable / tracecompass.git / blobdiff