lttng-ust.git
8 years agoIntroduce a verbose mode for the Java agent ust-app-context
Alexandre Montplaisir [Thu, 4 Feb 2016 23:37:37 +0000 (18:37 -0500)] 
Introduce a verbose mode for the Java agent

If the LTTNG_UST_DEBUG environment variable is defined, log messages
from the Java agent will be sent to stderr. This is in line with the
rest of UST.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoImplement Java agent application context retrieval
Alexandre Montplaisir [Thu, 7 Jan 2016 22:43:34 +0000 (17:43 -0500)] 
Implement Java agent application context retrieval

Java application can now register an IContextInfoRetriever to provide
context information. This information can be used for filtering:

  lttng enable-event -j myevent --filter '$app.retriever:context=="something"'

or for saving in the trace directly by enabling the context:

  lttng add-context -j -t '$app.retriever:context'

See the "ApplicationContextExample.java" program for an example of
utilization.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoTurn ISessiondCommand into an abstract class
Alexandre Montplaisir [Fri, 8 Jan 2016 20:28:07 +0000 (15:28 -0500)] 
Turn ISessiondCommand into an abstract class

This will allow us to define common methods to read strings passed on
the socket, which many commands (including upcoming ones) need to do.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoSplit the JNI APIs in separate classes
Alexandre Montplaisir [Mon, 4 Jan 2016 20:11:11 +0000 (15:11 -0500)] 
Split the JNI APIs in separate classes

Isolate the "native" methods in their own class, to better reflect
what is present in the equivalent C files.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoAdd app context support to ust-ctl protocol
Jérémie Galarneau [Wed, 27 Jan 2016 03:48:21 +0000 (22:48 -0500)] 
Add app context support to ust-ctl protocol

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoHandle application context cmd
Mathieu Desnoyers [Wed, 3 Feb 2016 22:07:27 +0000 (17:07 -0500)] 
Handle application context cmd

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoImplement dynamic types, and application context provider support
Mathieu Desnoyers [Wed, 13 Jan 2016 21:52:26 +0000 (16:52 -0500)] 
Implement dynamic types, and application context provider support

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Ensure the Java JUL messages are correctly formatted
Alexandre Montplaisir [Thu, 4 Feb 2016 05:32:01 +0000 (00:32 -0500)] 
Fix: Ensure the Java JUL messages are correctly formatted

It is possible for log records to contain messages that need some
formatting, for example if the string contains localized elements
or if the log(Level, String, Object[]) method is used.

In these cases, we need to make sure to format the string and not
pass the "raw" string to the tracepoint.

This only applies to the JUL API. log4j 1.2.x did not handle such
formatting, although log4j 2.x does.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: double-free on error sending fields
Mathieu Desnoyers [Fri, 15 Jan 2016 16:31:29 +0000 (11:31 -0500)] 
Fix: double-free on error sending fields

Found by Coverity:
*** CID 1348462:    (USE_AFTER_FREE)

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoAdd CTF enum type support to tracepoint event
Mathieu Desnoyers [Sat, 9 Jan 2016 19:44:30 +0000 (14:44 -0500)] 
Add CTF enum type support to tracepoint event

Derived from initial implementation by:
Geneviève Bastien <gbastien+lttng@versatic.net>

Bump UST communication protocol version to 6.1 (minor version increase)
since we're adding enumeration notification command.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: missing _GNU_SOURCE define
Jonathan Rajotte [Mon, 4 Jan 2016 22:31:57 +0000 (17:31 -0500)] 
Fix: missing _GNU_SOURCE define

O_CLOEXEC requires _GNU_SOURCE.

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agopython-lttngust/setup.py.in: update setup() fields
Philippe Proulx [Fri, 27 Nov 2015 18:04:44 +0000 (13:04 -0500)] 
python-lttngust/setup.py.in: update setup() fields

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoRefactor Python agent build and install
Philippe Proulx [Fri, 27 Nov 2015 17:39:51 +0000 (12:39 -0500)] 
Refactor Python agent build and install

Since the Python agent's tracepoint provider,
liblttng-ust-python-agent, does not depend on Python, it can
always be built and installed alongside LTTng-UST.

The Python package of this agent is completely independent
from the rest of the tree, thus it is isolated in its own
directory. This also eases the creation of distribution
packages because the packager can selectively build and
install the Python package without also building/installing the
tracepoint provider.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: remove debugging print() call from Python agent
Philippe Proulx [Tue, 27 Oct 2015 23:06:56 +0000 (19:06 -0400)] 
Fix: remove debugging print() call from Python agent

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: header size larger than 256 bytes
Mathieu Desnoyers [Sun, 25 Oct 2015 15:25:31 +0000 (11:25 -0400)] 
Fix: header size larger than 256 bytes

When adding large context (e.g. callstack), headers larger than 256
bytes cause discrepancy between calculated size and size written into
the trace buffers. This generates a corrupted trace and triggers a
warning in ring buffer backend, which triggers a safety net disabling
tracing for the current channel.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoPython agent: Support Agent protocol v2.0
Jérémie Galarneau [Fri, 23 Oct 2015 21:12:50 +0000 (17:12 -0400)] 
Python agent: Support Agent protocol v2.0

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Java agent protocol network endianness consistency
Jérémie Galarneau [Fri, 23 Oct 2015 21:12:51 +0000 (17:12 -0400)] 
Fix: Java agent protocol network endianness consistency

Considering the Agent Protocol is bumped to v2.0 as of 2.8.x,
this patch revisits the unfortunate decision of communicating
in host-endianness from the session daemon to the agents, and
in big endian from the agents to the session daemon.

This change does not affect the Python agent which was erroneously
(although quite reasonably) assuming communications were occurring
in network endianness.

This issue does not affect versions 2.7 and below because the loglevel
was not used.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: examples jul: add missing files to make dist
Mathieu Desnoyers [Thu, 22 Oct 2015 21:29:45 +0000 (17:29 -0400)] 
Fix: examples jul: add missing files to make dist

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Filer -> Filter typo in example
Mathieu Desnoyers [Thu, 22 Oct 2015 21:26:25 +0000 (17:26 -0400)] 
Fix: Filer -> Filter typo in example

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Return the correct list of available Java events
Alexandre Montplaisir [Wed, 9 Sep 2015 19:07:24 +0000 (15:07 -0400)] 
Fix: Return the correct list of available Java events

The "lttng list -j/-l" command should list the events that are currently
offered by Java application and available to be enabled.

Due to some confusion in the implementation of the corresponding agent
command response, it was actually returning the list of events that were
enabled in the tracing session.

Rectify this by sending the list of loggers of the corresponding domain
that have one or more LTTng log handlers attached. The interface method
was also renamed from listEnabledEvents() to listAvailableEvents() to
make it more representative.

Fixes: #933
Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoAdd a toString() to Java agent's EventRule
Alexandre Montplaisir [Tue, 15 Sep 2015 17:57:09 +0000 (13:57 -0400)] 
Add a toString() to Java agent's EventRule

Helps with debugging and eventually for pretty-printing.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoAdd API stubs for the Java context info retrievers
Alexandre Montplaisir [Thu, 3 Sep 2015 18:00:28 +0000 (14:00 -0400)] 
Add API stubs for the Java context info retrievers

A context info retriever will allow an application to define their own
context information, and have it available in the resulting UST traces.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoReceive the event filter string in the Java agent
Alexandre Montplaisir [Wed, 9 Sep 2015 23:46:08 +0000 (19:46 -0400)] 
Receive the event filter string in the Java agent

Update to agent protocol 2.0, which now sends the filter string as part
of the "enable event" command.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoDocument the Java filter notification model
Alexandre Montplaisir [Thu, 3 Sep 2015 16:42:15 +0000 (12:42 -0400)] 
Document the Java filter notification model

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoAdd filter change notification mechanism to the Java agent
Alexandre Montplaisir [Thu, 27 Aug 2015 23:50:36 +0000 (19:50 -0400)] 
Add filter change notification mechanism to the Java agent

Java applications can now register to receive notifications of event
filtering rules being changed in the tracing session. This can be
used to implement application-specific filtering on the Java side,
to reduce the amount of events sent through JNI.

To do so, they need to implement a IFilterChangeListener and register
it to the FilterChangeNotifier. The listener's callbacks will
be invoked by the LTTng agent when the tracing session(s) change.

A new example file is provided to demo this usage.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoManage complete "event rules" in the Java agent
Alexandre Montplaisir [Thu, 27 Aug 2015 00:10:50 +0000 (20:10 -0400)] 
Manage complete "event rules" in the Java agent

Instead of just tracking which event names are enabled in the tracing
session, we can track the complete name/filter/loglevel tuple. This
allows the same event name to be specified multiple times but with
different parameters.

Right now the sessiond does not send the filter string, a new protocol
version will be required to do so. But we can prepare for it in the
meantime.

The agent will continue to use the event names to decide if events
should be sent through JNI or not. However, full rules will be useable
for other purposes, like the upcoming filter notifications.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoIntroduce a new client listener interface for the Java agent
Alexandre Montplaisir [Sat, 29 Aug 2015 02:41:04 +0000 (22:41 -0400)] 
Introduce a new client listener interface for the Java agent

Decouple the TCP client from the implementation of the LTTng Java agent.
Instead of using AbstractLttngAgent directly, the TCP client (and the
command subclasses) can deal with the new ILttngTcpClientListener
interface. The agent will implement this interface.

This will also allow easier testing of the TCP client and its protocol,
since test classess can now implement their own listener and verify the
contents of each command.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoTurn ILttngAgentResponse into an abstract class
Alexandre Montplaisir [Tue, 25 Aug 2015 23:04:27 +0000 (19:04 -0400)] 
Turn ILttngAgentResponse into an abstract class

This allows defining a default behavior for the getBytes() method.
That behavior consists of only returning the integer return code,
which is what most (but not all) subclasses use.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoRemove stale tests/java-jul test
Mathieu Desnoyers [Thu, 22 Oct 2015 20:51:49 +0000 (16:51 -0400)] 
Remove stale tests/java-jul test

This test has been superseded by lttng-tools tests and
https://github.com/lttng/lttng-ust-java-tests .

This test was not in configure.ac nor had any automake Makefile.am
anyway.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: live timer calculation error
Mikael Beckius [Tue, 12 May 2015 09:04:34 +0000 (11:04 +0200)] 
Fix: live timer calculation error

There is an calculation error for live timer. Variable
chan->switch_timer_interval is based on microsecond, and it is not right
to assign chan->switch_timer_interval mod 1000000 to var tv_nsec which
is based on nanosecond.

Signed-off-by: Mikael Beckius <mikael.beckius@windriver.com>
Signed-off-by: Jianchuan Wang <jianchuan.wang@windriver.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix python agent build/install/uninstall with DESTDIR specified
Jonathan Rajotte [Fri, 16 Oct 2015 19:07:31 +0000 (15:07 -0400)] 
Fix python agent build/install/uninstall with DESTDIR specified

Remove the install_files.txt record since it's simpler to delete the
complete folder on uninstall.

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Don't (re)define STAP_PROBEV
Stelios Bounanos [Wed, 14 Oct 2015 16:31:36 +0000 (17:31 +0100)] 
Fix: Don't (re)define STAP_PROBEV

Define a new LTTNG_STAP_PROBEV macro to avoid clobbering STAP_PROBEV or
emitting unwanted sdt probes when lttng-ust has been built without sdt
support.

Signed-off-by: Stelios Bounanos <sb@enotty.net>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: don't dereference NULL pointers
Mathieu Desnoyers [Mon, 21 Sep 2015 20:44:38 +0000 (16:44 -0400)] 
Fix: don't dereference NULL pointers

Detected by scan-build.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoCleanup: Remove unused values
Mathieu Desnoyers [Mon, 21 Sep 2015 20:01:06 +0000 (16:01 -0400)] 
Cleanup: Remove unused values

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Value stored to 'has_waited' is never read
Mathieu Desnoyers [Mon, 21 Sep 2015 19:57:36 +0000 (15:57 -0400)] 
Fix: Value stored to 'has_waited' is never read

Reported by scan-build.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Argument with 'nonnull' attribute passed null
Mathieu Desnoyers [Mon, 21 Sep 2015 19:47:09 +0000 (15:47 -0400)] 
Fix: Argument with 'nonnull' attribute passed null

Reported by scan-build
API Argument with 'nonnull' attribute passed null libringbuffer
/ring_buffer_backend.c 380
API Argument with 'nonnull' attribute passed null libringbuffer
/ring_buffer_backend.c  420

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoCleanup: Unnecessary bit shift
Jonathan Rajotte [Mon, 21 Sep 2015 18:31:33 +0000 (14:31 -0400)] 
Cleanup: Unnecessary bit shift

Reported by cppcheck [1].

[1]
https://ci.lttng.org/view/Code%20quality/job/lttng-ust_master_cppcheck/5/cppcheckResult/source.10/

Proposed-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoClean-up: remove extraneous "found" parameters in ust-elf
Antoine Busque [Mon, 21 Sep 2015 18:19:43 +0000 (14:19 -0400)] 
Clean-up: remove extraneous "found" parameters in ust-elf

In lttng-ust-elf.c, static functions used to extract build ID or debug
link information had an extraneous `found` parameter, carrying no more
information than could be obtained by checking the other out
parameters against NULL. The resulting simplified logic should also
prevent static analysis tools from misidentifying resource leaks.

Signed-off-by: Antoine Busque <abusque@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: sysconf() unchecked return value
Mathieu Desnoyers [Mon, 21 Sep 2015 17:43:08 +0000 (13:43 -0400)] 
Fix: sysconf() unchecked return value

Fix Coverity bug:

CID 1021259 (#1 of 1): Improper use of negative value
(NEGATIVE_RETURNS)5. negative_returns: sysconf(_SC_PAGESIZE) is passed
to a parameter that cannot be negative.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agodoc: add Python example
Philippe Proulx [Sat, 5 Sep 2015 17:47:13 +0000 (13:47 -0400)] 
doc: add Python example

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Python agent: do not register twice to same port
Philippe Proulx [Sat, 5 Sep 2015 17:38:01 +0000 (13:38 -0400)] 
Fix: Python agent: do not register twice to same port

It is possible that one of the session daemons left its agent.port
file on the file system, for example when killed with SIGKILL. It
is also common that both those session daemons use the same port for
listening to agent connections. In this case, if one session daemon
is running, but two agent.port files exist, the Python agent would
connect its two threads to the same session daemon, leading to
everything done twice: list shows events twice, tracing records
events twice, etc.

This patch ensures that if two agent.port files are found and have
the same content, only one thread is used.

Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: potential leaks in error paths
Antoine Busque [Tue, 8 Sep 2015 21:33:18 +0000 (17:33 -0400)] 
Fix: potential leaks in error paths

Signed-off-by: Antoine Busque <abusque@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: double free in liblttng-ust-dl
Antoine Busque [Tue, 8 Sep 2015 21:24:16 +0000 (17:24 -0400)] 
Fix: double free in liblttng-ust-dl

Signed-off-by: Antoine Busque <abusque@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: make check in OOT build with absolute path
Michael Jeanson [Fri, 4 Sep 2015 21:31:31 +0000 (17:31 -0400)] 
Fix: make check in OOT build with absolute path

The previous patch fixed the out of tree build when the configure script
was called with a path relative to the builddir but still failed when
the path was absolute. This works with both.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: elf: leaks on error paths
Mathieu Desnoyers [Fri, 4 Sep 2015 06:00:09 +0000 (02:00 -0400)] 
Fix: elf: leaks on error paths

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoCleanup: coding style
Mathieu Desnoyers [Fri, 4 Sep 2015 05:50:56 +0000 (01:50 -0400)] 
Cleanup: coding style

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoCleanup: elf: use off_t for offsets, size_t for len
Mathieu Desnoyers [Fri, 4 Sep 2015 05:48:58 +0000 (01:48 -0400)] 
Cleanup: elf: use off_t for offsets, size_t for len

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoCleanup: eliminate implicit sign-extension
Mathieu Desnoyers [Fri, 4 Sep 2015 05:43:31 +0000 (01:43 -0400)] 
Cleanup: eliminate implicit sign-extension

Coverity reported:

CID 1321723 (#1 of 1): Unintended sign extension
(SIGN_EXTENSION)sign_extension: Suspicious implicit sign extension:
elf->ehdr->e_shentsize with type unsigned short (16 bits, unsigned) is
promoted in index * elf->ehdr->e_shentsize to type int (32 bits,
signed), then sign-extended to type unsigned long (64 bits, unsigned).
If index * elf->ehdr->e_shentsize is greater than 0x7FFFFFFF, the upper
bits of the result will all be 1.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: elf: NULL pointer dereference
Mathieu Desnoyers [Fri, 4 Sep 2015 05:35:47 +0000 (01:35 -0400)] 
Fix: elf: NULL pointer dereference

Coverity reported:
CID 1321730 (#1 of 1): Dereference null return value (NULL_RETURNS)15.
dereference: Dereferencing a pointer that might be null _build_id when
calling lttng_ust_read.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: add missing ust-elf.h header to dist
Michael Jeanson [Thu, 3 Sep 2015 20:14:41 +0000 (16:14 -0400)] 
Fix: add missing ust-elf.h header to dist

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: make check in out of tree build
Michael Jeanson [Thu, 3 Sep 2015 20:11:38 +0000 (16:11 -0400)] 
Fix: make check in out of tree build

Use the elf data files from the source dir when running make check in an
out of tree builddir and add the data files to dist.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: old gcc warnings
Mathieu Desnoyers [Tue, 1 Sep 2015 17:29:39 +0000 (13:29 -0400)] 
Fix: old gcc warnings

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: old gcc warning
Mathieu Desnoyers [Tue, 1 Sep 2015 16:55:02 +0000 (12:55 -0400)] 
Fix: old gcc warning

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: remove stale lttng-ust-elf.h from makefile
Mathieu Desnoyers [Mon, 31 Aug 2015 22:34:18 +0000 (18:34 -0400)] 
Fix: remove stale lttng-ust-elf.h from makefile

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: elf: uninitialized ret
Mathieu Desnoyers [Mon, 31 Aug 2015 22:25:02 +0000 (18:25 -0400)] 
Fix: elf: uninitialized ret

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: test elf: handle NULL debug file
Mathieu Desnoyers [Mon, 31 Aug 2015 21:27:39 +0000 (17:27 -0400)] 
Fix: test elf: handle NULL debug file

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agotest: elf move constants to top of implementation
Mathieu Desnoyers [Mon, 31 Aug 2015 21:25:07 +0000 (17:25 -0400)] 
test: elf move constants to top of implementation

Declare them static const.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: elf test prog arg checking
Mathieu Desnoyers [Mon, 31 Aug 2015 21:19:20 +0000 (17:19 -0400)] 
Fix: elf test prog arg checking

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agobaddr statedump: remove dependency on file streams
Mathieu Desnoyers [Mon, 31 Aug 2015 20:50:50 +0000 (16:50 -0400)] 
baddr statedump: remove dependency on file streams

None of the rest of UST used by applications use file streams (only the
sessiond uses a file streams for metadata). Therefore, use file
descriptors directly for baddr statedump.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agobaddr statedump: hold ust lock around allocations
Mathieu Desnoyers [Mon, 31 Aug 2015 20:50:03 +0000 (16:50 -0400)] 
baddr statedump: hold ust lock around allocations

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoAdd unit tests for lttng_ust_elf
Antoine Busque [Thu, 30 Jul 2015 20:37:04 +0000 (16:37 -0400)] 
Add unit tests for lttng_ust_elf

This adds unit tests for UST's ELF parser. Also included are test ELF
files for multiple architectures (x86, x86_64, armeb, aarch64_be). The
procedure to generate these test files is described in
`tests/ust-elf/README.md`.

Signed-off-by: Antoine Busque <abusque@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoAdd memory size, build id, and debug link info to statedump and dl
Antoine Busque [Thu, 2 Jul 2015 15:55:53 +0000 (11:55 -0400)] 
Add memory size, build id, and debug link info to statedump and dl

Implement a minimal ELF parser allowing to compute an executable's
in-memory size and to read its build id and debug link info, if any,
at runtime. This is performed during base address statedump, and on
dlopen when using the `liblttng-ust-dl.so` helper.

This adds a `memsz` field to `lttng_ust_statedump:soinfo` and
`lttng_ust_dl:dlopen`. Also adds two events to both providers,
`build_id` and `debug_link`, which are only traced when the
corresponding information is found in the executable file. They can be
matched with the corresponding `soinfo` or `dlopen` via the `baddr`
field shared amongst all the events. Build ID and debug link are
standard methods of identifying and retrieving debug information
corresponding to a specific version of an executable.

The fields `size` and `mtime` from the existing `soinfo` and `dlopen`
events have been removed as they provided no valuable information for
analysis, whereas build ID or debug link allow unambiguous retrieval
of the debug information, and the on-disk `size` is superseded by the
in-memory `memsz`.

Signed-off-by: Antoine Busque <abusque@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: close socket on protocol error, sendmsg MSG_NOSIGNAL
Mathieu Desnoyers [Wed, 26 Aug 2015 22:24:11 +0000 (18:24 -0400)] 
Fix: close socket on protocol error, sendmsg MSG_NOSIGNAL

Don't try to keep interacting with sessiond when a protocol error is
detected at the UST application side: this means we cannot trust the
protocol anymore, so there is no reason for keeping the socket open.
For instance, if the application is exiting and we receive a new stream,
we're effectively not reading the stream data, and we return an error.
Unfortunately, the session daemon may try to send us another command,
but we will try interpreting the stream data as a command, which is
invalid.

Also, use MSG_NOSIGNAL flag in the fds recvmsg, so the session daemon
don't get killed with SIGPIPE when it cannot send to the socket due to
connection closed.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoCleanup: more descriptive error message
Mathieu Desnoyers [Tue, 25 Aug 2015 11:12:51 +0000 (07:12 -0400)] 
Cleanup: more descriptive error message

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: add missing ust lock around objd_table_destroy()
Mathieu Desnoyers [Fri, 21 Aug 2015 23:11:56 +0000 (16:11 -0700)] 
Fix: add missing ust lock around objd_table_destroy()

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: application exit race with pthread cancel
Mathieu Desnoyers [Mon, 17 Aug 2015 16:50:34 +0000 (09:50 -0700)] 
Fix: application exit race with pthread cancel

Listener threads can be cancelled with ust lock held, which can hang the
following ust cleanup routine, because tracepoint probe unregister needs
to take the ust lock.

Fix this by disabling pthread cancellation for the entire duration of
the ust lock.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoCleanup: configure layout for python agent detection
Mathieu Desnoyers [Tue, 11 Aug 2015 16:11:57 +0000 (12:11 -0400)] 
Cleanup: configure layout for python agent detection

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoBuild: python agent: use setup.py over autoconf
Jonathan Rajotte [Mon, 10 Aug 2015 18:45:34 +0000 (14:45 -0400)] 
Build: python agent: use setup.py over autoconf

This change provides a valid way of installing the agent. The autoconf
python macro provides the wrong installation path for
python 3 under Debian and Ubuntu due to distro-specific packaging for
python 2.7 and 3.

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoBuild: add python interpreter used by python agent if built
Jonathan Rajotte [Mon, 10 Aug 2015 18:45:33 +0000 (14:45 -0400)] 
Build: add python interpreter used by python agent if built

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoDoc: basic lttng ust python agent documentation
Jonathan Rajotte [Mon, 10 Aug 2015 18:45:32 +0000 (14:45 -0400)] 
Doc: basic lttng ust python agent documentation

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoBuild: use AM_PATH_PYTHON to resolve python when building agent
Jonathan Rajotte [Mon, 10 Aug 2015 18:45:31 +0000 (14:45 -0400)] 
Build: use AM_PATH_PYTHON to resolve python when building agent

The agent can be built against a forced python version by exporting
PYTHON environment variable at configure time.

ex:
$ export PYTHON=python3
$ ./configure --enable-python-agent

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoBuild: only check if python exists in path
Jonathan Rajotte [Mon, 10 Aug 2015 18:45:30 +0000 (14:45 -0400)] 
Build: only check if python exists in path

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoUpdate version to 2.8.0-pre
Mathieu Desnoyers [Thu, 6 Aug 2015 23:49:26 +0000 (19:49 -0400)] 
Update version to 2.8.0-pre

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Send the correct Java agent return code when disabling events
Alexandre Montplaisir [Thu, 6 Aug 2015 17:41:50 +0000 (13:41 -0400)] 
Fix: Send the correct Java agent return code when disabling events

When receiving a "disable event" command from the sessiond for
an event that is not known, the Java agent should send the
"unknown logger name" return code, not the "invalid command" one.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Correctly handle the command to disable all events
Alexandre Montplaisir [Thu, 30 Jul 2015 23:52:14 +0000 (19:52 -0400)] 
Fix: Correctly handle the command to disable all events

Missed a return statement in the method to handle disable
event commands. The "*" event would get disabled correctly,
but the execution would fall-through and fail later on,
incorrectly reporting a failure.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoSmall refactor of the Java agent's TCP client
Alexandre Montplaisir [Thu, 30 Jul 2015 19:37:31 +0000 (15:37 -0400)] 
Small refactor of the Java agent's TCP client

Better separate the "commands" (sent from the sessiond to the
agent) from the "responses" (sent from the agent to the sessiond
as replies to commands) into distinct objects.

ISessiondCommand#execute() is now an interface method, and it
returns an ILttngAgentResponse.

This more rigorous handling of commands and responses will make
it easier to add support for additional commands in the future.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoRename Java Agent event names to "event"
Alexandre Montplaisir [Thu, 30 Jul 2015 20:11:55 +0000 (16:11 -0400)] 
Rename Java Agent event names to "event"

Previously, separate event names were used for events targeting
a root session daemon or a user one. Those were "sys_event" and
"user_event", respectively.

Now that the Java side only sends one event type, we can rename
it to simply "event".

This patch is dependant on a similar update on the lttng-tools
side.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoRename Python agent tracepoint to "event"
Jérémie Galarneau [Thu, 30 Jul 2015 20:43:21 +0000 (16:43 -0400)] 
Rename Python agent tracepoint to "event"

This rename comes as an effort to remain consistent with the tracepoint
naming scheme used by Log4J and JUL.

The associated change in lttng-tools is introduced as of d8c7bda0.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoAllow compiling the Java agent with Java 1.6
Alexandre Montplaisir [Wed, 29 Jul 2015 20:29:54 +0000 (16:29 -0400)] 
Allow compiling the Java agent with Java 1.6

Previous patch d60dfbe inadvertently made use of
ReflectiveOperationException, which was introduced in Java 7.
To continue being able to compile the agent with Java 6, we
can list individual exception types instead.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoRestore concurrent build when Java agent is enabled
Alexandre Montplaisir [Mon, 27 Jul 2015 19:41:01 +0000 (15:41 -0400)] 
Restore concurrent build when Java agent is enabled

Add a missing target dependency in lttng-ust-agent-common, to
make sure it does not try to package the .jar before the classes
are actually compiled.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoRefactor Java agent to let applications manage the log handlers
Alexandre Montplaisir [Wed, 22 Jul 2015 02:56:15 +0000 (22:56 -0400)] 
Refactor Java agent to let applications manage the log handlers

Major refactoring of the UST Java agent and its JUL and log4j
logging facilities.

Split up the big LTTngAgent class into two separate agent
singletons, one for JUL and one for log4j.
The large parts of common code remain in the -common package
in AbstractLttngAgent, with the API-specific implementations
in their respective -jul or -log4j package.
This removes the need for circular-dependencies (which were
worked around using reflection) between the common and the
other packages.

The applications are now expected to instantiate their
JUL Handler or log4j LogAppender themselves, and to attach
them to corresponding Logger objects. The LttngLogHandler and
LttngLogAppender classes are now made public to allow this.

Agents are now spawned automatically on demand, as Handler
objects are created. Once the last handler/appender is
close()'d, the agent is disposed as well. This reduces the
number of threads and socket connections to the minimum that
is necessary.

The two separate tracepoints in the JNI libraries, "tracepointS"
and "tracepointU", are now merged into one. Events sent to
either the root or user session daemon will use the same call.

The LTTngAgent.getLTTngAgent() API is kept for compatibility
with applications using it, but now marked @Deprecated. The
functionality underneath was re-implemented using the new
classes, but the behaviour should be kept the same.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoDoc: add LTTNG_UST_CLOCK_PLUGIN to man page
Jonathan Rajotte [Thu, 23 Jul 2015 20:02:51 +0000 (16:02 -0400)] 
Doc: add LTTNG_UST_CLOCK_PLUGIN to man page

Also add indications on how to use the plugin with lttng-tools.

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoDoc: add LTTNG_UST_GETCPU_PLUGIN to man page
Jonathan Rajotte [Thu, 23 Jul 2015 20:02:50 +0000 (16:02 -0400)] 
Doc: add LTTNG_UST_GETCPU_PLUGIN to man page

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoCleanup: Add Javadoc to all public methods and members
Alexandre Montplaisir [Wed, 22 Jul 2015 02:50:54 +0000 (22:50 -0400)] 
Cleanup: Add Javadoc to all public methods and members

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Small fixes to the Java agent example files
Alexandre Montplaisir [Wed, 22 Jul 2015 02:50:44 +0000 (22:50 -0400)] 
Fix: Small fixes to the Java agent example files

Remove unused imports.
Add missing dispose() in the log4j example.
Fix typos.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoUpdate doc/java-agent.txt to reflect the new packaging
Alexandre Montplaisir [Fri, 17 Jul 2015 23:07:50 +0000 (19:07 -0400)] 
Update doc/java-agent.txt to reflect the new packaging

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoSplit Java agent library in 3 separate jars
Alexandre Montplaisir [Tue, 30 Jun 2015 00:13:50 +0000 (20:13 -0400)] 
Split Java agent library in 3 separate jars

Instead of having one .jar whose content is dependent on the
configure flags used for the build, we can produce 3 separate
jars whose contents will always be the same:

lttng-ust-agent-common
lttng-ust-agent-jul
lttng-ust-agent-log4j

The configure flag will now determine if the jul/log4j packages
are built or not.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoRemove deprecated org.lttng.ust.jul.LTTngAgent class
Alexandre Montplaisir [Fri, 17 Jul 2015 17:00:41 +0000 (13:00 -0400)] 
Remove deprecated org.lttng.ust.jul.LTTngAgent class

This interface was deprecated back in 2.6. All users should now
use the org.lttng.ust.agent.jul.LTTngAgent class instead.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoUse tar-ustar format for the dist archive
Alexandre Montplaisir [Fri, 17 Jul 2015 22:06:27 +0000 (18:06 -0400)] 
Use tar-ustar format for the dist archive

By default, Automake uses the old tar v7 format, which does not
support file paths longer than 99 characters. With the upcoming
Java agent refactoring, this becomes a problem.

This patch switches to the newer tar-ustar format, which supports
longer file paths in the archive.

Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: remove generated file from git tree
Mathieu Desnoyers [Thu, 16 Jul 2015 16:07:27 +0000 (12:07 -0400)] 
Fix: remove generated file from git tree

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoVersion 2.7.0-rc1
Mathieu Desnoyers [Wed, 15 Jul 2015 18:42:05 +0000 (14:42 -0400)] 
Version 2.7.0-rc1

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Use $enableval with AC_ARG_ENABLE
Michael Jeanson [Wed, 8 Jul 2015 18:24:57 +0000 (14:24 -0400)] 
Fix: Use $enableval with AC_ARG_ENABLE

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: Cleanup local_apps sock_info in lttng_ust_cleanup
Jérémie Galarneau [Tue, 7 Jul 2015 18:35:54 +0000 (14:35 -0400)] 
Fix: Cleanup local_apps sock_info in lttng_ust_cleanup

LTTng-UST will deadlock after a fork while waiting on the
"constructor_wait" semaphore if local apps, handled the session daemon
running under the current UID, are disabled or "not_allowed".

This deadlock can be triggered by setting an infinite registration
timeout, clearing the HOME environment variable and launching an app
which calls FORK(3). This will cause setup_local_apps() to fail to
determine the local_apps sock_path, thus leaving
local_apps.allowed == 0.

This, in turn, would cause lttng_ust_cleanup to skip the cleanup
of the local_apps sock_info after a fork,
leaving local_apps.constructor_sem_posted == 1. This would cause
handle_register_done() in the child to skip over the decrementation
of sem_count and post of the constructor_wait semaphore.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: handle sys_futex EINTR and EWOULDBLOCK
Mathieu Desnoyers [Mon, 6 Jul 2015 21:09:32 +0000 (17:09 -0400)] 
Fix: handle sys_futex EINTR and EWOULDBLOCK

Handling sys_futex EINTR allows us to retry waiting on the futex
immediately if we are interrupted by a signal without having to try
connecting to the session daemon.

This should not impact correctness though, as the wait scheme would
simply try to connect, fail, and try waiting again.

Also handle errors on FUTEX_WAKE.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: update liburcu URL
Mathieu Desnoyers [Thu, 2 Jul 2015 21:11:22 +0000 (17:11 -0400)] 
Fix: update liburcu URL

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years ago.gitignore: ignore Python agent generated files
Philippe Proulx [Tue, 30 Jun 2015 23:45:10 +0000 (19:45 -0400)] 
.gitignore: ignore Python agent generated files

Reported-by: Antoine Busque <abusque@efficios.com>
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
8 years agoAdd ctf_sequence_hex() macro
Philippe Proulx [Tue, 30 Jun 2015 23:39:36 +0000 (19:39 -0400)] 
Add ctf_sequence_hex() macro

This patch adds the `_elem_type_base` parameter to
the internal _ctf_sequence_encoded() macro to
specify the sequence's element type's base.

ctf_sequence_hex() sets the `_elem_type_base`
parameter to 16.

This is convenient for recording byte buffers and
viewing them in hexadecimal.

Suggested-by: Antoine Busque <abusque@efficios.com>
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
8 years agoFix: set soinfo_data's vdso flag correctly in base address statedump
Antoine Busque [Tue, 30 Jun 2015 21:59:01 +0000 (17:59 -0400)] 
Fix: set soinfo_data's vdso flag correctly in base address statedump

The vdso flag is set to 0 when the executable can be found on disk.

Signed-off-by: Antoine Busque <abusque@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: running java examples out of tree
Michael Jeanson [Tue, 30 Jun 2015 21:32:56 +0000 (17:32 -0400)] 
Fix: running java examples out of tree

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
8 years agoFix: java class check when uudecode is not present
Michael Jeanson [Tue, 30 Jun 2015 20:59:54 +0000 (16:59 -0400)] 
Fix: java class check when uudecode is not present

On systems where 'uudecode' is present, the java class check will
run a precompiled class file in the local directory. If the CLASSPATH
variable doesn't contain "." the test will fail regardless of the
presence of the tested for class on the classpath.

This fix makes the behavior of the test consistant whether 'uudecode'
is present or not.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
This page took 0.043295 seconds and 5 git commands to generate.