Fix: libcompat is now part of libcommon
[lttng-tools.git] / doc / relayd-architecture.txt
CommitLineData
7591bab1
MD
1LTTng Relay Daemon Architecture
2Mathieu Desnoyers, August 2015
3
4This document describes the object model and architecture of the relay
5daemon, after the refactoring done within the commit "Fix: Relay daemon
6ownership and reference counting".
7
8We have the following object composition hierarchy:
9
10relay connection (main.c, for sessiond/consumer)
11 |
12 \-> 0 or 1 session
13 |
14 \-> 0 or many ctf-trace
15 |
16 \-> 0 or many stream
17 | |
18 | \-> 0 or many index
19 |
20 \-------> 0 or 1 viewer stream
21
22live connection (live.c, for client)
23 |
24 \-> 1 viewer session
25 |
26 \-> 0 or many session (actually a reference to session as created
27 | by the relay connection)
28 |
29 \-> ..... (ctf-trace, stream, index, viewer stream)
30
31There are global tables declared in lttng-relayd.h for sessions
32(sessions_ht, indexed by session id), streams (relay_streams_ht, indexed
33by stream handle), and viewer streams (viewer_streams_ht, indexed by
34stream handle). The purpose of those tables is to allow fast lookup of
35those objects using the IDs received in the communication protocols.
36
37There is also one connection hash table per worker thread. There is one
38worker thread to receive data (main.c), and one worker thread to
39interact with viewer clients (live.c). Those tables are indexed by
40socket file descriptor.
41
42A RCU lookup+refcounting scheme has been introduced for all objects
43(except viewer session which is still an exception at the moment). This
44scheme allows looking up the objects or doing a traversal on the RCU
45linked list or hash table in combination with a getter on the object.
46This getter validates that there is still at least one reference to the
ce4d4083 47object, else the lookup acts just as if the object does not exist.
7591bab1
MD
48
49The relay_connection (connection between the sessiond/consumer and the
50relayd) is the outermost object of its hierarchy.
51
52The live connection (connection between a live client and the relayd)
53is the outermost object of its hierarchy.
54
55There is also a "lock" mutex in each object. Those are used to
56synchronize between threads (currently the main.c relay thread and
57live.c client thread) when objects are shared. Locks can be nested from
58the outermost object to the innermost object. IOW, the ctf-trace lock can
59nest within the session lock.
60
7591bab1
MD
61RCU linked lists are used to iterate using RCU, and are protected by
62their own mutex for modifications. Iterations should be confirmed using
63the object "getter" to ensure its refcount is not 0 (except in cases
64where the caller actually owns the objects and therefore can assume its
65refcount is not 0).
66
67RCU hash tables are used to iterate using RCU. Iteration should be
68confirmed using the object "getter" to ensure its refcount is not 0
69(except again if we have ownership and can assume the object refcount is
70not 0).
71
72Object creation has a refcount of 1. Each getter increments the
73refcount, and needs to be paired with a "put" to decrement it. A final
74put on "self" (ownership) will allow refcount to reach 0, therefore
75triggering release, and thus free through call_rcu.
76
77In the composition scheme, we find back references from each composite
78to its container. Therefore, each composite holds a reference (refcount)
79on its container. This allows following pointers from e.g. viewer stream
80to stream to ctf-trace to session without performing any validation,
81due to transitive refcounting of those back-references.
82
83In addition to those back references, there are a few key ownership
84references held. The connection in the relay worker thread (main.c)
85holds ownership on the session, and on each stream it contains. The
86connection in the live worker thread (live.c) holds ownership on each
87viewer stream it creates. The rest is ensured by back references from
88composite to container objects. When a connection is closed, it puts all
89the ownership references it is holding. This will then eventually
90trigger destruction of the session, streams, and viewer streams
91associated with the connection when all the back references reach 0.
92
93RCU read-side locks are now only held during iteration on RCU lists and
94hash tables, and within the internals of the get (lookup) and put
95functions. Those functions then use refcounting to ensure existence of
96the object when returned to their caller.
This page took 0.040336 seconds and 5 git commands to generate.