[deliverable/linux.git] / Documentation / accounting / delay-accounting.txt

Delay accounting
----------------

Tasks encounter delays in execution when they wait
for some kernel resource to become available e.g. a
runnable task may wait for a free CPU to run on.

The per-task delay accounting functionality measures
the delays experienced by a task while

a) waiting for a CPU (while being runnable)
b) completion of synchronous block I/O initiated by the task
c) swapping in pages

and makes these statistics available to userspace through
the taskstats interface.

Such delays provide feedback for setting a task's cpu priority,
io priority and rss limit values appropriately. Long delays for
important tasks could be a trigger for raising its corresponding priority.

The functionality, through its use of the taskstats interface, also provides
delay statistics aggregated for all tasks (or threads) belonging to a
thread group (corresponding to a traditional Unix process). This is a commonly
needed aggregation that is more efficiently done by the kernel.

Userspace utilities, particularly resource management applications, can also
aggregate delay statistics into arbitrary groups. To enable this, delay
statistics of a task are available both during its lifetime as well as on its
exit, ensuring continuous and complete monitoring can be done.


Interface
---------

Delay accounting uses the taskstats interface which is described
in detail in a separate document in this directory. Taskstats returns a
generic data structure to userspace corresponding to per-pid and per-tgid
statistics. The delay accounting functionality populates specific fields of
this structure. See
     include/linux/taskstats.h
for a description of the fields pertaining to delay accounting.
It will generally be in the form of counters returning the cumulative
delay seen for cpu, sync block I/O, swapin etc.

Taking the difference of two successive readings of a given
counter (say cpu_delay_total) for a task will give the delay
experienced by the task waiting for the corresponding resource
in that interval.

When a task exits, records containing the per-task statistics
are sent to userspace without requiring a command. If it is the last exiting
task of a thread group, the per-tgid statistics are also sent. More details
are given in the taskstats interface description.

The getdelays.c userspace utility in this directory allows simple commands to
be run and the corresponding delay statistics to be displayed. It also serves
as an example of using the taskstats interface.

Usage
-----

Compile the kernel with
	CONFIG_TASK_DELAY_ACCT=y
	CONFIG_TASKSTATS=y

Delay accounting is enabled by default at boot up.
To disable, add
   nodelayacct
to the kernel boot options. The rest of the instructions
below assume this has not been done.

After the system has booted up, use a utility
similar to  getdelays.c to access the delays
seen by a given task or a task group (tgid).
The utility also allows a given command to be
executed and the corresponding delays to be
seen.

General format of the getdelays command

getdelays [-t tgid] [-p pid] [-c cmd...]


Get delays, since system boot, for pid 10
# ./getdelays -p 10
(output similar to next case)

Get sum of delays, since system boot, for all pids with tgid 5
# ./getdelays -t 5


CPU	count	real total	virtual total	delay total
	7876	92005750	100000000	24001500
IO	count	delay total
	0	0
MEM	count	delay total
	0	0

Get delays seen in executing a given simple command
# ./getdelays -c ls /

bin   data1  data3  data5  dev  home  media  opt   root  srv        sys  usr
boot  data2  data4  data6  etc  lib   mnt    proc  sbin  subdomain  tmp  var


CPU	count	real total	virtual total	delay total
	6	4000250		4000000		0
IO	count	delay total
	0	0
MEM	count	delay total
	0	0
Commit	Line	Data
a3baf649 SN	1	Delay accounting
	2	----------------
	3
	4	Tasks encounter delays in execution when they wait
	5	for some kernel resource to become available e.g. a
	6	runnable task may wait for a free CPU to run on.
	7
	8	The per-task delay accounting functionality measures
	9	the delays experienced by a task while
	10
	11	a) waiting for a CPU (while being runnable)
	12	b) completion of synchronous block I/O initiated by the task
	13	c) swapping in pages
	14
	15	and makes these statistics available to userspace through
	16	the taskstats interface.
	17
	18	Such delays provide feedback for setting a task's cpu priority,
	19	io priority and rss limit values appropriately. Long delays for
	20	important tasks could be a trigger for raising its corresponding priority.
	21
	22	The functionality, through its use of the taskstats interface, also provides
	23	delay statistics aggregated for all tasks (or threads) belonging to a
	24	thread group (corresponding to a traditional Unix process). This is a commonly
	25	needed aggregation that is more efficiently done by the kernel.
	26
	27	Userspace utilities, particularly resource management applications, can also
	28	aggregate delay statistics into arbitrary groups. To enable this, delay
	29	statistics of a task are available both during its lifetime as well as on its
	30	exit, ensuring continuous and complete monitoring can be done.
	31
	32
	33	Interface
	34	---------
	35
	36	Delay accounting uses the taskstats interface which is described
	37	in detail in a separate document in this directory. Taskstats returns a
	38	generic data structure to userspace corresponding to per-pid and per-tgid
	39	statistics. The delay accounting functionality populates specific fields of
	40	this structure. See
	41	include/linux/taskstats.h
	42	for a description of the fields pertaining to delay accounting.
	43	It will generally be in the form of counters returning the cumulative
	44	delay seen for cpu, sync block I/O, swapin etc.
	45
	46	Taking the difference of two successive readings of a given
	47	counter (say cpu_delay_total) for a task will give the delay
	48	experienced by the task waiting for the corresponding resource
	49	in that interval.
	50
ad4ecbcb SN	51	When a task exits, records containing the per-task statistics
	52	are sent to userspace without requiring a command. If it is the last exiting
	53	task of a thread group, the per-tgid statistics are also sent. More details
	54	are given in the taskstats interface description.
a3baf649 SN	55
	56	The getdelays.c userspace utility in this directory allows simple commands to
	57	be run and the corresponding delay statistics to be displayed. It also serves
	58	as an example of using the taskstats interface.
	59
	60	Usage
	61	-----
	62
	63	Compile the kernel with
	64	CONFIG_TASK_DELAY_ACCT=y
	65	CONFIG_TASKSTATS=y
	66
163ecdff SN	67	Delay accounting is enabled by default at boot up.
	68	To disable, add
	69	nodelayacct
	70	to the kernel boot options. The rest of the instructions
	71	below assume this has not been done.
a3baf649	72
163ecdff	73	After the system has booted up, use a utility
a3baf649 SN	74	similar to getdelays.c to access the delays
	75	seen by a given task or a task group (tgid).
	76	The utility also allows a given command to be
	77	executed and the corresponding delays to be
	78	seen.
	79
	80	General format of the getdelays command
	81
	82	getdelays [-t tgid] [-p pid] [-c cmd...]
	83
	84
	85	Get delays, since system boot, for pid 10
	86	# ./getdelays -p 10
	87	(output similar to next case)
	88
	89	Get sum of delays, since system boot, for all pids with tgid 5
	90	# ./getdelays -t 5
	91
	92
	93	CPU count real total virtual total delay total
	94	7876 92005750 100000000 24001500
	95	IO count delay total
	96	0 0
	97	MEM count delay total
	98	0 0
	99
	100	Get delays seen in executing a given simple command
	101	# ./getdelays -c ls /
	102
	103	bin data1 data3 data5 dev home media opt root srv sys usr
	104	boot data2 data4 data6 etc lib mnt proc sbin subdomain tmp var
	105
	106
	107	CPU count real total virtual total delay total
	108	6 4000250 4000000 0
	109	IO count delay total
	110	0 0
	111	MEM count delay total
	112	0 0