Documentation/media/uapi/v4l/dev-overlay.rst

   1 .. -*- coding: utf-8; mode: rst -*-
   2
   3 .. _overlay:
   4
   5 ***********************
   6 Video Overlay Interface
   7 ***********************
   8
   9 **Also known as Framebuffer Overlay or Previewing.**
  10
  11 Video overlay devices have the ability to genlock (TV-)video into the
  12 (VGA-)video signal of a graphics card, or to store captured images
  13 directly in video memory of a graphics card, typically with clipping.
  14 This can be considerable more efficient than capturing images and
  15 displaying them by other means. In the old days when only nuclear power
  16 plants needed cooling towers this used to be the only way to put live
  17 video into a window.
  18
  19 Video overlay devices are accessed through the same character special
  20 files as :ref:`video capture <capture>` devices.
  21
  22 .. note::
  23
  24    The default function of a ``/dev/video`` device is video
  25    capturing. The overlay function is only available after calling
  26    the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl.
  27
  28 The driver may support simultaneous overlay and capturing using the
  29 read/write and streaming I/O methods. If so, operation at the nominal
  30 frame rate of the video standard is not guaranteed. Frames may be
  31 directed away from overlay to capture, or one field may be used for
  32 overlay and the other for capture if the capture parameters permit this.
  33
  34 Applications should use different file descriptors for capturing and
  35 overlay. This must be supported by all drivers capable of simultaneous
  36 capturing and overlay. Optionally these drivers may also permit
  37 capturing and overlay with a single file descriptor for compatibility
  38 with V4L and earlier versions of V4L2. [#f1]_
  39
  40
  41 Querying Capabilities
  42 =====================
  43
  44 Devices supporting the video overlay interface set the
  45 ``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct
  46 :c:type:`v4l2_capability` returned by the
  47 :ref:`VIDIOC_QUERYCAP` ioctl. The overlay I/O
  48 method specified below must be supported. Tuners and audio inputs are
  49 optional.
  50
  51
  52 Supplemental Functions
  53 ======================
  54
  55 Video overlay devices shall support :ref:`audio input <audio>`,
  56 :ref:`tuner`, :ref:`controls <control>`,
  57 :ref:`cropping and scaling <crop>` and
  58 :ref:`streaming parameter <streaming-par>` ioctls as needed. The
  59 :ref:`video input <video>` and :ref:`video standard <standard>`
  60 ioctls must be supported by all video overlay devices.
  61
  62
  63 Setup
  64 =====
  65
  66 Before overlay can commence applications must program the driver with
  67 frame buffer parameters, namely the address and size of the frame buffer
  68 and the image format, for example RGB 5:6:5. The
  69 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and
  70 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and
  71 set these parameters, respectively. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is
  72 privileged because it allows to set up DMA into physical memory,
  73 bypassing the memory protection mechanisms of the kernel. Only the
  74 superuser can change the frame buffer address and size. Users are not
  75 supposed to run TV applications as root or with SUID bit set. A small
  76 helper application with suitable privileges should query the graphics
  77 system and program the V4L2 driver at the appropriate time.
  78
  79 Some devices add the video overlay to the output signal of the graphics
  80 card. In this case the frame buffer is not modified by the video device,
  81 and the frame buffer address and pixel format are not needed by the
  82 driver. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is not privileged. An application
  83 can check for this type of device by calling the :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`
  84 ioctl.
  85
  86 A driver may support any (or none) of five clipping/blending methods:
  87
  88 1. Chroma-keying displays the overlaid image only where pixels in the
  89    primary graphics surface assume a certain color.
  90
  91 2. A bitmap can be specified where each bit corresponds to a pixel in
  92    the overlaid image. When the bit is set, the corresponding video
  93    pixel is displayed, otherwise a pixel of the graphics surface.
  94
  95 3. A list of clipping rectangles can be specified. In these regions *no*
  96    video is displayed, so the graphics surface can be seen here.
  97
  98 4. The framebuffer has an alpha channel that can be used to clip or
  99    blend the framebuffer with the video.
 100
 101 5. A global alpha value can be specified to blend the framebuffer
 102    contents with video images.
 103
 104 When simultaneous capturing and overlay is supported and the hardware
 105 prohibits different image and frame buffer formats, the format requested
 106 first takes precedence. The attempt to capture
 107 (:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay
 108 (:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an ``EBUSY`` error
 109 code or return accordingly modified parameters..
 110
 111
 112 Overlay Window
 113 ==============
 114
 115 The overlaid image is determined by cropping and overlay window
 116 parameters. The former select an area of the video picture to capture,
 117 the latter how images are overlaid and clipped. Cropping initialization
 118 at minimum requires to reset the parameters to defaults. An example is
 119 given in :ref:`crop`.
 120
 121 The overlay window is described by a struct
 122 :c:type:`v4l2_window`. It defines the size of the image,
 123 its position over the graphics surface and the clipping to be applied.
 124 To get the current parameters applications set the ``type`` field of a
 125 struct :c:type:`v4l2_format` to
 126 ``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the
 127 :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the
 128 struct :c:type:`v4l2_window` substructure named ``win``. It is not
 129 possible to retrieve a previously programmed clipping list or bitmap.
 130
 131 To program the overlay window applications set the ``type`` field of a
 132 struct :c:type:`v4l2_format` to
 133 ``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and
 134 call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver
 135 adjusts the parameters against hardware limits and returns the actual
 136 parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does. Like :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`, the
 137 :ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn
 138 about driver capabilities without actually changing driver state. Unlike
 139 :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` this also works after the overlay has been enabled.
 140
 141 The scaling factor of the overlaid image is implied by the width and
 142 height given in struct :c:type:`v4l2_window` and the size
 143 of the cropping rectangle. For more information see :ref:`crop`.
 144
 145 When simultaneous capturing and overlay is supported and the hardware
 146 prohibits different image and window sizes, the size requested first
 147 takes precedence. The attempt to capture or overlay as well
 148 (:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an ``EBUSY`` error
 149 code or return accordingly modified parameters.
 150
 151
 152 .. c:type:: v4l2_window
 153
 154 struct v4l2_window
 155 ------------------
 156
 157 ``struct v4l2_rect w``
 158     Size and position of the window relative to the top, left corner of
 159     the frame buffer defined with
 160     :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the
 161     frame buffer width and height, the ``x`` and ``y`` coordinates can
 162     be negative, and it can lie completely outside the frame buffer. The
 163     driver clips the window accordingly, or if that is not possible,
 164     modifies its size and/or position.
 165
 166 ``enum v4l2_field field``
 167     Applications set this field to determine which video field shall be
 168     overlaid, typically one of ``V4L2_FIELD_ANY`` (0),
 169     ``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or
 170     ``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different
 171     field order and return the actual setting here.
 172
 173 ``__u32 chromakey``
 174     When chroma-keying has been negotiated with
 175     :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field
 176     to the desired pixel value for the chroma key. The format is the
 177     same as the pixel format of the framebuffer (struct
 178     :c:type:`v4l2_framebuffer` ``fmt.pixelformat``
 179     field), with bytes in host order. E. g. for
 180     :ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should
 181     be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host.
 182
 183 ``struct v4l2_clip * clips``
 184     When chroma-keying has *not* been negotiated and
 185     :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
 186     applications can set this field to point to an array of clipping
 187     rectangles.
 188
 189     Like the window coordinates w, clipping rectangles are defined
 190     relative to the top, left corner of the frame buffer. However
 191     clipping rectangles must not extend the frame buffer width and
 192     height, and they must not overlap. If possible applications
 193     should merge adjacent rectangles. Whether this must create
 194     x-y or y-x bands, or the order of rectangles, is not defined. When
 195     clip lists are not supported the driver ignores this field. Its
 196     contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`
 197     are undefined.
 198
 199 ``__u32 clipcount``
 200     When the application set the ``clips`` field, this field must
 201     contain the number of clipping rectangles in the list. When clip
 202     lists are not supported the driver ignores this field, its contents
 203     after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are undefined. When clip lists are
 204     supported but no clipping is desired this field must be set to zero.
 205
 206 ``void * bitmap``
 207     When chroma-keying has *not* been negotiated and
 208     :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability,
 209     applications can set this field to point to a clipping bit mask.
 210
 211 It must be of the same size as the window, ``w.width`` and ``w.height``.
 212 Each bit corresponds to a pixel in the overlaid image, which is
 213 displayed only when the bit is *set*. Pixel coordinates translate to
 214 bits like:
 215
 216
 217 .. code-block:: c
 218
 219     ((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7))
 220
 221 where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [#f2]_
 222
 223 When a clipping bit mask is not supported the driver ignores this field,
 224 its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are
 225 undefined. When a bit mask is supported but no clipping is desired this
 226 field must be set to ``NULL``.
 227
 228 Applications need not create a clip list or bit mask. When they pass
 229 both, or despite negotiating chroma-keying, the results are undefined.
 230 Regardless of the chosen method, the clipping abilities of the hardware
 231 may be limited in quantity or quality. The results when these limits are
 232 exceeded are undefined. [#f3]_
 233
 234 ``__u8 global_alpha``
 235     The global alpha value used to blend the framebuffer with video
 236     images, if global alpha blending has been negotiated
 237     (``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see
 238     :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`,
 239     :ref:`framebuffer-flags`).
 240
 241 .. note::
 242
 243    This field was added in Linux 2.6.23, extending the
 244    structure. However the :ref:`VIDIOC_[G|S|TRY]_FMT <VIDIOC_G_FMT>`
 245    ioctls, which take a pointer to a :c:type:`v4l2_format`
 246    parent structure with padding bytes at the end, are not affected.
 247
 248
 249 .. c:type:: v4l2_clip
 250
 251 struct v4l2_clip [#f4]_
 252 -----------------------
 253
 254 ``struct v4l2_rect c``
 255     Coordinates of the clipping rectangle, relative to the top, left
 256     corner of the frame buffer. Only window pixels *outside* all
 257     clipping rectangles are displayed.
 258
 259 ``struct v4l2_clip * next``
 260     Pointer to the next clipping rectangle, ``NULL`` when this is the last
 261     rectangle. Drivers ignore this field, it cannot be used to pass a
 262     linked list of clipping rectangles.
 263
 264
 265 .. c:type:: v4l2_rect
 266
 267 struct v4l2_rect
 268 ----------------
 269
 270 ``__s32 left``
 271     Horizontal offset of the top, left corner of the rectangle, in
 272     pixels.
 273
 274 ``__s32 top``
 275     Vertical offset of the top, left corner of the rectangle, in pixels.
 276     Offsets increase to the right and down.
 277
 278 ``__u32 width``
 279     Width of the rectangle, in pixels.
 280
 281 ``__u32 height``
 282     Height of the rectangle, in pixels.
 283
 284
 285 Enabling Overlay
 286 ================
 287
 288 To start or stop the frame buffer overlay applications call the
 289 :ref:`VIDIOC_OVERLAY` ioctl.
 290
 291 .. [#f1]
 292    A common application of two file descriptors is the XFree86
 293    :ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application.
 294    While the X server controls video overlay, the application can take
 295    advantage of memory mapping and DMA.
 296
 297    In the opinion of the designers of this API, no driver writer taking
 298    the efforts to support simultaneous capturing and overlay will
 299    restrict this ability by requiring a single file descriptor, as in
 300    V4L and earlier versions of V4L2. Making this optional means
 301    applications depending on two file descriptors need backup routines
 302    to be compatible with all drivers, which is considerable more work
 303    than using two fds in applications which do not. Also two fd's fit
 304    the general concept of one file descriptor for each logical stream.
 305    Hence as a complexity trade-off drivers *must* support two file
 306    descriptors and *may* support single fd operation.
 307
 308 .. [#f2]
 309    Should we require ``w.width`` to be a multiple of eight?
 310
 311 .. [#f3]
 312    When the image is written into frame buffer memory it will be
 313    undesirable if the driver clips out less pixels than expected,
 314    because the application and graphics system are not aware these
 315    regions need to be refreshed. The driver should clip out more pixels
 316    or not write the image at all.
 317
 318 .. [#f4]
 319    The X Window system defines "regions" which are vectors of ``struct
 320    BoxRec { short x1, y1, x2, y2; }`` with ``width = x2 - x1`` and
 321    ``height = y2 - y1``, so one cannot pass X11 clip lists directly.