--- /dev/null
+This is a collection of tests for GDB.
+
+The file gdb/README contains basic instructions on how to run the
+testsuite, while this file documents additional options and controls
+that are available. The GDB wiki may also have some pages with ideas
+and suggestions.
+
+
+Running the Testsuite
+*********************
+
+There are two ways to run the testsuite and pass additional parameters
+to DejaGnu. The first is to do `make check' in the main build
+directory and specifying the makefile variable `RUNTESTFLAGS':
+
+ make check RUNTESTFLAGS='TRANSCRIPT=y gdb.base/a2-run.exp'
+
+The second is to cd to the testsuite directory and invoke the DejaGnu
+`runtest' command directly.
+
+ cd testsuite
+ make site.exp
+ runtest TRANSCRIPT=y
+
+(The `site.exp' file contains a handful of useful variables like host
+and target triplets, and pathnames.)
+
+Testsuite Parameters
+********************
+
+The following parameters are DejaGNU variables that you can set to
+affect the testsuite run globally.
+
+TRANSCRIPT
+
+You may find it useful to have a transcript of the commands that the
+testsuite sends to GDB, for instance if GDB crashes during the run,
+and you want to reconstruct the sequence of commands.
+
+If the DejaGNU variable TRANSCRIPT is set (to any value), each
+invocation of GDB during the test run will get a transcript file
+written into the DejaGNU output directory. The file will have the
+name transcript.<n>, where <n> is an integer. The first line of the
+file shows the invocation command with all the options passed to it,
+while subsequent lines are the GDB commands. A `make check' might
+look like this:
+
+ make check RUNTESTFLAGS=TRANSCRIPT=y
+
+The transcript may not be complete, as for instance tests of command
+completion may show only partial command lines.
+
+GDB
+
+By default, the testsuite exercises the GDB in the build directory,
+but you can set GDB to be a pathname to a different version. For
+instance,
+
+ make check RUNTESTFLAGS=GDB=/usr/bin/gdb
+
+runs the testsuite on the GDB in /usr/bin.
+
+GDBSERVER
+
+You can set GDBSERVER to be a particular GDBserver of interest, so for
+instance
+
+ make check RUNTESTFLAGS="GDB=/usr/bin/gdb GDBSERVER=/usr/bin/gdbserver"
+
+checks both the installed GDB and GDBserver.
+
+INTERNAL_GDBFLAGS
+
+Command line options passed to all GDB invocations.
+
+The default is "-nw -nx".
+
+`-nw' disables any of the windowed interfaces.
+`-nx' disables ~/.gdbinit, so that it doesn't interfere with
+the tests.
+
+This is actually considered an internal variable, and you
+won't normally want to change it. However, in some situations,
+this may be tweaked as a last resort if the testsuite doesn't
+have direct support for the specifics of your environment.
+The testsuite does not override a value provided by the user.
+
+As an example, when testing an installed GDB that has been
+configured with `--with-system-gdbinit', like by default,
+you do not want ~/.gdbinit to interfere with tests, but, you
+may want the system .gdbinit file loaded. As there's no way to
+ask the testsuite, or GDB, to load the system gdbinit but
+not ~/.gdbinit, a workaround is then to remove `-nx' from
+INTERNAL_GDBFLAGS, and point $HOME at a directory without
+a .gdbinit. For example:
+
+ cd testsuite
+ HOME=`pwd` runtest \
+ GDB=/usr/bin/gdb \
+ GDBSERVER=/usr/bin/gdbserver \
+ INTERNAL_GDBFLAGS=-nw
+
+GDB_PARALLEL
+
+When testing natively (that is, not with a remote host), you can run
+the GDB test suite in a fully parallel mode. In this mode, each .exp
+file runs separately and maybe simultaneously. The test suite will
+ensure that all the temporary files created by the test suite do not
+clash, by putting them into separate directories. This mode is
+primarily intended for use by the Makefile.
+
+To use this mode, set the GDB_PARALLEL on the runtest command line.
+Before starting the tests, you must ensure that the directories cache,
+outputs, and temp in the test suite build directory are either empty
+or have been deleted. cache in particular is used to share data
+across invocations of runtest, and files there may affect the test
+results. Note that the Makefile automatically does these deletions.
+
+GDB_INOTIFY
+
+For debugging parallel mode, it is handy to be able to see when a test
+case writes to a file outside of its designated output directory.
+
+If you have the inotify-tools package installed, you can set the
+GDB_INOTIFY variable on the runtest command line. This will cause the
+test suite to watch for parallel-unsafe file creations and report
+them, both to stdout and in the test suite log file.
+
+This setting is only meaningful in conjunction with GDB_PARALLEL.
+
+
+Testsuite Configuration
+***********************
+
+It is possible to adjust the behavior of the testsuite by defining
+the global variables listed below, either in a `site.exp' file,
+or in a board file.
+
+gdb_test_timeout
+
+Defining this variable changes the default timeout duration used
+during communication with GDB. More specifically, the global variable
+used during testing is `timeout', but this variable gets reset to
+`gdb_test_timeout' at the beginning of each testcase, which ensures
+that any local change to `timeout' in a testcase does not affect
+subsequent testcases.
+
+This global variable comes in handy when the debugger is slower than
+normal due to the testing environment, triggering unexpected `TIMEOUT'
+test failures. Examples include when testing on a remote machine, or
+against a system where communications are slow.
+
+If not specifically defined, this variable gets automatically defined
+to the same value as `timeout' during the testsuite initialization.
+The default value of the timeout is defined in the file
+`testsuite/config/unix.exp' (at least for Unix hosts; board files may
+have their own values).
+
+
+Board Settings
+**************
+
+DejaGNU includes the concept of a "board file", which specifies
+testing details for a particular target (which are often bare circuit
+boards, thus the name).
+
+In the GDB testsuite specifically, the board file may include a
+number of "board settings" that test cases may check before deciding
+whether to exercise a particular feature. For instance, a board
+lacking any I/O devices, or perhaps simply having its I/O devices
+not wired up, should set `noinferiorio'.
+
+Here are the supported board settings:
+
+gdb,cannot_call_functions
+
+ The board does not support inferior call, that is, invoking inferior
+ functions in GDB.
+
+gdb,can_reverse
+
+ The board supports reverse execution.
+
+gdb,no_hardware_watchpoints
+
+ The board does not support hardware watchpoints.
+
+gdb,nofileio
+
+ GDB is unable to intercept target file operations in remote and
+ perform them on the host.
+
+gdb,noinferiorio
+
+ The board is unable to provide I/O capability to the inferior.
+
+gdb,noresults
+
+ A program will not return an exit code or result code (or the value
+ of the result is undefined, and should not be looked at).
+
+gdb,nosignals
+
+ The board does not support signals.
+
+gdb,skip_huge_test
+
+ Skip time-consuming tests on the board with slow connection.
+
+gdb,skip_float_tests
+
+ Skip tests related to floating point.
+
+gdb,use_precord
+
+ The board supports process record.
+
+gdb_server_prog
+
+ The location of GDBserver. If GDBserver somewhere other than its
+ default location is used in test, specify the location of GDBserver in
+ this variable. The location is a file name for GDBserver, and may be
+ either absolute or relative to the testsuite subdirectory of the build
+ directory.
+
+in_proc_agent
+
+ The location of the in-process agent (used for fast tracepoints and
+ other special tests). If the in-process agent of interest is anywhere
+ other than its default location, set this variable. The location is a
+ filename, and may be either absolute or relative to the testsuite
+ subdirectory of the build directory.
+
+noargs
+
+ GDB does not support argument passing for inferior.
+
+no_long_long
+
+ The board does not support type long long.
+
+use_cygmon
+
+ The board is running the monitor Cygmon.
+
+use_gdb_stub
+
+ The tests are running with a GDB stub.
+
+gdb,predefined_tsv
+
+ The predefined trace state variables the board has.
+
+
+Testsuite Organization
+**********************
+
+The testsuite is entirely contained in `gdb/testsuite'. The main
+directory of the testsuite includes some makefiles and configury, but
+these are minimal, and used for little besides cleaning up, since the
+tests themselves handle the compilation of the programs that GDB will
+run.
+
+The file `testsuite/lib/gdb.exp' contains common utility procs useful
+for all GDB tests, while the directory testsuite/config contains
+configuration-specific files, typically used for special-purpose
+definitions of procs like `gdb_load' and `gdb_start'.
+
+The tests themselves are to be found in directories named
+'testsuite/gdb.* and subdirectories of those. The names of the test
+files must always end with ".exp". DejaGNU collects the test files by
+wildcarding in the test directories, so both subdirectories and
+individual files typically get chosen and run in alphabetical order.
+
+The following lists some notable types of subdirectories and what they
+are for. Since DejaGNU finds test files no matter where they are
+located, and since each test file sets up its own compilation and
+execution environment, this organization is simply for convenience and
+intelligibility.
+
+gdb.base
+
+This is the base testsuite. The tests in it should apply to all
+configurations of GDB (but generic native-only tests may live here).
+The test programs should be in the subset of C that is both valid
+ANSI/ISO C, and C++.
+
+gdb.<lang>
+
+Language-specific tests for any language besides C. Examples are
+gdb.cp for C++ and gdb.java for Java.
+
+gdb.<platform>
+
+Non-portable tests. The tests are specific to a specific
+configuration (host or target), such as HP-UX or eCos. Example is
+gdb.hp, for HP-UX.
+
+gdb.arch
+
+Architecture-specific tests that are (usually) cross-platform.
+
+gdb.<subsystem>
+
+Tests that exercise a specific GDB subsystem in more depth. For
+instance, gdb.disasm exercises various disassemblers, while
+gdb.stabs tests pathways through the stabs symbol reader.
+
+Writing Tests
+*************
+
+In many areas, the GDB tests are already quite comprehensive; you
+should be able to copy existing tests to handle new cases. Be aware
+that older tests may use obsolete practices but have not yet been
+updated.
+
+You should try to use `gdb_test' whenever possible, since it includes
+cases to handle all the unexpected errors that might happen. However,
+it doesn't cost anything to add new test procedures; for instance,
+gdb.base/exprs.exp defines a `test_expr' that calls `gdb_test'
+multiple times.
+
+Only use `send_gdb' and `gdb_expect' when absolutely necessary. Even
+if GDB has several valid responses to a command, you can use
+`gdb_test_multiple'. Like `gdb_test', `gdb_test_multiple' recognizes
+internal errors and unexpected prompts.
+
+Do not write tests which expect a literal tab character from GDB. On
+some operating systems (e.g. OpenBSD) the TTY layer expands tabs to
+spaces, so by the time GDB's output reaches `expect' the tab is gone.
+
+The source language programs do *not* need to be in a consistent
+style. Since GDB is used to debug programs written in many different
+styles, it's worth having a mix of styles in the testsuite; for
+instance, some GDB bugs involving the display of source lines might
+never manifest themselves if the test programs used GNU coding style
+uniformly.
+
+Some testcase results need more detailed explanation:
+
+KFAIL
+
+Use KFAIL for known problem of GDB itself. You must specify the GDB
+bug report number, as in these sample tests:
+
+ kfail "gdb/13392" "continue to marker 2"
+
+or
+
+ setup_kfail gdb/13392 "*-*-*"
+ kfail "continue to marker 2"
+
+
+XFAIL
+
+Short for "expected failure", this indicates a known problem with the
+environment. This could include limitations of the operating system,
+compiler version, and other components.
+
+This example from gdb.base/attach-pie-misread.exp is a sanity check
+for the target environment:
+
+ # On x86_64 it is commonly about 4MB.
+ if {$stub_size > 25000000} {
+ xfail "stub size $stub_size is too large"
+ return
+ }
+
+You should provide bug report number for the failing component of the
+environment, if such bug report is available, as with this example
+referring to a GCC problem:
+
+ if {[test_compiler_info {gcc-[0-3]-*}]
+ || [test_compiler_info {gcc-4-[0-5]-*}]} {
+ setup_xfail "gcc/46955" *-*-*
+ }
+ gdb_test "python print ttype.template_argument(2)" "&C::c"
+
+Note that it is also acceptable, and often preferable, to avoid
+running the test at all. This is the better option if the limitation
+is intrinsic to the environment, rather than a bug expected to be
+fixed in the near future.
projects / deliverable / binutils-gdb.git / commitdiff
