(The `site.exp' file contains a handful of useful variables like host
and target triplets, and pathnames.)
+Parallel testing
+****************
+
+If not testing with a remote host (in DejaGnu's sense), you can run
+the GDB test suite in a fully parallel mode. In this mode, each .exp
+file runs separately and maybe simultaneously. The test suite ensures
+that all the temporary files created by the test suite do not clash,
+by putting them into separate directories. This mode is primarily
+intended for use by the Makefile.
+
+For GNU make, the Makefile tries to run the tests in parallel mode if
+any -j option is given. For a non-GNU make, tests are not
+parallelized.
+
+If RUNTESTFLAGS is not empty, then by default the tests are
+serialized. This can be overridden by either using the
+`check-parallel' target in the Makefile, or by setting FORCE_PARALLEL
+to any non-empty value:
+
+ make check-parallel RUNTESTFLAGS="--target_board=native-gdbserver"
+ make check RUNTESTFLAGS="--target_board=native-gdbserver" FORCE_PARALLEL=1
+
+If you want to use runtest directly instead of using the Makefile, see
+the description of GDB_PARALLEL below.
+
+Racy testcases
+**************
+
+Sometimes, new testcases are added to the testsuite that are not
+entirely deterministic, and can randomly pass or fail. We call them
+"racy testcases", and they can be bothersome when one is comparing
+different testsuite runs. In order to help identifying them, it is
+possible to run the tests several times in a row and ask the testsuite
+machinery to analyze the results. To do that, you need to specify the
+RACY_ITER environment variable to make:
+
+ make check RACY_ITER=5 -j4
+
+The value assigned to RACY_ITER represents the number of times you
+wish to run the tests in sequence (in the example above, the entire
+testsuite will be executed 5 times in a row, in parallel). It is also
+possible to check just a specific test:
+
+ make check TESTS='gdb.base/default.exp' RACY_ITER=3
+
+One can also decide to call the Makefile rules by hand inside the
+gdb/testsuite directory, e.g.:
+
+ make check-paralell-racy -j4
+
+In which case the value of the DEFAULT_RACY_ITER variable (inside
+gdb/testsuite/Makefile.in) will be used to determine how many
+iterations will be run.
+
+After running the tests, you shall see a file name 'racy.sum' in the
+gdb/testsuite directory. You can also inspect the generated *.log and
+*.sum files by looking into the gdb/testsuite/racy_ouputs directory.
+
+If you already have *.sum files generated from previous testsuite runs
+and you would like to analyze them without having to run the testsuite
+again, you can also use the 'analyze-racy-logs.py' script directly.
+It is located in the gdb/testsuite/ directory, and it expects a list
+of two or more *.sum files to be provided as its argument. For
+example:
+
+ ./gdb/testsuite/analyze-racy-logs.py testsuite-01/gdb.sum \
+ testsuite-02/gdb.sum testsuite-03/gdb.sum
+
+The script will output its analysis report to the standard output.
+
+Running the Performance Tests
+*****************************
+
+GDB Testsuite includes performance test cases, which are not run together
+with other test cases, because performance test cases are slow and need
+a quiet system. There are two ways to run the performance test cases.
+The first is to do `make check-perf' in the main build directory:
+
+ make check-perf RUNTESTFLAGS="solib.exp SOLIB_COUNT=8"
+
+The second is to cd to the testsuite directory and invoke the DejaGnu
+`runtest' command directly.
+
+ cd testsuite
+ make site.exp
+ runtest GDB_PERFTEST_MODE=both GDB_PERFTEST_TIMEOUT=4000 --directory=gdb.perf solib.exp SOLIB_COUNT=8
+
+Only "compile", "run" and "both" are valid to GDB_PERFTEST_MODE. They
+stand for "compile tests only", "run tests only", and "compile and run
+tests" respectively. "both" is the default. GDB_PERFTEST_TIMEOUT
+specify the timeout, which is 3000 in default. The result of
+performance test is appended in `testsuite/perftest.log'.
+
Testsuite Parameters
********************
GDB_PARALLEL
-When testing natively (that is, not with a remote host), you can run
-the GDB test suite in a fully parallel mode. In this mode, each .exp
-file runs separately and maybe simultaneously. The test suite will
-ensure that all the temporary files created by the test suite do not
-clash, by putting them into separate directories. This mode is
-primarily intended for use by the Makefile.
-
-To use this mode, set the GDB_PARALLEL on the runtest command line.
-Before starting the tests, you must ensure that the directories cache,
-outputs, and temp in the test suite build directory are either empty
-or have been deleted. cache in particular is used to share data
-across invocations of runtest, and files there may affect the test
-results. Note that the Makefile automatically does these deletions.
+To use parallel testing mode without using the the Makefile, set
+GDB_PARALLEL on the runtest command line to "yes". Before starting
+the tests, you must ensure that the directories cache, outputs, and
+temp in the test suite build directory are either empty or have been
+deleted. cache in particular is used to share data across invocations
+of runtest, and files there may affect the test results. The Makefile
+automatically does these deletions.
+
+FORCE_PARALLEL
+
+Setting FORCE_PARALLEL to any non-empty value forces parallel testing
+mode even if RUNTESTFLAGS is not empty.
+
+FORCE_SEPARATE_MI_TTY
+
+Setting FORCE_MI_SEPARATE_UI to 1 forces all MI testing to start GDB
+in console mode, with MI running on a separate TTY, on a secondary UI
+started with "new-ui".
GDB_INOTIFY
This setting is only meaningful in conjunction with GDB_PARALLEL.
+TESTS
+
+This variable is used to specify which set of tests to run.
+It is passed to make (not runtest) and its contents are a space separated
+list of tests to run.
+
+If using GNU make then the contents are wildcard-expanded using
+GNU make's $(wildcard) function. Test paths must be fully specified,
+relative to the "testsuite" subdirectory. This allows one to run all
+tests in a subdirectory by passing "gdb.subdir/*.exp", or more simply
+by using the check-gdb.subdir target in the Makefile.
+
+If for some strange reason one wanted to run all tests that begin with
+the letter "d" that is also possible: TESTS="*/d*.exp".
+
+Do not write */*.exp to specify all tests (assuming all tests are only
+nested one level deep, which is not necessarily true). This will pick up
+.exp files in ancillary directories like "lib" and "config".
+Instead write gdb.*/*.exp.
+
+Example:
+
+ make -j10 check TESTS="gdb.server/[s-w]*.exp */x*.exp"
+
+If not using GNU make then the value is passed directly to runtest.
+If not specified, all tests are run.
+
+READ1
+
+This make (not runtest) variable is used to specify whether the
+testsuite preloads the read1.so library into expect. Any non-empty
+value means true. See "Race detection" below.
+
+Race detection
+**************
+
+The testsuite includes a mechanism that helps detect test races.
+
+For example, say the program running under expect outputs "abcd", and
+a test does something like this:
+
+ expect {
+ "a.*c" {
+ }
+ "b" {
+ }
+ "a" {
+ }
+ }
+
+Which case happens to match depends on what expect manages to read
+into its internal buffer in one go. If it manages to read three bytes
+or more, then the first case matches. If it manages to read two
+bytes, then the second case matches. If it manages to read only one
+byte, then the third case matches.
+
+To help detect these cases, the race detection mechanism preloads a
+library into expect that forces the `read' system call to always
+return at most 1 byte.
+
+To enable this, either pass a non-empty value in the READ1 make
+variable, or use the check-read1 make target instead of check.
+
+Examples:
+
+ make -j10 check-read1 TESTS="*/paginate-*.exp"
+ make -j10 check READ1="1"
Testsuite Configuration
***********************
`testsuite/config/unix.exp' (at least for Unix hosts; board files may
have their own values).
+gdb_reverse_timeout
+
+Defining this variable changes the default timeout duration when tests
+under gdb.reverse directory are running. Process record and reverse
+debugging is so slow that its tests have unexpected `TIMEOUT' test
+failures. This global variable is useful to bump up the value of
+`timeout' for gdb.reverse tests and doesn't cause any delay where
+actual failures happen in the rest of the testsuite.
+
Board Settings
**************
The board supports process record.
+gdb_init_command
+gdb_init_commands
+
+ Commands to send to GDB every time a program is about to be run. The
+ first of these settings defines a single command as a string. The
+ second defines a TCL list of commands being a string each. The commands
+ are sent one by one in a sequence, first from `gdb_init_command', if any,
+ followed by individual commands from `gdb_init_command', if any, in this
+ list's order.
+
gdb_server_prog
The location of GDBserver. If GDBserver somewhere other than its
The tests are running with a GDB stub.
+exit_is_reliable
+
+ Set to true if GDB can assume that letting the program run to end
+ reliably results in program exits being reported as such, as opposed
+ to, e.g., the program ending in an infinite loop or the board
+ crashing/resetting. If not set, this defaults to $use_gdb_stub. In
+ other words, native targets are assumed reliable by default, and
+ remote stubs assumed unreliable.
+
gdb,predefined_tsv
The predefined trace state variables the board has.
+gdb,no_thread_names
+
+ The target doesn't support thread names.
Testsuite Organization
**********************
gdb.<lang>
Language-specific tests for any language besides C. Examples are
-gdb.cp for C++ and gdb.java for Java.
+gdb.cp for C++ and gdb.rust for Rust.
gdb.<platform>
Non-portable tests. The tests are specific to a specific
-configuration (host or target), such as HP-UX or eCos. Example is
-gdb.hp, for HP-UX.
+configuration (host or target), such as eCos.
gdb.arch
instance, gdb.disasm exercises various disassemblers, while
gdb.stabs tests pathways through the stabs symbol reader.
+gdb.perf
+
+GDB performance tests.
+
Writing Tests
*************
projects / deliverable / binutils-gdb.git / blobdiff