diff options
Diffstat (limited to 't/README')
| -rw-r--r-- | t/README | 326 |
1 files changed, 234 insertions, 92 deletions
@@ -1,7 +1,7 @@ -Core GIT Tests +Core Git Tests ============== -This directory holds many test scripts for core GIT tools. The +This directory holds many test scripts for core Git tools. The first part of this short document describes how to run the tests and read their output. @@ -32,7 +32,14 @@ the tests. ok 2 - plain with GIT_WORK_TREE ok 3 - plain bare -Since the tests all output TAP (see http://testanything.org) they can +t/Makefile defines a target for each test file, such that you can also use +shell pattern matching to run a subset of the tests: + + make *checkout* + +will run all tests with 'checkout' in their filename. + +Since the tests all output TAP (see https://testanything.org) they can be run with any TAP harness. Here's an example of parallel testing powered by a recent version of prove(1): @@ -258,26 +265,28 @@ For an individual test suite --run could be used to specify that only some tests should be run or that some tests should be excluded from a run. -The argument for --run is a list of individual test numbers or -ranges with an optional negation prefix that define what tests in -a test suite to include in the run. A range is two numbers -separated with a dash and matches a range of tests with both ends -been included. You may omit the first or the second number to -mean "from the first test" or "up to the very last test" -respectively. - -Optional prefix of '!' means that the test or a range of tests -should be excluded from the run. - -If --run starts with an unprefixed number or range the initial -set of tests to run is empty. If the first item starts with '!' +The argument for --run, <test-selector>, is a list of description +substrings or globs or individual test numbers or ranges with an +optional negation prefix (of '!') that define what tests in a test +suite to include (or exclude, if negated) in the run. A range is two +numbers separated with a dash and specifies an inclusive range of tests +to run. You may omit the first or the second number to +mean "from the first test" or "up to the very last test" respectively. + +The argument to --run is split on commas into separate strings, +numbers, and ranges, and picks all tests that match any of the +individual selection criteria. If the substring of the description +text that you want to match includes a comma, use the glob character +'?' instead. For example --run='rebase,merge?cherry-pick' would match +on all tests that match either the glob *rebase* or the glob +*merge?cherry-pick*. + +If --run starts with an unprefixed number or range, the initial +set of tests to run is empty. If the first item starts with '!', all the tests are added to the initial set. After initial set is -determined every test number or range is added or excluded from +determined, every test number or range is added or excluded from the set one by one, from left to right. -Individual numbers or ranges could be separated either by a space -or a comma. - For example, to run only tests up to a specific test (21), one could do this: @@ -290,7 +299,7 @@ or this: Common case is to run several setup tests (1, 2, 3) and then a specific test (21) that relies on that setup: - $ sh ./t9200-git-cvsexport-commit.sh --run='1 2 3 21' + $ sh ./t9200-git-cvsexport-commit.sh --run='1,2,3,21' or: @@ -298,17 +307,17 @@ or: or: - $ sh ./t9200-git-cvsexport-commit.sh --run='-3 21' + $ sh ./t9200-git-cvsexport-commit.sh --run='-3,21' As noted above, the test set is built by going through the items from left to right, so this: - $ sh ./t9200-git-cvsexport-commit.sh --run='1-4 !3' + $ sh ./t9200-git-cvsexport-commit.sh --run='1-4,!3' will run tests 1, 2, and 4. Items that come later have higher precedence. It means that this: - $ sh ./t9200-git-cvsexport-commit.sh --run='!3 1-4' + $ sh ./t9200-git-cvsexport-commit.sh --run='!3,1-4' would just run tests from 1 to 4, including 3. @@ -317,6 +326,18 @@ test in the test suite except from 7 up to 11: $ sh ./t9200-git-cvsexport-commit.sh --run='!7-11' +Sometimes there may be multiple tests with e.g. "setup" in their name +that are needed and rather than figuring out the number for all of them +we can just use "setup" as a substring/glob to match against the test +description: + + $ sh ./t0050-filesystem.sh --run=setup,9-11 + +or one could select both the setup tests and the rename ones (assuming all +relevant tests had those words in their descriptions): + + $ sh ./t0050-filesystem.sh --run=setup,rename + Some tests in a test suite rely on the previous tests performing certain actions, specifically some tests are designated as "setup" test, so you cannot _arbitrarily_ disable one test and @@ -344,15 +365,51 @@ whether this mode is active, and e.g. skip some tests that are hard to refactor to deal with it. The "SYMLINKS" prerequisite is currently excluded as so much relies on it, but this might change in the future. -GIT_TEST_GETTEXT_POISON=<boolean> turns all strings marked for -translation into gibberish if true. Used for spotting those tests that -need to be marked with a C_LOCALE_OUTPUT prerequisite when adding more -strings for translation. See "Testing marked strings" in po/README for -details. - GIT_TEST_SPLIT_INDEX=<boolean> forces split-index mode on the whole test suite. Accept any boolean values that are accepted by git-config. +GIT_TEST_PASSING_SANITIZE_LEAK=true skips those tests that haven't +declared themselves as leak-free by setting +"TEST_PASSES_SANITIZE_LEAK=true" before sourcing "test-lib.sh". This +test mode is used by the "linux-leaks" CI target. + +GIT_TEST_PASSING_SANITIZE_LEAK=check checks that our +"TEST_PASSES_SANITIZE_LEAK=true" markings are current. Rather than +skipping those tests that haven't set "TEST_PASSES_SANITIZE_LEAK=true" +before sourcing "test-lib.sh" this mode runs them with +"--invert-exit-code". This is used to check that there's a one-to-one +mapping between "TEST_PASSES_SANITIZE_LEAK=true" and those tests that +pass under "SANITIZE=leak". This is especially useful when testing a +series that fixes various memory leaks with "git rebase -x". + +GIT_TEST_SANITIZE_LEAK_LOG=true will log memory leaks to +"test-results/$TEST_NAME.leak/trace.*" files. The logs include a +"dedup_token" (see +"ASAN_OPTIONS=help=1 ./git") and other options to +make logs +machine-readable. + +With GIT_TEST_SANITIZE_LEAK_LOG=true we'll look at the leak logs +before exiting and exit on failure if the logs showed that we had a +memory leak, even if the test itself would have otherwise passed. This +allows us to catch e.g. missing &&-chaining. This is especially useful +when combined with "GIT_TEST_PASSING_SANITIZE_LEAK", see below. + +GIT_TEST_PASSING_SANITIZE_LEAK=check when combined with "--immediate" +will run to completion faster, and result in the same failing +tests. The only practical reason to run +GIT_TEST_PASSING_SANITIZE_LEAK=check without "--immediate" is to +combine it with "GIT_TEST_SANITIZE_LEAK_LOG=true". If we stop at the +first failing test case our leak logs won't show subsequent leaks we +might have run into. + +GIT_TEST_PASSING_SANITIZE_LEAK=(true|check) will not catch all memory +leaks unless combined with GIT_TEST_SANITIZE_LEAK_LOG=true. Some tests +run "git" (or "test-tool" etc.) without properly checking the exit +code, or git will invoke itself and fail to ferry the abort() exit +code to the original caller. When the two modes are combined we'll +look at the "test-results/$TEST_NAME.leak/trace.*" files at the end of +the test run to see if had memory leaks which the test itself didn't +catch. + GIT_TEST_PROTOCOL_VERSION=<n>, when set, makes 'protocol.version' default to n. @@ -385,13 +442,17 @@ every 'git commit-graph write', as if the `--changed-paths` option was passed in. GIT_TEST_FSMONITOR=$PWD/t7519/fsmonitor-all exercises the fsmonitor -code path for utilizing a file system monitor to speed up detecting -new or changed files. +code paths for utilizing a (hook based) file system monitor to speed up +detecting new or changed files. GIT_TEST_INDEX_VERSION=<n> exercises the index read/write code path for the index version specified. Can be set to any valid version (currently 2, 3, or 4). +GIT_TEST_PACK_USE_BITMAP_BOUNDARY_TRAVERSAL=<boolean> if enabled will +use the boundary-based bitmap traversal algorithm. See the documentation +of `pack.useBitmapBoundaryTraversal` for more details. + GIT_TEST_PACK_SPARSE=<boolean> if disabled will default the pack-objects builtin to use the non-sparse object walk. This can still be overridden by the --sparse command-line argument. @@ -399,10 +460,6 @@ the --sparse command-line argument. GIT_TEST_PRELOAD_INDEX=<boolean> exercises the preload-index code path by overriding the minimum number of cache entries required per thread. -GIT_TEST_ADD_I_USE_BUILTIN=<boolean>, when true, enables the -built-in version of git add -i. See 'add.interactive.useBuiltin' in -git-config(1). - GIT_TEST_INDEX_THREADS=<n> enables exercising the multi-threaded loading of the index for the whole test suite by bypassing the default number of cache entries and thread minimums. Setting this to 1 will make the @@ -412,6 +469,10 @@ GIT_TEST_MULTI_PACK_INDEX=<boolean>, when true, forces the multi-pack- index to be written after every 'git repack' command, and overrides the 'core.multiPackIndex' setting to true. +GIT_TEST_MULTI_PACK_INDEX_WRITE_BITMAP=<boolean>, when true, sets the +'--bitmap' option on all invocations of 'git multi-pack-index write', +and ignores pack-objects' '--write-bitmap-index'. + GIT_TEST_SIDEBAND_ALL=<boolean>, when true, overrides the 'uploadpack.allowSidebandAll' setting to true, and when false, forces fetch-pack to not request sideband-all (even if the server advertises @@ -421,6 +482,36 @@ GIT_TEST_DISALLOW_ABBREVIATED_OPTIONS=<boolean>, when true (which is the default when running tests), errors out when an abbreviated option is used. +GIT_TEST_DEFAULT_HASH=<hash-algo> specifies which hash algorithm to +use in the test scripts. Recognized values for <hash-algo> are "sha1" +and "sha256". + +GIT_TEST_DEFAULT_REF_FORMAT=<format> specifies which ref storage format +to use in the test scripts. Recognized values for <format> are "files". + +GIT_TEST_NO_WRITE_REV_INDEX=<boolean>, when true disables the +'pack.writeReverseIndex' setting. + +GIT_TEST_SPARSE_INDEX=<boolean>, when true enables index writes to use the +sparse-index format by default. + +GIT_TEST_CHECKOUT_WORKERS=<n> overrides the 'checkout.workers' setting +to <n> and 'checkout.thresholdForParallelism' to 0, forcing the +execution of the parallel-checkout code. + +GIT_TEST_FATAL_REGISTER_SUBMODULE_ODB=<boolean>, when true, makes +registering submodule ODBs as alternates a fatal action. Support for +this environment variable can be removed once the migration to +explicitly providing repositories when accessing submodule objects is +complete or needs to be abandoned for whatever reason (in which case the +migrated codepaths still retain their performance benefits). + +GIT_TEST_REQUIRE_PREREQ=<list> allows specifying a space separated list of +prereqs that are required to succeed. If a prereq in this list is triggered by +a test and then fails then the whole test run will abort. This can help to make +sure the expected tests are executed and not silently skipped when their +dependency breaks or is simply not present in a new environment. + Naming Tests ------------ @@ -496,13 +587,67 @@ This test harness library does the following things: consistently when command line arguments --verbose (or -v), --debug (or -d), and --immediate (or -i) is given. +Recommended style +----------------- + + - Keep the test_expect_* function call and test title on + the same line. + + For example, with test_expect_success, write it like: + + test_expect_success 'test title' ' + ... test body ... + ' + + Instead of: + + test_expect_success \ + 'test title' \ + '... test body ...' + + - End the line with an opening single quote. + + - Indent here-document bodies, and use "<<-" instead of "<<" + to strip leading TABs used for indentation: + + test_expect_success 'test something' ' + cat >expect <<-\EOF && + one + two + three + EOF + test_something > actual && + test_cmp expect actual + ' + + Instead of: + + test_expect_success 'test something' ' + cat >expect <<\EOF && + one + two + three + EOF + test_something > actual && + test_cmp expect actual + ' + + - Quote or escape the EOF delimiter that begins a here-document if + there is no parameter or other expansion in it, to signal readers + that they can skim it more casually: + + cmd <<-\EOF + literal here-document text without any expansion + EOF + + Do's & don'ts ------------- Here are a few examples of things you probably should and shouldn't do when writing tests. -Here are the "do's:" +The "do's:" - Put all code inside test_expect_success and other assertions. @@ -586,6 +731,26 @@ Here are the "do's:" Note that we still &&-chain the loop to propagate failures from earlier commands. + - Repeat tests with slightly different arguments in a loop. + + In some cases it may make sense to re-run the same set of tests with + different options or commands to ensure that the command behaves + despite the different parameters. This can be achieved by looping + around a specific parameter: + + for arg in '' "--foo" + do + test_expect_success "test command ${arg:-without arguments}" ' + command $arg + ' + done + + Note that while the test title uses double quotes ("), the test body + should continue to use single quotes (') to avoid breakage in case the + values contain e.g. quoting characters. The loop variable will be + accessible regardless of the single quotes as the test body is passed + to `eval`. + And here are the "don'ts:" @@ -726,7 +891,8 @@ Test harness library -------------------- There are a handful helper functions defined in the test harness -library for your script to use. +library for your script to use. Some of them are listed below; +see test-lib-functions.sh for the full list and their options. - test_expect_success [<prereq>] <message> <script> @@ -751,7 +917,7 @@ library for your script to use. rare case where your test depends on more than one: test_expect_success PERL,PYTHON 'yo dawg' \ - ' test $(perl -E 'print eval "1 +" . qx[python -c "print 2"]') == "4" ' + ' test $(perl -E '\''print eval "1 +" . qx[python -c "print(2)"]'\'') = "4" ' - test_expect_failure [<prereq>] <message> <script> @@ -772,10 +938,12 @@ library for your script to use. argument. This is primarily meant for use during the development of a new test script. - - debug <git-command> + - debug [options] <git-command> Run a git command inside a debugger. This is primarily meant for - use when debugging a failing test script. + use when debugging a failing test script. With '-t', use your + original TERM instead of test-lib.sh's "dumb", so that your + debugger interface has colors. - test_done @@ -826,32 +994,6 @@ library for your script to use. test_done fi - - test_external [<prereq>] <message> <external> <script> - - Execute a <script> with an <external> interpreter (like perl). This - was added for tests like t9700-perl-git.sh which do most of their - work in an external test script. - - test_external \ - 'GitwebCache::*FileCache*' \ - perl "$TEST_DIRECTORY"/t9503/test_cache_interface.pl - - If the test is outputting its own TAP you should set the - test_external_has_tap variable somewhere before calling the first - test_external* function. See t9700-perl-git.sh for an example. - - # The external test will outputs its own plan - test_external_has_tap=1 - - - test_external_without_stderr [<prereq>] <message> <external> <script> - - Like test_external but fail if there's any output on stderr, - instead of checking the exit code. - - test_external_without_stderr \ - 'Perl API' \ - perl "$TEST_DIRECTORY"/t9700/test.pl - - test_expect_code <exit-code> <command> Run a command and ensure that it exits with the given exit code. @@ -899,13 +1041,13 @@ library for your script to use. Check whether a file has the length it is expected to. - - test_path_is_file <path> [<diagnosis>] - test_path_is_dir <path> [<diagnosis>] - test_path_is_missing <path> [<diagnosis>] + - test_path_is_file <path> + test_path_is_dir <path> + test_path_is_missing <path> Check if the named path is a file, if the named path is a directory, or if the named path does not exist, respectively, - and fail otherwise, showing the <diagnosis> text. + and fail otherwise. - test_when_finished <script> @@ -962,7 +1104,7 @@ library for your script to use. EOF - - test_pause + - test_pause [options] This command is useful for writing and debugging tests and must be removed before submitting. It halts the execution of the test and @@ -989,6 +1131,12 @@ library for your script to use. the symbolic link in the file system and a part that does; then only the latter part need be protected by a SYMLINKS prerequisite (see below). + - test_path_is_executable + + This tests whether a file is executable and prints an error message + if not. This must be used only under the POSIXPERM prerequisite + (see below). + - test_oid_init This function loads facts and useful object IDs related to the hash @@ -1086,18 +1234,6 @@ use these, and "test_set_prereq" for how to define your own. Git was compiled with support for PCRE. Wrap any tests that use git-grep --perl-regexp or git-grep -P in these. - - LIBPCRE1 - - Git was compiled with PCRE v1 support via - USE_LIBPCRE1=YesPlease. Wrap any PCRE using tests that for some - reason need v1 of the PCRE library instead of v2 in these. - - - LIBPCRE2 - - Git was compiled with PCRE v2 support via - USE_LIBPCRE2=YesPlease. Wrap any PCRE using tests that for some - reason need v2 of the PCRE library instead of v1 in these. - - CASE_INSENSITIVE_FS Test is run on a case insensitive file system. @@ -1111,31 +1247,37 @@ use these, and "test_set_prereq" for how to define your own. Git wasn't compiled with NO_PTHREADS=YesPlease. + - REFFILES + + Test is specific to packed/loose ref storage, and should be + disabled for other ref storage backends + + Tips for Writing Tests ---------------------- As with any programming projects, existing programs are the best source of the information. However, do _not_ emulate t0000-basic.sh when writing your tests. The test is special in -that it tries to validate the very core of GIT. For example, it +that it tries to validate the very core of Git. For example, it knows that there will be 256 subdirectories under .git/objects/, and it knows that the object ID of an empty tree is a certain 40-byte string. This is deliberately done so in t0000-basic.sh because the things the very basic core test tries to achieve is -to serve as a basis for people who are changing the GIT internal +to serve as a basis for people who are changing the Git internals drastically. For these people, after making certain changes, -not seeing failures from the basic test _is_ a failure. And -such drastic changes to the core GIT that even changes these +not seeing failures from the basic test _is_ a failure. Any +Git core changes so drastic that they change even these otherwise supposedly stable object IDs should be accompanied by an update to t0000-basic.sh. However, other tests that simply rely on basic parts of the core -GIT working properly should not have that level of intimate -knowledge of the core GIT internals. If all the test scripts +Git working properly should not have that level of intimate +knowledge of the core Git internals. If all the test scripts hardcoded the object IDs like t0000-basic.sh does, that defeats the purpose of t0000-basic.sh, which is to isolate that level of validation in one place. Your test also ends up needing -updating when such a change to the internal happens, so do _not_ +an update whenever the internals change, so do _not_ do it and leave the low level of validation to t0000-basic.sh. Test coverage @@ -1166,7 +1308,7 @@ Devel::Cover module. To install it do: sudo aptitude install libdevel-cover-perl # From the CPAN with cpanminus - curl -L http://cpanmin.us | perl - --sudo --self-upgrade + curl -L https://cpanmin.us/ | perl - --sudo --self-upgrade cpanm --sudo Devel::Cover Then, at the top-level: |