path: root/t/Makefile
AgeCommit message (Collapse)Author
2018-07-17t/Makefile: add machinery to check correctness of chainlint.sedEric Sunshine
The --chain-lint option uses heuristics and knowledge of shell syntax to detect broken &&-chains in subshells by pure textual inspection. Although the heuristics work well, they are still best-guesses and future changes could accidentally break assumptions upon which they are based. To protect against this possibility, tests checking correctness of the linter itself will be added. As preparation, add a new makefile "check-chainlint" target and associated machinery. Signed-off-by: Eric Sunshine <> Signed-off-by: Junio C Hamano <>
2017-12-08t/Makefile: introduce TEST_SHELL_PATHJeff King
You may want to run the test suite with a different shell than you use to build Git. For instance, you may build with SHELL_PATH=/bin/sh (because it's faster, or it's what you expect to exist on systems where the build will be used) but want to run the test suite with bash (e.g., since that allows using "-x" reliably across the whole test suite). There's currently no good way to do this. You might think that doing two separate make invocations, like: make && make -C t SHELL_PATH=/bin/bash would work. And it _almost_ does. The second make will see our bash SHELL_PATH, and we'll use that to run the individual test scripts (or tell prove to use it to do so). So far so good. But this breaks down when "--tee" or "--verbose-log" is used. Those options cause the test script to actually re-exec itself using $SHELL_PATH. But wait, wouldn't our second make invocation have set SHELL_PATH correctly in the environment? Yes, but sources GIT-BUILD-OPTIONS, which we built during the first "make". And that overrides the environment, giving us the original SHELL_PATH again. Let's introduce a new variable that lets you specify a specific shell to be run for the test scripts. Note that we have to touch both the main and t/ Makefiles, since we have to record it in GIT-BUILD-OPTIONS in one, and use it in the latter. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2017-01-27t/Makefile: add a rule to re-run previously-failed testsJohannes Schindelin
This patch automates the process of determining which tests failed previously and re-running them. While developing patch series, it is a good practice to run the test suite from time to time, just to make sure that obvious bugs are caught early. With complex patch series, it is common to run `make -j15 -k test`, i.e. run the tests in parallel and *not* stop at the first failing test but continue. This has the advantage of identifying possibly multiple problems in one big test run. It is particularly important to reduce the turn-around time thusly on Windows, where the test suite spends 45 minutes on the computer on which this patch was developed. It is the most convenient way to determine which tests failed after running the entire test suite, in parallel, to look for left-over "trash directory.t*" subdirectories in the t/ subdirectory. However, those directories might live outside t/ when overridden using the --root=<directory> option, to which the Makefile has no access. The next best method is to grep explicitly for failed tests in the test-results/ directory, which the Makefile *can* access. Please note that the often-recommended `prove` tool requires Perl, and that opens a whole new can of worms on Windows. As no native Windows Perl comes with Subversion bindings, we have to use a Perl in Git for Windows that uses the POSIX emulation layer named MSYS2 (which is a portable version of Cygwin). When using this emulation layer under stress, e.g. when running massively-parallel tests, unexplicable crashes occur quite frequently, and instead of having a solution to the original problem, the developer now has an additional, quite huge problem. For that reason, this developer rejected `prove` as a solution and went with this patch instead. Signed-off-by: Johannes Schindelin <> Reviewed-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2016-08-16t/Makefile: ensure that paths are valid on platforms we careJohannes Schindelin
Some pathnames that are okay on ext4 and on HFS+ cannot be checked out on Windows. Tests that want to see operations on such paths on filesystems that support them must do so behind appropriate test prerequisites, and must not include them in the source tree (instead they should create them when they run). Otherwise, the source tree cannot even be checked out. Make sure that double-quotes, asterisk, colon, greater/less-than, question-mark, backslash, tab, vertical-bar, as well as any non-ASCII characters never appear in the pathnames with a new test-lint-* target as part of a `make test`. To that end, we call `git ls-files` (ensuring that the paths are quoted properly), relying on the fact that paths containing non-ASCII characters are quoted within double-quotes. In case that the source code does not actually live in a Git repository (e.g. when extracted from a .zip file), or that the `git` executable cannot be executed, we simply ignore the error for now; In that case, our trusty Continuous Integration will be the last line of defense and catch any problematic file name. Noticed when a topic wanted to add a pathname with '>' in it. A check like this will prevent a similar problems from happening in the future. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2016-02-23tests: remove no-op full-svn-test targetEric Wong
git-svn has not supported GIT_SVN_NO_OPTIMIZE_COMMITS for the "set-tree" sub-command in 9 years since commit 490f49ea5899 ("git-svn: remove optimized commit stuff for set-tree"). So remove this target and TSVN variable to avoid confusion. ref: Helped-by: Kazutoshi Satoda <> Signed-off-by: Eric Wong <> Signed-off-by: Junio C Hamano <>
2014-07-10t/Makefile: always test all lint targets when running testsJens Lehmann
Only the two targets "test-lint-duplicates" and "test-lint-executable" are currently executed when running the test target. This was done on purpose when the TEST_LINT variable was added in 81127d74 to avoid twisted shell scripting by developers only to avoid false positives that might result from the rather simple minded tests, e.g. test-lint-shell-syntax. But it looks like it might be better to include all lint tests to help developers to detect non portable shell constructs before the patch is sent to the list and reviewed there. Change the TEST_LINT variable to run all lint test unless the TEST_LINT variable is overridden. If we hit false positives more often than helping developers to avoid non-portable code (or add less accurate or slow tests later) we could still fall back to exclude them like 81127d74 proposed. Signed-off-by: Jens Lehmann <> Signed-off-by: Junio C Hamano <>
2014-07-10t/Makefile: check helper scripts for non-portable shell commands tooJens Lehmann
Currently only the "t[0-9][0-9][0-9][0-9]-*.sh" scripts are tested for shell incompatibilities using the script. This makes it easy to miss non-POSIX constructs added to one of the t/*lib*.sh helper scripts, as they aren't automatically detected. Fix that by adding a THELPERS variable containing all shell scripts that aren't tests and add these to the "test-lint-shell-syntax" target too. Signed-off-by: Jens Lehmann <> Signed-off-by: Junio C Hamano <>
2014-03-21t/Makefile: stop setting GIT_CONFIGJeff King
Once upon a time, the setting of GIT_CONFIG in the environment could affect how tests ran. Commit 9c3796f (Fix setting config variables with an alternative GIT_CONFIG, 2006-06-20) unconditionally set GIT_CONFIG in the Makefile when running tests to give us a known starting point. This is insufficient for running the tests outside of the Makefile, however, and 8565d2d (Make tests independent of global config files, 2007-02-15) later set GIT_CONFIG directly in At that point the Makefile setting was redundant, but we never removed it. Let's do so now. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2013-05-29Merge branch 'jk/test-output'Junio C Hamano
When TEST_OUTPUT_DIRECTORY setting is used, it was handled somewhat inconsistently between the test framework and t/Makefile, and logic to summarize the results looked at a wrong place. * jk/test-output: t/Makefile: don't define TEST_RESULTS_DIRECTORY recursively test output: respect $TEST_OUTPUT_DIRECTORY t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY
2013-05-06t/Makefile: don't define TEST_RESULTS_DIRECTORY recursivelyJohn Keeping
Commit 54bb901 (t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY - 2013-04-26) incorrectly defined TEST_RESULTS_DIRECTORY relative to itself, when it should be relative to TEST_OUTPUT_DIRECTORY. Fix this. Signed-off-by: John Keeping <> Signed-off-by: Junio C Hamano <>
2013-04-28t/Makefile: remove smoke test targetsJohn Keeping
Commit d24fbca (Remove Git's support for smoke testing - 2011-12-23) removed the smoke test support from the test suite but it was re-added by commit 342e9ef (Introduce a performance testing framework - 2012-02-17). This appears to be the result of a mis-rebase, since re-adding the smoke testing infrastructure does not relate to the subject of that commit. The current 'smoke' target is broken since the 'harness' script it uses no longer exists, so just reapply this section of commit d24fbca and remove all of the smoke testing section in the makefile. Signed-off-by: John Keeping <> Acked-by: Thomas Rast <> Signed-off-by: Junio C Hamano <>
2013-04-27t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORYJohn Keeping
When TEST_OUTPUT_DIRECTORY is set, the test results will be generated in "$TEST_OUTPUT_DIRECTORY/test-results", which may not be the same as "test-results" in t/Makefile. This causes the aggregate-results target to fail as it finds no count files. Fix this by introducing TEST_RESULTS_DIRECTORY and using it wherever test-results is referenced. Signed-off-by: John Keeping <> Signed-off-by: Junio C Hamano <>
2013-01-10Merge branch 'tb/test-shell-lint'Junio C Hamano
Check for common mistakes in the test scripts, based on simple pattern-matching. * tb/test-shell-lint: test: Add
2013-01-03tests: turn on test-lint by defaultJeff King
The test Makefile knows about a few "lint" checks for common errors. However, they are not enabled as part of "make test" by default, which means that many people do not bother running them. Since they are both quick to run and accurate (i.e., no false positives), there should be no harm in turning them on and helping submitters catch errors earlier. We could just set: TEST_LINT = test-lint to enable all tests. But that would be unnecessarily annoying later on if we add slower or less accurate tests that should not be part of the default. Instead, we name the tests individually. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2013-01-03test: Add check-non-portable-shell.plTorsten Bögershausen
Add the perl script "" to detect non-portable shell syntax. "echo -n" is an example of a shell command working on Linux, but not on Mac OS X. These shell commands are checked and reported as error: - "echo -n" (printf should be used) - "sed -i" (GNUism; use a temp file instead) - "declare" (bashism, often used with arrays) - "which" (unreliable exit status and output; use type instead) - "test a == b" (bashism for "test a = b") "make test-lint-shell-syntax" can be used to run only the check. Helped-By: Jeff King <> Signed-off-by: Torsten Bögershausen <> Signed-off-by: Junio C Hamano <>
2012-12-10Makefile: whitespace style fixes in macro definitionsStefano Lattarini
Consistently use a single space before and after the "=" (or ":=", "+=", etc.) in assignments to make macros. Granted, this was not a big deal, but I did find the needless inconsistency quite distracting. Signed-off-by: Stefano Lattarini <> Signed-off-by: Junio C Hamano <>
2012-05-20t/Makefile: retain cache t/.prove across prove runsMichael Haggerty
prove(1) can write a summary of its test results and timings into a cache file, t/.prove, then use this information during later runs for various purposes. But deleting t/.prove after every test run defeats this purpose. So do not delete t/.prove as part of "make DEFAILT_TEST_TARGET=prove test". (Continue to delete the file on "make clean".) Signed-off-by: Michael Haggerty <> Signed-off-by: Junio C Hamano <>
2012-02-17Introduce a performance testing frameworkThomas Rast
This introduces a performance testing framework under t/perf/. It tries to be as close to the infrastructure as possible, and thus should be easy to get used to for git developers. The following points were considered for the implementation: 1. You usually want to compare arbitrary revisions/build trees against each other. They may not have the performance test under consideration, or even the infrastructure. To cope with this, the 'run' script lets you specify arbitrary build dirs and revisions. It even automatically builds the revisions if it doesn't have them at hand yet. 2. Usually you would not want to run all tests. It would take too long anyway. The 'run' script lets you specify which tests to run; or you can also do it manually. There is a Makefile for discoverability and 'make clean', but it is not meant for real-world use. 3. Creating test repos from scratch in every test is extremely time-consuming, and shipping or downloading such large/weird repos is out of the question. We leave this decision to the user. Two different sizes of test repos can be configured, and the scripts just copy one or more of those (using hardlinks for the object store). By default it tries to use the build tree's git.git repository. This is fairly fast and versatile. Using a copy instead of a clone preserves many properties that the user may want to test for, such as lots of loose objects, unpacked refs, etc. Signed-off-by: Thomas Rast <> Signed-off-by: Junio C Hamano <>
2012-02-13Sync with maintJunio C Hamano
Signed-off-by: Junio C Hamano <>
2012-02-13Remove Git's support for smoke testingÆvar Arnfjörð Bjarmason
I'm no longer running the Git smoke testing service at due to Smolder being a fragile piece of software not having time to follow through on making it easy for third parties to run and submit their own smoke tests. So remove the support in Git for sending smoke tests to, it's still easy to modify the test suite to submit smokes somewhere else. This reverts the following commits: Revert "t/README: Add SMOKE_{COMMENT,TAGS}= to smoke_report target" -- e38efac87d Revert "t/README: Document the Smoke testing" -- d15e9ebc5c Revert "t/Makefile: Create test-results dir for smoke target" -- 617344d77b Revert "tests: Infrastructure for Git smoke testing" -- b6b84d1b74 Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2012-01-22t/Makefile: Use $(sort ...) explicitly where neededKirill Smelkov
Starting from GNU Make 3.82 $(wildcard ...) no longer sorts the result (from NEWS): * WARNING: Backward-incompatibility! Wildcards were not documented as returning sorted values, but the results have been sorted up until this release.. If your makefiles require sorted results from wildcard expansions, use the $(sort ...) function to request it explicitly. I usually watch test progress visually, and if tests are sorted, even with make -j4 they go more or less incrementally by their t number. On the other side, without sorting, tests are executed in seemingly random order even for -j1. Let's please maintain sane tests order for perceived prettyness. Another note is that in GNU Make sort also works as uniq, so after sort being removed, we might expect e.g. $(wildcard *.sh a.*) to produce duplicates for e.g. "". From this point of view, adding sort could be seen as hardening t/Makefile from accidentally introduced dups. It turned out that prevous releases of GNU Make did not perform full sort in $(wildcard), only sorting results for each pattern, that's why explicit sort-as-uniq is relevant even for older makes. Signed-off-by: Kirill Smelkov <> Signed-off-by: Junio C Hamano <>
2011-06-17t/Makefile: pass test opts to valgrind target properlyJeff King
The valgrind target just reinvokes make with GIT_TEST_OPTS set to "--valgrind". However, it does this using an environment variable, which means GIT_TEST_OPTS in your config.mak would override it, and "make valgrind" would simply run the test suite without valgrind on. Instead, we should pass GIT_TEST_OPTS on the command-line, overriding what's in config.mak, and take care to append to whatever the user has there already. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2010-12-13tests: add some script lint checksJeff King
There are some common but minor errors we tend to make in writing test scripts: 1. Scripts are left non-executable. This is not usually noticed immediately because "make test" does not need the bit, but it is a matter of git policy to make them executable (and is a slight convenience when running individual scripts). 2. Two scripts are allocated the same number. Usually this happens on separate branches, and the problem only comes about during a merge. But since there is no textual conflict, the merger would have to be very observant to notice. This is also a minor error, but can make GIT_SKIP_TESTS ambiguous. This patch introduces a "test-lint" target which checks both. It is not invoked by default. You can invoke it as "make test-lint", or you can make it a prerequisite of running the tests by specifying "TEST_LINT = test-lint" in your config.mak or on the command line. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2010-11-17Merge branch 'mg/make-prove'Junio C Hamano
* mg/make-prove: test: allow running the tests under "prove"
2010-10-26test: allow running the tests under "prove"Michael J Gruber
You can run "make DEFAULT_TEST_TARGET=prove test" to run the test under "prove" (or $(PROVE) if set). The output is a bit easier to read when running many tests in parallel. Signed-off-by: Michael J Gruber <> Liked-by: Ævar Arnfjörð Bjarmason <> Liked-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2010-09-30gitweb/Makefile: Add 'test' and 'test-installed' targetsJakub Narebski
The 'test-installed' target in gitweb/Makefile tests installed gitweb, using the same destination directory that 'install' target uses. The 'test' target is just a convenience wrapper invoking 'gitweb-test' target of t/Makefile. Signed-off-by: Jakub Narebski <> Signed-off-by: Junio C Hamano <>
2010-08-18cvs tests: do not touch test CVS repositories shipped with sourceJunio C Hamano
Some tests in t96xx series (cvsimport) want to write into the control area (CVSROOT) of their test CVS repositories, but this does not work well when the source area is made read-only (test trash directories are moved via --root=else/where option). Copy the supplied test CVS repository to a scratch place at the beginning of these tests. Signed-off-by: Junio C Hamano <>
2010-08-18t/Makefile: Create test-results dir for smoke targetÆvar Arnfjörð Bjarmason
Change the smoke target to create a test-results directory. This was done implicitly by the test-lib before my "test-lib: Don't write test-results when HARNESS_ACTIVE" patch, but after that smoking from the pu branch hasn't worked. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2010-08-18t/README: Add SMOKE_{COMMENT,TAGS}= to smoke_report targetÆvar Arnfjörð Bjarmason
The smoke server supports a free form text field with comments about a report, and a comma delimited list of tags. Change the smoke_report target to expose this functionality. Now smokers can send more data that explains and categorizes the reports they're submitting. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2010-08-18t/Makefile: Can't include GIT-BUILD-OPTIONS, it's a .shÆvar Arnfjörð Bjarmason
Change the smoke testing portion of t/Makefile not to include GIT-BUILD-OPTIONS. It's a shellscript, not a Makefile snippet, so it had the nasty side-effect of sneaking e.g. SHELL_PATH = '/bin/sh' (with quotes) everywhere. Just add our own PERL_PATH variable as a workaround. The t/Makefile already has e.g. an equivalent SHELL_PATH and TAR option which duplicate the definitions in GIT-BUILD-OPTIONS. Reported-by: Thomas Rast <> Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2010-08-18tests: Infrastructure for Git smoke testingÆvar Arnfjörð Bjarmason
Add the capability to send smoke reports from the Git test suite. Currently we only notice bugs in the test suite when it's run manually. Bugs in Git that only occur on obscure platforms or setups that the core developers aren't using can thus go unnoticed. This series aims to change that. With it, anyone that's interested in avoiding bitrot in Git can volunteer to run a smoke tester. A smoke tester periodically compiles the latest version of Git, runs the test suite, and submits a report to a central server indicating how the test run went. A smoke tester might run something like this in cron: #!/bin/sh cd ~/g/git git fetch for branch in maint master next pu; do git checkout origin/$i && make clean all && cd t && make smoke_report done The smoker might want to compile git with non-default flags, include bisecting functionality or run the tests under valgrind. Doing that is outside the scope of this patch, this just adds a report submission mechanism. But including a canonical smoke runner is something we'll want to include eventually. What this does now is add smoke and smoke_report targets to t/Makefile (this example only uses a few tests for demonstration): $ make clean smoke rm -f -r 'trash directory'.* test-results rm -f t????/cvsroot/CVSROOT/?* rm -f -r valgrind/bin rm -f .prove perl ./harness --git-version="" \ --no-verbose \ --archive="test-results/git-smoke.tar.gz" \ ....... ok ........ ok ..... ok .. ok .. ok ..... ok ........ ok All tests successful. Test Summary Report ------------------- (Wstat: 0 Tests: 46 Failed: 0) TODO passed: 5 Files=7, Tests=134, 3 wallclock secs ( 0.06 usr 0.05 sys + 0.23 cusr 1.33 csys = 1.67 CPU) Result: PASS TAP Archive created at /home/avar/g/git/t/test-results/git-smoke.tar.gz The smoke target uses TAP::Harness::Archive to aggregate the test results into a tarball. The tarball contains two things, the output of every test file that was run, and a metadata file: Tarball contents: $ tar xzvf git-smoke.tar.gz meta.yml A test report: $ cat ok 1 - sigchain works # passed all 1 test(s) 1..1 A metadata file: --- extra_properties: file_attributes: - description: end_time: 1280437324.61398 start_time: 1280437324.22186 - description: end_time: 1280437325.12346 start_time: 1280437324.62393 - description: end_time: 1280437325.29428 start_time: 1280437325.13646 - description: end_time: 1280437325.59678 start_time: 1280437325.30565 - description: end_time: 1280437325.77376 start_time: 1280437325.61003 - description: end_time: 1280437325.85426 start_time: 1280437325.78727 - description: end_time: 1280437326.2362 start_time: 1280437325.86768 file_order: - - - - - - - start_time: 1280437324 stop_time: 1280437326 The "extra_properties" hash is where we'll stick Git-specific info, like whether Git was compiled with gettext or the fallback regex engine, and what branch we're compiling. Currently no metadata like this is included. The entire tarball is then submitted to a central smokebox at This is done with curl(1) via the "smoke_report" target: $ make smoke_report curl \ -H "Expect: " \ -F project=Git \ -F architecture=x86_64 \ -F platform=Linux \ -F revision="" \ -F report_file=@test-results/git-smoke.tar.gz \ \ | grep -v ^Redirecting % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 117k 100 63 100 117k 3 6430 0:00:21 0:00:18 0:00:03 0 Reported #8 added. Reports are then made available on the smokebox via a web interface: The smoke reports are also mirrored to a Git repository hosted on GitHub: The Smolder SQLite database that contains metadata about the reports is also made available: Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2010-07-26tests: Ignore the Test::Harness .prove fileÆvar Arnfjörð Bjarmason
We document how to run prove with the --state option in t/README. This produces a .prove YAML file in the current directory. Change the t/ gitignore to ignore it, and clean it up on `make clean'. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2010-07-16Only run aggregate-results over actual countsThomas Rast
The current make target 'aggregate-results' scanned all files matching test-results/t*-*. Normally these are only the test counts (and the exit values, which are ignored), but with --tee the suite also dumps all output. Furthermore, with --verbose t1450 contains several lines starting with "broken link from ..." which matches the criteria used by Rename the counts output files to *.counts, and only scan those. Signed-off-by: Thomas Rast <> Acked-by: Sverre Rabbelier <> Signed-off-by: Junio C Hamano <>
2010-06-21Merge branch 'gv/portable'Junio C Hamano
* gv/portable: test-lib: use DIFF definition from GIT-BUILD-OPTIONS build: propagate $DIFF to scripts Makefile: Tru64 portability fix Makefile: HP-UX 10.20 portability fixes Makefile: HPUX11 portability fixes Makefile: SunOS 5.6 portability fix inline declaration does not work on AIX Allow disabling "inline" Some platforms lack socklen_t type Make NO_{INET_NTOP,INET_PTON} configured independently Makefile: some platforms do not have hstrerror anywhere git-compat-util.h: some platforms with mmap() lack MAP_FAILED definition test_cmp: do not use "diff -u" on platforms that lack one fixup: do not unconditionally disable "diff -u" tests: use "test_cmp", not "diff", when verifying the result Do not use "diff" found on PATH while building and installing enums: omit trailing comma for portability Makefile: -lpthread may still be necessary when libc has only pthread stubs Rewrite dynamic structure initializations to runtime assignment Makefile: pass CPPFLAGS through to fllow customization Conflicts: Makefile wt-status.h
2010-06-02t/aggregate-results: accomodate systems with small max argument list lengthBrandon Casey
IRIX 6.5 has a default maximum argument list length of 20480. The file glob that is passed to aggregate-results currently exceeds this length, and so the script cannot run successfully. Work around this issue by passing the file names in via the standard input rather than the argument list. Signed-off-by: Brandon Casey <> Signed-off-by: Junio C Hamano <>
2010-06-01test_cmp: do not use "diff -u" on platforms that lack oneGary V. Vaughan
By default the testsuite calls 'diff -u' whenever a file comparison is called for. Unfortunately that throws a "diff: unknown option '-u'" error for most non-GNU diffs. This patch sets GIT_TEST_CMP to 'cmp' on all the architectures where that happens. The previous version of this patch forgot to export GIT_TEST_CMP from t/Makefile, which is why 'make test' continued to fail most tests on most architectures - was falling back on its default of `diff -u' for GIT_TEST_CMP. This version of this patch shows a vast improvement in testsuite results where either GNU diff is in the path at configure time, or where Makefile knows that GIT_TEST_CMP=cmp is required. Signed-off-by: Gary V. Vaughan <> Signed-off-by: Junio C Hamano <>
2010-06-01fixup: do not unconditionally disable "diff -u"Junio C Hamano
Signed-off-by: Junio C Hamano <>
2010-05-31tests: use "test_cmp", not "diff", when verifying the resultGary V. Vaughan
In tests, call test_cmp rather than raw diff where possible (i.e. if the output does not go to a pipe), to allow the use of, say, 'cmp' when the default 'diff -u' is not compatible with a vendor diff. When that is not possible, use $DIFF, as set in GIT-BUILD-OPTIONS. Signed-off-by: Gary V. Vaughan <> Signed-off-by: Junio C Hamano <>
2010-03-13t/Makefile: remove test artifacts upon "make clean"Junio C Hamano
Signed-off-by: Junio C Hamano <>
2009-08-09t/Makefile: include config.makJeff King
This is useful if you want to specify GIT_TEST_OPTS that you always use. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2009-03-28Merge branch 'maint'Junio C Hamano
* maint: test-lib: Clean up comments and Makefile. diff --no-index: Do not generate patch output if other output is requested
2009-03-28test-lib: Clean up comments and Makefile.Emil Sit
Bring documentation in test-lib and clean target in Makefile in-line with abc5d372. Signed-off-by: Emil Sit <> Signed-off-by: Junio C Hamano <>
2009-02-04Tests: let --valgrind imply --verbose and --teeJohannes Schindelin
It does not make much sense to run the (expensive) valgrind tests and not look at the output. To prevent output from scrolling out of reach, the parameter --tee is implied, too. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2009-02-04t/Makefile: provide a 'valgrind' targetJohannes Schindelin
It is easy to forget running valgrinded tests without -v, and it is also easy to forget to redirect the output to "tee" (lest the output scroll out of the terminal's buffer). Running "make valgrind" will take care of all that. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2008-08-09Enable parallel testsJohannes Schindelin
On multiprocessor machines, or with I/O heavy tests (that leave the CPU waiting a lot), it makes sense to parallelize the tests. However, care has to be taken that the different jobs use different trash directories. This commit does so, by creating the trash directories with a suffix that is unique with regard to the test, as it is the test's base name. Further, the trash directory is removed in the test itself if everything went fine, so that the trash directories do not pile up only to be removed at the very end. If a test failed, the trash directory is not removed. Chances are that the exact error message is lost in the clutter, but you can still see what test failed from the name of the trash directory, and repeat the test (without -j). If all was good, you will see the aggregated results. Suggestions to simplify this commit came from Junio and René. There still is an issue with tests that want to run a server process and listen to a fixed port (http and svn) --- they cannot run in parallel but this patch does not address this issue. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2008-08-09tests: Clarify dependencies between tests, 'aggregate-results' and 'clean'Johannes Schindelin
The Makefile targets 'aggregate-results' and 'clean' pretended to be independent. This is not true, of course, since aggregate-results needs the results _before_ they are removed. Likewise, the tests should have been run already when the results are to be aggregated. However, as it is legitimate to run only a few tests, and then aggregate just those results, so another target is introduced, that depends on all tests, then aggregates the results, and only then removes the results. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2008-07-16t/Makefile: use specified shell when running aggregation scriptSungHyun Nam
Signed-off-by: SungHyun Nam <> Signed-off-by: Junio C Hamano <>
2008-06-08Hook up the result aggregation in the test makefile.Sverre Rabbelier
This patch makes 'make' output the aggregated results at the end of each build. The 'git-test-result' file is removed both before and after each build. Signed-off-by: Sverre Rabbelier <> Signed-off-by: Junio C Hamano <>
2008-06-08Modify to output stats to t/test-results/*Sverre Rabbelier
This change is needed order to aggregate data on the test run later on. Signed-off-by: Sverre Rabbelier <> Signed-off-by: Junio C Hamano <>
2008-05-19t/Makefile: "trash" directory was renamed recentlyBrandon Casey
Signed-off-by: Brandon Casey <> Signed-off-by: Junio C Hamano <>