path: root/t/perf/README
AgeCommit message (Collapse)Author
2020-10-20t/perf/README: elaborate on output formatNipunn Koorapati
Signed-off-by: Nipunn Koorapati <> Signed-off-by: Junio C Hamano <>
2020-08-21p5302: disable thread-count parameter tests by defaultJeff King
The primary function of the perf suite is to detect regressions (or improvements) between versions of Git. The only numbers we show a direct comparison for are timings between the same test run on two different versions. However, it can sometimes be used to collect other information. For instance, p5302 runs the same index-pack operation with different thread counts. The output doesn't directly compare these, but anybody interested in working on index-pack can manually compare the results. For a normal regression run of the full perf-suite, though, this incurs a significant cost to generate numbers nobody will actually look at; about 25% of the total time of the test suite is spent in p5302. And the low-thread-count runs are the most expensive part of it, since they're (unsurprisingly) not using as many threads. Let's skip these tests by default, but make it possible for people working on index-pack to still run them by setting an environment variable. Rather than make this specific to p5302, let's introduce a generic mechanism. This makes it possible to run the full suite with every possible test if somebody really wants to burn some CPU. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2019-05-08perf README: correct docs for 3c8f12c96c regressionÆvar Arnfjörð Bjarmason
Since 3c8f12c96c ("test-lib: reorder and include GIT-BUILD-OPTIONS a lot earlier", 2012-06-24) the suggested advice of overriding GIT_BUILD_DIR has not worked. We've printed a hard error like this given e.g. GIT_BUILD_DIR=/home/avar/g/git: /bin-wrappers/git is not executable; using GIT_EXEC_PATH error: You haven't built things yet, have you? Let's just suggest that the user run other gits via the "run" script. That'll do the right thing for setting the path to the other git, and running the "aggregate.perl" scripts afterwards will work. As an aside, if setting GIT_BUILD_DIR had still worked, then the MODERN_GIT feature/fix added in 1a0962dee5 ("t/perf: fix regression in testing older versions of git", 2016-06-22) would have broke. Signed-off-by: Ævar Arnfjörð Bjarmason <>
2018-08-20t/perf: add infrastructure for measuring sizesJeff King
The main objective of scripts in the perf framework is to run "test_perf", which measures the time it takes to run some operation. However, it can also be interesting to see the change in the output size of certain operations. This patch introduces test_size, which records a single numeric output from the test and shows it in the aggregated output (with pretty printing and relative size comparison). Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2017-06-02Merge branch 'ab/grep-preparatory-cleanup'Junio C Hamano
The internal implementation of "git grep" has seen some clean-up. * ab/grep-preparatory-cleanup: (31 commits) grep: assert that threading is enabled when calling grep_{lock,unlock} grep: given --threads with NO_PTHREADS=YesPlease, warn pack-objects: fix buggy warning about threads pack-objects & index-pack: add test for --threads warning test-lib: add a PTHREADS prerequisite grep: move is_fixed() earlier to avoid forward declaration grep: change internal *pcre* variable & function names to be *pcre1* grep: change the internal PCRE macro names to be PCRE1 grep: factor test for \0 in grep patterns into a function grep: remove redundant regflags assignments grep: catch a missing enum in switch statement perf: add a comparison test of log --grep regex engines with -F perf: add a comparison test of log --grep regex engines perf: add a comparison test of grep regex engines with -F perf: add a comparison test of grep regex engines perf: emit progress output when unpacking & building perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't do grep: add tests to fix blind spots with \0 patterns grep: prepare for testing binary regexes containing rx metacharacters grep: add a test helper function for less verbose -f \0 tests ...
2017-05-20perf: add a GIT_PERF_MAKE_COMMAND for when *_MAKE_OPTS won't doÆvar Arnfjörð Bjarmason
Add a git GIT_PERF_MAKE_COMMAND variable to compliment the existing GIT_PERF_MAKE_OPTS facility. This allows specifying an arbitrary shell command to execute instead of 'make'. This is useful e.g. in cases where the name, semantics or defaults of a Makefile flag have changed over time. It can even be used to change the contents of the tree, useful for monkeypatching ancient versions of git to get them to build. This opens Pandora's box in some ways, it's now possible to "jailbreak" the perf environment and e.g. modify the source tree via this arbitrary instead of just issuing a custom "make" command, such a command has to be re-entrant in the sense that subsequent perf runs will re-use the possibly modified tree. It would be pointless to try to mitigate or work around that caveat in a tool purely aimed at Git developers, so this change makes no attempt to do so. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2017-05-12perf: add function to setup a fresh test repoÆvar Arnfjörð Bjarmason
Add a function to setup a fresh test repo via 'git init' to compliment the existing functions to copy over a normal & large repo. Some performance tests don't need any existing repository data at all to be significant, e.g. tests which stress glob matches against single pathological revisions or files, which I'm about to add in a subsequent commit. Signed-off-by: Ævar Arnfjörð Bjarmason <> Reviewed-by: Jonathan Nieder <> Signed-off-by: Junio C Hamano <>
2016-06-22t/perf: fix regression in testing older versions of gitJeff King
Commit 7501b59 (perf: make the tests work in worktrees, 2016-05-13) introduced the use of "git rev-parse --git-path" in the perf-lib setup code. Because the to-be-tested version of git is at the front of the $PATH when this code runs, this means we cannot use modern versions of t/perf to test versions of git older than v2.5.0 (when that option was introduced). This is a symptom of a more general problem. The t/perf suite is essentially independent of git versions, and ideally we would be able to run the most modern and complete set of tests across many historical versions (to see how they compare). But any setup code they run is therefore required to use the lowest common denominator we expect to test. So let's introduce a new variable, $MODERN_GIT, that we can use both in perf-lib and in the test setup to get a reliable set of git features (we might change git and break some tests, of course, but $MODERN_GIT is tied to the same version of git as the t/perf scripts, so they can be fixed or adjusted together). This commit fixes the "--git-path" case, but does not mass-convert existing setup code to use $MODERN_GIT. Most setup code is fairly vanilla and will work with effectively all versions. But now the tool is there to fix any other issues we find going forward. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2013-06-23Documentation: Update 'linux-2.6.git' -> 'linux.git'W. Trevor King
The 3.x tree has been out for a while now. The -2.6 repository name survived the initial release [1], but now only lists 'linux.git' (for aegl as well as torvalds) [2]. [1]: On 2011-05-30 01:47:57 GMT, Linus Torvalds wrote: > ... yes, that means that my git tree is still called > "linux-2.6.git" on [2]: Signed-off-by: W. Trevor King <> Signed-off-by: Junio C Hamano <>
2013-03-09perf: update documentation of GIT_PERF_REPEAT_COUNTAntoine Pelisse
Currently the documentation of GIT_PERF_REPEAT_COUNT says the default is five while "" uses a value of three as a default. Update the documentation so that it is consistent with the code. Signed-off-by: Antoine Pelisse <> Signed-off-by: Junio C Hamano <>
2012-02-17Introduce a performance testing frameworkThomas Rast
This introduces a performance testing framework under t/perf/. It tries to be as close to the infrastructure as possible, and thus should be easy to get used to for git developers. The following points were considered for the implementation: 1. You usually want to compare arbitrary revisions/build trees against each other. They may not have the performance test under consideration, or even the infrastructure. To cope with this, the 'run' script lets you specify arbitrary build dirs and revisions. It even automatically builds the revisions if it doesn't have them at hand yet. 2. Usually you would not want to run all tests. It would take too long anyway. The 'run' script lets you specify which tests to run; or you can also do it manually. There is a Makefile for discoverability and 'make clean', but it is not meant for real-world use. 3. Creating test repos from scratch in every test is extremely time-consuming, and shipping or downloading such large/weird repos is out of the question. We leave this decision to the user. Two different sizes of test repos can be configured, and the scripts just copy one or more of those (using hardlinks for the object store). By default it tries to use the build tree's git.git repository. This is fairly fast and versatile. Using a copy instead of a clone preserves many properties that the user may want to test for, such as lots of loose objects, unpacked refs, etc. Signed-off-by: Thomas Rast <> Signed-off-by: Junio C Hamano <>