summaryrefslogtreecommitdiff
path: root/t/perf
AgeCommit message (Collapse)Author
9 daysMerge branch 'jk/perf-lib-test-titles'Junio C Hamano
Show test titles to the performance test output again. * jk/perf-lib-test-titles: perf-lib: fix missing test titles in output
10 daysperf-lib: fix missing test titles in outputJeff King
Commit 5dccd9155f (t/perf: add iteration setup mechanism to perf-lib, 2022-04-04) modified the parameter parsing of test_wrapper() such that the test title was no longer in $1, and is instead in $test_title_. We correctly pass the new variable to the code which outputs the title to the log, but missed the spot in test_wrapper() where the title is written to the ".descr" file which is used to produce the final output table. As a result, all of the titles are missing from that table (or worse, using whatever was left in $1): $ ./p0000-perf-lib-sanity.sh [...] Test this tree ------------------------------ 0000.1: 0.01(0.01+0.00) 0000.2: 0.01(0.00+0.01) 0000.4: 0.00(0.00+0.00) 0000.5: true 0.00(0.00+0.00) 0000.7: 0.00(0.00+0.00) 0000.8: 0.00(0.00+0.00) After this patch, we get the pre-5dccd9155f output: Test this tree -------------------------------------------------------------------------- 0000.1: test_perf_default_repo works 0.00(0.00+0.00) 0000.2: test_checkout_worktree works 0.01(0.00+0.01) 0000.4: export a weird var 0.00(0.00+0.00) 0000.5: éḿíẗ ńöń-ÁŚĆÍÍ ćḧáŕáćẗéŕś 0.00(0.00+0.00) 0000.7: important variables available in subshells 0.00(0.00+0.00) 0000.8: test-lib-functions correctly loaded in subshells 0.00(0.00+0.00) Signed-off-by: Jeff King <peff@peff.net> Acked-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-10Merge branch 'jh/builtin-fsmonitor-part3'Junio C Hamano
More fsmonitor--daemon. * jh/builtin-fsmonitor-part3: (30 commits) t7527: improve implicit shutdown testing in fsmonitor--daemon fsmonitor--daemon: allow --super-prefix argument t7527: test Unicode NFC/NFD handling on MacOS t/lib-unicode-nfc-nfd: helper prereqs for testing unicode nfc/nfd t/helper/hexdump: add helper to print hexdump of stdin fsmonitor: on macOS also emit NFC spelling for NFD pathname t7527: test FSMonitor on case insensitive+preserving file system fsmonitor: never set CE_FSMONITOR_VALID on submodules t/perf/p7527: add perf test for builtin FSMonitor t7527: FSMonitor tests for directory moves fsmonitor: optimize processing of directory events fsm-listen-darwin: shutdown daemon if worktree root is moved/renamed fsm-health-win32: force shutdown daemon if worktree root moves fsm-health-win32: add polling framework to monitor daemon health fsmonitor--daemon: stub in health thread fsmonitor--daemon: rename listener thread related variables fsmonitor--daemon: prepare for adding health thread fsmonitor--daemon: cd out of worktree root fsm-listen-darwin: ignore FSEvents caused by xattr changes on macOS unpack-trees: initialize fsmonitor_has_run_once in o->result ...
2022-06-03Merge branch 'ds/sparse-sparse-checkout'Junio C Hamano
"sparse-checkout" learns to work well with the sparse-index feature. * ds/sparse-sparse-checkout: sparse-checkout: integrate with sparse index p2000: add test for 'git sparse-checkout [add|set]' sparse-index: complete partial expansion sparse-index: partially expand directories sparse-checkout: --no-sparse-index needs a full index cache-tree: implement cache_tree_find_path() sparse-index: introduce partially-sparse indexes sparse-index: create expand_index() t1092: stress test 'git sparse-checkout set' t1092: refactor 'sparse-index contents' test
2022-06-03Merge branch 'ns/batch-fsync'Junio C Hamano
Introduce a filesystem-dependent mechanism to optimize the way the bits for many loose object files are ensured to hit the disk platter. * ns/batch-fsync: core.fsyncmethod: performance tests for batch mode t/perf: add iteration setup mechanism to perf-lib core.fsyncmethod: tests for batch mode test-lib-functions: add parsing helpers for ls-files and ls-tree core.fsync: use batch mode and sync loose objects by default on Windows unpack-objects: use the bulk-checkin infrastructure update-index: use the bulk-checkin infrastructure builtin/add: add ODB transaction around add_files_to_cache cache-tree: use ODB transaction around writing a tree core.fsyncmethod: batched disk flushes for loose-objects bulk-checkin: rebrand plug/unplug APIs as 'odb transactions' bulk-checkin: rename 'state' variable and separate 'plugged' boolean
2022-05-26t/perf/p7527: add perf test for builtin FSMonitorJeff Hostetler
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-23p2000: add test for 'git sparse-checkout [add|set]'Derrick Stolee
The sparse-checkout builtin is almost completely integrated with the sparse index, allowing the sparse-checkout boundary to be modified without expanding a sparse index to a full one. Add a test to p2000-sparse-operations.sh that adds a directory to the sparse-checkout definition, then removes it. Using both operations is important to ensure that the operation is doing the same work in each repetition as well as leaving the test repo in a good state for later tests. Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-10stash: expand sparse-checkout compatibility testingVictoria Dye
Add tests verifying expected 'git stash' behavior in 't1092-sparse-checkout-compatibility'. These cases establish the expected behavior of 'git stash' in a sparse-checkout and verify consistency both with and without a sparse index. Although no sparse index compatibility has been integrated into 'git stash' yet, the tests are all 'expect_success' - we don't want the cone-mode sparse-checkout behavior to change depending on whether it is using a sparse index or not. Therefore, we expect these tests to continue passing once sparse index is integrated with 'git stash'. Additionally, add performance test cases for 'git stash' both with and without untracked files. Note that, unlike the other tests in 'p2000-sparse-operations.sh', the tests added for 'stash' are combination operations. This is done to ensure the stash/unstash is not blocked by the modification of '$SPARSE_CONE/a' performed as part of 'test_perf_on_all'. Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-06t/perf: add iteration setup mechanism to perf-libNeeraj Singh
Tests that affect the repo in stateful ways are easier to write if we can run setup steps outside of the measured portion of perf iteration. This change adds a "--setup 'setup-script'" parameter to test_perf. To make invocations easier to understand, I also moved the prerequisites to a new --prereq parameter. The setup facility will be used in the upcoming perf tests for batch mode, but it already helps in some existing tests, like t5302 and t7820. Signed-off-by: Neeraj Singh <neerajsi@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-06core.fsyncmethod: performance tests for batch modeNeeraj Singh
Add basic performance tests for git commands that can add data to the object database. We cover: * git add * git stash * git update-index (via git stash) * git unpack-objects * git commit --all We cover all currently available fsync methods as well. Signed-off-by: Neeraj Singh <neerajsi@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-04Merge branch 'jh/builtin-fsmonitor-part2'Junio C Hamano
Built-in fsmonitor (part 2). * jh/builtin-fsmonitor-part2: (30 commits) t7527: test status with untracked-cache and fsmonitor--daemon fsmonitor: force update index after large responses fsmonitor--daemon: use a cookie file to sync with file system fsmonitor--daemon: periodically truncate list of modified files t/perf/p7519: add fsmonitor--daemon test cases t/perf/p7519: speed up test on Windows t/perf/p7519: fix coding style t/helper/test-chmtime: skip directories on Windows t/perf: avoid copying builtin fsmonitor files into test repo t7527: create test for fsmonitor--daemon t/helper/fsmonitor-client: create IPC client to talk to FSMonitor Daemon help: include fsmonitor--daemon feature flag in version info fsmonitor--daemon: implement handle_client callback compat/fsmonitor/fsm-listen-darwin: implement FSEvent listener on MacOS compat/fsmonitor/fsm-listen-darwin: add MacOS header files for FSEvent compat/fsmonitor/fsm-listen-win32: implement FSMonitor backend on Windows fsmonitor--daemon: create token-based changed path cache fsmonitor--daemon: define token-ids fsmonitor--daemon: add pathname classification fsmonitor--daemon: implement 'start' command ...
2022-03-25t/perf/p7519: add fsmonitor--daemon test casesJeff Hostetler
Repeat all of the fsmonitor perf tests using `git fsmonitor--daemon` and the "Simple IPC" interface. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-25t/perf/p7519: speed up test on WindowsJeff Hostetler
Change p7519 to use `test_seq` and `xargs` rather than a `for` loop to touch thousands of files. This takes minutes off of test runs on Windows because of process creation overhead. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-25t/perf/p7519: fix coding styleJeff Hostetler
Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-25t/perf: avoid copying builtin fsmonitor files into test repoJeff Hostetler
Do not copy any of the various fsmonitor--daemon files from the .git directory of the (GIT_PREF_REPO or GIT_PERF_LARGE_REPO) source repo into the test's trash directory. When perf tests start, they copy the contents of the source repo into the test's trash directory. If fsmonitor is running in the source repo, there may be control files, such as the IPC socket and/or fsmonitor cookie files. These should not be copied into the test repo. Unix domain sockets cannot be copied in the manner used by the test setup, so if present, the test setup fails. Cookie files are harmless, but we should avoid them. The builtin fsmonitor keeps all such control files/sockets in .git/fsmonitor--daemon*, so it is simple to exclude them. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-23Merge branch 'jc/cat-file-batch-default-format-optim'Junio C Hamano
Optimize away strbuf_expand() call with a hardcoded formatting logic specific for the default format in the --batch and --batch-check options of "git cat-file". * jc/cat-file-batch-default-format-optim: cat-file: skip expanding default format
2022-03-17Merge branch 'vd/sparse-read-tree'Junio C Hamano
"git read-tree" has been made to be aware of the sparse-index feature. * vd/sparse-read-tree: read-tree: make three-way merge sparse-aware read-tree: make two-way merge sparse-aware read-tree: narrow scope of index expansion for '--prefix' read-tree: integrate with sparse index read-tree: expand sparse checkout test coverage read-tree: explicitly disallow prefixes with a leading '/' status: fix nested sparse directory diff in sparse index sparse-index: prevent repo root from becoming sparse
2022-03-15cat-file: skip expanding default formatJohn Cai
When format is passed into --batch, --batch-check, --batch-command, the format gets expanded. When nothing is passed in, the default format is set and the expand_format() gets called. We can save on these cycles by hardcoding how to print the information when nothing is passed as the format, or when the default format is passed. There is no need for the fully expanded format with the default. Since batch_object_write() happens on every object provided in batch mode, we get a nice performance improvement. git rev-list --all > /tmp/all-obj.txt git cat-file --batch-check </tmp/all-obj.txt with HEAD^: Time (mean ± σ): 57.6 ms ± 1.7 ms [User: 51.5 ms, System: 6.2 ms] Range (min … max): 54.6 ms … 64.7 ms 50 runs with HEAD: Time (mean ± σ): 49.8 ms ± 1.7 ms [User: 42.6 ms, System: 7.3 ms] Range (min … max): 46.9 ms … 55.9 ms 56 runs If nothing is provided as a format argument, or if the default format is passed, skip expanding of the format and print the object info with a default format. See https://lore.kernel.org/git/87eecf8ork.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: John Cai <johncai86@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-03Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to itÆvar Arnfjörð Bjarmason
We have various behavior that's shared across our Makefiles, or that really should be (e.g. via defined templates). Let's create a top-level "shared.mak" to house those sorts of things, and start by adding the ".DELETE_ON_ERROR" flag to it. See my own 7b76d6bf221 (Makefile: add and use the ".DELETE_ON_ERROR" flag, 2021-06-29) and db10fc6c09f (doc: simplify Makefile using .DELETE_ON_ERROR, 2021-05-21) for the addition and use of the ".DELETE_ON_ERROR" flag. I.e. this changes the behavior of existing rules in the altered Makefiles (except "Makefile" & "Documentation/Makefile"). I'm confident that this is safe having read the relevant rules in those Makfiles, and as the GNU make manual notes that it isn't the default behavior is out of an abundance of backwards compatibility caution. From edition 0.75 of its manual, covering GNU make 4.3: [Enabling '.DELETE_ON_ERROR' is] almost always what you want 'make' to do, but it is not historical practice; so for compatibility, you must explicitly request it. This doesn't introduce a bug by e.g. having this ".DELETE_ON_ERROR" flag only apply to this new shared.mak, Makefiles have no such scoping semantics. It does increase the danger that any Makefile without an explicit "The default target of this Makefile is..." snippet to define the default target as "all" could have its default rule changed if our new shared.mak ever defines a "real" rule. In subsequent commits we'll be careful not to do that, and such breakage would be obvious e.g. in the case of "make -C t". We might want to make that less fragile still (e.g. by using ".DEFAULT_GOAL" as noted in the preceding commit), but for now let's simply include "shared.mak" without adding that boilerplate to all the Makefiles that don't have it already. Most of those are already exposed to that potential caveat e.g. due to including "config.mak*". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-03-01read-tree: expand sparse checkout test coverageVictoria Dye
Add tests focused on how 'git read-tree' behaves in sparse checkouts. Extra emphasis is placed on interactions with files outside the sparse cone, e.g. merges with out-of-cone conflicts. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-18Merge branch 'vd/sparse-clean-etc'Junio C Hamano
"git update-index", "git checkout-index", and "git clean" are taught to work better with the sparse checkout feature. * vd/sparse-clean-etc: update-index: reduce scope of index expansion in do_reupdate update-index: integrate with sparse index update-index: add tests for sparse-checkout compatibility checkout-index: integrate with sparse index checkout-index: add --ignore-skip-worktree-bits option checkout-index: expand sparse checkout compatibility tests clean: integrate with sparse index reset: reorder wildcard pathspec conditions reset: fix validation in sparse index test
2022-01-13update-index: add tests for sparse-checkout compatibilityVictoria Dye
Introduce tests for a variety of `git update-index` use cases, including performance scenarios. Tests are intended to exercise `update-index` with options that change the commands interaction with the index (e.g., `--again`) and with files/directories inside and outside a sparse checkout cone. Of note is that these tests clearly establish the behavior of `git update-index --add` with untracked, outside-of-cone files. Unlike `git add`, which fails with an error when provided with such files, `update-index` succeeds in adding them to the index. Additionally, the `skip-worktree` flag is *not* automatically added to the new entry. Although this is pre-existing behavior, there are a couple of reasons to avoid changing it in favor of consistency with e.g. `git add`: * `update-index` is low-level command for modifying the index; while it can perform operations similar to those of `add`, it traditionally has fewer "guardrails" preventing a user from doing something they may not want to do (in this case, adding an outside-of-cone, non-`skip-worktree` file to the index) * `update-index` typically only exits with an error code if it is incapable of performing an operation (e.g., if an internal function call fails); adding a new file outside the sparse checkout definition is still a valid operation, albeit an inadvisable one * `update-index` does not implicitly set flags (e.g., `skip-worktree`) when creating new index entries with `--add`; if flags need to be updated, options like `--[no-]skip-worktree` allow a user to intentionally set them All this to say that, while there are valid reasons to consider changing the treatment of outside-of-cone files in `update-index`, there are also sufficient reasons for leaving it as-is. Co-authored-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Victoria Dye <vdye@github.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-13checkout-index: expand sparse checkout compatibility testsVictoria Dye
Add tests to cover `checkout-index`, with a focus on cases interesting in a sparse checkout (e.g., files specified outside sparse checkout definition). New tests are intended to serve as a baseline for existing and/or expected behavior and performance when integrating `checkout-index` with the sparse index. Note that the test 'checkout-index --all' is marked as 'test_expect_failure', indicating that `update-index --all` will be modified in a subsequent patch to behave as the test expects. Signed-off-by: Victoria Dye <vdye@github.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-10Merge branch 'ja/perf-use-specified-shell'Junio C Hamano
Perf tests were run with end-user's shell, but it has been corrected to use the shell specified by $TEST_SHELL_PATH. * ja/perf-use-specified-shell: t/perf: do not run tests in user's $SHELL
2022-01-05Merge branch 'pw/diff-color-moved-fix'Junio C Hamano
Correctness and performance update to "diff --color-moved" feature. * pw/diff-color-moved-fix: diff --color-moved: intern strings diff: use designated initializers for emitted_diff_symbol diff --color-moved-ws=allow-indentation-change: improve hash lookups diff --color-moved: stop clearing potential moved blocks diff --color-moved: shrink potential moved blocks as we go diff --color-moved: unify moved block growth functions diff --color-moved: call comparison function directly diff --color-moved-ws=allow-indentation-change: simplify and optimize diff: simplify allow-indentation-change delta calculation diff --color-moved: avoid false short line matches and bad zebra coloring diff --color-moved=zebra: fix alternate coloring diff --color-moved: rewind when discarding pmb diff --color-moved: factor out function diff --color-moved: clear all flags on blocks that are too short diff --color-moved: add perf tests
2022-01-04Merge branch 'es/test-chain-lint'Junio C Hamano
Broken &&-chains in the test scripts have been corrected. * es/test-chain-lint: t6000-t9999: detect and signal failure within loop t5000-t5999: detect and signal failure within loop t4000-t4999: detect and signal failure within loop t0000-t3999: detect and signal failure within loop tests: simplify by dropping unnecessary `for` loops tests: apply modern idiom for exiting loop upon failure tests: apply modern idiom for signaling test failure tests: fix broken &&-chains in `{...}` groups tests: fix broken &&-chains in `$(...)` command substitutions tests: fix broken &&-chains in compound statements tests: use test_write_lines() to generate line-oriented output tests: simplify construction of large blocks of text t9107: use shell parameter expansion to avoid breaking &&-chain t6300: make `%(raw:size) --shell` test more robust t5516: drop unnecessary subshell and command invocation t4202: clarify intent by creating expected content less cleverly t1020: avoid aborting entire test script when one test fails t1010: fix unnoticed failure on Windows t/lib-pager: use sane_unset() to avoid breaking &&-chain
2021-12-25t/perf: do not run tests in user's $SHELLJohannes Altmanninger
The environment variable $SHELL is usually set to the user's interactive shell. Our build and test scripts never use $SHELL because there are no guarantees about its input language. Instead, we use /bin/sh which should be a POSIX shell. For systems with a broken /bin/sh, we allow to override that path via SHELL_PATH. To run tests in yet another shell we allow to override SHELL_PATH with TEST_SHELL_PATH. Perf tests run in $SHELL via a wrapper defined in t/perf/perf-lib.sh, so they break with e.g. SHELL=python. Use TEST_SHELL_PATH like in other tests. TEST_SHELL_PATH is always defined because t/perf/perf-lib.sh includes t/test-lib.sh, which includes GIT-BUILD-OPTIONS. Acked-by: Jeff King <peff@peff.net> Signed-off-by: Johannes Altmanninger <aclopte@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-21Merge branch 'ld/sparse-diff-blame'Junio C Hamano
Teach diff and blame to work well with sparse index. * ld/sparse-diff-blame: blame: enable and test the sparse index diff: enable and test the sparse index diff: replace --staged with --cached in t1092 tests repo-settings: prepare_repo_settings only in git repos test-read-cache: set up repo after git directory commit-graph: return if there is no git directory git: ensure correct git directory setup with -h
2021-12-13t0000-t3999: detect and signal failure within loopEric Sunshine
Failures within `for` and `while` loops can go unnoticed if not detected and signaled manually since the loop itself does not abort when a contained command fails, nor will a failure necessarily be detected when the loop finishes since the loop returns the exit code of the last command it ran on the final iteration, which may not be the command which failed. Therefore, detect and signal failures manually within loops using the idiom `|| return 1` (or `|| exit 1` within subshells). Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-13tests: fix broken &&-chains in compound statementsEric Sunshine
The top-level &&-chain checker built into t/test-lib.sh causes tests to magically exit with code 117 if the &&-chain is broken. However, it has the shortcoming that the magic does not work within `{...}` groups, `(...)` subshells, `$(...)` substitutions, or within bodies of compound statements, such as `if`, `for`, `while`, `case`, etc. `chainlint.sed` partly fills in the gap by catching broken &&-chains in `(...)` subshells, but bugs can still lurk behind broken &&-chains in the other cases. Fix broken &&-chains in compound statements in order to reduce the number of possible lurking bugs. Signed-off-by: Eric Sunshine <sunshine@sunshineco.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-10Merge branch 'vd/sparse-reset'Junio C Hamano
Various operating modes of "git reset" have been made to work better with the sparse index. * vd/sparse-reset: unpack-trees: improve performance of next_cache_entry reset: make --mixed sparse-aware reset: make sparse-aware (except --mixed) reset: integrate with sparse index reset: expand test coverage for sparse checkouts sparse-index: update command for expand/collapse test reset: preserve skip-worktree bit in mixed reset reset: rename is_missing to !is_in_reset_tree
2021-12-09diff --color-moved: add perf testsPhillip Wood
Add some tests so we can monitor changes to the performance of the move detection code. The tests record the performance --color-moved and --color-moved-ws=allow-indentation-change for a large diff and a sequence of smaller diffs. The range of commits used for the large diff can be customized by exporting TEST_REV_A and TEST_REV_B when running the test. Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-06blame: enable and test the sparse indexLessley Dennington
Enable the sparse index for the 'git blame' command. The index was already not expanded with this command, so the most interesting thing to do is to add tests that verify that 'git blame' behaves correctly when the sparse index is enabled and that its performance improves. More specifically, these cases are: 1. The index is not expanded for 'blame' when given paths in the sparse checkout cone at multiple levels. 2. Performance measurably improves for 'blame' with sparse index when given paths in the sparse checkout cone at multiple levels. The `p2000` tests demonstrate a ~60% execution time reduction when running 'blame' for a file two levels deep and and a ~30% execution time reduction for a file three levels deep. Test before after ---------------------------------------------------------------- 2000.62: git blame f2/f4/a (full-v3) 0.31 0.32 +3.2% 2000.63: git blame f2/f4/a (full-v4) 0.29 0.31 +6.9% 2000.64: git blame f2/f4/a (sparse-v3) 0.55 0.23 -58.2% 2000.65: git blame f2/f4/a (sparse-v4) 0.57 0.23 -59.6% 2000.66: git blame f2/f4/f3/a (full-v3) 0.77 0.85 +10.4% 2000.67: git blame f2/f4/f3/a (full-v4) 0.78 0.81 +3.8% 2000.68: git blame f2/f4/f3/a (sparse-v3) 1.07 0.72 -32.7% 2000.99: git blame f2/f4/f3/a (sparse-v4) 1.05 0.73 -30.5% We do not include paths outside the sparse checkout cone because blame does not support blaming files that are not present in the working directory. This is true in both sparse and full checkouts. Signed-off-by: Lessley Dennington <lessleydennington@gmail.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-06diff: enable and test the sparse indexLessley Dennington
Enable the sparse index within the 'git diff' command. Its implementation already safely integrates with the sparse index because it shares code with the 'git status' and 'git checkout' commands that were already integrated. For more details see: d76723ee53 (status: use sparse-index throughout, 2021-07-14) 1ba5f45132 (checkout: stop expanding sparse indexes, 2021-06-29) The most interesting thing to do is to add tests that verify that 'git diff' behaves correctly when the sparse index is enabled. These cases are: 1. The index is not expanded for 'diff' and 'diff --staged' 2. 'diff' and 'diff --staged' behave the same in full checkout, sparse checkout, and sparse index repositories in the following partially-staged scenarios (i.e. the index, HEAD, and working directory differ at a given path): 1. Path is within sparse-checkout cone 2. Path is outside sparse-checkout cone 3. A merge conflict exists for paths outside sparse-checkout cone The `p2000` tests demonstrate a ~44% execution time reduction for 'git diff' and a ~86% execution time reduction for 'git diff --staged' using a sparse index: Test before after ------------------------------------------------------------- 2000.30: git diff (full-v3) 0.33 0.34 +3.0% 2000.31: git diff (full-v4) 0.33 0.35 +6.1% 2000.32: git diff (sparse-v3) 0.53 0.31 -41.5% 2000.33: git diff (sparse-v4) 0.54 0.29 -46.3% 2000.34: git diff --cached (full-v3) 0.07 0.07 +0.0% 2000.35: git diff --cached (full-v4) 0.07 0.08 +14.3% 2000.36: git diff --cached (sparse-v3) 0.28 0.04 -85.7% 2000.37: git diff --cached (sparse-v4) 0.23 0.03 -87.0% Co-authored-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Lessley Dennington <lessleydennington@gmail.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-29Merge branch 'vd/sparse-reset' into ld/sparse-diff-blameJunio C Hamano
* vd/sparse-reset: unpack-trees: improve performance of next_cache_entry reset: make --mixed sparse-aware reset: make sparse-aware (except --mixed) reset: integrate with sparse index reset: expand test coverage for sparse checkouts sparse-index: update command for expand/collapse test reset: preserve skip-worktree bit in mixed reset reset: rename is_missing to !is_in_reset_tree
2021-11-29reset: expand test coverage for sparse checkoutsVictoria Dye
Add new tests for `--merge` and `--keep` modes, as well as mixed reset with pathspecs. New performance test cases exercise various execution paths for `reset`. Co-authored-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-25Merge branch 'rs/disable-gc-during-perf-tests'Junio C Hamano
Avoid performance measurements from getting ruined by gc and other housekeeping pauses interfering in the middle. * rs/disable-gc-during-perf-tests: perf: disable automatic housekeeping
2021-10-25Merge branch 'jh/perf-remove-test-times'Junio C Hamano
Perf test fix. * jh/perf-remove-test-times: t/perf/perf-lib.sh: remove test_times.* at the end test_perf_()
2021-10-18Merge branch 'rs/mergesort'Junio C Hamano
The mergesort implementation used to sort linked list has been optimized. * rs/mergesort: test-mergesort: use repeatable random numbers mergesort: use ranks stack p0071: test performance of llist_mergesort() p0071: measure sorting of already sorted and reversed files test-mergesort: add unriffle_skewed mode test-mergesort: add unriffle mode test-mergesort: add generate subcommand test-mergesort: add test subcommand test-mergesort: add sort subcommand test-mergesort: use strbuf_getline()
2021-10-11perf: disable automatic housekeepingRené Scharfe
Turn off automatic background maintenance for perf tests by default to avoid interference with performance measurements. Do that by using the new file t/perf/config and using it as the system config file for perf tests. Future tests intended to measure gc performance can override the setting locally or call "git gc" explicitly. This fixes a breakage in p2000 caused by gc automatically emptying the reflog due its fake dates from 2005 being older than 90 days. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-11Merge branch 'tb/aggregate-ignore-leading-whitespaces'Junio C Hamano
Test portability update. * tb/aggregate-ignore-leading-whitespaces: t/perf/aggregate.perl: tolerate leading spaces
2021-10-11Merge branch 'rs/p3400-lose-tac'Junio C Hamano
Test portability update. * rs/p3400-lose-tac: p3400: stop using tac(1)
2021-10-11Merge branch 'tb/midx-write-propagate-namehash'Junio C Hamano
"git multi-pack-index write --bitmap" learns to propagate the hashcache from original bitmap to resulting bitmap. * tb/midx-write-propagate-namehash: t5326: test propagating hashcache values p5326: generate pack bitmaps before writing the MIDX bitmap p5326: don't set core.multiPackIndex unnecessarily p5326: create missing 'perf-tag' tag midx.c: respect 'pack.writeBitmapHashcache' when writing bitmaps pack-bitmap.c: propagate namehash values from existing bitmaps t/helper/test-bitmap.c: add 'dump-hashes' mode
2021-10-05t/perf/perf-lib.sh: remove test_times.* at the end test_perf_()Jeff Hostetler
Teach test_perf_() to remove the temporary test_times.* files at the end of each test. test_perf_() runs a particular GIT_PERF_REPEAT_COUNT times and creates ./test_times.[123...]. It then uses a perl script to find the minimum over "./test_times.*" (note the wildcard) and writes that time to "test-results/<testname>.<testnumber>.result". If the repeat count is changed during the pXXXX test script, stale test_times.* files (from previous steps) may be included in the min() computation. For example: ... GIT_PERF_REPEAT_COUNT=3 \ test_perf "status" " git status " GIT_PERF_REPEAT_COUNT=1 \ test_perf "checkout other" " git checkout other " ... The time reported in the summary for "XXXX.2 checkout other" would be "min( checkout[1], status[2], status[3] )". We prevent that error by removing the test_times.* files at the end of each test. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-04t/perf/aggregate.perl: tolerate leading spacesTaylor Blau
When using `test_size` with `wc -c`, users on certain platforms can run into issues when `wc` emits leading space characters in its output, which confuses get_times. Callers could switch to use test_file_size instead of `wc -c` (the former never prints leading space characters, so will always work with test_size regardless of platform), but this is an easy enough spot to miss that we should teach get_times to be more tolerant of the input it accepts. Teach get_times to do just that by stripping any leading space characters. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-04p3400: stop using tac(1)René Scharfe
b3dfeebb92 (rebase: avoid computing unnecessary patch IDs, 2016-07-29) added a perf test that calls tac(1) from GNU core utilities. Support systems without it by reversing the generated list using sort -nr instead. sort(1) with options -n and -r is already used in other tests. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-01p0071: test performance of llist_mergesort()René Scharfe
Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-01p0071: measure sorting of already sorted and reversed filesRené Scharfe
Check if sorting takes advantage of already sorted or reversed content, or if that corner case actually decreases performance, like it would for a simplistic quicksort implementation. Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-22t/perf/run: fix bin-wrappers computationDerrick Stolee
The GIT_TEST_INSTALLED was moved from perf-lib.sh to run in df0f5021 (perf-lib.sh: remove GIT_TEST_INSTALLED from perf-lib.sh, 2019-05-07) and that included a change to how it inspected the existence of a bin-wrappers directory. However, that included a typo that made the match of bin-wrappers never work. Specifically, the assignment was mydir_abs_wrappers="$mydir_abs_wrappers/bin-wrappers" which uses the same variable before it is initialized. By changing it to mydir_abs_wrappers="$mydir_abs/bin-wrappers" We can correctly use the bin-wrappers directory. This is critical to successfully computing performance of commands that execute subcommands. The bin-wrappers ensure that the --exec-path is set correctly. Reported-by: Victoria Dye <vdye@github.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-17p5326: generate pack bitmaps before writing the MIDX bitmapTaylor Blau
To help test the performance of permuting the contents of the hash-cache when generating a MIDX bitmap, we need a bitmap which has its hash-cache populated. And since multi-pack bitmaps don't add *new* values to the hash-cache, we have to rely on a single-pack bitmap to generate those values for us. Therefore, generate a pack bitmap before the MIDX one in order to ensure that the MIDX bitmap has entries in its hash-cache. Since we don't want to time generating the pack bitmap, move that to a non-perf test run before we try to generate the MIDX bitmap. Likewise, get rid of the pack bitmap afterwords, to make certain that we are not accidentally using it in the performance tests run later on. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>