summaryrefslogtreecommitdiff
path: root/t/perf/p5303-many-packs.sh
diff options
context:
space:
mode:
authorColin Stolley <cstolley@runbox.com>2019-11-27 22:24:53 (GMT)
committerJunio C Hamano <gitster@pobox.com>2019-12-03 15:59:45 (GMT)
commitec48540fe8c387cf7424d5387ddbd53e89bb9d51 (patch)
tree1b2111770e24c35a2591d4458c6c6e3fba124e8b /t/perf/p5303-many-packs.sh
parentd9f6f3b6195a0ca35642561e530798ad1469bd41 (diff)
downloadgit-ec48540fe8c387cf7424d5387ddbd53e89bb9d51.zip
git-ec48540fe8c387cf7424d5387ddbd53e89bb9d51.tar.gz
git-ec48540fe8c387cf7424d5387ddbd53e89bb9d51.tar.bz2
packfile.c: speed up loading lots of packfiles
When loading packfiles on start-up, we traverse the internal packfile list once per file to avoid reloading packfiles that have already been loaded. This check runs in quadratic time, so for poorly maintained repos with a large number of packfiles, it can be pretty slow. Add a hashmap containing the packfile names as we load them so that the average runtime cost of checking for already-loaded packs becomes constant. Add a perf test to p5303 to show speed-up. The existing p5303 test runtimes are dominated by other factors and do not show an appreciable speed-up. The new test in p5303 clearly exposes a speed-up in bad cases. In this test we create 10,000 packfiles and measure the start-up time of git rev-parse, which does little else besides load in the packs. Here are the numbers for the new p5303 test: Test HEAD^ HEAD --------------------------------------------------------------------- 5303.12: load 10,000 packs 1.03(0.92+0.10) 0.12(0.02+0.09) -88.3% Signed-off-by: Colin Stolley <cstolley@runbox.com> Helped-by: Jeff King <peff@peff.net> [jc: squashed the change to call hashmap in install_packed_git() by peff] Signed-off-by: Junio C Hamano <gitster@pobox.com>
Diffstat (limited to 't/perf/p5303-many-packs.sh')
-rwxr-xr-xt/perf/p5303-many-packs.sh18
1 files changed, 18 insertions, 0 deletions
diff --git a/t/perf/p5303-many-packs.sh b/t/perf/p5303-many-packs.sh
index 3779851..ede78e1 100755
--- a/t/perf/p5303-many-packs.sh
+++ b/t/perf/p5303-many-packs.sh
@@ -84,4 +84,22 @@ do
'
done
+# Measure pack loading with 10,000 packs.
+test_expect_success 'generate lots of packs' '
+ for i in $(test_seq 10000); do
+ echo "blob"
+ echo "data <<EOF"
+ echo "blob $i"
+ echo "EOF"
+ echo "checkpoint"
+ done |
+ git -c fastimport.unpackLimit=0 fast-import
+'
+
+# The purpose of this test is to evaluate load time for a large number
+# of packs while doing as little other work as possible.
+test_perf "load 10,000 packs" '
+ git rev-parse --verify "HEAD^{commit}"
+'
+
test_done