summaryrefslogtreecommitdiff
path: root/t/perf/p5326-multi-pack-bitmaps.sh
AgeCommit message (Collapse)Author
2021-09-17p5326: generate pack bitmaps before writing the MIDX bitmapTaylor Blau
To help test the performance of permuting the contents of the hash-cache when generating a MIDX bitmap, we need a bitmap which has its hash-cache populated. And since multi-pack bitmaps don't add *new* values to the hash-cache, we have to rely on a single-pack bitmap to generate those values for us. Therefore, generate a pack bitmap before the MIDX one in order to ensure that the MIDX bitmap has entries in its hash-cache. Since we don't want to time generating the pack bitmap, move that to a non-perf test run before we try to generate the MIDX bitmap. Likewise, get rid of the pack bitmap afterwords, to make certain that we are not accidentally using it in the performance tests run later on. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-14p5326: don't set core.multiPackIndex unnecessarilyTaylor Blau
When this performance test was originally written, `core.multiPackIndex` was not the default and thus had to be enabled. But now that we have 18e449f86b (midx: enable core.multiPackIndex by default, 2020-09-25), we no longer need this. Drop the unnecessary setup (even though it's not hurting anything, it is unnecessary at best and confusing at worst). Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-14p5326: create missing 'perf-tag' tagTaylor Blau
Some of the tests in test_full_bitmap rely on having a tag named perf-tag in place. We could create it in test_full_bitmap(), but we want to have it in place before the repack starts. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-01p5326: perf tests for MIDX bitmapsTaylor Blau
These new performance tests demonstrate effectively the same behavior as p5310, but use a multi-pack bitmap instead of a single-pack one. Notably, p5326 does not create a MIDX bitmap with multiple packs. This is so we can measure a direct comparison between it and p5310. Any difference between the two is measuring just the overhead of using MIDX bitmaps. Here are the results of p5310 and p5326 together, measured at the same time and on the same machine (using a Xenon W-2255 CPU): Test HEAD ------------------------------------------------------------------------ 5310.2: repack to disk 96.78(93.39+11.33) 5310.3: simulated clone 9.98(9.79+0.19) 5310.4: simulated fetch 1.75(4.26+0.19) 5310.5: pack to file (bitmap) 28.20(27.87+8.70) 5310.6: rev-list (commits) 0.41(0.36+0.05) 5310.7: rev-list (objects) 1.61(1.54+0.07) 5310.8: rev-list count with blob:none 0.25(0.21+0.04) 5310.9: rev-list count with blob:limit=1k 2.65(2.54+0.10) 5310.10: rev-list count with tree:0 0.23(0.19+0.04) 5310.11: simulated partial clone 4.34(4.21+0.12) 5310.13: clone (partial bitmap) 11.05(12.21+0.48) 5310.14: pack to file (partial bitmap) 31.25(34.22+3.70) 5310.15: rev-list with tree filter (partial bitmap) 0.26(0.22+0.04) versus the same tests (this time using a multi-pack index): Test HEAD ------------------------------------------------------------------------ 5326.2: setup multi-pack index 78.99(75.29+11.58) 5326.3: simulated clone 11.78(11.56+0.22) 5326.4: simulated fetch 1.70(4.49+0.13) 5326.5: pack to file (bitmap) 28.02(27.72+8.76) 5326.6: rev-list (commits) 0.42(0.36+0.06) 5326.7: rev-list (objects) 1.65(1.58+0.06) 5326.8: rev-list count with blob:none 0.26(0.21+0.05) 5326.9: rev-list count with blob:limit=1k 2.97(2.86+0.10) 5326.10: rev-list count with tree:0 0.25(0.20+0.04) 5326.11: simulated partial clone 5.65(5.49+0.16) 5326.13: clone (partial bitmap) 12.22(13.43+0.38) 5326.14: pack to file (partial bitmap) 30.05(31.57+7.25) 5326.15: rev-list with tree filter (partial bitmap) 0.24(0.20+0.04) There is slight overhead in "simulated clone", "simulated partial clone", and "clone (partial bitmap)". Unsurprisingly, that overhead is due to using the MIDX's reverse index to map between bit positions and MIDX positions. This can be reproduced by running "git repack -adb" along with "git multi-pack-index write --bitmap" in a large-ish repository. Then run: $ perf record -o pack.perf git -c core.multiPackIndex=false \ pack-objects --all --stdout >/dev/null </dev/null $ perf record -o midx.perf git -c core.multiPackIndex=true \ pack-objects --all --stdout >/dev/null </dev/null and compare the two with "perf diff -c delta -o 1 pack.perf midx.perf". The most notable results are below (the next largest positive delta is +0.14%): # Event 'cycles' # # Baseline Delta Shared Object Symbol # ........ ....... .................. .......................... # +5.86% git [.] nth_midxed_offset +5.24% git [.] nth_midxed_pack_int_id 3.45% +0.97% git [.] offset_to_pack_pos 3.30% +0.57% git [.] pack_pos_to_offset +0.30% git [.] pack_pos_to_midx Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>