path: root/contrib
AgeCommit message (Collapse)Author
2023-01-09use DUP_ARRAYRené Scharfe
Add a semantic patch for replace ALLOC_ARRAY+COPY_ARRAY with DUP_ARRAY to reduce code duplication and apply its results. Signed-off-by: René Scharfe <> Signed-off-by: Junio C Hamano <>
2022-12-19Merge branch 'aw/complete-case-insensitive'Junio C Hamano
Introduce a case insensitive mode to the Bash completion helpers. * aw/complete-case-insensitive: completion: add case-insensitive match of pseudorefs completion: add optional ignore-case when matching refs
2022-12-14Merge branch 'yn/git-jump-emacs'Junio C Hamano
"git jump" (in contrib/) learned to present the "quickfix list" to its standard output (instead of letting it consumed by the editor it invokes), and learned to also drive emacs/emacsclient. * yn/git-jump-emacs: git-jump: invoke emacs/emacsclient git-jump: move valid-mode check earlier git-jump: add an optional argument '--stdout'
2022-12-01Merge branch 'ab/fewer-the-index-macros'Junio C Hamano
Squelch warnings from Coccinelle * ab/fewer-the-index-macros: cocci: avoid "should ... be a metavariable" warnings
2022-11-30cocci: avoid "should ... be a metavariable" warningsÆvar Arnfjörð Bjarmason
Since [1] running "make coccicheck" has resulted in [2] being emitted to the *.log files for the "spatch" run, and in the case of "make coccicheck-test" we'd emit these to the user's terminal. Nothing was broken as a result, but let's refactor the relevant rules to eliminate the ambiguity between a possible variable and an identifier. 1. 0e6550a2c63 (cocci: add a index-compatibility.pending.cocci, 2022-11-19) 2. warning: line 257: should active_cache be a metavariable? warning: line 260: should active_cache_changed be a metavariable? warning: line 263: should active_cache_tree be a metavariable? warning: line 271: should active_nr be a metavariable? Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2022-11-30completion: add case-insensitive match of pseudorefsAlison Winters
When GIT_COMPLETION_IGNORE_CASE is set, also allow lowercase completion text like "head" to match uppercase HEAD and other pseudorefs. Signed-off-by: Alison Winters <> Signed-off-by: Junio C Hamano <>
2022-11-30completion: add optional ignore-case when matching refsAlison Winters
If GIT_COMPLETION_IGNORE_CASE is set, --ignore-case will be added to git for-each-ref calls so that refs can be matched case insensitively, even when running on case sensitive filesystems. Signed-off-by: Alison Winters <> Signed-off-by: Junio C Hamano <>
2022-11-28Merge branch 'ab/fewer-the-index-macros'Junio C Hamano
Progress on removing 'the_index' convenience wrappers. * ab/fewer-the-index-macros: cocci: apply "pending" index-compatibility to some "builtin/*.c" cache.h & test-tool.h: add & use "USE_THE_INDEX_VARIABLE" {builtin/*,repository}.c: add & use "USE_THE_INDEX_VARIABLE" cocci: apply "pending" index-compatibility to "t/helper/*.c" cocci & cache.h: apply variable section of "pending" index-compatibility cocci & cache.h: apply a selection of "pending" index-compatibility cocci: add a index-compatibility.pending.cocci read-cache API & users: make discard_index() return void cocci & cache.h: remove rarely used "the_index" compat macros builtin/{grep,log}.: don't define "USE_THE_INDEX_COMPATIBILITY_MACROS" cache.h: remove unused "the_index" compat macros
2022-11-27git-jump: invoke emacs/emacsclientYoichi Nakayama
It works with GIT_EDITOR="emacs", "emacsclient" or "emacsclient -t" Signed-off-by: Yoichi Nakayama <> Signed-off-by: Junio C Hamano <>
2022-11-27git-jump: move valid-mode check earlierJeff King
We check if the "mode" argument supplied by the user is valid by seeing if we have a mode_$mode function defined. But we don't do that until after creating the tempfile. This is wasteful (we create a tempfile but never use it), and makes it harder to add new options (the recent stdout option exits before creating the tempfile, so it misses the check and "git jump --stdout foo" will produce "git-jump: 92: mode_foo: not found" rather than the regular usage message). Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2022-11-27git-jump: add an optional argument '--stdout'Yoichi Nakayama
It can be used with M-x grep on Emacs. Signed-off-by: Yoichi Nakayama <> Signed-off-by: Junio C Hamano <>
2022-11-23Merge branch 'ab/coccicheck-incremental'Junio C Hamano
"make coccicheck" is time consuming. It has been made to run more incrementally. * ab/coccicheck-incremental: Makefile: don't create a ".build/.build/" for cocci, fix output spatchcache: add a ccache-alike for "spatch" cocci: run against a generated ALL.cocci cocci rules: remove <id>'s from rules that don't need them Makefile: copy contrib/coccinelle/*.cocci to build/ cocci: optimistically use COMPUTE_HEADER_DEPENDENCIES cocci: make "coccicheck" rule incremental cocci: split off "--all-includes" from SPATCH_FLAGS cocci: split off include-less "tests" from SPATCH_FLAGS Makefile: split off SPATCH_BATCH_SIZE comment from "cocci" heading Makefile: have "coccicheck" re-run if flags change Makefile: add ability to TAB-complete cocci *.patch rules cocci rules: remove unused "F" metavariable from pending rule Makefile + shared.mak: rename and indent $(QUIET_SPATCH_T)
2022-11-21cocci: apply "pending" index-compatibility to some "builtin/*.c"Ævar Arnfjörð Bjarmason
Apply "index-compatibility.pending.cocci" rule to "builtin/*", but exclude those where we conflict with in-flight changes. As a result some of them end up using only "the_index", so let's have them use the more narrow "USE_THE_INDEX_VARIABLE" rather than "USE_THE_INDEX_COMPATIBILITY_MACROS". Manual changes not made by coccinelle, that were squashed in: * Whitespace-wrap argument lists for repo_hold_locked_index(), repo_read_index_preload() and repo_refresh_and_write_index(), in cases where the line became too long after the transformation. * Change "refresh_cache()" to "refresh_index()" in a comment in "builtin/update-index.c". * For those whose call was followed by perror("<macro-name>"), change it to perror("<function-name>"), referring to the new function. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2022-11-21cocci & cache.h: apply variable section of "pending" index-compatibilityÆvar Arnfjörð Bjarmason
Mostly apply the part of "index-compatibility.pending.cocci" that renames the global variables like "active_nr", which are a shorthand to referencing (in that case) a struct member as "the_index.cache_nr". In doing so move more of "index-compatibility.pending.cocci" to "index-compatibility.cocci". In the case of "active_nr" we'd have a textual conflict with "ab/various-leak-fixes" in "next"[1]. Let's exclude that specific case while moving the rule over from "pending". 1. 407b94280f8 (commit: discard partial cache before (re-)reading it, 2022-11-08) Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2022-11-21cocci & cache.h: apply a selection of "pending" index-compatibilityÆvar Arnfjörð Bjarmason
Apply a selection of rules in "index-compatibility.pending.cocci" tree-wide, and in doing so migrate them to "index-compatibility.cocci". As in preceding commits the only manual changes here are the macro removals in "cache.h", and the update to the '*.cocci" rules. The rest of the C code changes are the result of applying those updated rules. Move rules for some rarely used cache compatibility macros from "index-compatibility.pending.cocci" to "index-compatibility.cocci" and apply them. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2022-11-21cocci: add a index-compatibility.pending.cocciÆvar Arnfjörð Bjarmason
Add a coccinelle rule which covers the rest of the macros guarded by "USE_THE_INDEX_COMPATIBILITY_MACROS" cache.h. If the result of this were applied it can be reduced down to just: #ifdef USE_THE_INDEX_COMPATIBILITY_MACROS extern struct index_state the_index; #endif But that patch is just under 2000 lines, so let's first add this as a "pending", and then incrementally pick changes from it in subsequent commits. In doing that we'll migrate rules from this "index-compatibility.pending.cocci" to the "index-compatibility.cocci" created in a preceding commit. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2022-11-21cocci & cache.h: remove rarely used "the_index" compat macrosÆvar Arnfjörð Bjarmason
Since 4aab5b46f44 (Make read-cache.c "the_index" free., 2007-04-01) we've been undergoing a slow migration away from these macros, but haven't made much progress since f8adbec9fea (cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch, 2019-01-24). Let's move forward a bit by changing the users of those macros that are rare enough that we can convert them in one go, and then remove the compatibility shim. The only manual change to the C code here is to "cache.h", the rest is all the result of applying the new "index-compatibility.cocci". Even though it's a one-off, let's keep the coccinelle rules for now. We'll extend them in subsequent commits, and this will help anything that's in-flight or out-of-tree to migrate. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2022-11-08Makefile & test-tool: replace "DC_SHA1" variable with a "define"Ævar Arnfjörð Bjarmason
Address the root cause of technical debt we've been carrying since sha1collisiondetection was made the default in [1]. In a preceding commit we narrowly fixed a bug where the "DC_SHA1" variable would be unset (in combination with "NO_APPLE_COMMON_CRYPTO=" on OSX), even though we had the sha1collisiondetection library enabled. But the only reason we needed to have such a user-exposed knob went away with [1], and it's been doing nothing useful since then. We don't care if you define DC_SHA1=*, we only care that you don't ask for any other SHA-1 implementation. If it turns out that you didn't, we'll use sha1collisiondetection, whether you had "DC_SHA1" set or not. As a result of this being confusing we had e.g. [2] for cmake and the recent [3] for ci/ setting "DC_SHA1" explicitly, even though this was always a NOOP. A much simpler way to do this is to stop having the Makefile and CMakeLists.txt set "DC_SHA1" to be picked up by the, let's instead add a trivial "test-tool sha1-is-sha1dc". It returns zero if we're using sha1collisiondetection, non-zero otherwise. 1. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17) 2. c4b2f41b5f5 (cmake: support for testing git with ctest, 2020-06-26) 3. 1ad5c3df35a (ci: use DC_SHA1=YesPlease on osx-clang job for CI, 2022-10-20) Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Taylor Blau <>
2022-11-03spatchcache: add a ccache-alike for "spatch"Ævar Arnfjörð Bjarmason
Add a rather trivial "spatchcache", with this running e.g.: make cocciclean make contrib/coccinelle/free.cocci.patch \ SPATCH=contrib/coccicheck/spatchcache \ SPATCH_FLAGS=--very-quiet Is cut down from ~20s to ~5s on my system. Much of that is either fixable shell overhead, or the around 40 files we "CANTCACHE" (see the implementation). This uses "redis" as a cache by default, but it's configurable. See the embedded documentation. This is *not* like ccache in that we won't cache failed spatch invocations, or those where spatch suggests changes for us. Those cases are so rare that I didn't think it was worth the bother, by far the most common case is that it has no suggested changes. We'll also refuse to cache any "spatch" invocation that has output on stderr, which means that "--very-quiet" must be added to "SPATCH_FLAGS". Because we narrow the cache to that we don't need to save away stdout, stderr & the exit code. We simply cache the cases where we had no suggested changes. Another benchmark is to compare this with the previous SPATCH_BATCH_SIZE=N, as noted in [1]. Before this (on my 8 core system) running: make clean; time make contrib/coccinelle/array.cocci.patch SPATCH_BATCH_SIZE=0 Would take 33s, but with the preceding changes running without this "spatchcache" is slightly slower, or around 35s: make clean; time make contrib/coccinelle/array.cocci.patch Now doing the same with SPATCH=contrib/coccinelle/spatchcache will take around 6s, but we'll need to compile the *.o files first to take full advantage of it (which can be fast with "ccache"): make clean; make; time make contrib/coccinelle/array.cocci.patch SPATCH=contrib/coccinelle/spatchcache 1. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Taylor Blau <>
2022-11-03cocci: run against a generated ALL.cocciÆvar Arnfjörð Bjarmason
The preceding commits to make the "coccicheck" target incremental made it slower in some cases. As an optimization let's not have the many=many mapping of <*.cocci>=<*.[ch]>, but instead concat the <*.cocci> into an ALL.cocci, and then run one-to-many ALL.cocci=<*.[ch]>. A "make coccicheck" is now around 2x as fast as it was on "master", and around 1.5x as fast as the preceding change to make the run incremental: $ git hyperfine -L rev origin/master,HEAD~,HEAD -p 'make clean' 'make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' -r 3 Benchmark 1: make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'origin/master Time (mean ± σ): 4.258 s ± 0.015 s [User: 27.432 s, System: 1.532 s] Range (min … max): 4.241 s … 4.268 s 3 runs Benchmark 2: make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'HEAD~ Time (mean ± σ): 5.365 s ± 0.079 s [User: 36.899 s, System: 1.810 s] Range (min … max): 5.281 s … 5.436 s 3 runs Benchmark 3: make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'HEAD Time (mean ± σ): 2.725 s ± 0.063 s [User: 14.796 s, System: 0.233 s] Range (min … max): 2.667 s … 2.792 s 3 runs Summary 'make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'HEAD' ran 1.56 ± 0.04 times faster than 'make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'origin/master' 1.97 ± 0.05 times faster than 'make coccicheck SPATCH=spatch COCCI_SOURCES="$(echo $(ls o*.c builtin/h*.c))"' in 'HEAD~' This can be turned off with SPATCH_CONCAT_COCCI, but as the beneficiaries of "SPATCH_CONCAT_COCCI=" would mainly be those developing the *.cocci rules themselves, let's leave this optimization on by default. For more information see my "Optimizing *.cocci rules by concat'ing them" (<>) on the mailing list. This potentially changes the results of our *.cocci rules, but as noted in that discussion it should be safe for our use. We don't name rules, or if we do their names don't conflict across our *.cocci files. To the extent that we'd have any inter-dependencies between rules this doesn't make that worse, as we'd have them now if we ran "make coccicheck", applied the results, and would then have (due to hypothetical interdependencies) suggested changes on the subsequent "make coccicheck". Our "coccicheck-test" target makes use of the ALL.cocci when running tests, e.g. when testing unused.{c,out} we test it against ALL.cocci, not unused.cocci. We thus assert (to the extent that we have test coverage) that this concatenation doesn't change the expected results of running these rules. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Taylor Blau <>
2022-11-03cocci rules: remove <id>'s from rules that don't need themÆvar Arnfjörð Bjarmason
The <id> in the <rulename> part of the coccinelle syntax[1] is for our purposes there to declares if we have inter-dependencies between different rules. But such <id>'s must be unique within a given semantic patch file. As we'll be processing a concatenated version of our rules in the subsequent commit let's remove these names. They weren't being used for the semantic patches themselves, and equated to a short comment about the rule. Both the filename and context of the rules makes it clear what they're doing, so we're not gaining anything from keeping these. Retaining them goes against recommendations that "contrib/coccinelle/README" will be making in the subsequent commit. This leaves only one named rule in our sources, where it's needed for a "<id> <-> <extends> <id>" relationship: $ git -P grep '^@ ' -- contrib/coccinelle/ contrib/coccinelle/swap.cocci:@ swap @ contrib/coccinelle/swap.cocci:@ extends swap @ 1. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Taylor Blau <>
2022-11-03cocci: optimistically use COMPUTE_HEADER_DEPENDENCIESÆvar Arnfjörð Bjarmason
Improve the incremental rebuilding support of "coccicheck" by piggy-backing on the computed dependency information of the corresponding *.o file, rather than rebuilding all <RULE>/<FILE> pairs if either their corresponding file changes, or if any header changes. This in effect uses the same method that the "sparse" target was made to use in c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY, 2021-09-23), except that the dependency on the *.o file isn't a hard one, we check with $(wildcard) if the *.o file exists, and if so we'll depend on it. This means that the common case of: make make coccicheck Will benefit from incremental rebuilding, now changing e.g. a header will only re-run "spatch" on those those *.c files that make use of it: By depending on the *.o we piggy-back on COMPUTE_HEADER_DEPENDENCIES. See c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY, 2021-09-23) for prior art of doing that for the *.sp files. E.g.: make contrib/coccinelle/free.cocci.patch make -W column.h contrib/coccinelle/free.cocci.patch Will take around 15 seconds for the second command on my 8 core box if I didn't run "make" beforehand to create the *.o files. But around 2 seconds if I did and we have those "*.o" files. Notes about the approach of piggy-backing on *.o for dependencies: * It *is* a trade-off since we'll pay the extra cost of running the C compiler, but we're probably doing that anyway. The compiler is much faster than "spatch", so even though we need to re-compile the *.o to create the dependency info for the *.c for "spatch" it's faster (especially if using "ccache"). * There *are* use-cases where some would like to have *.o files around, but to have the "make coccicheck" ignore them. See: For those users a: make make coccicheck SPATCH_USE_O_DEPENDENCIES= Will avoid considering the *.o files. * If that *.o file doesn't exist we'll depend on an intermediate file of ours which in turn depends on $(FOUND_H_SOURCES). This covers both an initial build, or where "coccicheck" is run without running "all" beforehand, and because we run "coccicheck" on e.g. files in compat/* that we don't know how to build unless the requisite flag was provided to the Makefile. Most of the runtime of "incremental" runs is now spent on various compat/* files, i.e. we conditionally add files to COMPAT_OBJS, and therefore conflate whether we *can* compile an object and generate dependency information for it with whether we'd like to link it into our binary. Before this change the distinction didn't matter, but now one way to make this even faster on incremental builds would be to peel those concerns apart so that we can see that e.g. compat/mmap.c doesn't depend on column.h. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Taylor Blau <>
2022-11-03cocci: make "coccicheck" rule incrementalÆvar Arnfjörð Bjarmason
Optimize the very slow "coccicheck" target to take advantage of incremental rebuilding, and fix outstanding dependency problems with the existing rule. The rule is now faster both on the initial run as we can make better use of GNU make's parallelism than the old ad-hoc combination of make's parallelism combined with $(SPATCH_BATCH_SIZE) and/or the "--jobs" argument to "spatch(1)". It also makes us *much* faster when incrementally building, it's now viable to "make coccicheck" as topic branches are merged down. The rule didn't use FORCE (or its equivalents) before, so a: make coccicheck make coccicheck Would report nothing to do on the second iteration. But all of our patch output depended on all $(COCCI_SOURCES) files, therefore e.g.: make -W grep.c coccicheck Would do a full re-run, i.e. a a change in a single file would force us to do a full re-run. The reason for this (not the initial rationale, but my analysis) is: * Since we create a single "*.cocci.patch+" we don't know where to pick up where we left off, or how to incrementally merge e.g. a "grep.c" change with an existing *.cocci.patch. * We've been carrying forward the dependency on the *.c files since 63f0a758a06 (add coccicheck make target, 2016-09-15) the rule was initially added as a sort of poor man's dependency discovery. As we don't include other *.c files depending on other *.c files has always been broken, as could be trivially demonstrated e.g. with: make coccicheck make -W strbuf.h coccicheck However, depending on the corresponding *.c files has been doing something, namely that *if* an API change modified both *.c and *.h files we'd catch the change to the *.h we care about via the *.c being changed. For API changes that happened only via *.h files we'd do the wrong thing before this change, but e.g. for function additions (not "static inline" ones) catch the *.h change by proxy. Now we'll instead: * Create a <RULE>/<FILE> pair in the .build directory, E.g. for swap.cocci and grep.c we'll create .build/contrib/coccinelle/swap.cocci.patch/grep.c. That file is the diff we'll apply for that <RULE>-<FILE> combination, if there's no changes to me made (the common case) it'll be an empty file. * Our generated *.patch file (e.g. contrib/coccinelle/swap.cocci.patch) is now a simple "cat $^" of all of all of the <RULE>/<FILE> files for a given <RULE>. In the case discussed above of "grep.c" being changed we'll do the full "cat" every time, so they resulting *.cocci.patch will always be correct and up-to-date, even if it's "incrementally updated". See 1cc0425a27c (Makefile: have "make pot" not "reset --hard", 2022-05-26) for another recent rule that used that technique. As before we'll: * End up generating a contrib/coccinelle/swap.cocci.patch, if we "fail" by creating a non-empty patch we'll still exit with a zero exit code. Arguably we should move to a more Makefile-native way of doing this, i.e. fail early, and if we want all of the "failed" changes we can use "make -k", but as the current "ci/" expects us to behave this way let's keep the existing behavior of exhaustively discovering all cocci changes, and only failing if spatch itself errors out. Further implementation details & notes: * Before this change running "make coccicheck" would by default end up pegging just one CPU at the very end for a while, usually as we'd finish whichever *.cocci rule was the most expensive. This could be mitigated by combining "make -jN" with SPATCH_BATCH_SIZE, see 960154b9c17 (coccicheck: optionally batch spatch invocations, 2019-05-06). There will be cases where getting rid of "SPATCH_BATCH_SIZE" makes things worse, but a from-scratch "make coccicheck" with the default of SPATCH_BATCH_SIZE=1 (and tweaking it doesn't make a difference) is faster (~3m36s v.s. ~3m56s) with this approach, as we can feed the CPU more work in a less staggered way. * Getting rid of "SPATCH_BATCH_SIZE" particularly helps in cases where the default of 1 yields parallelism under "make coccicheck", but then running e.g.: make -W contrib/coccinelle/swap.cocci coccicheck I.e. before that would use only one CPU core, until the user remembered to adjust "SPATCH_BATCH_SIZE" differently than the setting that makes sense when doing a non-incremental run of "make coccicheck". * Before the "make coccicheck" rule would have to clean "contrib/coccinelle/*.cocci.patch*", since we'd create "*+" and "*.log" files there. Now those are created in .build/contrib/coccinelle/, which is covered by the "cocciclean" rule already. Outstanding issues & future work: * We could get rid of "--all-includes" in favor of manually specifying a list of includes to give to "spatch(1)". As noted upthread of [1] a naïve removal of "--all-includes" will result in broken *.cocci patches, but if we know the exhaustive list of includes via COMPUTE_HEADER_DEPENDENCIES we don't need to re-scan for them, we could grab the headers to include from the .depend.d/<file>.o.d and supply them with the "--include" option to spatch(1).q 1. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Taylor Blau <>
2022-11-03cocci rules: remove unused "F" metavariable from pending ruleÆvar Arnfjörð Bjarmason
Fix an issue with a rule added in 9b45f499818 (object-store: prepare has_{sha1, object}_file to handle any repo, 2018-11-13). We've been spewing out this warning into our $@.log since that rule was added: warning: rule starting on line 21: metavariable F not used in the - or context code We should do a better job of scouring our coccinelle log files for such issues, but for now let's fix this as a one-off. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Taylor Blau <>
2022-10-31Merge branch 'pb/subtree-split-and-merge-after-squashing-tag-fix'Taylor Blau
A bugfix to "git subtree" in its split and merge features. * pb/subtree-split-and-merge-after-squashing-tag-fix: subtree: fix split after annotated tag was squashed merged subtree: fix squash merging after annotated tag was squashed merged subtree: process 'git-subtree-split' trailer in separate function subtree: use named variables instead of "$@" in cmd_pull subtree: define a variable before its first use in 'find_latest_squash' subtree: prefix die messages with 'fatal' subtree: add 'die_incompatible_opt' function to reduce duplication subtree: use 'git rev-parse --verify [--quiet]' for better error messages test-lib-functions: mark 'test_commit' variables as 'local'
2022-10-27Merge branch 'js/cmake-updates'Junio C Hamano
Update to build procedure with VS using CMake/CTest. * js/cmake-updates: cmake: increase time-out for a long-running test cmake: avoid editing t/ add -p: avoid ambiguous signed/unsigned comparison cmake: copy the merge tools for testing cmake: make it easier to diagnose regressions in CTest runs
2022-10-21subtree: fix split after annotated tag was squashed mergedPhilippe Blain
The previous commit fixed a failure in 'git subtree merge --squash' when the previous squash-merge merged an annotated tag of the subtree repository which is missing locally. The same failure happens in 'git subtree split', either directly or when called by 'git subtree push', under the same circumstances: 'cmd_split' invokes 'find_existing_splits', which loops through previous commits and invokes 'git rev-parse' (via 'process_subtree_split_trailer') on the value of any 'git subtree-split' trailer it finds. This fails if this value is the hash of an annotated tag which is missing locally. Add a new optional argument 'repository' to 'cmd_split' and 'find_existing_splits', and invoke 'cmd_split' with that argument from 'cmd_push'. This allows 'process_subtree_split_trailer' to try to fetch the missing tag from the 'repository' if it's not available locally, mirroring the new behaviour of 'git subtree pull' and 'git subtree merge'. Signed-off-by: Philippe Blain <> Signed-off-by: Junio C Hamano <>
2022-10-21subtree: fix squash merging after annotated tag was squashed mergedPhilippe Blain
When 'git subtree merge --squash $ref' is invoked, either directly or through 'git subtree pull --squash $repo $ref', the code looks for the latest squash merge of the subtree in order to create the new merge commit as a child of the previous squash merge. This search is done in function 'process_subtree_split_trailer', invoked by 'find_latest_squash', which looks for the most recent commit with a 'git-subtree-split' trailer; that trailer's value is the object name in the subtree repository of the ref that was last squash-merged. The function verifies that this object is present locally with 'git rev-parse', and aborts if it's not. The hash referenced by the 'git-subtree-split' trailer is guaranteed to correspond to a commit since it is the result of running 'git rev-parse -q --verify "$1^{commit}"' on the first argument of 'cmd_merge' (this corresponds to 'rev' in 'cmd_merge' which is passed through to 'new_squash_commit' and 'squash_msg'). But this is only the case since e4f8baa88a (subtree: parse revs in individual cmd_ functions, 2021-04-27), which went into Git 2.32. Before that commit, 'cmd_merge' verified the revision it was given using 'git rev-parse --revs-only "$@"'. Such an invocation, when fed the name of an annotated tag, would return the hash of the tag, not of the commit referenced by the tag. This leads to a failure in 'find_latest_squash' when squash-merging if the most recent squash-merge merged an annotated tag of the subtree repository, using a pre-2.32 version of 'git subtree', unless that previous annotated tag is present locally (which is not usually the case). We can fix this by fetching the object directly by its hash in 'process_subtree_split_trailer' when 'git rev-parse' fails, but in order to do so we need to know the name or URL of the subtree repository. This is not possible in general for 'git subtree merge', but is easy when it is invoked through 'git subtree pull' since in that case the subtree repository is passed by the user at the command line. Allow the 'git subtree pull' scenario to work out-of-the-box by adding an optional 'repository' argument to functions 'cmd_merge', 'find_latest_squash' and 'process_subtree_split_trailer', and invoke 'cmd_merge' with that 'repository' argument in 'cmd_pull'. If 'repository' is absent in 'process_subtree_split_trailer', instruct the user to try fetching the missing object directly. Signed-off-by: Philippe Blain <> Signed-off-by: Junio C Hamano <>
2022-10-21subtree: process 'git-subtree-split' trailer in separate functionPhilippe Blain
Both functions 'find_latest_squash' (called by 'git subtree merge --squash' and 'git subtree split --rejoin') and 'find_existing_splits' (called by git 'subtree split') loop through commits that have a 'git-subtree-dir' trailer, and then process the 'git-subtree-mainline' and 'git-subtree-split' trailers for those commits. The processing done for the 'git-subtree-split' trailer is simple: we check if the object exists with 'rev-parse' and set the variable 'sub' to the object name, or we die if the object does not exist. In a future commit we will add more steps to the processing of this trailer in order to make the code more robust. To reduce code duplication, move the processing of the 'git-subtree-split' trailer to a dedicated function, 'process_subtree_split_trailer'. Signed-off-by: Philippe Blain <> Signed-off-by: Junio C Hamano <>
2022-10-21subtree: use named variables instead of "$@" in cmd_pullPhilippe Blain
'cmd_pull' already checks that only two arguments are given, 'repository' and 'ref'. Define variables with these names instead of using the positional parameter $2 and "$@". This will allow a subsequent commit to pass 'repository' to 'cmd_merge'. Signed-off-by: Philippe Blain <> Signed-off-by: Junio C Hamano <>
2022-10-21subtree: define a variable before its first use in 'find_latest_squash'Philippe Blain
The function 'find_latest_squash' takes a single argument, 'dir', but a debug statement uses this variable before it takes its value from $1. This statement thus gets the value of 'dir' from the calling function, which currently is the same as the 'dir' argument, so it works but it is confusing. Move the definition of 'dir' before its first use. Signed-off-by: Philippe Blain <> Signed-off-by: Junio C Hamano <>
2022-10-21subtree: prefix die messages with 'fatal'Philippe Blain
Just as was done in 0008d12284 (submodule: prefix die messages with 'fatal', 2021-07-10) for '', make the 'die' messages outputed by '' more in line with the rest of the code base by prefixing them with "fatal: ", and do not capitalize their first letter. Signed-off-by: Philippe Blain <> Signed-off-by: Junio C Hamano <>
2022-10-21subtree: add 'die_incompatible_opt' function to reduce duplicationPhilippe Blain
9a3e3ca2ba (subtree: be stricter about validating flags, 2021-04-27) added validation code to check that options given to 'git subtree <cmd>' made sense with the command being used. Refactor these checks by adding a 'die_incompatible_opt' function to reduce code duplication. Signed-off-by: Philippe Blain <> Signed-off-by: Junio C Hamano <>
2022-10-21subtree: use 'git rev-parse --verify [--quiet]' for better error messagesPhilippe Blain
There are three occurences of 'git rev-parse <rev>' in '' where the command expects a revision and the script dies or exits if the revision can't be found. In that case, the error message from 'git rev-parse' is: $ git rev-parse <bad rev> <bad rev> fatal: ambiguous argument '<bad rev>': unknown revision or path not in the working tree. Use '--' to separate paths from revisions, like this: 'git <command> [<revision>...] -- [<file>...]' This is a little confusing to the user, since this error message is outputed by 'git subtree'. At these points in the script, we know that we are looking for a single revision, so be explicit by using '--verify', resulting in a little better error message: $ git rev-parse --verify <bad rev> fatal: Needed a single revision In the two occurences where we 'die' if 'git rev-parse' fails, 'git subtree' outputs "could not rev-parse split hash $b from commit $sq", so we actually do not need the supplementary error message from 'git rev-parse'; add '--quiet' to silence it. In the third occurence, we 'exit', so keep the error message from 'git rev-parse'. Note that this messsage is still suboptimal since it can be understood to mean that 'git rev-parse' did not receive a single revision as argument, which is not the case here: the command did receive a single revision, but the revision is not resolvable to an available object. The alternative would be to use '--' after the revision, as suggested by the first error message, resulting in a clearer error message: $ git rev-parse <bad rev> -- fatal: bad revision '<bad rev>' Unfortunately we can't use that syntax because in the more common case of the revision resolving to a known object, the command outputs the object's hash, a newline, and the dashdash, which breaks the 'git subtree' script. Signed-off-by: Philippe Blain <> Signed-off-by: Junio C Hamano <>
2022-10-19cmake: increase time-out for a long-running testJohannes Schindelin
As suggested in, t7112 can run for well over one hour, which seems to be the default maximum run time at least when running CTest-based tests in Visual Studio. Let's increase the time-out as a stop gap to unblock developers wishing to run Git's test suite in Visual Studio. Note: The actual run time is highly dependent on the circumstances. For example, in Git's CI runs, the Windows-based tests typically take a bit over 5 minutes to run. CI runs have the added benefit that Windows Defender (the common anti-malware scanner on Windows) is turned off, something many developers are not at liberty to do on their work stations. When Defender is turned on, even on this developer's high-end Ryzen system, t7112 takes over 15 minutes to run. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2022-10-19cmake: avoid editing t/test-lib.shJohannes Schindelin
In 7f5397a07c6c (cmake: support for testing git when building out of the source tree, 2020-06-26), we implemented support for running Git's test scripts even after building Git in a different directory than the source directory. The way we did this was to edit the file `t/` to override `GIT_BUILD_DIR` to point somewhere else than the parent of the `t/` directory. This is unideal because it always leaves a tracked file marked as modified, and it is all too easy to commit that change by mistake. Let's change the strategy by teaching `t/` to detect the presence of a file called `GIT-BUILD-DIR` in the source directory. If it exists, the contents are interpreted as the location to the _actual_ build directory. We then write this file as part of the CTest definition. To support building Git via a regular `make` invocation after building it using CMake, we ensure that the `GIT-BUILD-DIR` file is deleted (for convenience, this is done as part of the Makefile rule that is already run with every `make` invocation to ensure that `GIT-BUILD-OPTIONS` is up to date). Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2022-10-19cmake: copy the merge tools for testingJohannes Schindelin
Even when running the tests via CTest, t7609 and t7610 rely on more than only a few mergetools to be copied to the build directory. Let's make it so. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2022-10-19cmake: make it easier to diagnose regressions in CTest runsJohannes Schindelin
When a test script fails in Git's test suite, the usual course of action is to re-run it using options to increase the verbosity of the output, e.g. `-v` and `-x`. Like in Git's CI runs, when running the tests in Visual Studio via the CTest route, it is cumbersome or at least requires a very unintuitive approach to pass options to the test scripts: the CMakeLists.txt file would have to be modified, passing the desired options to _all_ test scripts, and then the CMake Cache would have to be reconfigured before running the test in question individually. Unintuitive at best, and opposite to the niceties IDE users expect. So let's just pass those options by default: This will not clutter any output window but the log that is written to a log file will have information necessary to figure out test failures. While at it, also imitate what the Windows jobs in Git's CI runs do to accelerate running the test scripts: pass the `--no-bin-wrappers` and `--no-chain-lint` options. This makes the test runs noticeably faster because the `bin-wrappers/` scripts as well as the `chain-lint` code make heavy use of POSIX shell scripting, which is really, really slow on Windows due to the need to emulate POSIX behavior via the MSYS2 runtime. In a test by Eric Sunshine, it added two minutes (!) just to perform the chain-lint task. The idea of adding a CMake config option (á la `GIT_TEST_OPTS`) was considered during the development of this patch, but then dropped: such a setting is global, across _all_ tests, where e.g. `--run=...` would not make sense. Users wishing to override these new defaults are better advised running the test script manually, in a Git Bash, with full control over the command line. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2022-10-17Merge branch 'ed/fsmonitor-on-networked-macos'Junio C Hamano
By default, use of fsmonitor on a repository on networked filesystem is disabled. Add knobs to make it workable on macOS. * ed/fsmonitor-on-networked-macos: fsmonitor: fix leak of warning message fsmonitor: add documentation for allowRemote and socketDir options fsmonitor: check for compatability before communicating with fsmonitor fsmonitor: deal with synthetic firmlinks on macOS fsmonitor: avoid socket location check if using hook fsmonitor: relocate socket file if .git directory is remote fsmonitor: refactor filesystem checks to common interface
2022-10-05fsmonitor: relocate socket file if .git directory is remoteEric DeCosta
If the .git directory is on a remote filesystem, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME. Signed-off-by: Eric DeCosta <> Signed-off-by: Junio C Hamano <>
2022-10-05fsmonitor: refactor filesystem checks to common interfaceEric DeCosta
Provide a common interface for getting basic filesystem information including filesystem type and whether the filesystem is remote. Refactor existing code for getting basic filesystem info and detecting remote file systems to the new interface. Refactor filesystem checks to leverage new interface. For macOS, error-out if the Unix Domain socket (UDS) file is on a remote filesystem. Signed-off-by: Eric DeCosta <> Signed-off-by: Junio C Hamano <>
2022-09-22osxkeychain: clarify that we ignore unknown linesMatthew John Cheetham
Like in all the other credential helpers, the osxkeychain helper ignores unknown credential lines. Add a comment (a la the other helpers) to make it clear and explicit that this is the desired behaviour. Signed-off-by: Matthew John Cheetham <> Signed-off-by: Junio C Hamano <>
2022-09-22netrc: ignore unknown lines (do not die)Matthew John Cheetham
Contrary to the documentation on credential helpers, as well as the help text for git-credential-netrc itself, this helper will `die` when presented with an unknown property/attribute/token. Correct the behaviour here by skipping and ignoring any tokens that are unknown. This means all helpers in the tree are consistent and ignore any unknown credential properties/attributes. Signed-off-by: Matthew John Cheetham <> Signed-off-by: Junio C Hamano <>
2022-09-22wincred: ignore unknown lines (do not die)Matthew John Cheetham
It is the expectation that credential helpers be liberal in what they accept and conservative in what they return, to allow for future growth and evolution of the protocol/interaction. All of the other helpers (store, cache, osxkeychain, libsecret, gnome-keyring) except `netrc` currently ignore any credential lines that are not recognised, whereas the Windows helper (wincred) instead dies. Fix the discrepancy and ignore unknown lines in the wincred helper. Signed-off-by: Matthew John Cheetham <> Signed-off-by: Junio C Hamano <>
2022-09-21t/Makefile: remove 'test-results' on 'make clean'SZEDER Gábor
The 't/test-results' directory and its contents are by-products of the test process, so 'make clean' should remove them, but, alas, this has been broken since fee65b194d (t/Makefile: don't remove test-results in "clean-except-prove-cache", 2022-07-28). The 'clean' target in 't/Makefile' was not directly responsible for removing the 'test-results' directory, but relied on its dependency 'clean-except-prove-cache' to do that [1]. ee65b194d broke this, because it only removed the 'rm -r test-results' command from the 'clean-except-prove-cache' target instead of moving it to the 'clean' target, resulting in stray 't/test-results' directories. Add that missing cleanup command to 't/Makefile', and to all sub-Makefiles touched by that commit as well. [1] 60f26f6348 (t/Makefile: retain cache t/.prove across prove runs, 2012-05-02) Signed-off-by: SZEDER Gábor <> Signed-off-by: Junio C Hamano <>
2022-09-19Merge branch 'vd/scalar-to-main'Junio C Hamano
Hoist the remainder of "scalar" out of contrib/ to the main part of the codebase. * vd/scalar-to-main: Documentation/technical: include Scalar technical doc t/perf: add 'GIT_PERF_USE_SCALAR' run option t/perf: add Scalar performance tests scalar-clone: add test coverage scalar: add to 'git help -a' command list scalar: implement the `help` subcommand git help: special-case `scalar` scalar: include in standard Git build & installation scalar: fix command documentation section header
2022-09-02scalar: include in standard Git build & installationVictoria Dye
Move 'scalar' out of 'contrib/' and into the root of the Git tree. The goal of this change is to build 'scalar' as part of the standard Git build & install processes. This patch includes both the physical move of Scalar's files out of 'contrib/' ('scalar.c', 'scalar.txt', and ''), and the changes to the build definitions in 'Makefile' and 'CMakelists.txt' to accommodate the new program. At a high level, Scalar is built so that: - there is a 'scalar-objs' target (similar to those created in 029bac01a8 (Makefile: add {program,xdiff,test,git,fuzz}-objs & objects targets, 2021-02-23)) for debugging purposes. - it appears in the root of the install directory (rather than the gitexecdir). - it is included in the 'bin-wrappers/' directory for use in tests. - it receives a platform-specific executable suffix (e.g., '.exe'), if applicable. - 'scalar.txt' is installed as 'man1' documentation. - the 'clean' target removes the 'scalar' executable. Additionally, update the root level '.gitignore' file to ignore the Scalar executable. Signed-off-by: Victoria Dye <> Signed-off-by: Junio C Hamano <>
2022-09-02scalar: fix command documentation section headerVictoria Dye
Rename the last section header in 'contrib/scalar/scalar.txt' from "Scalar" to "GIT". The linting rules of the 'documentation' CI build enforce the existence of a "GIT" section in command documentation. Although 'scalar.txt' is not yet checked, it will be in a future patch. Here, changing the header name is more appropriate than making a Scalar-specific exception to the linting rule. The existing "Scalar" section contains only a link back to the main Git documentation, essentially the same as the "GIT" section in builtin documentation. Changing the section name further clarifies the Scalar-Git association and maintains consistency with the rest of Git. Signed-off-by: Victoria Dye <> Signed-off-by: Junio C Hamano <>
2022-09-01test-lib: replace chainlint.sed with chainlint.plEric Sunshine
By automatically invoking chainlint.sed upon each test it runs, `test_run_` in ensures that broken &&-chains will be detected early as tests are modified or new are tests created since it is typical to run a test script manually (i.e. `./`) during test development. Now that the implementation of is complete, modify to invoke it automatically instead of chainlint.sed each time a test script is run. This change reduces the number of "linter" invocations from 26800+ (once per test run) down to 1050+ (once per test script), however, a subsequent change will drop the number of invocations to 1 per `make test`, thus fully realizing the benefit of the new linter. Note that the "magic exit code 117" &&-chain checker added by bb79af9d09 (t/test-lib: introduce --chain-lint option, 2015-03-20) which is built into t/ is retained since it has near zero-cost and (theoretically) may catch a broken &&-chain not caught by Signed-off-by: Eric Sunshine <> Signed-off-by: Junio C Hamano <>
2022-08-29Merge branch 'jd/prompt-show-conflict'Junio C Hamano
The bash prompt (in contrib/) learned to optionally indicate when the index is unmerged. * jd/prompt-show-conflict: git-prompt: show presence of unresolved conflicts at command prompt