summaryrefslogtreecommitdiff
path: root/builtin/submodule--helper.c
AgeCommit message (Collapse)Author
2016-06-01submodule: remove bashism from shell scriptStefan Beller
Junio pointed out `relative_path` was using bashisms via the local variables. As the longer term goal is to rewrite most of the submodule code in C, do it now. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-01submodule--helper: offer a consistent APIStefan Beller
In 48308681 (2016-02-29, git submodule update: have a dedicated helper for cloning), the helper communicated errors back only via exit code, and dance with printing '#unmatched' in case of error was left to git-submodule.sh as it uses the output of the helper and pipes it into shell commands. This change makes the helper consistent by never printing '#unmatched' in the helper but always handling these piping issues in the actual shell script. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-17Merge branch 'jk/submodule-c-credential'Junio C Hamano
An earlier addition of "sanitize_submodule_env" with 14111fc4 (git: submodule honor -c credential.* from command line, 2016-02-29) turned out to be a convoluted no-op; implement what it wanted to do correctly, and stop filtering settings given via "git -c var=val". * jk/submodule-c-credential: submodule: stop sanitizing config options submodule: use prepare_submodule_repo_env consistently submodule--helper: move config-sanitizing to submodule.c submodule: export sanitized GIT_CONFIG_PARAMETERS t5550: break submodule config test into multiple sub-tests t5550: fix typo in $HTTPD_URL
2016-05-17Merge branch 'sb/submodule-init'Junio C Hamano
Update of "git submodule" to move pieces of logic to C continues. * sb/submodule-init: submodule init: redirect stdout to stderr submodule--helper update-clone: abort gracefully on missing .gitmodules submodule init: fail gracefully with a missing .gitmodules file submodule: port init from shell to C submodule: port resolve_relative_url from shell to C
2016-05-06submodule: stop sanitizing config optionsJeff King
The point of having a whitelist of command-line config options to pass to submodules was two-fold: 1. It prevented obvious nonsense like using core.worktree for multiple repos. 2. It could prevent surprise when the user did not mean for the options to leak to the submodules (e.g., http.sslverify=false). For case 1, the answer is mostly "if it hurts, don't do that". For case 2, we can note that any such example has a matching inverted surprise (e.g., a user who meant http.sslverify=true to apply everywhere, but it didn't). So this whitelist is probably not giving us any benefit, and is already creating a hassle as people propose things to put on it. Let's just drop it entirely. Note that we still need to keep a special code path for "prepare the submodule environment", because we still have to take care to pass through $GIT_CONFIG_PARAMETERS (and block the rest of the repo-specific environment variables). We can do this easily from within the submodule shell script, which lets us drop the submodule--helper option entirely (and it's OK to do so because as a "--" program, it is entirely a private implementation detail). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-03submodule init: redirect stdout to stderrStefan Beller
Reroute the output of stdout to stderr as it is just informative messages, not to be consumed by machines. This should not regress any scripts that try to parse the current output, as the output is already internationalized and therefore unstable. We want to init submodules from the helper for `submodule update` in a later patch and the stdout output of said helper is consumed by the parts of `submodule update` which are still written in shell. So we have to be careful which messages are on stdout. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-29submodule--helper update-clone: abort gracefully on missing .gitmodulesStefan Beller
When there is no .gitmodules file availabe to initialize a submodule from, `submodule_from_path` just returns NULL. We need to check for that and abort gracefully. When `git submodule update` was implemented in shell, this error out with the warning Submodule path '%s' not initialized Maybe you want to use 'update --init'? Replicate that behavior for now instead of crashing. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-29submodule init: fail gracefully with a missing .gitmodules fileStefan Beller
When there is no .gitmodules file availabe to initialize a submodule from, `submodule_from_path` just returns NULL. We need to check for that and abort gracefully. When `submodule init` was implemented in shell, a missing .gitmodules file would result in an error message No url found for submodule path '%s' in .gitmodules Replicate that error message for now. When the .gitmodules file is missing we can probably fail even earlier for all of the submodules with an improved error message. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-28submodule--helper: move config-sanitizing to submodule.cJeff King
These functions should be used by any code which spawns a submodule process, which may happen in submodule.c (e.g., for spawning fetch). Let's move them there and make them public so that submodule--helper can continue to use them. Since they're now public, let's also provide a basic overview of their intended use. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-22Merge branch 'sb/submodule-helper-clone-regression-fix'Junio C Hamano
A partial rewrite of "git submodule" in the 2.7 timeframe changed the way the gitdir: pointer in the submodules point at the real repository location to use absolute paths by accident. This has been corrected. * sb/submodule-helper-clone-regression-fix: submodule--helper, module_clone: catch fprintf failure submodule--helper: do not borrow absolute_path() result for too long submodule--helper, module_clone: always operate on absolute paths submodule--helper clone: create the submodule path just once submodule--helper: fix potential NULL-dereference recursive submodules: test for relative paths
2016-04-17submodule: port init from shell to CStefan Beller
By having the `submodule init` functionality in C, we can reference it easier from other parts in the code in later patches. The code is split up to have one function to initialize one submodule and a calling function that takes care of the rest, such as argument handling and translating the arguments to the paths of the submodules. This is the first submodule subcommand that is fully converted to C except for the usage string, so this is actually removing a call to the `submodule--helper list` function, which is supposed to be used in this transition. Instead we'll make a direct call to `module_list_compute`. An explanation why we need to edit the prefixes in cmd_update in git-submodule.sh in this patch: By having no processing in the shell part, we need to convey the notion of wt_prefix and prefix to the C parts, which former patches punted on and did the processing of displaying path in the shell. `wt_prefix` used to hold the path from the repository root to the current directory, e.g. wt_prefix would be t/ if the user invoked the `git submodule` command in ~/repo/t and ~repo is the GIT_DIR. `prefix` used to hold the relative path from the repository root to the operation, e.g. if you have recursive submodules, the shell script would modify the `prefix` in each recursive step by adding the submodule path. We will pass `wt_prefix` into the C helper via `git -C <dir>` as that will setup git in the directory the user actually called git-submodule.sh from. The `prefix` will be passed in via the `--prefix` option. Having `prefix` and `wt_prefix` relative to the GIT_DIR of the calling superproject is unfortunate with this patch as the C code doesn't know about a possible recursion from a superproject via `submodule update --init --recursive`. To fix this, we change the meaning of `wt_prefix` to point to the current project instead of the superproject and `prefix` to include any relative paths issues in the superproject. That way `prefix` will become the leading part for displaying paths and `wt_prefix` will be empty in recursive calls for now. The new notion of `wt_prefix` and `prefix` still allows us to reconstruct the calling directory in the superproject by just traveling reverse of `prefix`. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-17submodule: port resolve_relative_url from shell to CStefan Beller
Later on we want to automatically call `git submodule init` from other commands, such that the users don't have to initialize the submodule themselves. As these other commands are written in C already, we'd need the init functionality in C, too. The `resolve_relative_url` function is a large part of that init functionality, so start by porting this function to C. To create the tests in t0060, the function `resolve_relative_url` was temporarily enhanced to write all inputs and output to disk when running the test suite. The added tests in this patch are a small selection thereof. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-14Merge branch 'sb/submodule-path-misc-bugs' into sb/submodule-initJunio C Hamano
"git submodule" reports the paths of submodules the command recurses into, but this was incorrect when the command was not run from the root level of the superproject. Any further comments? Otherwise will merge to 'next'. * sb/submodule-path-misc-bugs: (600 commits) t7407: make expectation as clear as possible submodule update: test recursive path reporting from subdirectory submodule update: align reporting path for custom command execution submodule status: correct path handling in recursive submodules submodule update --init: correct path handling in recursive submodules submodule foreach: correct path display in recursive submodules Git 2.8 Documentation: fix git-p4 AsciiDoc formatting mingw: skip some tests in t9115 due to file name issues t1300: fix the new --show-origin tests on Windows t1300-repo-config: make it resilient to being run via 'sh -x' config --show-origin: report paths with forward slashes submodule: fix regression for deinit without submodules l10n: pt_PT: Update and add new translations l10n: ca.po: update translation Git 2.8-rc4 Documentation: fix broken linkgit to git-config Documentation: use ASCII quotation marks in git-p4 Revert "config.mak.uname: use clang for Mac OS X 10.6" git-compat-util: st_add4: work around gcc 4.2.x compiler crash ...
2016-04-14Merge branch 'sb/submodule-helper-clone-regression-fix' into sb/submodule-initJunio C Hamano
* sb/submodule-helper-clone-regression-fix: submodule--helper, module_clone: catch fprintf failure submodule--helper: do not borrow absolute_path() result for too long submodule--helper, module_clone: always operate on absolute paths submodule--helper clone: create the submodule path just once submodule--helper: fix potential NULL-dereference recursive submodules: test for relative paths
2016-04-06Merge branch 'jk/submodule-c-credential'Junio C Hamano
"git -c credential.<var>=<value> submodule" can now be used to propagate configuration variables related to credential helper down to the submodules. * jk/submodule-c-credential: git_config_push_parameter: handle empty GIT_CONFIG_PARAMETERS git: submodule honor -c credential.* from command line quote: implement sq_quotef() submodule: fix segmentation fault in submodule--helper clone submodule: fix submodule--helper clone usage submodule: check argc count for git submodule--helper clone submodule: don't pass empty string arguments to submodule--helper clone
2016-04-06Merge branch 'sb/submodule-parallel-update'Junio C Hamano
A major part of "git submodule update" has been ported to C to take advantage of the recently added framework to run download tasks in parallel. * sb/submodule-parallel-update: clone: allow an explicit argument for parallel submodule clones submodule update: expose parallelism to the user submodule helper: remove double 'fatal: ' prefix git submodule update: have a dedicated helper for cloning run_processes_parallel: rename parameters for the callbacks run_processes_parallel: treat output of children as byte array submodule update: direct error message to stderr fetching submodules: respect `submodule.fetchJobs` config option submodule-config: drop check against NULL submodule-config: keep update strategy around
2016-04-01submodule--helper, module_clone: catch fprintf failureStefan Beller
The return value of fprintf is unchecked, which may lead to unreported errors. Use fprintf_or_die to report the error to the user. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-01submodule--helper: do not borrow absolute_path() result for too longJunio C Hamano
absolute_path() is designed to allow its callers to take a brief peek of the result (typically, to be fed to functions like strbuf_add() and relative_path() as a parameter) without having to worry about freeing it, but the other side of the coin of that memory model is that the caller shouldn't rely too much on the result living forever--there may be a helper function the caller subsequently calls that makes its own call to absolute_path(), invalidating the earlier result. Use xstrdup() to make our own copy, and free(3) it when we are done. While at it, remove an unnecessary sm_gitdir_rel variable that was only used to as a parameter to call absolute_path() and never used again. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-01submodule--helper, module_clone: always operate on absolute pathsStefan Beller
When giving relative paths to `relative_path` to compute a relative path from one directory to another, this may fail in `relative_path`. Make sure both arguments to `relative_path` are always absolute. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-31submodule--helper clone: create the submodule path just onceStefan Beller
We make sure that the parent directory of path exists (or create it otherwise) and then do the same for path + "/.git". That is equivalent to just making sure that the parent directory of path + "/.git" exists (or create it otherwise). Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-31submodule--helper: fix potential NULL-dereferenceStefan Beller
Don't dereference NULL 'path' if it was never assigned. Also protect against an empty --path argument. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-23submodule: fix regression for deinit without submodulesStefan Beller
Per Cederqvist wrote: > It used to be possible to run > > git submodule deinit -f . > > to remove any submodules, no matter how many submodules you had. That > is no longer possible in projects that don't have any submodules at > all. The command will fail with: > > error: pathspec '.' did not match any file(s) known to git. This regression was introduced in 74703a1e4dfc (submodule: rewrite `module_list` shell function in C, 2015-09-02), as we changed the order of checking in new module listing to first check whether it is a gitlin before feeding it to match_pathspec(). It used to be that a pathspec that does not match any path were diagnosed as an error, but the new code complains for a pathspec that does not match any submodule path. Arguably the new behaviour may give us a better diagnosis, but that is inconsistent with the suggestion "deinit" gives, and also this was an unintended accident. The new behaviour hopefully can be redesigned and implemented better in future releases, but for now, switch these two checks to restore the same behavior as before. In an empty repository, giving the pathspec '.' will still get the same "did not match" error, but that is the same bug we had before 1.7.0. Reported-by: Per Cederqvist <cederp@opera.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01git: submodule honor -c credential.* from command lineJacob Keller
Due to the way that the git-submodule code works, it clears all local git environment variables before entering submodules. This is normally a good thing since we want to clear settings such as GIT_WORKTREE and other variables which would affect the operation of submodule commands. However, GIT_CONFIG_PARAMETERS is special, and we actually do want to preserve these settings. However, we do not want to preserve all configuration as many things should be left specific to the parent project. Add a git submodule--helper function, sanitize-config, which shall be used to sanitize GIT_CONFIG_PARAMETERS, removing all key/value pairs except a small subset that are known to be safe and necessary. Replace all the calls to clear_local_git_env with a wrapped function that filters GIT_CONFIG_PARAMETERS using the new helper and then restores it to the filtered subset after clearing the rest of the environment. Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01submodule: fix segmentation fault in submodule--helper cloneJacob Keller
The git submodule--helper clone command will fail with a segmentation fault when given a null url or null path variable. Since these are required for proper functioning of the submodule--helper clone subcommand, add checks to prevent running and fail gracefully when missing. Update the usage string to reflect the requirement that the --url and --path "options" are required. Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01submodule: fix submodule--helper clone usageJacob Keller
git submodule--helper clone usage stated that paths were added after the [--] argument. The actual implementation required use of --path argument and only supports one path at a time. Update the usage string to match the current implementation. Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01submodule: check argc count for git submodule--helper cloneJacob Keller
Extra unused arguments to git submodule--helper clone subcommand were being silently ignored. Add a check to the argc count after options handling to ensure that no extra arguments were left on the argv array. Signed-off-by: Jacob Keller <jacob.keller@gmail.com> Reviewed-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01submodule update: expose parallelism to the userStefan Beller
Expose possible parallelism either via the "--jobs" CLI parameter or the "submodule.fetchJobs" setting. By having the variable initialized to -1, we make sure 0 can be passed into the parallel processing machine, which will then pick as many parallel workers as there are CPUs. Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01submodule helper: remove double 'fatal: ' prefixStefan Beller
The prefix is added by die(...), so we don't have to do it. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01git submodule update: have a dedicated helper for cloningStefan Beller
This introduces a new helper function in git submodule--helper which takes care of cloning all submodules, which we want to parallelize eventually. Some tests (such as empty URL, update_mode=none) are required in the helper to make the decision for cloning. These checks have been moved into the C function as well (no need to repeat them in the shell script). Reviewed-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-24submodule helper list: respect correct path prefixStefan Beller
This is a regression introduced by 74703a1e4d (submodule: rewrite `module_list` shell function in C, 2015-09-02). Add a test to ensure we list the right submodule when giving a specific pathspec. Reported-By: Caleb Jorden <cjorden@gmail.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-08submodule: rewrite `module_clone` shell function in CStefan Beller
This reimplements the helper function `module_clone` in shell in C as `clone`. This functionality is needed for converting `git submodule update` later on, which we want to add threading to. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-03submodule: rewrite `module_name` shell function in CStefan Beller
This implements the helper `name` in C instead of shell, yielding a nice performance boost. Before this patch, I measured a time (best out of three): $ time ./t7400-submodule-basic.sh >/dev/null real 0m11.066s user 0m3.348s sys 0m8.534s With this patch applied I measured (also best out of three) $ time ./t7400-submodule-basic.sh >/dev/null real 0m10.063s user 0m3.044s sys 0m7.487s Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-03submodule: rewrite `module_list` shell function in CStefan Beller
Most of the submodule operations work on a set of submodules. Calculating and using this set is usually done via: module_list "$@" | { while read mode sha1 stage sm_path do # the actual operation done } Currently the function `module_list` is implemented in the git-submodule.sh as a shell script wrapping a perl script. The rewrite is in C, such that it is faster and can later be easily adapted when other functions are rewritten in C. git-submodule.sh, similar to the builtin commands, will navigate to the top-most directory of the repository and keep the subdirectory as a variable. As the helper is called from within the git-submodule.sh script, we are already navigated to the root level, but the path arguments are still relative to the subdirectory we were in when calling git-submodule.sh. That's why there is a `--prefix` option pointing to an alternative path which to anchor relative path arguments. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>