path: root/repository.c
AgeCommit message (Collapse)Author
2020-02-24repository: require a build flag to use SHA-256brian m. carlson
At this point, SHA-256 support is experimental and some behavior may change. To avoid surprising unsuspecting users, require a build flag, ENABLE_SHA256, to allow use of a non-SHA-1 algorithm. Enable this flag by default when the DEVELOPER make flag is set so that contributors can test this case adequately. Signed-off-by: brian m. carlson <> Signed-off-by: Junio C Hamano <>
2019-11-10Fix spelling errors in code commentsElijah Newren
Reported-by: Jens Schleusener <> Signed-off-by: Elijah Newren <> Signed-off-by: Junio C Hamano <>
2019-03-20Merge branch 'ma/clear-repository-format'Junio C Hamano
The setup code has been cleaned up to avoid leaks around the repository_format structure. * ma/clear-repository-format: setup: fix memory leaks with `struct repository_format` setup: free old value before setting `work_tree`
2019-02-28setup: fix memory leaks with `struct repository_format`Martin Ågren
After we set up a `struct repository_format`, it owns various pieces of allocated memory. We then either use those members, because we decide we want to use the "candidate" repository format, or we discard the candidate / scratch space. In the first case, we transfer ownership of the memory to a few global variables. In the latter case, we just silently drop the struct and end up leaking memory. Introduce an initialization macro `REPOSITORY_FORMAT_INIT` and a function `clear_repository_format()`, to be used on each side of `read_repository_format()`. To have a clear and simple memory ownership, let all users of `struct repository_format` duplicate the strings that they take from it, rather than stealing the pointers. Call `clear_...()` at the start of `read_...()` instead of just zeroing the struct, since we sometimes enter the function multiple times. Thus, it is important to initialize the struct before calling `read_...()`, so document that. It's also important because we might not even call `read_...()` before we call `clear_...()`, see, e.g., builtin/init-db.c. Teach `read_...()` to clear the struct on error, so that it is reset to a safe state, and document this. (In `setup_git_directory_gently()`, we look at `repo_fmt.hash_algo` even if `repo_fmt.version` is -1, which we weren't actually supposed to do per the API. After this commit, that's ok.) We inherit the existing code's combining "error" and "no version found". Both are signalled through `version == -1` and now both cause us to clear any partial configuration we have picked up. For "extensions.*", that's fine, since they require a positive version number. For "core.bare" and "core.worktree", we're already verifying that we have a non-negative version number before using them. Signed-off-by: Martin Ågren <> Signed-off-by: Junio C Hamano <>
2019-02-22trace2: create new combined trace facilityJeff Hostetler
Create a new unified tracing facility for git. The eventual intent is to replace the current trace_printf* and trace_performance* routines with a unified set of git_trace2* routines. In addition to the usual printf-style API, trace2 provides higer-level event verbs with fixed-fields allowing structured data to be written. This makes post-processing and analysis easier for external tools. Trace2 defines 3 output targets. These are set using the environment variables "GIT_TR2", "GIT_TR2_PERF", and "GIT_TR2_EVENT". These may be set to "1" or to an absolute pathname (just like the current GIT_TRACE). * GIT_TR2 is intended to be a replacement for GIT_TRACE and logs command summary data. * GIT_TR2_PERF is intended as a replacement for GIT_TRACE_PERFORMANCE. It extends the output with columns for the command process, thread, repo, absolute and relative elapsed times. It reports events for child process start/stop, thread start/stop, and per-thread function nesting. * GIT_TR2_EVENT is a new structured format. It writes event data as a series of JSON records. Calls to trace2 functions log to any of the 3 output targets enabled without the need to call different trace_printf* or trace_performance* routines. Signed-off-by: Jeff Hostetler <> Signed-off-by: Junio C Hamano <>
2019-02-07Merge branch 'nd/the-index-final'Junio C Hamano
The assumption to work on the single "in-core index" instance has been reduced from the library-ish part of the codebase. * nd/the-index-final: cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch read-cache.c: remove the_* from index_has_changes() merge-recursive.c: remove implicit dependency on the_repository merge-recursive.c: remove implicit dependency on the_index sha1-name.c: remove implicit dependency on the_index read-cache.c: replace update_index_if_able with repo_& read-cache.c: kill read_index() checkout: avoid the_index when possible repository.c: replace hold_locked_index() with repo_hold_locked_index() notes-utils.c: remove the_repository references grep: use grep_opt->repo instead of explict repo argument
2019-01-29Merge branch 'sb/submodule-recursive-fetch-gets-the-tip'Junio C Hamano
"git fetch --recurse-submodules" may not fetch the necessary commit that is bound to the superproject, which is getting corrected. * sb/submodule-recursive-fetch-gets-the-tip: fetch: ensure submodule objects fetched submodule.c: fetch in submodules git directory instead of in worktree submodule: migrate get_next_submodule to use repository structs repository: repo_submodule_init to take a submodule struct submodule: store OIDs in changed_submodule_names submodule.c: tighten scope of changed_submodule_names struct submodule.c: sort changed_submodule_names before searching it submodule.c: fix indentation sha1-array: provide oid_array_filter
2019-01-24cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switchNguyễn Thái Ngọc Duy
By default, index compat macros are off from now on, because they could hide the_index dependency. Only those in builtin can use it. Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2019-01-14repository.c: replace hold_locked_index() with repo_hold_locked_index()Nguyễn Thái Ngọc Duy
hold_locked_index() assumes the index path at $GIT_DIR/index. This is not good for places that take an arbitrary index_state instead of the_index, which is basically everywhere except builtin/. Replace it with repo_hold_locked_index(). hold_locked_index() remains as a wrapper around repo_hold_locked_index() to reduce changes in builtin/ Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2018-12-05repository: repo_submodule_init to take a submodule structStefan Beller
When constructing a struct repository for a submodule for some revision of the superproject where the submodule is not contained in the index, it may not be present in the working tree currently either. In that situation giving a 'path' argument is not useful. Upgrade the repo_submodule_init function to take a struct submodule instead. The submodule struct can be obtained via submodule_from_{path, name} or an artificial submodule struct can be passed in. While we are at it, rename the repository struct in the repo_submodule_init function, which is to be initialized, to a name that is not confused with the struct submodule as easily. Perform such renames in similar functions as well. Also move its documentation into the header file. Reviewed-by: Jonathan Tan <> Signed-off-by: Stefan Beller <> Signed-off-by: Junio C Hamano <>
2018-11-13sha1-file: use an object_directory for the main object dirJeff King
Our handling of alternate object directories is needlessly different from the main object directory. As a result, many places in the code basically look like this: do_something(r->objects->objdir); for (odb = r->objects->alt_odb_list; odb; odb = odb->next) do_something(odb->path); That gets annoying when do_something() is non-trivial, and we've resorted to gross hacks like creating fake alternates (see find_short_object_filename()). Instead, let's give each raw_object_store a unified list of object_directory structs. The first will be the main store, and everything after is an alternate. Very few callers even care about the distinction, and can just loop over the whole list (and those who care can just treat the first element differently). A few observations: - we don't need r->objects->objectdir anymore, and can just mechanically convert that to r->objects->odb->path - object_directory's path field needs to become a real pointer rather than a FLEX_ARRAY, in order to fill it with expand_base_dir() - we'll call prepare_alt_odb() earlier in many functions (i.e., outside of the loop). This may result in us calling it even when our function would be satisfied looking only at the main odb. But this doesn't matter in practice. It's not a very expensive operation in the first place, and in the majority of cases it will be a noop. We call it already (and cache its results) in prepare_packed_git(), and we'll generally check packs before loose objects. So essentially every program is going to call it immediately once per program. Arguably we should just prepare_alt_odb() immediately upon setting up the repository's object directory, which would save us sprinkling calls throughout the code base (and forgetting to do so has been a source of subtle bugs in the past). But I've stopped short of that here, since there are already a lot of other moving parts in this patch. - Most call sites just get shorter. The check_and_freshen() functions are an exception, because they have entry points to handle local and nonlocal directories separately. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2018-06-25Merge branch 'sb/object-store-alloc'Junio C Hamano
The conversion to pass "the_repository" and then "a_repository" throughout the object access API continues. * sb/object-store-alloc: alloc: allow arbitrary repositories for alloc functions object: allow create_object to handle arbitrary repositories object: allow grow_object_hash to handle arbitrary repositories alloc: add repository argument to alloc_commit_index alloc: add repository argument to alloc_report alloc: add repository argument to alloc_object_node alloc: add repository argument to alloc_tag_node alloc: add repository argument to alloc_commit_node alloc: add repository argument to alloc_tree_node alloc: add repository argument to alloc_blob_node object: add repository argument to grow_object_hash object: add repository argument to create_object repository: introduce parsed objects field
2018-05-30Merge branch 'nd/repo-clear-keep-the-index'Junio C Hamano
the_repository->index is not a allocated piece of memory but repo_clear() indiscriminately attempted to free(3) it, which has been corrected. * nd/repo-clear-keep-the-index: repository: fix free problem with repo_clear(the_repository)
2018-05-10repository: fix free problem with repo_clear(the_repository)Nguyễn Thái Ngọc Duy
the_repository is special. One of the special things about it is that it does not allocate a new index_state object like submodules but points to the global the_index variable instead. As a global variable, the_index cannot be free()'d. Add an exception for this in repo_clear(). In the future perhaps we would be able to allocate the_repository's index on heap too. Then we can revert this. the_repository->index remains pointed to a clean the_index even after repo_clear() so that it could still be used next time (e.g. in a crazy use case where a dev switches repo in the same process). Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2018-05-09repository: introduce parsed objects fieldStefan Beller
Convert the existing global cache for parsed objects (obj_hash) into repository-specific parsed object caches. Existing code that uses obj_hash are modified to use the parsed object cache of the_repository; future patches will use the parsed object caches of other repositories. Another future use case for a pool of objects is ease of memory management in revision walking: If we can free the rev-list related memory early in pack-objects (e.g. part of repack operation) then it could lower memory pressure significantly when running on large repos. While this has been discussed on the mailing list lately, this series doesn't implement this. Signed-off-by: Stefan Beller <> Signed-off-by: Junio C Hamano <>
2018-03-29submodule: fixup nested submodules after moving the submoduleStefan Beller
connect_work_tree_and_git_dir is used to connect a submodule worktree with its git directory and vice versa after events that require a reconnection such as moving around the working tree. As submodules can have nested submodules themselves, we'd also want to fix the nested submodules when asked to. Add an option to recurse into the nested submodules and connect them as well. As submodules are identified by their name (which determines their git directory in relation to their superproject's git directory) internally and by their path in the working tree of the superproject, we need to make sure that the mapping of name <-> path is kept intact. We can do that in the git-mv command by writing out the gitmodules file first and then forcing a reload of the submodule config machinery. Signed-off-by: Stefan Beller <> Reviewed-by: Jonathan Tan <> Signed-off-by: Junio C Hamano <>
2018-03-29submodule-config: remove submodule_from_cacheStefan Beller
This continues the story of bf12fcdf5e (submodule-config: store the_submodule_cache in the_repository, 2017-06-22). The previous patch taught submodule_from_path to take a repository into account, such that submodule_from_{path, cache} are the same now. Remove submodule_from_cache, migrating all its callers to submodule_from_path. Reviewed-by: Jonathan Tan <> Signed-off-by: Stefan Beller <> Reviewed-by: Jonathan Tan <> Signed-off-by: Junio C Hamano <>
2018-03-23repository: introduce raw object store fieldStefan Beller
The raw object store field will contain any objects needed for access to objects in a given repository. This patch introduces the raw object store and populates it with the `objectdir`, which used to be part of the repository struct. As the struct gains members, we'll also populate the function to clear the memory for these members. In a later step, we'll introduce a struct object_parser, that will complement the object parsing in a repository struct: The raw object parser is the layer that will provide access to raw object content, while the higher level object parser code will parse raw objects and keeps track of parenthood and other object relationships using 'struct object'. For now only add the lower level to the repository struct. Signed-off-by: Stefan Beller <> Signed-off-by: Jonathan Nieder <> Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2018-03-05repository: delete ignore_env memberNguyễn Thái Ngọc Duy
This variable was added because the repo_set_gitdir() was created to cover both submodule and main repos, but these two are initialized a bit differently so ignore_env == 0 means main repo, while ignore_env != 0 is submodules. Since the difference part (env variables) has been moved out of repo_set_gitdir(), this function works the same way for both repo types and ignore_env is not needed anymore. Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2018-03-05sha1_file.c: move delayed getenv(altdb) back to setup_git_env()Nguyễn Thái Ngọc Duy
getenv() is supposed to work on the main repository only. This delayed getenv() code in sha1_file.c makes it more difficult to convert sha1_file.c to a generic object store that could be used by both submodule and main repositories. Move the getenv() back in setup_git_env() where other env vars are also fetched. Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2018-03-05repository.c: delete dead functionsNguyễn Thái Ngọc Duy
Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2018-03-05repository.c: move env-related setup code back to environment.cNguyễn Thái Ngọc Duy
It does not make sense that generic repository code contains handling of environment variables, which are specific for the main repository only. Refactor repo_set_gitdir() function to take $GIT_DIR and optionally _all_ other customizable paths. These optional paths can be NULL and will be calculated according to the default directory layout. Note that some dead functions are left behind to reduce diff noise. They will be deleted in the next patch. Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2018-03-05repository: initialize the_repository in main()Nguyễn Thái Ngọc Duy
This simplifies initialization of struct repository and anything inside. Easier to read. Easier to add/remove fields. Everything will go through main() common-main.c so this should cover all programs, including t/helper. Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2018-02-13Merge branch 'tg/split-index-fixes'Junio C Hamano
The split-index mode had a few corner case bugs fixed. * tg/split-index-fixes: travis: run tests with GIT_TEST_SPLIT_INDEX split-index: don't write cache tree with null oid entries read-cache: fix reading the shared index for other repos
2018-01-19repository: pre-initialize hash algo pointerbrian m. carlson
There are various git subcommands (among them, clone) which don't set up the repository (that is, they lack RUN_SETUP or RUN_SETUP_GENTLY) but end up needing to have information about the hash algorithm in use. Because the hash algorithm is part of struct repository and it's only initialized in repository setup, we can end up dereferencing a NULL pointer in some cases if we call one of these subcommands and look up the empty blob or empty tree values. A "git clone" of a project that has two paths that differ only in case suffers from this if it is run on a case insensitive platform. When the command attempts to check out one of these two paths after checking out the other one, the checkout codepath needs to see if the version that is already on the filesystem (which should not happen if the FS were case sensitive) is dirty, and it needs to exercise the hashing code at that point. In the future, we can add a command line option for this or read it from the configuration, but until we're ready to expose that functionality to the user, simply initialize the repository structure to use the current hash algorithm, SHA-1. Signed-off-by: brian m. carlson <> Signed-off-by: Junio C Hamano <>
2018-01-19read-cache: fix reading the shared index for other reposThomas Gummerer
read_index_from() takes a path argument for the location of the index file. For reading the shared index in split index mode however it just ignores that path argument, and reads it from the gitdir of the current repository. This works as long as an index in the_repository is read. Once that changes, such as when we read the index of a submodule, or of a different working tree than the current one, the gitdir of the_repository will no longer contain the appropriate shared index, and git will fail to read it. For example was broken with GIT_TEST_SPLIT_INDEX set in 188dce131f ("ls-files: use repository object", 2017-06-22), and was also broken in a similar manner, probably by introducing struct repository there, although I didn't track down the exact commit for that. be489d02d2 ("revision.c: --indexed-objects add objects from all worktrees", 2017-08-23) breaks with split index mode in a similar manner, not erroring out when it can't read the index, but instead carrying on with pruning, without taking the index of the worktree into account. Fix this by passing an additional gitdir parameter to read_index_from, to indicate where it should look for and read the shared index from. read_cache_from() defaults to using the gitdir of the_repository. As it is mostly a convenience macro, having to pass get_git_dir() for every call seems overkill, and if necessary users can have more control by using read_index_from(). Helped-by: Brandon Williams <> Signed-off-by: Thomas Gummerer <> Signed-off-by: Junio C Hamano <>
2017-11-28repository: fix a sparse 'using integer as NULL pointer' warningRamsay Jones
Commit 78a6766802 ("Integrate hash algorithm support with repo setup", 2017-11-12) added a 'const struct git_hash_algo *hash_algo' field to the repository structure, without modifying the initializer of the 'the_repo' variable. This does not actually introduce a bug, since the '0' initializer for the 'ignore_env:1' bit-field is interpreted as a NULL pointer (hence the warning), and the final field (now with no initializer) receives a default '0'. Signed-off-by: Ramsay Jones <> Signed-off-by: Junio C Hamano <>
2017-11-13Integrate hash algorithm support with repo setupbrian m. carlson
In future versions of Git, we plan to support an additional hash algorithm. Integrate the enumeration of hash algorithms with repository setup, and store a pointer to the enumerated data in struct repository. Of course, we currently only support SHA-1, so hard-code this value in read_repository_format. In the future, we'll enumerate this value from the configuration. Add a constant, the_hash_algo, which points to the hash_algo structure pointer in the repository global. Note that this is the hash which is used to serialize data to disk, not the hash which is used to display items to the user. The transition plan anticipates that these may be different. We can add an additional element in the future (say, ui_hash_algo) to provide for this case. Include repository.h in cache.h since we now need to have access to these struct and variable definitions. Signed-off-by: brian m. carlson <> Signed-off-by: Junio C Hamano <>
2017-10-02repository: use FREE_AND_NULLRené Scharfe
Use the macro FREE_AND_NULL to release allocated objects and clear their pointers. This is shorter and documents the intent better by combining the two related operations into one. Patch generated with Coccinelle and contrib/coccinelle/free.cocci. Signed-off-by: Rene Scharfe <> Signed-off-by: Junio C Hamano <>
2017-09-06set_git_dir: handle feeding gitdir to itselfJeff King
Ideally we'd free the existing gitdir field before assigning the new one, to avoid a memory leak. But we can't do so safely because some callers do the equivalent of: set_git_dir(get_git_dir()); We can detect that case as a noop, but there are even more complicated cases like: set_git_dir(remove_leading_path(worktree, get_git_dir()); where we really do need to do some work, but the original string must remain valid. Rather than put the burden on callers to make a copy of the string (only to free it later, since we'll make a copy of it ourselves), let's solve the problem inside set_git_dir(). We can make a copy of the pointer for the old gitdir, and then avoid freeing it until after we've made our new copy. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2017-09-06repository: free fields before overwriting themJeff King
It's possible that the repository data may be initialized twice (e.g., after doing a chdir() to the top of the worktree we may have to adjust a relative git_dir path). We should free() any existing fields before assigning to them to avoid leaks. This should be safe, as the fields are set based on the environment or on other strings like the gitdir or commondir. That makes it impossible that we are feeding an alias to the just-freed string. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2017-08-22Merge branch 'bw/grep-recurse-submodules'Junio C Hamano
"git grep --recurse-submodules" has been reworked to give a more consistent output across submodule boundary (and do its thing without having to fork a separate process). * bw/grep-recurse-submodules: grep: recurse in-process using 'struct repository' submodule: merge repo_read_gitmodules and gitmodules_config submodule: check for unmerged .gitmodules outside of config parsing submodule: check for unstaged .gitmodules outside of config parsing submodule: remove fetch.recursesubmodules from submodule-config parsing submodule: remove submodule.fetchjobs from submodule-config parsing config: add config_from_gitmodules cache.h: add GITMODULES_FILE macro repository: have the_repository use the_index repo_read_index: don't discard the index
2017-07-18repository: have the_repository use the_indexBrandon Williams
Have the index state which is stored in 'the_repository' be a pointer to the in-core index 'the_index'. This makes it easier to begin transitioning more parts of the code base to operate on a 'struct repository'. Signed-off-by: Brandon Williams <> Signed-off-by: Junio C Hamano <>
2017-07-18repo_read_index: don't discard the indexBrandon Williams
Have 'repo_read_index()' behave more like the other read_index family of functions and don't discard the index if it has already been populated and instead rely on the quick return of read_index_from which has: /* istate->initialized covers both .git/index and .git/ */ if (istate->initialized) return istate->cache_nr; Signed-off-by: Brandon Williams <> Signed-off-by: Junio C Hamano <>
2017-07-17submodule: convert submodule config lookup to use object_idbrian m. carlson
Signed-off-by: brian m. carlson <> Signed-off-by: Junio C Hamano <>
2017-06-24repository: enable initialization of submodulesBrandon Williams
Introduce 'repo_submodule_init()' which performs initialization of a 'struct repository' as a submodule of another 'struct repository'. The resulting submodule 'struct repository' can be in one of three states: 1. The submodule is initialized and has a worktree. 2. The submodule is initialized but does not have a worktree. This would occur when the submodule's gitdir is present in the superproject's 'gitdir/modules/' directory yet the submodule has not been checked out in superproject's worktree. 3. The submodule remains uninitialized due to an error in the initialization process or there is no matching submodule at the provided path in the superproject. Signed-off-by: Brandon Williams <> Signed-off-by: Junio C Hamano <>
2017-06-24submodule-config: store the_submodule_cache in the_repositoryBrandon Williams
Refactor how 'the_submodule_cache' is handled so that it can be stored inside of a repository object. Also migrate 'the_submodule_cache' to be stored in 'the_repository'. Signed-off-by: Brandon Williams <> Signed-off-by: Junio C Hamano <>
2017-06-24repository: add index_state to struct repoBrandon Williams
Signed-off-by: Brandon Williams <> Signed-off-by: Junio C Hamano <>
2017-06-24config: read config from a repository objectBrandon Williams
Teach the config machinery to read config information from a repository object. This involves storing a 'struct config_set' inside the repository object and adding a number of functions (repo_config*) to be able to query a repository's config. The current config API enables lazy-loading of the config. This means that when 'git_config_get_int()' is called, if the_config_set hasn't been populated yet, then it will be populated and properly initialized by reading the necessary config files (system wide .gitconfig, user's home .gitconfig, and the repository's config). To maintain this paradigm, the new API to read from a repository object's config will also perform this lazy-initialization. Since both APIs (git_config_get* and repo_config_get*) have the same semantics we can migrate the default config to be stored within 'the_repository' and just have the 'git_config_get*' family of functions redirect to the 'repo_config_get*' functions. Signed-off-by: Brandon Williams <> Signed-off-by: Junio C Hamano <>
2017-06-24repository: introduce the repository objectBrandon Williams
Introduce the repository object 'struct repository' which can be used to hold all state pertaining to a git repository. Some of the benefits of object-ifying a repository are: 1. Make the code base more readable and easier to reason about. 2. Allow for working on multiple repositories, specifically submodules, within the same process. Currently the process for working on a submodule involves setting up an argv_array of options for a particular command and then launching a child process to execute the command in the context of the submodule. This is clunky and can require lots of little hacks in order to ensure correctness. Ideally it would be nice to simply pass a repository and an options struct to a command. 3. Eliminating reliance on global state will make it easier to enable the use of threading to improve performance. Signed-off-by: Brandon Williams <> Signed-off-by: Junio C Hamano <>