summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2018-06-28commit-graph: fix documentation inconsistenciesDerrick Stolee
The commit-graph feature shipped in Git 2.18 has some inconsistencies in the constants used by the implementation and specified by the format document. The commit data chunk uses the key "CDAT" in the file format, but was previously documented to say "CGET". The commit data chunk stores commit parents using two 32-bit fields that typically store the integer position of the parent in the list of commit ids within the commit-graph file. When a parent does not exist, we had documented the value 0xffffffff, but implemented the value 0x70000000. This swap is easy to correct in the documentation, but unfortunately reduces the number of commits that we can store in the commit-graph. Update that estimate, too. Reported-by: Grant Welch <gwelch925@gmail.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-11commit-graph: implement "--append" optionDerrick Stolee
Teach git-commit-graph to add all commits from the existing commit-graph file to the file about to be written. This should be used when adding new commits without performing garbage collection. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-11commit-graph: build graph from starting commitsDerrick Stolee
Teach git-commit-graph to read commits from stdin when the --stdin-commits flag is specified. Commits reachable from these commits are added to the graph. This is a much faster way to construct the graph than inspecting all packed objects, but is restricted to known tips. For the Linux repository, 700,000+ commits were added to the graph file starting from 'master' in 7-9 seconds, depending on the number of packfiles in the repo (1, 24, or 120). Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-11commit-graph: read only from specific pack-indexesDerrick Stolee
Teach git-commit-graph to inspect the objects only in a certain list of pack-indexes within the given pack directory. This allows updating the commit graph iteratively. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-11commit: integrate commit graph with commit parsingDerrick Stolee
Teach Git to inspect a commit graph file to supply the contents of a struct commit when calling parse_commit_gently(). This implementation satisfies all post-conditions on the struct commit, including loading parents, the root tree, and the commit date. If core.commitGraph is false, then do not check graph files. In test script t5318-commit-graph.sh, add output-matching conditions on read-only graph operations. By loading commits from the graph instead of parsing commit buffers, we save a lot of time on long commit walks. Here are some performance results for a copy of the Linux repository where 'master' has 678,653 reachable commits and is behind 'origin/master' by 59,929 commits. | Command | Before | After | Rel % | |----------------------------------|--------|--------|-------| | log --oneline --topo-order -1000 | 8.31s | 0.94s | -88% | | branch -vv | 1.02s | 0.14s | -86% | | rev-list --all | 5.89s | 1.07s | -81% | | rev-list --all --objects | 66.15s | 58.45s | -11% | Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-11commit-graph: close under reachabilityDerrick Stolee
Teach write_commit_graph() to walk all parents from the commits discovered in packfiles. This prevents gaps given by loose objects or previously-missed packfiles. Also automatically add commits from the existing graph file, if it exists. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-11commit-graph: add core.commitGraph settingDerrick Stolee
The commit graph feature is controlled by the new core.commitGraph config setting. This defaults to 0, so the feature is opt-in. The intention of core.commitGraph is that a user can always stop checking for or parsing commit graph files if core.commitGraph=0. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-11commit-graph: implement git commit-graph readDerrick Stolee
Teach git-commit-graph to read commit graph files and summarize their contents. Use the read subcommand to verify the contents of a commit graph file in the tests. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-02commit-graph: implement git-commit-graph writeDerrick Stolee
Teach git-commit-graph to write graph files. Create new test script to verify this command succeeds without failure. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-02commit-graph: implement write_commit_graph()Derrick Stolee
Teach Git to write a commit graph file by checking all packed objects to see if they are commits, then store the file in the given object directory. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-02commit-graph: create git-commit-graph builtinDerrick Stolee
Teach git the 'commit-graph' builtin that will be used for writing and reading packed graph files. The current implementation is mostly empty, except for an '--object-dir' option. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-02graph: add commit graph design documentDerrick Stolee
Add Documentation/technical/commit-graph.txt with details of the planned commit graph feature, including future plans. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-02commit-graph: add format documentDerrick Stolee
Add document specifying the binary format for commit graphs. This format allows for: * New versions. * New hash functions and hash lengths. * Optional extensions. Basic header information is followed by a binary table of contents into "chunks" that include: * An ordered list of commit object IDs. * A 256-entry fanout into that list of OIDs. * A list of metadata for the commits. * A list of "large edges" to enable octopus merges. The format automatically includes two parent positions for every commit. This favors speed over space, since using only one position per commit would cause an extra level of indirection for every merge commit. (Octopus merges suffer from this indirection, but they are very rare.) Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-02csum-file: refactor finalize_hashfile() methodDerrick Stolee
If we want to use a hashfile on the temporary file for a lockfile, then we need finalize_hashfile() to fully write the trailing hash but also keep the file descriptor open. Do this by adding a new CSUM_HASH_IN_STREAM flag along with a functional change that checks this flag before writing the checksum to the stream. This differs from previous behavior since it would be written if either CSUM_CLOSE or CSUM_FSYNC is provided. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-02csum-file: rename hashclose() to finalize_hashfile()Derrick Stolee
The hashclose() method behaves very differently depending on the flags parameter. In particular, the file descriptor is not always closed. Perform a simple rename of "hashclose()" to "finalize_hashfile()" in preparation for functional changes. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-13Merge branch 'jk/cached-commit-buffer' into HEADJunio C Hamano
* jk/cached-commit-buffer: revision: drop --show-all option commit: drop uses of get_cached_commit_buffer() Git 2.16.2
2018-03-13Merge branch 'jt/binsearch-with-fanout' into HEADJunio C Hamano
* jt/binsearch-with-fanout: packfile: refactor hash search with fanout table packfile: remove GIT_DEBUG_LOOKUP log statements
2018-02-22revision: drop --show-all optionJeff King
This was an undocumented debugging aid that does not seem to have come in handy in the past decade, judging from its lack of mentions on the mailing list. Let's drop it in the name of simplicity. This is morally a revert of 3131b71301 (Add "--show-all" revision walker flag for debugging, 2008-02-09), but note that I did leave in the mapping of UNINTERESTING to "^" in get_revision_mark(). I don't think this would be possible to trigger with the current code, but it's the only sensible marker. We'll skip the usual deprecation period because this was explicitly a debugging aid that was never documented. Signed-off-by: Jeff King <peff@peff.net> Acked-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-22commit: drop uses of get_cached_commit_buffer()Jeff King
The "--show-all" revision option shows UNINTERESTING commits. Some of these commits may be unparsed when we try to show them (since we may or may not need to walk their parents to fulfill the request). Commit 3131b71301 (Add "--show-all" revision walker flag for debugging, 2008-02-09) resolved this by just skipping pretty-printing for commits without their object contents cached, saying: Because we now end up listing commits we may not even have been parsed at all "show_log" and "show_commit" need to protect against commits that don't have a commit buffer entry. That was the easy fix to avoid the pretty-printer segfaulting, but: 1. It doesn't work for all formats. E.g., --oneline prints the oid for each such commit but not a trailing newline, leading to jumbled output. 2. It only affects some commits, depending on whether we happened to parse them or not (so if they were at the tip of an UNINTERESTING starting point, or if we happened to traverse over them, you'd see more data). 3. It unncessarily ties the decision to show the verbose header to whether the commit buffer was cached. That makes it harder to change the logic around caching (e.g., if we could traverse without actually loading the full commit objects). These days it's safe to feed such a commit to the pretty-print code. Since be5c9fb904 (logmsg_reencode: lazily load missing commit buffers, 2013-01-26), we'll load it on demand in such a case. So let's just always show the verbose headers. This does change the behavior of plumbing, but: a. The --show-all option was explicitly introduced as a debugging aid, and was never documented (and has rarely even been mentioned on the list by git devs). b. Avoiding the commits was already not deterministic due to (2) above. So the caller might have seen full headers for these commits anyway, and would need to be prepared for it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-15Git 2.16.2v2.16.2Junio C Hamano
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-15Merge branch 'ab/doc-cat-file-e-still-shows-errors' into maintJunio C Hamano
Doc update. * ab/doc-cat-file-e-still-shows-errors: cat-file doc: document that -e will return some output
2018-02-15Merge branch 'as/read-tree-prefix-doc-fix' into maintJunio C Hamano
Doc update. * as/read-tree-prefix-doc-fix: doc/read-tree: remove obsolete remark
2018-02-15Merge branch 'nd/add-i-ignore-submodules' into maintJunio C Hamano
"git add -p" was taught to ignore local changes to submodules as they do not interfere with the partial addition of regular changes anyway. * nd/add-i-ignore-submodules: add--interactive: ignore submodule changes except HEAD
2018-02-15Merge branch 'tg/stash-with-pathspec-fix' into maintJunio C Hamano
"git stash -- <pathspec>" incorrectly blew away untracked files in the directory that matched the pathspec, which has been corrected. * tg/stash-with-pathspec-fix: stash: don't delete untracked files that match pathspec
2018-02-15Merge branch 'jk/abort-clone-with-existing-dest' into maintJunio C Hamano
"git clone $there $here" is allowed even when here directory exists as long as it is an empty directory, but the command incorrectly removed it upon a failure of the operation. * jk/abort-clone-with-existing-dest: clone: do not clean up directories we didn't create clone: factor out dir_exists() helper t5600: modernize style t5600: fix outdated comment about unborn HEAD
2018-02-15Merge branch 'jc/merge-symlink-ours-theirs' into maintJunio C Hamano
"git merge -Xours/-Xtheirs" learned to use our/their version when resolving a conflicting updates to a symbolic link. * jc/merge-symlink-ours-theirs: merge: teach -Xours/-Xtheirs to symbolic link merge
2018-02-15Merge branch 'rs/lose-leak-pending' into maintJunio C Hamano
API clean-up around revision traversal. * rs/lose-leak-pending: commit: remove unused function clear_commit_marks_for_object_array() revision: remove the unused flag leak_pending checkout: avoid using the rev_info flag leak_pending bundle: avoid using the rev_info flag leak_pending bisect: avoid using the rev_info flag leak_pending object: add clear_commit_marks_all() ref-filter: use clear_commit_marks_many() in do_merge_filter() commit: use clear_commit_marks_many() in remove_redundant() commit: avoid allocation in clear_commit_marks_many()
2018-02-15Merge branch 'jm/svn-pushmergeinfo-fix' into maintJunio C Hamano
"git svn dcommit" did not take into account the fact that a svn+ssh:// URL with a username@ (typically used for pushing) refers to the same SVN repository without the username@ and failed when svn.pushmergeinfo option is set. * jm/svn-pushmergeinfo-fix: git-svn: fix svn.pushmergeinfo handling of svn+ssh usernames.
2018-02-15Merge branch 'dk/describe-all-output-fix' into maintJunio C Hamano
An old regression in "git describe --all $annotated_tag^0" has been fixed. * dk/describe-all-output-fix: describe: prepend "tags/" when describing tags with embedded name
2018-02-15Merge branch 'ab/perf-grep-threads' into maintJunio C Hamano
More perf tests for threaded grep * ab/perf-grep-threads: perf: amend the grep tests to test grep.threads
2018-02-15Merge branch 'bc/hash-algo'Junio C Hamano
More abstraction of hash function from the codepath. * bc/hash-algo: hash: update obsolete reference to SHA1_HEADER bulk-checkin: abstract SHA-1 usage csum-file: abstract uses of SHA-1 csum-file: rename sha1file to hashfile read-cache: abstract away uses of SHA-1 pack-write: switch various SHA-1 values to abstract forms pack-check: convert various uses of SHA-1 to abstract forms fast-import: switch various uses of SHA-1 to the_hash_algo sha1_file: switch uses of SHA-1 to the_hash_algo builtin/unpack-objects: switch uses of SHA-1 to the_hash_algo builtin/index-pack: improve hash function abstraction hash: create union for hash context allocation hash: move SHA-1 macros to hash.h
2018-02-15Merge branch 'nd/ignore-glob-doc-update'Junio C Hamano
Doc update. * nd/ignore-glob-doc-update: gitignore.txt: elaborate shell glob syntax
2018-02-15Merge branch 'tg/reset-hard-show-head-with-pretty'Junio C Hamano
The way "git reset --hard" reports the commit the updated HEAD points at is made consistent with the way how the commit title is generated by the other parts of the system. This matters when the title is spread across physically multiple lines. * tg/reset-hard-show-head-with-pretty: reset --hard: make use of the pretty machinery
2018-02-15Merge branch 'rs/cocci-strbuf-addf-to-addstr'Junio C Hamano
* rs/cocci-strbuf-addf-to-addstr: cocci: simplify check for trivial format strings
2018-02-15Merge branch 'nd/trace-index-ops'Junio C Hamano
* nd/trace-index-ops: trace: measure where the time is spent in the index-heavy operations
2018-02-15Merge branch 'cc/perf-aggregate'Junio C Hamano
"make perf" enhancement. * cc/perf-aggregate: perf/aggregate: sort JSON fields in output perf/aggregate: add --reponame option perf/aggregate: add --subsection option
2018-02-15Merge branch 'ab/wildmatch-tests'Junio C Hamano
More tests for wildmatch functions. * ab/wildmatch-tests: wildmatch test: mark test as EXPENSIVE_ON_WINDOWS test-lib: add an EXPENSIVE_ON_WINDOWS prerequisite wildmatch test: create & test files on disk in addition to in-memory wildmatch test: perform all tests under all wildmatch() modes wildmatch test: use test_must_fail, not ! for test-wildmatch wildmatch test: remove dead fnmatch() test code wildmatch test: use a paranoia pattern from nul_match() wildmatch test: don't try to vertically align our output wildmatch test: use more standard shell style wildmatch test: indent with tabs, not spaces
2018-02-15Merge branch 'po/object-id'Junio C Hamano
Conversion from uchar[20] to struct object_id continues. * po/object-id: sha1_file: rename hash_sha1_file_literally sha1_file: convert write_loose_object to object_id sha1_file: convert force_object_loose to object_id sha1_file: convert write_sha1_file to object_id notes: convert write_notes_tree to object_id notes: convert combine_notes_* to object_id commit: convert commit_tree* to object_id match-trees: convert splice_tree to object_id cache: clear whole hash buffer with oidclr sha1_file: convert hash_sha1_file to object_id dir: convert struct sha1_stat to use object_id sha1_file: convert pretend_sha1_file to object_id
2018-02-15Merge branch 'sb/pull-rebase-submodule'Junio C Hamano
"git pull --rebase" did not pass verbosity setting down when recursing into a submodule. * sb/pull-rebase-submodule: builtin/pull: respect verbosity settings in submodules
2018-02-15Merge branch 'kg/packed-ref-cache-fix'Junio C Hamano
Avoid mmapping small files while using packed refs (especially ones with zero size, which would cause later munmap() to fail). * kg/packed-ref-cache-fix: packed_ref_cache: don't use mmap() for small files load_contents(): don't try to mmap an empty file packed_ref_iterator_begin(): make optimization more general find_reference_location(): make function safe for empty snapshots create_snapshot(): use `xmemdupz()` rather than a strbuf struct snapshot: store `start` rather than `header_len`
2018-02-15Merge branch 'jt/fsck-code-cleanup'Junio C Hamano
Plug recently introduced leaks in fsck. * jt/fsck-code-cleanup: fsck: fix leak when traversing trees
2018-02-15Merge branch 'en/merge-recursive-fixes'Junio C Hamano
* en/merge-recursive-fixes: merge-recursive: add explanation for src_entry and dst_entry merge-recursive: fix logic ordering issue Tighten and correct a few testcases for merging and cherry-picking
2018-02-15Merge branch 'jc/worktree-add-short-help'Junio C Hamano
Error message fix. * jc/worktree-add-short-help: worktree: say that "add" takes an arbitrary commit in short-help
2018-02-15Merge branch 'ab/sha1dc-build'Junio C Hamano
Push the submodule version of collision-detecting SHA-1 hash implementation a bit harder on builders. * ab/sha1dc-build: sha1dc_git.h: re-arrange an ifdef chain for a subsequent change Makefile: under "make dist", include the sha1collisiondetection submodule Makefile: don't error out under DC_SHA1_EXTERNAL if DC_SHA1_SUBMODULE=auto
2018-02-15packfile: refactor hash search with fanout tableJonathan Tan
Subsequent patches will introduce file formats that make use of a fanout array and a sorted table containing hashes, just like packfiles. Refactor the hash search in packfile.c into its own function, so that those patches can make use of it as well. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-15packfile: remove GIT_DEBUG_LOOKUP log statementsJonathan Tan
In commit 628522ec1439 ("sha1-lookup: more memory efficient search in sorted list of SHA-1", 2008-04-09), a different algorithm for searching a sorted list was introduced, together with a set of log statements guarded by GIT_DEBUG_LOOKUP that are invoked both when using that algorithm and when using the existing binary search. Those log statements was meant for experiments and debugging, but with the removal of the aforementioned different algorithm in commit f1068efefe6d ("sha1_file: drop experimental GIT_USE_LOOKUP search", 2017-08-09), those log statements are probably no longer necessary. Remove those statements. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-14Second batch for 2.17Junio C Hamano
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-02-13Merge branch 'tz/doc-show-defaults-to-head'Junio C Hamano
Doc update. * tz/doc-show-defaults-to-head: doc: mention 'git show' defaults to HEAD
2018-02-13Merge branch 'ew/svn-branch-segfault-fix'Junio C Hamano
Workaround for segfault with more recent versions of SVN. * ew/svn-branch-segfault-fix: git-svn: control destruction order to avoid segfault
2018-02-13Merge branch 'sg/travis-linux32-sanity'Junio C Hamano
Travis updates. * sg/travis-linux32-sanity: travis-ci: don't fail if user already exists on 32 bit Linux build job travis-ci: don't run the test suite as root in the 32 bit Linux build travis-ci: don't repeat the path of the cache directory travis-ci: use 'set -e' in the 32 bit Linux build job travis-ci: use 'set -x' for the commands under 'su' in the 32 bit Linux build