diff options
Diffstat (limited to 'Documentation')
60 files changed, 2043 insertions, 230 deletions
diff --git a/Documentation/CodingGuidelines b/Documentation/CodingGuidelines index 390ceec..45465bc 100644 --- a/Documentation/CodingGuidelines +++ b/Documentation/CodingGuidelines @@ -95,10 +95,6 @@ For shell scripts specifically (not exhaustive): - We use Arithmetic Expansion $(( ... )). - - Inside Arithmetic Expansion, spell shell variables with $ in front - of them, as some shells do not grok $((x)) while accepting $(($x)) - just fine (e.g. dash older than 0.5.4). - - We do not use Process Substitution <(list) or >(list). - Do not write control structures on a single line with semicolon. @@ -236,6 +232,18 @@ For C programs: while( condition ) func (bar+1); + - Do not explicitly compare an integral value with constant 0 or '\0', + or a pointer value with constant NULL. For instance, to validate that + counted array <ptr, cnt> is initialized but has no elements, write: + + if (!ptr || cnt) + BUG("empty array expected"); + + and not: + + if (ptr == NULL || cnt != 0); + BUG("empty array expected"); + - We avoid using braces unnecessarily. I.e. if (bla) { @@ -481,16 +489,11 @@ For Python scripts: - We follow PEP-8 (http://www.python.org/dev/peps/pep-0008/). - - As a minimum, we aim to be compatible with Python 2.6 and 2.7. + - As a minimum, we aim to be compatible with Python 2.7. - Where required libraries do not restrict us to Python 2, we try to also be compatible with Python 3.1 and later. - - When you must differentiate between Unicode literals and byte string - literals, it is OK to use the 'b' prefix. Even though the Python - documentation for version 2.6 does not mention this prefix, it has - been supported since version 2.6.0. - Error Messages - Do not end error messages with a full stop. diff --git a/Documentation/Makefile b/Documentation/Makefile index 15d9d04..ecd0b34 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -93,6 +93,7 @@ TECH_DOCS += technical/protocol-capabilities TECH_DOCS += technical/protocol-common TECH_DOCS += technical/protocol-v2 TECH_DOCS += technical/racy-git +TECH_DOCS += technical/reftable TECH_DOCS += technical/send-pack-pipeline TECH_DOCS += technical/shallow TECH_DOCS += technical/signature-format diff --git a/Documentation/MyFirstContribution.txt b/Documentation/MyFirstContribution.txt index 427274d..d85c9b5 100644 --- a/Documentation/MyFirstContribution.txt +++ b/Documentation/MyFirstContribution.txt @@ -1179,8 +1179,8 @@ look at the section below this one for some context.) [[after-approval]] === After Review Approval -The Git project has four integration branches: `pu`, `next`, `master`, and -`maint`. Your change will be placed into `pu` fairly early on by the maintainer +The Git project has four integration branches: `seen`, `next`, `master`, and +`maint`. Your change will be placed into `seen` fairly early on by the maintainer while it is still in the review process; from there, when it is ready for wider testing, it will be merged into `next`. Plenty of early testers use `next` and may report issues. Eventually, changes in `next` will make it to `master`, diff --git a/Documentation/RelNotes/2.27.0.txt b/Documentation/RelNotes/2.27.0.txt index 3f5c0bd..15518d0 100644 --- a/Documentation/RelNotes/2.27.0.txt +++ b/Documentation/RelNotes/2.27.0.txt @@ -24,6 +24,10 @@ Backward compatibility notes users may find annoying---those who prefer not to rebase need to set the variable to false to squelch the warning. + * The transport protocol version 2, which was promoted to the default + in Git 2.26 release, turned out to have some remaining rough edges, + so it has been demoted from the default. + UI, Workflows & Features @@ -96,6 +100,25 @@ UI, Workflows & Features check for the paths that were modified at each commit using Bloom filters. + * The approxidate parser learns to parse seconds with fraction and + ignore fractional part. + + * The userdiff patterns for Markdown documents have been added. + + * The sparse-checkout patterns have been forbidden from excluding all + paths, leaving an empty working tree, for a long time. This + limitation has been lifted. + + * "git restore --staged --worktree" now defaults to take the contents + out of "HEAD", instead of erring out. + + * "git p4" learned to recover from a (broken) state where a directory + and a file are recorded at the same path in the Perforce repository + the same way as their clients do. + + * "git multi-pack-index repack" has been taught to honor some + repack.* configuration variables. + Performance, Internal Implementation, Development Support etc. @@ -149,6 +172,21 @@ Performance, Internal Implementation, Development Support etc. * The "bugreport" tool has been added. + * The object walk with object filter "--filter=tree:0" can now take + advantage of the pack bitmap when available. + + * Instead of always building all branches at GitHub via Actions, + users can specify which branches to build. + + * Codepaths that show progress meter have been taught to also use the + start_progress() and the stop_progress() calls as a "region" to be + traced. + + * Instead of downloading Windows SDK for CI jobs for windows builds + from an external site (wingit.blob.core.windows.net), use the one + created in the windows-build job, to work around quota issues at + the external site. + Fixes since v2.26 ----------------- @@ -355,6 +393,107 @@ Fixes since v2.26 credential material embedded in URLs. (merge d192fa5006 js/anonymise-push-url-in-errors later to maint). + * Update the parser used for credential.<URL>.<variable> + configuration, to handle <URL>s with '/' in them correctly. + (merge b44d0118ac bc/wildcard-credential later to maint). + + * Recent updates broke parsing of "credential.<url>.<key>" where + <url> is not a full URL (e.g. [credential "https://"] helper = ...) + stopped working, which has been corrected. + (merge 9a121b0d22 js/partial-urlmatch-2.17 later to maint). + (merge cd93e6c029 js/partial-urlmatch later to maint). + + * Some of the files commit-graph subsystem keeps on disk did not + correctly honor the core.sharedRepository settings and some were + left read-write. + + * In error messages that "git switch" mentions its option to create a + new branch, "-b/-B" options were shown, where "-c/-C" options + should be, which has been corrected. + (merge 7c16ef7577 dl/switch-c-option-in-error-message later to maint). + + * With the recent tightening of the code that is used to parse + various parts of a URL for use in the credential subsystem, a + hand-edited credential-store file causes the credential helper to + die, which is a bit too harsh to the users. Demote the error + behaviour to just ignore and keep using well-formed lines instead. + (merge c03859a665 cb/credential-store-ignore-bogus-lines later to maint). + + * The samples in the credential documentation has been updated to + make it clear that we depict what would appear in the .git/config + file, by adding appropriate quotes as needed.. + (merge 177681a07e jk/credential-sample-update later to maint). + + * "git branch" and other "for-each-ref" variants accepted multiple + --sort=<key> options in the increasing order of precedence, but it + had a few breakages around "--ignore-case" handling, and tie-breaking + with the refname, which have been fixed. + (merge 7c5045fc18 jk/for-each-ref-multi-key-sort-fix later to maint). + + * The coding guideline for shell scripts instructed to refer to a + variable with dollar-sign inside arithmetic expansion to work + around a bug in old versions of dash, which is a thing of the past. + Now we are not forbidden from writing $((var+1)). + (merge 32b5fe7f0e jk/arith-expansion-coding-guidelines later to maint). + + * The <stdlib.h> header on NetBSD brings in its own definition of + hmac() function (eek), which conflicts with our own and unrelated + function with the same name. Our function has been renamed to work + around the issue. + (merge 3013118eb8 cb/avoid-colliding-with-netbsd-hmac later to maint). + + * The basic test did not honor $TEST_SHELL_PATH setting, which has + been corrected. + (merge 0555e4af58 cb/t0000-use-the-configured-shell later to maint). + + * Minor in-code comments and documentation updates around credential + API. + (merge 1aed817f99 cb/credential-doc-fixes later to maint). + + * Teach "am", "commit", "merge" and "rebase", when they are run with + the "--quiet" option, to pass "--quiet" down to "gc --auto". + (merge 7c3e9e8cfb jc/auto-gc-quiet later to maint). + + * The code to skip unmerged paths in the index when sparse checkout + is in use would have made out-of-bound access of the in-core index + when the last path was unmerged, which has been corrected. + + * Serving a "git fetch" client over "git://" and "ssh://" protocols + using the on-wire protocol version 2 was buggy on the server end + when the client needs to make a follow-up request to + e.g. auto-follow tags. + (merge 08450ef791 cc/upload-pack-v2-fetch-fix later to maint). + + * "git bisect replay" had trouble with input files when they used + CRLF line ending, which has been corrected. + (merge 6c722cbe5a cw/bisect-replay-with-dos later to maint). + + * "rebase -i" segfaulted when rearranging a sequence that has a + fix-up that applies another fix-up (which may or may not be a + fix-up of yet another step). + (merge 02471e7e20 js/rebase-autosquash-double-fixup-fix later to maint). + + * "git fsck" ensures that the paths recorded in tree objects are + sorted and without duplicates, but it failed to notice a case where + a blob is followed by entries that sort before a tree with the same + name. This has been corrected. + (merge 9068cfb20f rs/fsck-duplicate-names-in-trees later to maint). + + * Code clean-up by removing a compatibility implementation of a + function we no longer use. + (merge 84b0115f0d cb/no-more-gmtime later to maint). + + * When a binary file gets modified and renamed on both sides of history + to different locations, both files would be written to the working + tree but both would have the contents from "ours". This has been + corrected so that the path from each side gets their original content. + + * Fix for a copy-and-paste error introduced during 2.20 era. + (merge e68a5272b1 ds/multi-pack-verify later to maint). + + * Update an unconditional use of "grep -a" with a perl script in a test. + (merge 1eb7371236 dd/t5703-grep-a-fix later to maint). + * Other code cleanup, docfix, build fix, etc. (merge 564956f358 jc/maintain-doc later to maint). (merge 7422b2a0a1 sg/commit-slab-clarify-peek later to maint). @@ -381,3 +520,6 @@ Fixes since v2.26 (merge 39102cf4fe ms/doc-revision-illustration-fix later to maint). (merge 4d9378bfad eb/gitweb-more-trailers later to maint). (merge bdccbf7047 mt/doc-worktree-ref later to maint). + (merge ce9baf234f dl/push-recurse-submodules-fix later to maint). + (merge 4153274052 bc/doc-credential-helper-value later to maint). + (merge 5c7bb0146e jc/codingstyle-compare-with-null later to maint). diff --git a/Documentation/RelNotes/2.28.0.txt b/Documentation/RelNotes/2.28.0.txt new file mode 100644 index 0000000..c664771 --- /dev/null +++ b/Documentation/RelNotes/2.28.0.txt @@ -0,0 +1,227 @@ +Git 2.28 Release Notes +====================== + +Updates since v2.27 +------------------- + +Backward compatibility notes + + * "fetch.writeCommitGraph" is deemed to be still a bit too risky and + is no longer part of the "feature.experimental" set. + + * It used to be that setting extensions.* configuration variables + alone, while leaving core.repositoryFormatVersion=0, made these + settings effective, which was a wrong thing to do. In version 0, + there was no special meaning in extensions.* configuration + variables. This has been corrected. If you need these repository + extensions to be effective, the core.repositoryFormatVersion + variable needs to be updated to 1 after vetting these extensions.* + variables are set correctly. + + +UI, Workflows & Features + + * The commands in the "diff" family learned to honor "diff.relative" + configuration variable. + + * The check in "git fsck" to ensure that the tree objects are sorted + still had corner cases it missed unsorted entries. + + * The interface to redact sensitive information in the trace output + has been simplified. + + * The command line completion (in contrib/) learned to complete + options that the "git switch" command takes. + + * "git diff" used to take arguments in random and nonsense range + notation, e.g. "git diff A..B C", "git diff A..B C...D", etc., + which has been cleaned up. + + * "git diff-files" has been taught to say paths that are marked as + intent-to-add are new files, not modified from an empty blob. + + * "git status" learned to report the status of sparse checkout. + + * "git difftool" has trouble dealing with paths added to the index + with the intent-to-add bit. + + * "git fast-export --anonymize" learned to take customized mapping to + allow its users to tweak its output more usable for debugging. + + * The command line completion support (in contrib/) used to be + prepared to work with "set -u" but recent changes got a bit more + sloppy. This has been corrected. + + +Performance, Internal Implementation, Development Support etc. + + * Code optimization for a common case. + (merge 8777616e4d an/merge-single-strategy-optim later to maint). + + * We've adopted a convention that any on-stack structure can be + initialized to have zero values in all fields with "= { 0 }", + even when the first field happens to be a pointer, but sparse + complained that a null pointer should be spelled NULL for a long + time. Start using -Wno-universal-initializer option to squelch + it (the latest sparse has it on by default). + + * "git log -L..." now takes advantage of the "which paths are touched + by this commit?" info stored in the commit-graph system. + + * As FreeBSD is not the only platform whose regexp library reports + a REG_ILLSEQ error when fed invalid UTF-8, add logic to detect that + automatically and skip the affected tests. + + * "git bugreport" learns to report what shell is in use. + + * Support for GIT_CURL_VERBOSE has been rewritten in terms of + GIT_TRACE_CURL. + + * Preliminary clean-ups around refs API, plus file format + specification documentation for the reftable backend. + + * Workaround breakage in MSVC build, where "curl-config --cflags" + gives settings appropriate for GCC build. + + * Code clean-up of "git clean" resulted in a fix of recent + performance regression. + + * Code clean-up in the codepath that serves "git fetch" continues. + + * "git merge-base --is-ancestor" is taught to take advantage of the + commit graph. + + * Rewrite of parts of the scripted "git submodule" Porcelain command + continues; this time it is "git submodule set-branch" subcommand's + turn. + + * The "fetch/clone" protocol has been updated to allow the server to + instruct the clients to grab pre-packaged packfile(s) in addition + to the packed object data coming over the wire. + + * A misdesigned strbuf_write_fd() function has been retired. + + * SHA-256 migration work continues, including CVS/SVN interface. + + * A few fields in "struct commit" that do not have to always be + present have been moved to commit slabs. + + * API cleanup for get_worktrees() + + * By renumbering object flag bits, "struct object" managed to lose + bloated inter-field padding. + + * The name of the primary branch in existing repositories, and the + default name used for the first branch in newly created + repositories, is made configurable, so that we can eventually wean + ourselves off of the hardcoded 'master'. + + * The effort to avoid using test_must_fail on non-git command continues. + + +Fixes since v2.27 +----------------- + + * The "--prepare-p4-only" option of "git p4" is supposed to stop + after replaying one changeset, but kept going (by mistake?) + + * The error message from "git checkout -b foo -t bar baz" was + confusing. + + * Some repositories in the wild have commits that record nonsense + committer timezone (e.g. rails.git); "git fast-import" learned an + option to pass these nonsense timestamps intact to allow recreating + existing repositories as-is. + (merge d42a2fb72f en/fast-import-looser-date later to maint). + + * The command line completion script (in contrib/) tried to complete + "git stash -p" as if it were "git stash push -p", but it was too + aggressive and also affected "git stash show -p", which has been + corrected. + (merge fffd0cf520 vs/complete-stash-show-p-fix later to maint). + + * On-the-wire protocol v2 easily falls into a deadlock between the + remote-curl helper and the fetch-pack process when the server side + prematurely throws an error and disconnects. The communication has + been updated to make it more robust. + + * "git checkout -p" did not handle a newly added path at all. + (merge 2c8bd8471a js/checkout-p-new-file later to maint). + + * The code to parse "git bisect start" command line was lax in + validating the arguments. + (merge 4d9005ff5d cb/bisect-helper-parser-fix later to maint). + + * Reduce memory usage during "diff --quiet" in a worktree with too + many stat-unmatched paths. + (merge d2d7fbe129 jk/diff-memuse-optim-with-stat-unmatch later to maint). + + * The reflog entries for "git clone" and "git fetch" did not + anonymize the URL they operated on. + (merge 46da295a77 js/reflog-anonymize-for-clone-and-fetch later to maint). + + * The behaviour of "sparse-checkout" in the state "git clone + --no-checkout" left was changed accidentally in 2.27, which has + been corrected. + + * Use of negative pathspec, while collecting paths including + untracked ones in the working tree, was broken. + + * The same worktree directory must be registered only once, but + "git worktree move" allowed this invariant to be violated, which + has been corrected. + (merge 810382ed37 es/worktree-duplicate-paths later to maint). + + * The effect of sparse checkout settings on submodules is documented. + (merge e7d7c73249 en/sparse-with-submodule-doc later to maint). + + * Code clean-up around "git branch" with a minor bugfix. + (merge dc44639904 dl/branch-cleanup later to maint). + + * A branch name used in a test has been clarified to match what is + going on. + (merge 08dc26061f pb/t4014-unslave later to maint). + + * An in-code comment in "git diff" has been updated. + (merge c592fd4c83 dl/diff-usage-comment-update later to maint). + + * The documentation and some tests have been adjusted for the recent + renaming of "pu" branch to "seen". + (merge 6dca5dbf93 js/pu-to-seen later to maint). + + * The code to push changes over "dumb" HTTP had a bad interaction + with the commit reachability code due to incorrect allocation of + object flag bits, which has been corrected. + (merge 64472d15e9 bc/http-push-flagsfix later to maint). + + * "git send-email --in-reply-to=<msg>" did not use the In-Reply-To: + header with the value given from the command line, and let it be + overridden by the value on In-Reply-To: header in the messages + being sent out (if exists). + (merge f9f60d7066 ra/send-email-in-reply-to-from-command-line-wins later to maint). + + * "git log -Lx,y:path --before=date" lost track of where the range + should be because it didn't take the changes made by the youngest + commits that are omitted from the output into account. + + * When "fetch.writeCommitGraph" configuration is set in a shallow + repository and a fetch moves the shallow boundary, we wrote out + broken commit-graph files that do not match the reality, which has + been corrected. + + * "git checkout" failed to catch an error from fstat() after updating + a path in the working tree. + (merge 35e6e212fd mt/entry-fstat-fallback-fix later to maint). + + * Other code cleanup, docfix, build fix, etc. + (merge 2c31a7aa44 jx/pkt-line-doc-count-fix later to maint). + (merge d63ae31962 cb/t5608-cleanup later to maint). + (merge 788db145c7 dl/t-readme-spell-git-correctly later to maint). + (merge 45a87a83bb dl/python-2.7-is-the-floor-version later to maint). + (merge b75a219904 es/advertise-contribution-doc later to maint). + (merge 0c9a4f638a rs/pull-leakfix later to maint). + (merge d546fe2874 rs/commit-reach-leakfix later to maint). + (merge 087bf5409c mk/pb-pretty-email-without-domain-part-fix later to maint). + (merge 5f4ee57ad9 es/worktree-code-cleanup later to maint). + (merge 0172f7834a cc/cat-file-usage-update later to maint). + (merge 81de0c01cf ma/rebase-doc-typofix later to maint). diff --git a/Documentation/SubmittingPatches b/Documentation/SubmittingPatches index 4515cab..291b61e 100644 --- a/Documentation/SubmittingPatches +++ b/Documentation/SubmittingPatches @@ -3,8 +3,9 @@ Submitting Patches == Guidelines -Here are some guidelines for people who want to contribute their code -to this software. +Here are some guidelines for people who want to contribute their code to this +software. There is also a link:MyFirstContribution.html[step-by-step tutorial] +available which covers many of these same guidelines. [[base-branch]] === Decide what to base your work on. @@ -18,7 +19,7 @@ change is relevant to. base your work on the tip of the topic. * A new feature should be based on `master` in general. If the new - feature depends on a topic that is in `pu`, but not in `master`, + feature depends on a topic that is in `seen`, but not in `master`, base your work on the tip of that topic. * Corrections and enhancements to a topic not yet in `master` should @@ -27,7 +28,7 @@ change is relevant to. into the series. * In the exceptional case that a new feature depends on several topics - not in `master`, start working on `next` or `pu` privately and send + not in `master`, start working on `next` or `seen` privately and send out patches for discussion. Before the final merge, you may have to wait until some of the dependent topics graduate to `master`, and rebase your work. @@ -37,7 +38,7 @@ change is relevant to. these parts should be based on their trees. To find the tip of a topic branch, run `git log --first-parent -master..pu` and look for the merge commit. The second parent of this +master..seen` and look for the merge commit. The second parent of this commit is the tip of the topic branch. [[separate-commits]] @@ -423,7 +424,7 @@ help you find out who they are. and cooked further and eventually graduates to `master`. In any time between the (2)-(3) cycle, the maintainer may pick it up -from the list and queue it to `pu`, in order to make it easier for +from the list and queue it to `seen`, in order to make it easier for people play with it without having to pick up and apply the patch to their trees themselves. @@ -434,7 +435,7 @@ their trees themselves. master. `git pull --rebase` will automatically skip already-applied patches, and will let you know. This works only if you rebase on top of the branch in which your patch has been merged (i.e. it will not - tell you if your patch is merged in pu if you rebase on top of + tell you if your patch is merged in `seen` if you rebase on top of master). * Read the Git mailing list, the maintainer regularly posts messages diff --git a/Documentation/config/credential.txt b/Documentation/config/credential.txt index 60fb318..9d01641 100644 --- a/Documentation/config/credential.txt +++ b/Documentation/config/credential.txt @@ -1,9 +1,13 @@ credential.helper:: Specify an external helper to be called when a username or password credential is needed; the helper may consult external - storage to avoid prompting the user for the credentials. Note - that multiple helpers may be defined. See linkgit:gitcredentials[7] - for details. + storage to avoid prompting the user for the credentials. This is + normally the name of a credential helper with possible + arguments, but may also be an absolute path with arguments or, if + preceded by `!`, shell commands. ++ +Note that multiple helpers may be defined. See linkgit:gitcredentials[7] +for details and examples. credential.useHttpPath:: When acquiring credentials, consider the "path" component of an http diff --git a/Documentation/config/diff.txt b/Documentation/config/diff.txt index ff09f1c..c3ae136 100644 --- a/Documentation/config/diff.txt +++ b/Documentation/config/diff.txt @@ -105,6 +105,10 @@ diff.mnemonicPrefix:: diff.noprefix:: If set, 'git diff' does not show any source or destination prefix. +diff.relative:: + If set to 'true', 'git diff' does not show changes outside of the directory + and show pathnames relative to the current directory. + diff.orderFile:: File indicating how to order files within a diff. See the '-O' option to linkgit:git-diff[1] for details. diff --git a/Documentation/config/feature.txt b/Documentation/config/feature.txt index 4e3a5c0..c0cbf2b 100644 --- a/Documentation/config/feature.txt +++ b/Documentation/config/feature.txt @@ -15,13 +15,9 @@ feature.experimental:: * `fetch.negotiationAlgorithm=skipping` may improve fetch negotiation times by skipping more commits at a time, reducing the number of round trips. + -* `fetch.writeCommitGraph=true` writes a commit-graph after every `git fetch` -command that downloads a pack-file from a remote. Using the `--split` option, -most executions will create a very small commit-graph file on top of the -existing commit-graph file(s). Occasionally, these files will merge and the -write may take longer. Having an updated commit-graph file helps performance -of many Git commands, including `git merge-base`, `git push -f`, and -`git log --graph`. +* `protocol.version=2` speeds up fetches from repositories with many refs by +allowing the client to specify which refs to list before the server lists +them. feature.manyFiles:: Enable config options that optimize for repos with many files in the diff --git a/Documentation/config/fetch.txt b/Documentation/config/fetch.txt index b1a9b14..b203940 100644 --- a/Documentation/config/fetch.txt +++ b/Documentation/config/fetch.txt @@ -90,5 +90,4 @@ fetch.writeCommitGraph:: the existing commit-graph file(s). Occasionally, these files will merge and the write may take longer. Having an updated commit-graph file helps performance of many Git commands, including `git merge-base`, - `git push -f`, and `git log --graph`. Defaults to false, unless - `feature.experimental` is true. + `git push -f`, and `git log --graph`. Defaults to false. diff --git a/Documentation/config/init.txt b/Documentation/config/init.txt index 46fa8c6..dc77f8c 100644 --- a/Documentation/config/init.txt +++ b/Documentation/config/init.txt @@ -1,3 +1,7 @@ init.templateDir:: Specify the directory from which templates will be copied. (See the "TEMPLATE DIRECTORY" section of linkgit:git-init[1].) + +init.defaultBranch:: + Allows overriding the default branch name e.g. when initializing + a new repository or when cloning an empty repository. diff --git a/Documentation/config/protocol.txt b/Documentation/config/protocol.txt index 0b40141..c46e9b3 100644 --- a/Documentation/config/protocol.txt +++ b/Documentation/config/protocol.txt @@ -48,7 +48,8 @@ protocol.version:: If set, clients will attempt to communicate with a server using the specified protocol version. If the server does not support it, communication falls back to version 0. - If unset, the default is `0`. + If unset, the default is `0`, unless `feature.experimental` + is enabled, in which case the default is `2`. Supported versions: + -- diff --git a/Documentation/date-formats.txt b/Documentation/date-formats.txt index 6926e0a..f1097fa 100644 --- a/Documentation/date-formats.txt +++ b/Documentation/date-formats.txt @@ -20,7 +20,9 @@ RFC 2822:: ISO 8601:: Time and date specified by the ISO 8601 standard, for example `2005-04-07T22:13:13`. The parser accepts a space instead of the - `T` character as well. + `T` character as well. Fractional parts of a second will be ignored, + for example `2005-04-07T22:13:13.019` will be treated as + `2005-04-07T22:13:13`. + NOTE: In addition, the date part is accepted in the following formats: `YYYY.MM.DD`, `MM/DD/YYYY` and `DD.MM.YYYY`. diff --git a/Documentation/diff-options.txt b/Documentation/diff-options.txt index bb31f0c..7987d72 100644 --- a/Documentation/diff-options.txt +++ b/Documentation/diff-options.txt @@ -643,15 +643,18 @@ ifndef::git-format-patch[] -R:: Swap two inputs; that is, show differences from index or on-disk file to tree contents. +endif::git-format-patch[] --relative[=<path>]:: +--no-relative:: When run from a subdirectory of the project, it can be told to exclude changes outside the directory and show pathnames relative to it with this option. When you are not in a subdirectory (e.g. in a bare repository), you can name which subdirectory to make the output relative to by giving a <path> as an argument. -endif::git-format-patch[] + `--no-relative` can be used to countermand both `diff.relative` config + option and previous `--relative`. -a:: --text:: diff --git a/Documentation/git-branch.txt b/Documentation/git-branch.txt index 135206f..03c0824 100644 --- a/Documentation/git-branch.txt +++ b/Documentation/git-branch.txt @@ -12,7 +12,7 @@ SYNOPSIS [-v [--abbrev=<length> | --no-abbrev]] [--column[=<options>] | --no-column] [--sort=<key>] [(--merged | --no-merged) [<commit>]] - [--contains [<commit]] [--no-contains [<commit>]] + [--contains [<commit>]] [--no-contains [<commit>]] [--points-at <object>] [--format=<format>] [(-r | --remotes) | (-a | --all)] [--list] [<pattern>...] diff --git a/Documentation/git-bugreport.txt b/Documentation/git-bugreport.txt index 643d1b2..66e88c2 100644 --- a/Documentation/git-bugreport.txt +++ b/Documentation/git-bugreport.txt @@ -28,6 +28,8 @@ The following information is captured automatically: - 'git version --build-options' - uname sysname, release, version, and machine strings - Compiler-specific info string + - A list of enabled hooks + - $SHELL This tool is invoked via the typical Git setup process, which means that in some cases, it might not be able to launch - for example, if a relevant config file @@ -45,7 +47,7 @@ OPTIONS --suffix <format>:: Specify an alternate suffix for the bugreport name, to create a file named 'git-bugreport-<formatted suffix>'. This should take the form of a - link:strftime[3] format string; the current local time will be used. + strftime(3) format string; the current local time will be used. GIT --- diff --git a/Documentation/git-cat-file.txt b/Documentation/git-cat-file.txt index 8eca671..8e192d8 100644 --- a/Documentation/git-cat-file.txt +++ b/Documentation/git-cat-file.txt @@ -10,7 +10,7 @@ SYNOPSIS -------- [verse] 'git cat-file' (-t [--allow-unknown-type]| -s [--allow-unknown-type]| -e | -p | <type> | --textconv | --filters ) [--path=<path>] <object> -'git cat-file' (--batch | --batch-check) [ --textconv | --filters ] [--follow-symlinks] +'git cat-file' (--batch[=<format>] | --batch-check[=<format>]) [ --textconv | --filters ] [--follow-symlinks] DESCRIPTION ----------- diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt index 08d6045..c898310 100644 --- a/Documentation/git-clone.txt +++ b/Documentation/git-clone.txt @@ -259,7 +259,7 @@ maintain a branch with no references other than a single cloned branch. This is useful e.g. to maintain minimal clones of the default branch of some repository for search indexing. ---recurse-submodules[=<pathspec]:: +--recurse-submodules[=<pathspec>]:: After the clone is created, initialize and clone submodules within based on the provided pathspec. If no pathspec is provided, all submodules are initialized and cloned. diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt index fcac7d1..8ca1764 100644 --- a/Documentation/git-commit-graph.txt +++ b/Documentation/git-commit-graph.txt @@ -60,7 +60,7 @@ With the `--append` option, include all commits that are present in the existing commit-graph file. + With the `--changed-paths` option, compute and write information about the -paths changed between a commit and it's first parent. This operation can +paths changed between a commit and its first parent. This operation can take a while on large repositories. It provides significant performance gains for getting history of a directory or a file with `git log -- <path>`. + @@ -70,6 +70,7 @@ chain of multiple commit-graph files stored in strategy and other splitting options. The new commits not already in the commit-graph are added in a new "tip" file. This file is merged with the existing file if the following merge conditions are met: ++ * If `--split=no-merge` is specified, a merge is never performed, and the remaining options are ignored. `--split=replace` overwrites the existing chain with a new one. A bare `--split` defers to the remaining diff --git a/Documentation/git-credential-store.txt b/Documentation/git-credential-store.txt index 693dd9d..76b0798 100644 --- a/Documentation/git-credential-store.txt +++ b/Documentation/git-credential-store.txt @@ -94,6 +94,10 @@ stored on its own line as a URL like: https://user:pass@example.com ------------------------------ +No other kinds of lines (e.g. empty lines or comment lines) are +allowed in the file, even though some may be silently ignored. Do +not view or edit the file with editors. + When Git needs authentication for a particular URL context, credential-store will consider that context a pattern to match against each entry in the credentials file. If the protocol, hostname, and diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt index 6f0c7ca..31c81c4 100644 --- a/Documentation/git-credential.txt +++ b/Documentation/git-credential.txt @@ -103,17 +103,20 @@ INPUT/OUTPUT FORMAT `git credential` reads and/or writes (depending on the action used) credential information in its standard input/output. This information can correspond either to keys for which `git credential` will obtain -the login/password information (e.g. host, protocol, path), or to the -actual credential data to be obtained (login/password). +the login information (e.g. host, protocol, path), or to the actual +credential data to be obtained (username/password). The credential is split into a set of named attributes, with one -attribute per line. Each attribute is -specified by a key-value pair, separated by an `=` (equals) sign, -followed by a newline. The key may contain any bytes except `=`, -newline, or NUL. The value may contain any bytes except newline or NUL. +attribute per line. Each attribute is specified by a key-value pair, +separated by an `=` (equals) sign, followed by a newline. + +The key may contain any bytes except `=`, newline, or NUL. The value may +contain any bytes except newline or NUL. + In both cases, all bytes are treated as-is (i.e., there is no quoting, and one cannot transmit a value with newline or NUL in it). The list of attributes is terminated by a blank line or end-of-file. + Git understands the following attributes: `protocol`:: @@ -123,7 +126,8 @@ Git understands the following attributes: `host`:: - The remote hostname for a network credential. + The remote hostname for a network credential. This includes + the port number if one was specified (e.g., "example.com:8088"). `path`:: @@ -134,7 +138,7 @@ Git understands the following attributes: `username`:: The credential's username, if we already have one (e.g., from a - URL, from the user, or from a previously run helper). + URL, the configuration, the user, or from a previously run helper). `password`:: @@ -146,8 +150,12 @@ Git understands the following attributes: value is parsed as a URL and treated as if its constituent parts were read (e.g., `url=https://example.com` would behave as if `protocol=https` and `host=example.com` had been provided). This - can help callers avoid parsing URLs themselves. Note that any - components which are missing from the URL (e.g., there is no - username in the example above) will be set to empty; if you want - to provide a URL and override some attributes, provide the URL - attribute first, followed by any overrides. + can help callers avoid parsing URLs themselves. ++ +Note that specifying a protocol is mandatory and if the URL +doesn't specify a hostname (e.g., "cert:///path/to/file") the +credential will contain a hostname attribute whose value is an +empty string. ++ +Components which are missing from the URL (e.g., there is no +username in the example above) will be left unset. diff --git a/Documentation/git-diff.txt b/Documentation/git-diff.txt index 37781cf..727f24d 100644 --- a/Documentation/git-diff.txt +++ b/Documentation/git-diff.txt @@ -11,15 +11,17 @@ SYNOPSIS [verse] 'git diff' [<options>] [<commit>] [--] [<path>...] 'git diff' [<options>] --cached [<commit>] [--] [<path>...] -'git diff' [<options>] <commit> <commit> [--] [<path>...] +'git diff' [<options>] <commit> [<commit>...] <commit> [--] [<path>...] +'git diff' [<options>] <commit>...<commit> [--] [<path>...] 'git diff' [<options>] <blob> <blob> 'git diff' [<options>] --no-index [--] <path> <path> DESCRIPTION ----------- Show changes between the working tree and the index or a tree, changes -between the index and a tree, changes between two trees, changes between -two blob objects, or changes between two files on disk. +between the index and a tree, changes between two trees, changes resulting +from a merge, changes between two blob objects, or changes between two +files on disk. 'git diff' [<options>] [--] [<path>...]:: @@ -61,9 +63,19 @@ two blob objects, or changes between two files on disk. This is to view the changes between two arbitrary <commit>. +'git diff' [<options>] <commit> <commit>... <commit> [--] [<path>...]:: + + This form is to view the results of a merge commit. The first + listed <commit> must be the merge itself; the remaining two or + more commits should be its parents. A convenient way to produce + the desired set of revisions is to use the {caret}@ suffix. + For instance, if `master` names a merge commit, `git diff master + master^@` gives the same combined diff as `git show master`. + 'git diff' [<options>] <commit>..<commit> [--] [<path>...]:: - This is synonymous to the previous form. If <commit> on + This is synonymous to the earlier form (without the "..") for + viewing the changes between two arbitrary <commit>. If <commit> on one side is omitted, it will have the same effect as using HEAD instead. @@ -196,7 +208,8 @@ linkgit:git-difftool[1], linkgit:git-log[1], linkgit:gitdiffcore[7], linkgit:git-format-patch[1], -linkgit:git-apply[1] +linkgit:git-apply[1], +linkgit:git-show[1] GIT --- diff --git a/Documentation/git-fast-export.txt b/Documentation/git-fast-export.txt index e8950de..1978dbd 100644 --- a/Documentation/git-fast-export.txt +++ b/Documentation/git-fast-export.txt @@ -119,6 +119,11 @@ by keeping the marks the same across runs. the shape of the history and stored tree. See the section on `ANONYMIZING` below. +--anonymize-map=<from>[:<to>]:: + Convert token `<from>` to `<to>` in the anonymized output. If + `<to>` is omitted, map `<from>` to itself (i.e., do not + anonymize it). See the section on `ANONYMIZING` below. + --reference-excluded-parents:: By default, running a command such as `git fast-export master~5..master` will not include the commit master{tilde}5 @@ -238,6 +243,30 @@ collapse "User 0", "User 1", etc into "User X"). This produces a much smaller output, and it is usually easy to quickly confirm that there is no private data in the stream. +Reproducing some bugs may require referencing particular commits or +paths, which becomes challenging after refnames and paths have been +anonymized. You can ask for a particular token to be left as-is or +mapped to a new value. For example, if you have a bug which reproduces +with `git rev-list sensitive -- secret.c`, you can run: + +--------------------------------------------------- +$ git fast-export --anonymize --all \ + --anonymize-map=sensitive:foo \ + --anonymize-map=secret.c:bar.c \ + >stream +--------------------------------------------------- + +After importing the stream, you can then run `git rev-list foo -- bar.c` +in the anonymized repository. + +Note that paths and refnames are split into tokens at slash boundaries. +The command above would anonymize `subdir/secret.c` as something like +`path123/bar.c`; you could then search for `bar.c` in the anonymized +repository to determine the final pathname. + +To make referencing the final pathname simpler, you can map each path +component; so if you also anonymize `subdir` to `publicdir`, then the +final pathname would be `publicdir/bar.c`. LIMITATIONS ----------- diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index 77c6b3d..7d9aad2 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -293,7 +293,14 @@ by users who are located in the same location and time zone. In this case a reasonable offset from UTC could be assumed. + Unlike the `rfc2822` format, this format is very strict. Any -variation in formatting will cause fast-import to reject the value. +variation in formatting will cause fast-import to reject the value, +and some sanity checks on the numeric values may also be performed. + +`raw-permissive`:: + This is the same as `raw` except that no sanity checks on + the numeric epoch and local offset are performed. This can + be useful when trying to filter or import an existing history + with e.g. bogus timezone values. `rfc2822`:: This is the standard email format as described by RFC 2822. diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt index 5b1909f..45b6d8e 100644 --- a/Documentation/git-fetch.txt +++ b/Documentation/git-fetch.txt @@ -255,14 +255,14 @@ refspec. * Using refspecs explicitly: + ------------------------------------------------ -$ git fetch origin +pu:pu maint:tmp +$ git fetch origin +seen:seen maint:tmp ------------------------------------------------ + -This updates (or creates, as necessary) branches `pu` and `tmp` in +This updates (or creates, as necessary) branches `seen` and `tmp` in the local repository by fetching from the branches (respectively) -`pu` and `maint` from the remote repository. +`seen` and `maint` from the remote repository. + -The `pu` branch will be updated even if it does not fast-forward, +The `seen` branch will be updated even if it does not fast-forward, because it is prefixed with a plus sign; `tmp` will not be. * Peek at a remote's branch, without configuring the remote in your local diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt index 666b042..4deb489 100644 --- a/Documentation/git-http-fetch.txt +++ b/Documentation/git-http-fetch.txt @@ -9,7 +9,7 @@ git-http-fetch - Download from a remote Git repository via HTTP SYNOPSIS -------- [verse] -'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin] <commit> <url> +'git http-fetch' [-c] [-t] [-a] [-d] [-v] [-w filename] [--recover] [--stdin | --packfile=<hash> | <commit>] <url> DESCRIPTION ----------- @@ -40,6 +40,13 @@ commit-id:: <commit-id>['\t'<filename-as-in--w>] +--packfile=<hash>:: + Instead of a commit id on the command line (which is not expected in + this case), 'git http-fetch' fetches the packfile directly at the given + URL and uses index-pack to generate corresponding .idx and .keep files. + The hash is used to determine the name of the temporary file and is + arbitrary. The output of index-pack is printed to stdout. + --recover:: Verify that everything reachable from target is fetched. Used after an earlier fetch is interrupted. diff --git a/Documentation/git-index-pack.txt b/Documentation/git-index-pack.txt index d5b7560..9316d9a 100644 --- a/Documentation/git-index-pack.txt +++ b/Documentation/git-index-pack.txt @@ -93,6 +93,14 @@ OPTIONS --max-input-size=<size>:: Die, if the pack is larger than <size>. +--object-format=<hash-algorithm>:: + Specify the given object format (hash algorithm) for the pack. The valid + values are 'sha1' and (if enabled) 'sha256'. The default is the algorithm for + the current repository (set by `extensions.objectFormat`), or 'sha1' if no + value is set or outside a repository. ++ +This option cannot be used with --stdin. + NOTES ----- diff --git a/Documentation/git-init.txt b/Documentation/git-init.txt index adc6adf..ddfe265 100644 --- a/Documentation/git-init.txt +++ b/Documentation/git-init.txt @@ -10,7 +10,8 @@ SYNOPSIS -------- [verse] 'git init' [-q | --quiet] [--bare] [--template=<template_directory>] - [--separate-git-dir <git dir>] [--object-format=<format] + [--separate-git-dir <git dir>] [--object-format=<format>] + [-b <branch-name> | --initial-branch=<branch-name>] [--shared[=<permissions>]] [directory] @@ -67,6 +68,12 @@ repository. + If this is reinitialization, the repository will be moved to the specified path. +-b <branch-name:: +--initial-branch=<branch-name>:: + +Use the specified name for the initial branch in the newly created repository. +If not specified, fall back to the default name: `master`. + --shared[=(false|true|umask|group|all|world|everybody|0xxx)]:: Specify that the Git repository is to be shared amongst several users. This diff --git a/Documentation/git-ls-remote.txt b/Documentation/git-ls-remote.txt index 0a5c8b7..492e573 100644 --- a/Documentation/git-ls-remote.txt +++ b/Documentation/git-ls-remote.txt @@ -101,9 +101,9 @@ f25a265a342aed6041ab0cc484224d9ca54b6f41 refs/tags/v0.99.1 7ceca275d047c90c0c7d5afb13ab97efdf51bd6e refs/tags/v0.99.3 c5db5456ae3b0873fc659c19fafdde22313cc441 refs/tags/v0.99.2 0918385dbd9656cab0d1d81ba7453d49bbc16250 refs/tags/junio-gpg-pub -$ git ls-remote http://www.kernel.org/pub/scm/git/git.git master pu rc +$ git ls-remote http://www.kernel.org/pub/scm/git/git.git master seen rc 5fe978a5381f1fbad26a80e682ddd2a401966740 refs/heads/master -c781a84b5204fb294c9ccc79f8b3baceeb32c061 refs/heads/pu +c781a84b5204fb294c9ccc79f8b3baceeb32c061 refs/heads/seen $ git remote add korg http://www.kernel.org/pub/scm/git/git.git $ git ls-remote --tags korg v\* d6602ec5194c87b0fc87103ca4d67251c76f233a refs/tags/v0.99 diff --git a/Documentation/git-merge.txt b/Documentation/git-merge.txt index ec06b2f..3819fad 100644 --- a/Documentation/git-merge.txt +++ b/Documentation/git-merge.txt @@ -106,12 +106,12 @@ commit or stash your changes before running 'git merge'. `MERGE_HEAD` is present unless `MERGE_AUTOSTASH` is also present in which case 'git merge --abort' applies the stash entry to the worktree whereas 'git reset --merge' will save the stashed changes in the stash -reflog. +list. --quit:: Forget about the current merge in progress. Leave the index and the working tree as-is. If `MERGE_AUTOSTASH` is present, the - stash entry will be saved to the stash reflog. + stash entry will be saved to the stash list. --continue:: After a 'git merge' stops due to conflicts you can conclude the diff --git a/Documentation/git-multi-pack-index.txt b/Documentation/git-multi-pack-index.txt index 642d9ac..0c66194 100644 --- a/Documentation/git-multi-pack-index.txt +++ b/Documentation/git-multi-pack-index.txt @@ -56,6 +56,9 @@ repack:: file is created, rewrite the multi-pack-index to reference the new pack-file. A later run of 'git multi-pack-index expire' will delete the pack-files that were part of this batch. ++ +If `repack.packKeptObjects` is `false`, then any pack-files with an +associated `.keep` file will not be selected for the batch to repack. EXAMPLES diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index 794f2f3..374d248 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -257,9 +257,9 @@ See also INCOMPATIBLE OPTIONS below. Abort the rebase operation but HEAD is not reset back to the original branch. The index and working tree are also left unchanged as a result. If a temporary stash entry was created - using --autostash, it will be saved to the stash reflog. + using --autostash, it will be saved to the stash list. ---apply: +--apply:: Use applying strategies to rebase (calling `git-am` internally). This option may become a no-op in the future once the merge backend handles everything the apply one does. diff --git a/Documentation/git-restore.txt b/Documentation/git-restore.txt index 8e3b339..84c6c40 100644 --- a/Documentation/git-restore.txt +++ b/Documentation/git-restore.txt @@ -22,9 +22,8 @@ The command can also be used to restore the content in the index with `--staged`, or restore both the working tree and the index with `--staged --worktree`. -By default, the restore sources for working tree and the index are the -index and `HEAD` respectively. `--source` could be used to specify a -commit as the restore source. +By default, if `--staged` is given, the contents are restored from `HEAD`, +otherwise from the index. Use `--source` to restore from a different commit. See "Reset, restore and revert" in linkgit:git[1] for the differences between the three commands. @@ -39,10 +38,8 @@ OPTIONS tree. It is common to specify the source tree by naming a commit, branch or tag associated with it. + -If not specified, the default restore source for the working tree is -the index, and the default restore source for the index is -`HEAD`. When both `--staged` and `--worktree` are specified, -`--source` must also be specified. +If not specified, the contents are restored from `HEAD` if `--staged` is +given, otherwise from the index. -p:: --patch:: diff --git a/Documentation/git-show-index.txt b/Documentation/git-show-index.txt index 424e4ba..39b1d8e 100644 --- a/Documentation/git-show-index.txt +++ b/Documentation/git-show-index.txt @@ -9,7 +9,7 @@ git-show-index - Show packed archive index SYNOPSIS -------- [verse] -'git show-index' +'git show-index' [--object-format=<hash-algorithm>] DESCRIPTION @@ -36,6 +36,15 @@ Note that you can get more information on a packfile by calling linkgit:git-verify-pack[1]. However, as this command considers only the index file itself, it's both faster and more flexible. +OPTIONS +------- + +--object-format=<hash-algorithm>:: + Specify the given object format (hash algorithm) for the index file. The + valid values are 'sha1' and (if enabled) 'sha256'. The default is the + algorithm for the current repository (set by `extensions.objectFormat`), or + 'sha1' if no value is set or outside a repository.. + GIT --- Part of the linkgit:git[1] suite diff --git a/Documentation/git-sparse-checkout.txt b/Documentation/git-sparse-checkout.txt index 1a3ace6..a0eeaeb 100644 --- a/Documentation/git-sparse-checkout.txt +++ b/Documentation/git-sparse-checkout.txt @@ -70,7 +70,7 @@ C-style quoted strings. `core.sparseCheckoutCone` is enabled, the given patterns are interpreted as directory names as in the 'set' subcommand. -'reapply:: +'reapply':: Reapply the sparsity pattern rules to paths in the working tree. Commands like merge or rebase can materialize paths to do their work (e.g. in order to show you a conflict), and other @@ -200,10 +200,32 @@ directory. SUBMODULES ---------- -If your repository contains one or more submodules, then those submodules will -appear based on which you initialized with the `git submodule` command. If -your sparse-checkout patterns exclude an initialized submodule, then that -submodule will still appear in your working directory. +If your repository contains one or more submodules, then submodules +are populated based on interactions with the `git submodule` command. +Specifically, `git submodule init -- <path>` will ensure the submodule +at `<path>` is present, while `git submodule deinit [-f] -- <path>` +will remove the files for the submodule at `<path>` (including any +untracked files, uncommitted changes, and unpushed history). Similar +to how sparse-checkout removes files from the working tree but still +leaves entries in the index, deinitialized submodules are removed from +the working directory but still have an entry in the index. + +Since submodules may have unpushed changes or untracked files, +removing them could result in data loss. Thus, changing sparse +inclusion/exclusion rules will not cause an already checked out +submodule to be removed from the working copy. Said another way, just +as `checkout` will not cause submodules to be automatically removed or +initialized even when switching between branches that remove or add +submodules, using `sparse-checkout` to reduce or expand the scope of +"interesting" files will not cause submodules to be automatically +deinitialized or initialized either. + +Further, the above facts mean that there are multiple reasons that +"tracked" files might not be present in the working copy: sparsity +pattern application from sparse-checkout, and submodule initialization +state. Thus, commands like `git grep` that work on tracked files in +the working copy may return results that are limited by either or both +of these restrictions. SEE ALSO diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index c9ed2bf..7e5f995 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -183,7 +183,7 @@ set-branch (-d|--default) [--] <path>:: Sets the default remote tracking branch for the submodule. The `--branch` option allows the remote branch to be specified. The `--default` option removes the submodule.<name>.branch configuration - key, which causes the tracking branch to default to 'master'. + key, which causes the tracking branch to default to the remote 'HEAD'. set-url [--] <path> <newurl>:: Sets the URL of the specified submodule to <newurl>. Then, it will @@ -284,7 +284,7 @@ OPTIONS `.gitmodules` for `update --remote`. A special value of `.` is used to indicate that the name of the branch in the submodule should be the same name as the current branch in the current repository. If the - option is not specified, it defaults to 'master'. + option is not specified, it defaults to the remote 'HEAD'. -f:: --force:: @@ -322,10 +322,10 @@ OPTIONS the superproject's recorded SHA-1 to update the submodule, use the status of the submodule's remote-tracking branch. The remote used is branch's remote (`branch.<name>.remote`), defaulting to `origin`. - The remote branch used defaults to `master`, but the branch name may - be overridden by setting the `submodule.<name>.branch` option in - either `.gitmodules` or `.git/config` (with `.git/config` taking - precedence). + The remote branch used defaults to the remote `HEAD`, but the branch + name may be overridden by setting the `submodule.<name>.branch` + option in either `.gitmodules` or `.git/config` (with `.git/config` + taking precedence). + This works for any of the supported update procedures (`--checkout`, `--rebase`, etc.). The only change is the source of the target SHA-1. diff --git a/Documentation/git-worktree.txt b/Documentation/git-worktree.txt index 85d92c9..4796c3c 100644 --- a/Documentation/git-worktree.txt +++ b/Documentation/git-worktree.txt @@ -126,7 +126,9 @@ OPTIONS locked working tree path, specify `--force` twice. + `move` refuses to move a locked working tree unless `--force` is specified -twice. +twice. If the destination is already assigned to some other working tree but is +missing (for instance, if `<new-path>` was deleted manually), then `--force` +allows the move to proceed; use --force twice if the destination is locked. + `remove` refuses to remove an unclean working tree unless `--force` is used. To remove a locked working tree, specify `--force` twice. diff --git a/Documentation/git.txt b/Documentation/git.txt index 9d6769e..3e50065 100644 --- a/Documentation/git.txt +++ b/Documentation/git.txt @@ -493,7 +493,7 @@ double-quotes and respecting backslash escapes. E.g., the value details. This variable has lower precedence than other path variables such as GIT_INDEX_FILE, GIT_OBJECT_DIRECTORY... -`GIT_DEFAULT_HASH_ALGORITHM`:: +`GIT_DEFAULT_HASH`:: If this variable is set, the default hash algorithm for new repositories will be set to this value. This value is currently ignored when cloning; the setting of the remote repository @@ -721,8 +721,6 @@ of clones and fetches. Enables a curl full trace dump of all incoming and outgoing data, including descriptive information, of the git transport protocol. This is similar to doing curl `--trace-ascii` on the command line. - This option overrides setting the `GIT_CURL_VERBOSE` environment - variable. See `GIT_TRACE` for available trace output options. `GIT_TRACE_CURL_NO_DATA`:: @@ -777,11 +775,10 @@ for full details. See `GIT_TRACE2` for available trace output options and link:technical/api-trace2.html[Trace2 documentation] for full details. -`GIT_REDACT_COOKIES`:: - This can be set to a comma-separated list of strings. When a curl trace - is enabled (see `GIT_TRACE_CURL` above), whenever a "Cookies:" header - sent by the client is dumped, values of cookies whose key is in that - list (case-sensitive) are redacted. +`GIT_TRACE_REDACT`:: + By default, when tracing is activated, Git redacts the values of + cookies, the "Authorization:" header, and the "Proxy-Authorization:" + header. Set this variable to `0` to prevent this redaction. `GIT_LITERAL_PATHSPECS`:: Setting this variable to `1` will cause Git to treat all diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt index 508fe71..2d0a037 100644 --- a/Documentation/gitattributes.txt +++ b/Documentation/gitattributes.txt @@ -824,6 +824,8 @@ patterns are available: - `java` suitable for source code in the Java language. +- `markdown` suitable for Markdown documents. + - `matlab` suitable for source code in the MATLAB and Octave languages. - `objc` suitable for source code in the Objective-C language. diff --git a/Documentation/gitcredentials.txt b/Documentation/gitcredentials.txt index 1814d2d..9e481ae 100644 --- a/Documentation/gitcredentials.txt +++ b/Documentation/gitcredentials.txt @@ -216,20 +216,26 @@ Here are some example specifications: ---------------------------------------------------- # run "git credential-foo" -foo +[credential] + helper = foo # same as above, but pass an argument to the helper -foo --bar=baz +[credential] + helper = "foo --bar=baz" # the arguments are parsed by the shell, so use shell # quoting if necessary -foo --bar="whitespace arg" +[credential] + helper = "foo --bar='whitespace arg'" # you can also use an absolute path, which will not use the git wrapper -/path/to/my/helper --with-arguments +[credential] + helper = "/path/to/my/helper --with-arguments" # or you can specify your own shell snippet -!f() { echo "password=`cat $HOME/.secret`"; }; f +[credential "https://example.com"] + username = your_user + helper = "!f() { test \"$1\" = get && echo \"password=$(cat $HOME/.secret)\"; }; f" ---------------------------------------------------- Generally speaking, rule (3) above is the simplest for users to specify. @@ -262,16 +268,26 @@ For a `get` operation, the helper should produce a list of attributes on stdout in the same format (see linkgit:git-credential[1] for common attributes). A helper is free to produce a subset, or even no values at all if it has nothing useful to provide. Any provided attributes will -overwrite those already known about by Git. If a helper outputs a -`quit` attribute with a value of `true` or `1`, no further helpers will -be consulted, nor will the user be prompted (if no credential has been -provided, the operation will then fail). +overwrite those already known about by Git's credential subsystem. + +While it is possible to override all attributes, well behaving helpers +should refrain from doing so for any attribute other than username and +password. + +If a helper outputs a `quit` attribute with a value of `true` or `1`, +no further helpers will be consulted, nor will the user be prompted +(if no credential has been provided, the operation will then fail). + +Similarly, no more helpers will be consulted once both username and +password had been provided. For a `store` or `erase` operation, the helper's output is ignored. -If it fails to perform the requested operation, it may complain to -stderr to inform the user. If it does not support the requested -operation (e.g., a read-only store), it should silently ignore the -request. + +If a helper fails to perform the requested operation or needs to notify +the user of a potential issue, it may write to stderr. + +If it does not support the requested operation (e.g., a read-only store), +it should silently ignore the request. If a helper receives any other operation, it should silently ignore the request. This leaves room for future operations to be added (older diff --git a/Documentation/giteveryday.txt b/Documentation/giteveryday.txt index 1bd919f..faba2ef 100644 --- a/Documentation/giteveryday.txt +++ b/Documentation/giteveryday.txt @@ -278,13 +278,13 @@ $ git am -3 -i -s ./+to-apply <4> $ compile/test $ git switch -c hold/linus && git am -3 -i -s ./+hold-linus <5> $ git switch topic/one && git rebase master <6> -$ git switch -C pu next <7> +$ git switch -C seen next <7> $ git merge topic/one topic/two && git merge hold/linus <8> $ git switch maint $ git cherry-pick master~4 <9> $ compile/test $ git tag -s -m "GIT 0.99.9x" v0.99.9x <10> -$ git fetch ko && for branch in master maint next pu <11> +$ git fetch ko && for branch in master maint next seen <11> do git show-branch ko/$branch $branch <12> done @@ -294,14 +294,14 @@ $ git push --follow-tags ko <13> <1> see what you were in the middle of doing, if anything. <2> see which branches haven't been merged into `master` yet. Likewise for any other integration branches e.g. `maint`, `next` -and `pu` (potential updates). +and `seen`. <3> read mails, save ones that are applicable, and save others that are not quite ready (other mail readers are available). <4> apply them, interactively, with your sign-offs. <5> create topic branch as needed and apply, again with sign-offs. <6> rebase internal topic branch that has not been merged to the master or exposed as a part of a stable branch. -<7> restart `pu` every time from the next. +<7> restart `seen` every time from the next. <8> and bundle topic branches still cooking. <9> backport a critical fix. <10> create a signed tag. @@ -323,7 +323,7 @@ repository at kernel.org, and looks like this: fetch = refs/heads/*:refs/remotes/ko/* push = refs/heads/master push = refs/heads/next - push = +refs/heads/pu + push = +refs/heads/seen push = refs/heads/maint ------------ diff --git a/Documentation/gitfaq.txt b/Documentation/gitfaq.txt index 1cf83df..9cd7a59 100644 --- a/Documentation/gitfaq.txt +++ b/Documentation/gitfaq.txt @@ -223,6 +223,24 @@ a file checked into the repository which is a template or set of defaults which can then be copied alongside and modified as appropriate. This second, modified file is usually ignored to prevent accidentally committing it. +[[files-in-gitignore-are-tracked]] +I asked Git to ignore various files, yet they are still tracked:: + A `gitignore` file ensures that certain file(s) which are not + tracked by Git remain untracked. However, sometimes particular + file(s) may have been tracked before adding them into the + `.gitignore`, hence they still remain tracked. To untrack and + ignore files/patterns, use `git rm --cached <file/pattern>` + and add a pattern to `.gitignore` that matches the <file>. + See linkgit:gitignore[5] for details. + +[[fetching-and-pulling]] +How do I know if I want to do a fetch or a pull?:: + A fetch stores a copy of the latest changes from the remote + repository, without modifying the working tree or current branch. + You can then at your leisure inspect, merge, rebase on top of, or + ignore the upstream changes. A pull consists of a fetch followed + immediately by either a merge or rebase. See linkgit:git-pull[1]. + Hooks ----- diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt index 81f2a87..6424711 100644 --- a/Documentation/githooks.txt +++ b/Documentation/githooks.txt @@ -404,6 +404,35 @@ Both standard output and standard error output are forwarded to `git send-pack` on the other end, so you can simply `echo` messages for the user. +ref-transaction +~~~~~~~~~~~~~~~ + +This hook is invoked by any Git command that performs reference +updates. It executes whenever a reference transaction is prepared, +committed or aborted and may thus get called multiple times. + +The hook takes exactly one argument, which is the current state the +given reference transaction is in: + + - "prepared": All reference updates have been queued to the + transaction and references were locked on disk. + + - "committed": The reference transaction was committed and all + references now have their respective new value. + + - "aborted": The reference transaction was aborted, no changes + were performed and the locks have been released. + +For each reference update that was added to the transaction, the hook +receives on standard input a line of the format: + + <old-value> SP <new-value> SP <ref-name> LF + +The exit status of the hook is ignored for any state except for the +"prepared" state. In the "prepared" state, a non-zero exit status will +cause the transaction to be aborted. The hook will not be called with +"aborted" state in that case. + push-to-checkout ~~~~~~~~~~~~~~~~ diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt index 67275fd..539b4e1 100644 --- a/Documentation/gitmodules.txt +++ b/Documentation/gitmodules.txt @@ -49,9 +49,9 @@ submodule.<name>.update:: submodule.<name>.branch:: A remote branch name for tracking updates in the upstream submodule. - If the option is not specified, it defaults to 'master'. A special - value of `.` is used to indicate that the name of the branch in the - submodule should be the same name as the current branch in the + If the option is not specified, it defaults to the remote 'HEAD'. + A special value of `.` is used to indicate that the name of the branch + in the submodule should be the same name as the current branch in the current repository. See the `--remote` documentation in linkgit:git-submodule[1] for details. diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt index f48a031..6f1e269 100644 --- a/Documentation/gitremote-helpers.txt +++ b/Documentation/gitremote-helpers.txt @@ -238,6 +238,9 @@ the remote repository. `--signed-tags=verbatim` to linkgit:git-fast-export[1]. In the absence of this capability, Git will use `--signed-tags=warn-strip`. +'object-format':: + This indicates that the helper is able to interact with the remote + side using an explicit hash algorithm extension. COMMANDS @@ -257,12 +260,14 @@ Support for this command is mandatory. 'list':: Lists the refs, one per line, in the format "<value> <name> [<attr> ...]". The value may be a hex sha1 hash, "@<dest>" for - a symref, or "?" to indicate that the helper could not get the - value of the ref. A space-separated list of attributes follows - the name; unrecognized attributes are ignored. The list ends - with a blank line. + a symref, ":<keyword> <value>" for a key-value pair, or + "?" to indicate that the helper could not get the value of the + ref. A space-separated list of attributes follows the name; + unrecognized attributes are ignored. The list ends with a + blank line. + See REF LIST ATTRIBUTES for a list of currently defined attributes. +See REF LIST KEYWORDS for a list of currently defined keywords. + Supported if the helper has the "fetch" or "import" capability. @@ -405,7 +410,9 @@ Supported if the helper has the "connect" capability. trying to fall back). After line feed terminating the positive (empty) response, the output of the service starts. Messages (both request and response) must consist of zero or more - PKT-LINEs, terminating in a flush packet. The client must not + PKT-LINEs, terminating in a flush packet. Response messages will + then have a response end packet after the flush packet to + indicate the end of a response. The client must not expect the server to store any state in between request-response pairs. After the connection ends, the remote helper exits. + @@ -430,6 +437,18 @@ attributes are defined. This ref is unchanged since the last import or fetch, although the helper cannot necessarily determine what value that produced. +REF LIST KEYWORDS +----------------- + +The 'list' command may produce a list of key-value pairs. +The following keys are defined. + +'object-format':: + The refs are using the given hash algorithm. This keyword is only + used if the server and client both support the object-format + extension. + + OPTIONS ------- @@ -514,6 +533,14 @@ set by Git if the remote helper has the 'option' capability. transaction. If successful, all refs will be updated, or none will. If the remote side does not support this capability, the push will fail. +'option object-format' {'true'|algorithm}:: + If 'true', indicate that the caller wants hash algorithm information + to be passed back from the remote. This mode is used when fetching + refs. ++ +If set to an algorithm, indicate that the caller wants to interact with +the remote side using that algorithm. + SEE ALSO -------- linkgit:git-remote[1] diff --git a/Documentation/gitworkflows.txt b/Documentation/gitworkflows.txt index abc0dc6..2db7ba7 100644 --- a/Documentation/gitworkflows.txt +++ b/Documentation/gitworkflows.txt @@ -85,15 +85,15 @@ As a given feature goes from experimental to stable, it also There is a fourth official branch that is used slightly differently: -* 'pu' (proposed updates) is an integration branch for things that are - not quite ready for inclusion yet (see "Integration Branches" - below). +* 'seen' (patches seen by the maintainer) is an integration branch for + things that are not quite ready for inclusion yet (see "Integration + Branches" below). Each of the four branches is usually a direct descendant of the one above it. Conceptually, the feature enters at an unstable branch (usually 'next' -or 'pu'), and "graduates" to 'master' for the next release once it is +or 'seen'), and "graduates" to 'master' for the next release once it is considered stable enough. @@ -207,7 +207,7 @@ If you make it (very) clear that this branch is going to be deleted right after the testing, you can even publish this branch, for example to give the testers a chance to work with it, or other developers a chance to see if their in-progress work will be compatible. `git.git` -has such an official throw-away integration branch called 'pu'. +has such an official throw-away integration branch called 'seen'. Branch management for a release @@ -291,7 +291,7 @@ This will not happen if the content of the branches was verified as described in the previous section. -Branch management for next and pu after a feature release +Branch management for next and seen after a feature release ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ After a feature release, the integration branch 'next' may optionally be @@ -319,8 +319,8 @@ so. If you do this, then you should make a public announcement indicating that 'next' was rewound and rebuilt. -The same rewind and rebuild process may be followed for 'pu'. A public -announcement is not necessary since 'pu' is a throw-away branch, as +The same rewind and rebuild process may be followed for 'seen'. A public +announcement is not necessary since 'seen' is a throw-away branch, as described above. diff --git a/Documentation/howto/maintain-git.txt b/Documentation/howto/maintain-git.txt index 73be8b4..a67130d 100644 --- a/Documentation/howto/maintain-git.txt +++ b/Documentation/howto/maintain-git.txt @@ -66,7 +66,7 @@ this mailing list after each feature release is made. demonstrated to be regression free. New changes are tested in 'next' before merged to 'master'. - - 'pu' branch is used to publish other proposed changes that do + - 'seen' branch is used to publish other proposed changes that do not yet pass the criteria set for 'next'. - The tips of 'master' and 'maint' branches will not be rewound to @@ -76,7 +76,7 @@ this mailing list after each feature release is made. of the cycle. - Usually 'master' contains all of 'maint' and 'next' contains all - of 'master'. 'pu' contains all the topics merged to 'next', but + of 'master'. 'seen' contains all the topics merged to 'next', but is rebuilt directly on 'master'. - The tip of 'master' is meant to be more stable than any @@ -229,12 +229,12 @@ by doing the following: series?) - Prepare 'jch' branch, which is used to represent somewhere - between 'master' and 'pu' and often is slightly ahead of 'next'. + between 'master' and 'seen' and often is slightly ahead of 'next'. - $ Meta/Reintegrate master..pu >Meta/redo-jch.sh + $ Meta/Reintegrate master..seen >Meta/redo-jch.sh The result is a script that lists topics to be merged in order to - rebuild 'pu' as the input to Meta/Reintegrate script. Remove + rebuild 'seen' as the input to Meta/Reintegrate script. Remove later topics that should not be in 'jch' yet. Add a line that consists of '### match next' before the name of the first topic in the output that should be in 'jch' but not in 'next' yet. @@ -291,29 +291,29 @@ by doing the following: merged to 'master'. This may lose '### match next' marker; add it again to the appropriate place when it happens. - - Rebuild 'pu'. + - Rebuild 'seen'. - $ Meta/Reintegrate master..pu >Meta/redo-pu.sh + $ Meta/Reintegrate master..seen >Meta/redo-seen.sh - Edit the result by adding new topics that are not still in 'pu' + Edit the result by adding new topics that are not still in 'seen' in the script. Then - $ git checkout -B pu jch - $ sh Meta/redo-pu.sh + $ git checkout -B seen jch + $ sh Meta/redo-seen.sh - When all is well, clean up the redo-pu.sh script with + When all is well, clean up the redo-seen.sh script with - $ sh Meta/redo-pu.sh -u + $ sh Meta/redo-seen.sh -u Double check by running - $ git branch --no-merged pu + $ git branch --no-merged seen to see there is no unexpected leftover topics. At this point, build-test the result for semantic conflicts, and if there are, prepare an appropriate merge-fix first (see - appendix), and rebuild the 'pu' branch from scratch, starting at + appendix), and rebuild the 'seen' branch from scratch, starting at the tip of 'jch'. - Update "What's cooking" message to review the updates to @@ -323,14 +323,14 @@ by doing the following: $ Meta/cook - This script inspects the history between master..pu, finds tips + This script inspects the history between master..seen, finds tips of topic branches, compares what it found with the current contents in Meta/whats-cooking.txt, and updates that file. - Topics not listed in the file but are found in master..pu are + Topics not listed in the file but are found in master..seen are added to the "New topics" section, topics listed in the file that - are no longer found in master..pu are moved to the "Graduated to + are no longer found in master..seen are moved to the "Graduated to master" section, and topics whose commits changed their states - (e.g. used to be only in 'pu', now merged to 'next') are updated + (e.g. used to be only in 'seen', now merged to 'next') are updated with change markers "<<" and ">>". Look for lines enclosed in "<<" and ">>"; they hold contents from @@ -360,7 +360,7 @@ Observations Some observations to be made. * Each topic is tested individually, and also together with other - topics cooking first in 'pu', then in 'jch' and then in 'next'. + topics cooking first in 'seen', then in 'jch' and then in 'next'. Until it matures, no part of it is merged to 'master'. * A topic already in 'next' can get fixes while still in @@ -411,7 +411,7 @@ new use of the variable under its old name. When these two topics are merged together, the reference to the variable newly added by the latter topic will still use the old name in the result. -The Meta/Reintegrate script that is used by redo-jch and redo-pu +The Meta/Reintegrate script that is used by redo-jch and redo-seen scripts implements a crude but usable way to work this issue around. When the script merges branch $X, it checks if "refs/merge-fix/$X" exists, and if so, the effect of it is squashed into the result of @@ -431,14 +431,14 @@ commit that can be squashed into a result of mechanical merge to correct semantic conflicts. After finding that the result of merging branch "ai/topic" to an -integration branch had such a semantic conflict, say pu~4, check the +integration branch had such a semantic conflict, say seen~4, check the problematic merge out on a detached HEAD, edit the working tree to fix the semantic conflict, and make a separate commit to record the fix-up: - $ git checkout pu~4 + $ git checkout seen~4 $ git show -s --pretty=%s ;# double check - Merge branch 'ai/topic' to pu + Merge branch 'ai/topic' to seen $ edit $ git commit -m 'merge-fix/ai/topic' -a @@ -450,9 +450,9 @@ result: Then double check the result by asking Meta/Reintegrate to redo the merge: - $ git checkout pu~5 ;# the parent of the problem merge + $ git checkout seen~5 ;# the parent of the problem merge $ echo ai/topic | Meta/Reintegrate - $ git diff pu~4 + $ git diff seen~4 This time, because you prepared refs/merge-fix/ai/topic, the resulting merge should have been tweaked to include the fix for the @@ -464,7 +464,7 @@ branch needs this merge-fix is because another branch merged earlier to the integration branch changed the underlying assumption ai/topic branch made (e.g. ai/topic branch added a site to refer to a variable, while the other branch renamed that variable and adjusted -existing use sites), and if you changed redo-jch (or redo-pu) script +existing use sites), and if you changed redo-jch (or redo-seen) script to merge ai/topic branch before the other branch, then the above merge-fix should not be applied while merging ai/topic, but should instead be applied while merging the other branch. You would need diff --git a/Documentation/howto/rebase-from-internal-branch.txt b/Documentation/howto/rebase-from-internal-branch.txt index 02cb5f7..f2e10a7 100644 --- a/Documentation/howto/rebase-from-internal-branch.txt +++ b/Documentation/howto/rebase-from-internal-branch.txt @@ -4,7 +4,7 @@ Cc: Petr Baudis <pasky@suse.cz>, Linus Torvalds <torvalds@osdl.org> Subject: Re: sending changesets from the middle of a git tree Date: Sun, 14 Aug 2005 18:37:39 -0700 Abstract: In this article, JC talks about how he rebases the - public "pu" branch using the core Git tools when he updates + public "seen" branch using the core Git tools when he updates the "master" branch, and how "rebase" works. Also discussed is how this applies to individual developers who sends patches upstream. @@ -20,8 +20,8 @@ Petr Baudis <pasky@suse.cz> writes: > where Junio C Hamano <junkio@cox.net> told me that... >> Linus Torvalds <torvalds@osdl.org> writes: >> ->> > Junio, maybe you want to talk about how you move patches from your "pu" ->> > branch to the real branches. +>> > Junio, maybe you want to talk about how you move patches from your +>> > "seen" branch to the real branches. >> > Actually, wouldn't this be also precisely for what StGIT is intended to? -------------------------------------- @@ -33,12 +33,12 @@ the kind of task StGIT is designed to do. I just have done a simpler one, this time using only the core Git tools. -I had a handful of commits that were ahead of master in pu, and I +I had a handful of commits that were ahead of master in 'seen', and I wanted to add some documentation bypassing my usual habit of -placing new things in pu first. At the beginning, the commit +placing new things in 'seen' first. At the beginning, the commit ancestry graph looked like this: - *"pu" head + *"seen" head master --> #1 --> #2 --> #3 So I started from master, made a bunch of edits, and committed: @@ -50,7 +50,7 @@ So I started from master, made a bunch of edits, and committed: After the commit, the ancestry graph would look like this: - *"pu" head + *"seen" head master^ --> #1 --> #2 --> #3 \ \---> master @@ -58,31 +58,31 @@ After the commit, the ancestry graph would look like this: The old master is now master^ (the first parent of the master). The new master commit holds my documentation updates. -Now I have to deal with "pu" branch. +Now I have to deal with "seen" branch. This is the kind of situation I used to have all the time when Linus was the maintainer and I was a contributor, when you look -at "master" branch being the "maintainer" branch, and "pu" +at "master" branch being the "maintainer" branch, and "seen" branch being the "contributor" branch. Your work started at the tip of the "maintainer" branch some time ago, you made a lot of progress in the meantime, and now the maintainer branch has some other commits you do not have yet. And "git rebase" was written with the explicit purpose of helping to maintain branches like -"pu". You _could_ merge master to pu and keep going, but if you +"seen". You _could_ merge master to 'seen' and keep going, but if you eventually want to cherrypick and merge some but not necessarily all changes back to the master branch, it often makes later operations for _you_ easier if you rebase (i.e. carry forward -your changes) "pu" rather than merge. So I ran "git rebase": +your changes) "seen" rather than merge. So I ran "git rebase": - $ git checkout pu - $ git rebase master pu + $ git checkout seen + $ git rebase master seen What this does is to pick all the commits since the current -branch (note that I now am on "pu" branch) forked from the +branch (note that I now am on "seen" branch) forked from the master branch, and forward port these changes. master^ --> #1 --> #2 --> #3 - \ *"pu" head + \ *"seen" head \---> master --> #1' --> #2' --> #3' The diff between master^ and #1 is applied to master and @@ -92,7 +92,7 @@ commits are made similarly out of #2 and #3 commits. Old #3 is not recorded in any of the .git/refs/heads/ file anymore, so after doing this you will have dangling commit if -you ran fsck-cache, which is normal. After testing "pu", you +you ran fsck-cache, which is normal. After testing "seen", you can run "git prune" to get rid of those original three commits. While I am talking about "git rebase", I should talk about how diff --git a/Documentation/howto/revert-branch-rebase.txt b/Documentation/howto/revert-branch-rebase.txt index 149508e..a3e5595 100644 --- a/Documentation/howto/revert-branch-rebase.txt +++ b/Documentation/howto/revert-branch-rebase.txt @@ -15,7 +15,7 @@ One of the changes I pulled into the 'master' branch turns out to break building Git with GCC 2.95. While they were well-intentioned portability fixes, keeping things working with gcc-2.95 was also important. Here is what I did to revert the change in the 'master' -branch and to adjust the 'pu' branch, using core Git tools and +branch and to adjust the 'seen' branch, using core Git tools and barebone Porcelain. First, prepare a throw-away branch in case I screw things up. @@ -104,11 +104,11 @@ $ git diff master..revert-c99 says nothing. -Then we rebase the 'pu' branch as usual. +Then we rebase the 'seen' branch as usual. ------------------------------------------------ -$ git checkout pu -$ git tag pu-anchor pu +$ git checkout seen +$ git tag seen-anchor seen $ git rebase master * Applying: Redo "revert" using three-way merge machinery. First trying simple merge strategy to cherry-pick. @@ -127,11 +127,11 @@ First trying simple merge strategy to cherry-pick. First trying simple merge strategy to cherry-pick. ------------------------------------------------ -The temporary tag 'pu-anchor' is me just being careful, in case 'git +The temporary tag 'seen-anchor' is me just being careful, in case 'git rebase' screws up. After this, I can do these for sanity check: ------------------------------------------------ -$ git diff pu-anchor..pu ;# make sure we got the master fix. +$ git diff seen-anchor..seen ;# make sure we got the master fix. $ make CC=gcc-2.95 clean test ;# make sure it fixed the breakage. $ make clean test ;# make sure it did not cause other breakage. ------------------------------------------------ @@ -140,7 +140,7 @@ Everything is in the good order. I do not need the temporary branch or tag anymore, so remove them: ------------------------------------------------ -$ rm -f .git/refs/tags/pu-anchor +$ rm -f .git/refs/tags/seen-anchor $ git branch -d revert-c99 ------------------------------------------------ @@ -168,18 +168,18 @@ Committed merge 7fb9b7262a1d1e0a47bbfdcbbcf50ce0635d3f8f And the final repository status looks like this: ------------------------------------------------ -$ git show-branch --more=1 master pu rc +$ git show-branch --more=1 master seen rc ! [master] Revert "Replace zero-length array decls with []." - ! [pu] git-repack: Add option to repack all objects. + ! [seen] git-repack: Add option to repack all objects. * [rc] Merge refs/heads/master from . --- - + [pu] git-repack: Add option to repack all objects. - + [pu~1] More documentation updates. - + [pu~2] Show commits in topo order and name all commits. - + [pu~3] mailinfo and applymbox updates - + [pu~4] Document "git cherry-pick" and "git revert" - + [pu~5] Remove git-apply-patch-script. - + [pu~6] Redo "revert" using three-way merge machinery. + + [seen] git-repack: Add option to repack all objects. + + [seen~1] More documentation updates. + + [seen~2] Show commits in topo order and name all commits. + + [seen~3] mailinfo and applymbox updates + + [seen~4] Document "git cherry-pick" and "git revert" + + [seen~5] Remove git-apply-patch-script. + + [seen~6] Redo "revert" using three-way merge machinery. - [rc] Merge refs/heads/master from . ++* [master] Revert "Replace zero-length array decls with []." - [rc~1] Merge refs/heads/master from . diff --git a/Documentation/howto/update-hook-example.txt b/Documentation/howto/update-hook-example.txt index 89821ec..151ee84 100644 --- a/Documentation/howto/update-hook-example.txt +++ b/Documentation/howto/update-hook-example.txt @@ -179,7 +179,7 @@ allowed-groups, to describe which heads can be pushed into by whom. The format of each file would look like this: refs/heads/master junio - +refs/heads/pu junio + +refs/heads/seen junio refs/heads/cogito$ pasky refs/heads/bw/.* linus refs/heads/tmp/.* .* @@ -187,6 +187,6 @@ whom. The format of each file would look like this: With this, Linus can push or create "bw/penguin" or "bw/zebra" or "bw/panda" branches, Pasky can do only "cogito", and JC can -do master and pu branches and make versioned tags. And anybody -can do tmp/blah branches. The '+' sign at the pu record means +do master and "seen" branches and make versioned tags. And anybody +can do tmp/blah branches. The '+' sign at the "seen" record means that JC can make non-fast-forward pushes on it. diff --git a/Documentation/pretty-formats.txt b/Documentation/pretty-formats.txt index 547a552..84bbc74 100644 --- a/Documentation/pretty-formats.txt +++ b/Documentation/pretty-formats.txt @@ -196,8 +196,8 @@ The placeholders are: '%ce':: committer email '%cE':: committer email (respecting .mailmap, see linkgit:git-shortlog[1] or linkgit:git-blame[1]) -'%cl':: author email local-part (the part before the '@' sign) -'%cL':: author local-part (see '%cl') respecting .mailmap, see +'%cl':: committer email local-part (the part before the '@' sign) +'%cL':: committer local-part (see '%cl') respecting .mailmap, see linkgit:git-shortlog[1] or linkgit:git-blame[1]) '%cd':: committer date (format respects --date= option) '%cD':: committer date, RFC2822 style diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index 04ad7dd..b01b2b6 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -581,12 +581,12 @@ option does. Applied to the 'D..M' range, it results in: Before discussing another option, `--show-pulls`, we need to create a new example history. -+ + A common problem users face when looking at simplified history is that a commit they know changed a file somehow does not appear in the file's simplified history. Let's demonstrate a new example and show how options such as `--full-history` and `--simplify-merges` works in that case: -+ + ----------------------------------------------------------------------- .-A---M-----C--N---O---P / / \ \ \/ / / @@ -595,7 +595,7 @@ such as `--full-history` and `--simplify-merges` works in that case: \ / /\ / `---X--' `---Y--' ----------------------------------------------------------------------- -+ + For this example, suppose `I` created `file.txt` which was modified by `A`, `B`, and `X` in different ways. The single-parent commits `C`, `Z`, and `Y` do not change `file.txt`. The merge commit `M` was created by @@ -607,19 +607,19 @@ the contents of `file.txt` at `X`. Hence, `R` is TREESAME to `X` but not contents of `file.txt` at `R`, so `N` is TREESAME to `R` but not `C`. The merge commits `O` and `P` are TREESAME to their first parents, but not to their second parents, `Z` and `Y` respectively. -+ + When using the default mode, `N` and `R` both have a TREESAME parent, so those edges are walked and the others are ignored. The resulting history graph is: -+ + ----------------------------------------------------------------------- I---X ----------------------------------------------------------------------- -+ + When using `--full-history`, Git walks every edge. This will discover the commits `A` and `B` and the merge `M`, but also will reveal the merge commits `O` and `P`. With parent rewriting, the resulting graph is: -+ + ----------------------------------------------------------------------- .-A---M--------N---O---P / / \ \ \/ / / @@ -628,21 +628,21 @@ merge commits `O` and `P`. With parent rewriting, the resulting graph is: \ / /\ / `---X--' `------' ----------------------------------------------------------------------- -+ + Here, the merge commits `O` and `P` contribute extra noise, as they did not actually contribute a change to `file.txt`. They only merged a topic that was based on an older version of `file.txt`. This is a common issue in repositories using a workflow where many contributors work in parallel and merge their topic branches along a single trunk: manu unrelated merges appear in the `--full-history` results. -+ + When using the `--simplify-merges` option, the commits `O` and `P` disappear from the results. This is because the rewritten second parents of `O` and `P` are reachable from their first parents. Those edges are removed and then the commits look like single-parent commits that are TREESAME to their parent. This also happens to the commit `N`, resulting in a history view as follows: -+ + ----------------------------------------------------------------------- .-A---M--. / / \ @@ -651,18 +651,18 @@ in a history view as follows: \ / / `---X--' ----------------------------------------------------------------------- -+ + In this view, we see all of the important single-parent changes from `A`, `B`, and `X`. We also see the carefully-resolved merge `M` and the not-so-carefully-resolved merge `R`. This is usually enough information to determine why the commits `A` and `B` "disappeared" from history in the default view. However, there are a few issues with this approach. -+ + The first issue is performance. Unlike any previous option, the `--simplify-merges` option requires walking the entire commit history before returning a single result. This can make the option difficult to use for very large repositories. -+ + The second issue is one of auditing. When many contributors are working on the same repository, it is important which merge commits introduced a change into an important branch. The problematic merge `R` above is @@ -671,10 +671,13 @@ important branch. Instead, the merge `N` was used to merge `R` and `X` into the important branch. This commit may have information about why the change `X` came to override the changes from `A` and `B` in its commit message. + +--show-pulls:: + In addition to the commits shown in the default history, show + each merge commit that is not TREESAME to its first parent but + is TREESAME to a later parent. + -The `--show-pulls` option helps with both of these issues by adding more -merge commits to the history results. If a merge is not TREESAME to its -first parent but is TREESAME to a later parent, then that merge is +When a merge commit is included by `--show-pulls`, the merge is treated as if it "pulled" the change from another branch. When using `--show-pulls` on this example (and no other options) the resulting graph is: diff --git a/Documentation/technical/commit-graph-format.txt b/Documentation/technical/commit-graph-format.txt index de56f9f..1beef17 100644 --- a/Documentation/technical/commit-graph-format.txt +++ b/Documentation/technical/commit-graph-format.txt @@ -97,10 +97,10 @@ CHUNK DATA: bit on. The other bits correspond to the position of the last parent. Bloom Filter Index (ID: {'B', 'I', 'D', 'X'}) (N * 4 bytes) [Optional] - * The ith entry, BIDX[i], stores the number of 8-byte word blocks in all - Bloom filters from commit 0 to commit i (inclusive) in lexicographic - order. The Bloom filter for the i-th commit spans from BIDX[i-1] to - BIDX[i] (plus header length), where BIDX[-1] is 0. + * The ith entry, BIDX[i], stores the number of bytes in all Bloom filters + from commit 0 to commit i (inclusive) in lexicographic order. The Bloom + filter for the i-th commit spans from BIDX[i-1] to BIDX[i] (plus header + length), where BIDX[-1] is 0. * The BIDX chunk is ignored if the BDAT chunk is not present. Bloom Filter Data (ID: {'B', 'D', 'A', 'T'}) [Optional] diff --git a/Documentation/technical/http-protocol.txt b/Documentation/technical/http-protocol.txt index 9c5b6f0..51a79e6 100644 --- a/Documentation/technical/http-protocol.txt +++ b/Documentation/technical/http-protocol.txt @@ -216,7 +216,7 @@ smart server reply: S: 001e# service=git-upload-pack\n S: 0000 S: 004895dcfa3633004da0049d3d0fa03f80589cbcaf31 refs/heads/maint\0multi_ack\n - S: 0042d049f6c27a2244e12041955e262a404c7faba355 refs/heads/master\n + S: 003fd049f6c27a2244e12041955e262a404c7faba355 refs/heads/master\n S: 003c2cb58b79488a98d2721cea644875a8dd0026b115 refs/tags/v1.0\n S: 003fa3c2e2402b99163d1d59756e5f207ae21cccba4c refs/tags/v1.0^{}\n S: 0000 diff --git a/Documentation/technical/pack-protocol.txt b/Documentation/technical/pack-protocol.txt index d5ce4ee..a4573d1 100644 --- a/Documentation/technical/pack-protocol.txt +++ b/Documentation/technical/pack-protocol.txt @@ -96,7 +96,7 @@ Basically what the Git client is doing to connect to an 'upload-pack' process on the server side over the Git protocol is this: $ echo -e -n \ - "0039git-upload-pack /schacon/gitbook.git\0host=example.com\0" | + "003agit-upload-pack /schacon/gitbook.git\0host=example.com\0" | nc -v example.com 9418 @@ -171,9 +171,9 @@ with a version number (if "version=1" is sent as an Extra Parameter), and a listing of each reference it has (all branches and tags) along with the object name that each reference currently points to. - $ echo -e -n "0044git-upload-pack /schacon/gitbook.git\0host=example.com\0\0version=1\0" | + $ echo -e -n "0045git-upload-pack /schacon/gitbook.git\0host=example.com\0\0version=1\0" | nc -v example.com 9418 - 000aversion 1 + 000eversion 1 00887217a7c7e582c46cec22a130adf4b9d7d950fba0 HEAD\0multi_ack thin-pack side-band side-band-64k ofs-delta shallow no-progress include-tag 00441d3fcd5ced445d1abc402225c0b8a1299641f497 refs/heads/integration diff --git a/Documentation/technical/packfile-uri.txt b/Documentation/technical/packfile-uri.txt new file mode 100644 index 0000000..318713a --- /dev/null +++ b/Documentation/technical/packfile-uri.txt @@ -0,0 +1,78 @@ +Packfile URIs +============= + +This feature allows servers to serve part of their packfile response as URIs. +This allows server designs that improve scalability in bandwidth and CPU usage +(for example, by serving some data through a CDN), and (in the future) provides +some measure of resumability to clients. + +This feature is available only in protocol version 2. + +Protocol +-------- + +The server advertises the `packfile-uris` capability. + +If the client then communicates which protocols (HTTPS, etc.) it supports with +a `packfile-uris` argument, the server MAY send a `packfile-uris` section +directly before the `packfile` section (right after `wanted-refs` if it is +sent) containing URIs of any of the given protocols. The URIs point to +packfiles that use only features that the client has declared that it supports +(e.g. ofs-delta and thin-pack). See protocol-v2.txt for the documentation of +this section. + +Clients should then download and index all the given URIs (in addition to +downloading and indexing the packfile given in the `packfile` section of the +response) before performing the connectivity check. + +Server design +------------- + +The server can be trivially made compatible with the proposed protocol by +having it advertise `packfile-uris`, tolerating the client sending +`packfile-uris`, and never sending any `packfile-uris` section. But we should +include some sort of non-trivial implementation in the Minimum Viable Product, +at least so that we can test the client. + +This is the implementation: a feature, marked experimental, that allows the +server to be configured by one or more `uploadpack.blobPackfileUri=<sha1> +<uri>` entries. Whenever the list of objects to be sent is assembled, all such +blobs are excluded, replaced with URIs. The client will download those URIs, +expecting them to each point to packfiles containing single blobs. + +Client design +------------- + +The client has a config variable `fetch.uriprotocols` that determines which +protocols the end user is willing to use. By default, this is empty. + +When the client downloads the given URIs, it should store them with "keep" +files, just like it does with the packfile in the `packfile` section. These +additional "keep" files can only be removed after the refs have been updated - +just like the "keep" file for the packfile in the `packfile` section. + +The division of work (initial fetch + additional URIs) introduces convenient +points for resumption of an interrupted clone - such resumption can be done +after the Minimum Viable Product (see "Future work"). + +Future work +----------- + +The protocol design allows some evolution of the server and client without any +need for protocol changes, so only a small-scoped design is included here to +form the MVP. For example, the following can be done: + + * On the server, more sophisticated means of excluding objects (e.g. by + specifying a commit to represent that commit and all objects that it + references). + * On the client, resumption of clone. If a clone is interrupted, information + could be recorded in the repository's config and a "clone-resume" command + can resume the clone in progress. (Resumption of subsequent fetches is more + difficult because that must deal with the user wanting to use the repository + even after the fetch was interrupted.) + +There are some possible features that will require a change in protocol: + + * Additional HTTP headers (e.g. authentication) + * Byte range support + * Different file formats referenced by URIs (e.g. raw object) diff --git a/Documentation/technical/protocol-capabilities.txt b/Documentation/technical/protocol-capabilities.txt index 2b267c0..36ccd14 100644 --- a/Documentation/technical/protocol-capabilities.txt +++ b/Documentation/technical/protocol-capabilities.txt @@ -176,6 +176,21 @@ agent strings are purely informative for statistics and debugging purposes, and MUST NOT be used to programmatically assume the presence or absence of particular features. +object-format +------------- + +This capability, which takes a hash algorithm as an argument, indicates +that the server supports the given hash algorithms. It may be sent +multiple times; if so, the first one given is the one used in the ref +advertisement. + +When provided by the client, this indicates that it intends to use the +given hash algorithm to communicate. The algorithm provided must be one +that the server supports. + +If this capability is not provided, it is assumed that the only +supported algorithm is SHA-1. + symref ------ diff --git a/Documentation/technical/protocol-v2.txt b/Documentation/technical/protocol-v2.txt index 7e3766c..e597b74 100644 --- a/Documentation/technical/protocol-v2.txt +++ b/Documentation/technical/protocol-v2.txt @@ -33,6 +33,8 @@ In protocol v2 these special packets will have the following semantics: * '0000' Flush Packet (flush-pkt) - indicates the end of a message * '0001' Delimiter Packet (delim-pkt) - separates sections of a message + * '0002' Message Packet (response-end-pkt) - indicates the end of a response + for stateless connections Initial Client Request ---------------------- @@ -323,13 +325,26 @@ included in the client's request: indicating its sideband (1, 2, or 3), and the server may send "0005\2" (a PKT-LINE of sideband 2 with no payload) as a keepalive packet. +If the 'packfile-uris' feature is advertised, the following argument +can be included in the client's request as well as the potential +addition of the 'packfile-uris' section in the server's response as +explained below. + + packfile-uris <comma-separated list of protocols> + Indicates to the server that the client is willing to receive + URIs of any of the given protocols in place of objects in the + sent packfile. Before performing the connectivity check, the + client should download from all given URIs. Currently, the + protocols supported are "http" and "https". + The response of `fetch` is broken into a number of sections separated by delimiter packets (0001), with each section beginning with its section -header. +header. Most sections are sent only when the packfile is sent. - output = *section - section = (acknowledgments | shallow-info | wanted-refs | packfile) - (flush-pkt | delim-pkt) + output = acknowledgements flush-pkt | + [acknowledgments delim-pkt] [shallow-info delim-pkt] + [wanted-refs delim-pkt] [packfile-uris delim-pkt] + packfile flush-pkt acknowledgments = PKT-LINE("acknowledgments" LF) (nak | *ack) @@ -347,13 +362,17 @@ header. *PKT-LINE(wanted-ref LF) wanted-ref = obj-id SP refname + packfile-uris = PKT-LINE("packfile-uris" LF) *packfile-uri + packfile-uri = PKT-LINE(40*(HEXDIGIT) SP *%x20-ff LF) + packfile = PKT-LINE("packfile" LF) *PKT-LINE(%x01-03 *%x00-ff) acknowledgments section - * If the client determines that it is finished with negotiations - by sending a "done" line, the acknowledgments sections MUST be - omitted from the server's response. + * If the client determines that it is finished with negotiations by + sending a "done" line (thus requiring the server to send a packfile), + the acknowledgments sections MUST be omitted from the server's + response. * Always begins with the section header "acknowledgments" @@ -404,9 +423,6 @@ header. which the client has not indicated was shallow as a part of its request. - * This section is only included if a packfile section is also - included in the response. - wanted-refs section * This section is only included if the client has requested a ref using a 'want-ref' line and if a packfile section is also @@ -420,6 +436,20 @@ header. * The server MUST NOT send any refs which were not requested using 'want-ref' lines. + packfile-uris section + * This section is only included if the client sent + 'packfile-uris' and the server has at least one such URI to + send. + + * Always begins with the section header "packfile-uris". + + * For each URI the server sends, it sends a hash of the pack's + contents (as output by git index-pack) followed by the URI. + + * The hashes are 40 hex characters long. When Git upgrades to a new + hash algorithm, this might need to be updated. (It should match + whatever index-pack outputs after "pack\t" or "keep\t". + packfile section * This section is only included if the client has sent 'want' lines in its request and either requested that no more @@ -453,3 +483,12 @@ included in a request. This is done by sending each option as a a request. The provided options must not contain a NUL or LF character. + + object-format +~~~~~~~~~~~~~~~ + +The server can advertise the `object-format` capability with a value `X` (in the +form `object-format=X`) to notify the client that the server is able to deal +with objects using hash algorithm X. If not specified, the server is assumed to +only handle SHA-1. If the client would like to use a hash algorithm other than +SHA-1, it should specify its object-format string. diff --git a/Documentation/technical/reftable.txt b/Documentation/technical/reftable.txt new file mode 100644 index 0000000..2951840 --- /dev/null +++ b/Documentation/technical/reftable.txt @@ -0,0 +1,1083 @@ +reftable +-------- + +Overview +~~~~~~~~ + +Problem statement +^^^^^^^^^^^^^^^^^ + +Some repositories contain a lot of references (e.g. android at 866k, +rails at 31k). The existing packed-refs format takes up a lot of space +(e.g. 62M), and does not scale with additional references. Lookup of a +single reference requires linearly scanning the file. + +Atomic pushes modifying multiple references require copying the entire +packed-refs file, which can be a considerable amount of data moved +(e.g. 62M in, 62M out) for even small transactions (2 refs modified). + +Repositories with many loose references occupy a large number of disk +blocks from the local file system, as each reference is its own file +storing 41 bytes (and another file for the corresponding reflog). This +negatively affects the number of inodes available when a large number of +repositories are stored on the same filesystem. Readers can be penalized +due to the larger number of syscalls required to traverse and read the +`$GIT_DIR/refs` directory. + + +Objectives +^^^^^^^^^^ + +* Near constant time lookup for any single reference, even when the +repository is cold and not in process or kernel cache. +* Near constant time verification if an object name is referred to by at least +one reference (for allow-tip-sha1-in-want). +* Efficient enumeration of an entire namespace, such as `refs/tags/`. +* Support atomic push with `O(size_of_update)` operations. +* Combine reflog storage with ref storage for small transactions. +* Separate reflog storage for base refs and historical logs. + +Description +^^^^^^^^^^^ + +A reftable file is a portable binary file format customized for +reference storage. References are sorted, enabling linear scans, binary +search lookup, and range scans. + +Storage in the file is organized into variable sized blocks. Prefix +compression is used within a single block to reduce disk space. Block +size and alignment is tunable by the writer. + +Performance +^^^^^^^^^^^ + +Space used, packed-refs vs. reftable: + +[cols=",>,>,>,>,>",options="header",] +|=============================================================== +|repository |packed-refs |reftable |% original |avg ref |avg obj +|android |62.2 M |36.1 M |58.0% |33 bytes |5 bytes +|rails |1.8 M |1.1 M |57.7% |29 bytes |4 bytes +|git |78.7 K |48.1 K |61.0% |50 bytes |4 bytes +|git (heads) |332 b |269 b |81.0% |33 bytes |0 bytes +|=============================================================== + +Scan (read 866k refs), by reference name lookup (single ref from 866k +refs), and by SHA-1 lookup (refs with that SHA-1, from 866k refs): + +[cols=",>,>,>,>",options="header",] +|========================================================= +|format |cache |scan |by name |by SHA-1 +|packed-refs |cold |402 ms |409,660.1 usec |412,535.8 usec +|packed-refs |hot | |6,844.6 usec |20,110.1 usec +|reftable |cold |112 ms |33.9 usec |323.2 usec +|reftable |hot | |20.2 usec |320.8 usec +|========================================================= + +Space used for 149,932 log entries for 43,061 refs, reflog vs. reftable: + +[cols=",>,>",options="header",] +|================================ +|format |size |avg entry +|$GIT_DIR/logs |173 M |1209 bytes +|reftable |5 M |37 bytes +|================================ + +Details +~~~~~~~ + +Peeling +^^^^^^^ + +References stored in a reftable are peeled, a record for an annotated +(or signed) tag records both the tag object, and the object it refers +to. This is analogous to storage in the packed-refs format. + +Reference name encoding +^^^^^^^^^^^^^^^^^^^^^^^ + +Reference names are an uninterpreted sequence of bytes that must pass +linkgit:git-check-ref-format[1] as a valid reference name. + +Key unicity +^^^^^^^^^^^ + +Each entry must have a unique key; repeated keys are disallowed. + +Network byte order +^^^^^^^^^^^^^^^^^^ + +All multi-byte, fixed width fields are in network byte order. + +Varint encoding +^^^^^^^^^^^^^^^ + +Varint encoding is identical to the ofs-delta encoding method used +within pack files. + +Decoder works such as: + +.... +val = buf[ptr] & 0x7f +while (buf[ptr] & 0x80) { + ptr++ + val = ((val + 1) << 7) | (buf[ptr] & 0x7f) +} +.... + +Ordering +^^^^^^^^ + +Blocks are lexicographically ordered by their first reference. + +Directory/file conflicts +^^^^^^^^^^^^^^^^^^^^^^^^ + +The reftable format accepts both `refs/heads/foo` and +`refs/heads/foo/bar` as distinct references. + +This property is useful for retaining log records in reftable, but may +confuse versions of Git using `$GIT_DIR/refs` directory tree to maintain +references. Users of reftable may choose to continue to reject `foo` and +`foo/bar` type conflicts to prevent problems for peers. + +File format +~~~~~~~~~~~ + +Structure +^^^^^^^^^ + +A reftable file has the following high-level structure: + +.... +first_block { + header + first_ref_block +} +ref_block* +ref_index* +obj_block* +obj_index* +log_block* +log_index* +footer +.... + +A log-only file omits the `ref_block`, `ref_index`, `obj_block` and +`obj_index` sections, containing only the file header and log block: + +.... +first_block { + header +} +log_block* +log_index* +footer +.... + +in a log-only file the first log block immediately follows the file +header, without padding to block alignment. + +Block size +^^^^^^^^^^ + +The file's block size is arbitrarily determined by the writer, and does +not have to be a power of 2. The block size must be larger than the +longest reference name or log entry used in the repository, as +references cannot span blocks. + +Powers of two that are friendly to the virtual memory system or +filesystem (such as 4k or 8k) are recommended. Larger sizes (64k) can +yield better compression, with a possible increased cost incurred by +readers during access. + +The largest block size is `16777215` bytes (15.99 MiB). + +Block alignment +^^^^^^^^^^^^^^^ + +Writers may choose to align blocks at multiples of the block size by +including `padding` filled with NUL bytes at the end of a block to round +out to the chosen alignment. When alignment is used, writers must +specify the alignment with the file header's `block_size` field. + +Block alignment is not required by the file format. Unaligned files must +set `block_size = 0` in the file header, and omit `padding`. Unaligned +files with more than one ref block must include the link:#Ref-index[ref +index] to support fast lookup. Readers must be able to read both aligned +and non-aligned files. + +Very small files (e.g. a single ref block) may omit `padding` and the ref +index to reduce total file size. + +Header (version 1) +^^^^^^^^^^^^^^^^^^ + +A 24-byte header appears at the beginning of the file: + +.... +'REFT' +uint8( version_number = 1 ) +uint24( block_size ) +uint64( min_update_index ) +uint64( max_update_index ) +.... + +Aligned files must specify `block_size` to configure readers with the +expected block alignment. Unaligned files must set `block_size = 0`. + +The `min_update_index` and `max_update_index` describe bounds for the +`update_index` field of all log records in this file. When reftables are +used in a stack for link:#Update-transactions[transactions], these +fields can order the files such that the prior file's +`max_update_index + 1` is the next file's `min_update_index`. + +Header (version 2) +^^^^^^^^^^^^^^^^^^ + +A 28-byte header appears at the beginning of the file: + +.... +'REFT' +uint8( version_number = 2 ) +uint24( block_size ) +uint64( min_update_index ) +uint64( max_update_index ) +uint32( hash_id ) +.... + +The header is identical to `version_number=1`, with the 4-byte hash ID +("sha1" for SHA1 and "s256" for SHA-256) append to the header. + +For maximum backward compatibility, it is recommended to use version 1 when +writing SHA1 reftables. + +First ref block +^^^^^^^^^^^^^^^ + +The first ref block shares the same block as the file header, and is 24 +bytes smaller than all other blocks in the file. The first block +immediately begins after the file header, at position 24. + +If the first block is a log block (a log-only file), its block header +begins immediately at position 24. + +Ref block format +^^^^^^^^^^^^^^^^ + +A ref block is written as: + +.... +'r' +uint24( block_len ) +ref_record+ +uint24( restart_offset )+ +uint16( restart_count ) + +padding? +.... + +Blocks begin with `block_type = 'r'` and a 3-byte `block_len` which +encodes the number of bytes in the block up to, but not including the +optional `padding`. This is always less than or equal to the file's +block size. In the first ref block, `block_len` includes 24 bytes for +the file header. + +The 2-byte `restart_count` stores the number of entries in the +`restart_offset` list, which must not be empty. Readers can use +`restart_count` to binary search between restarts before starting a +linear scan. + +Exactly `restart_count` 3-byte `restart_offset` values precedes the +`restart_count`. Offsets are relative to the start of the block and +refer to the first byte of any `ref_record` whose name has not been +prefix compressed. Entries in the `restart_offset` list must be sorted, +ascending. Readers can start linear scans from any of these records. + +A variable number of `ref_record` fill the middle of the block, +describing reference names and values. The format is described below. + +As the first ref block shares the first file block with the file header, +all `restart_offset` in the first block are relative to the start of the +file (position 0), and include the file header. This forces the first +`restart_offset` to be `28`. + +ref record +++++++++++ + +A `ref_record` describes a single reference, storing both the name and +its value(s). Records are formatted as: + +.... +varint( prefix_length ) +varint( (suffix_length << 3) | value_type ) +suffix +varint( update_index_delta ) +value? +.... + +The `prefix_length` field specifies how many leading bytes of the prior +reference record's name should be copied to obtain this reference's +name. This must be 0 for the first reference in any block, and also must +be 0 for any `ref_record` whose offset is listed in the `restart_offset` +table at the end of the block. + +Recovering a reference name from any `ref_record` is a simple concat: + +.... +this_name = prior_name[0..prefix_length] + suffix +.... + +The `suffix_length` value provides the number of bytes available in +`suffix` to copy from `suffix` to complete the reference name. + +The `update_index` that last modified the reference can be obtained by +adding `update_index_delta` to the `min_update_index` from the file +header: `min_update_index + update_index_delta`. + +The `value` follows. Its format is determined by `value_type`, one of +the following: + +* `0x0`: deletion; no value data (see transactions, below) +* `0x1`: one object name; value of the ref +* `0x2`: two object names; value of the ref, peeled target +* `0x3`: symbolic reference: `varint( target_len ) target` + +Symbolic references use `0x3`, followed by the complete name of the +reference target. No compression is applied to the target name. + +Types `0x4..0x7` are reserved for future use. + +Ref index +^^^^^^^^^ + +The ref index stores the name of the last reference from every ref block +in the file, enabling reduced disk seeks for lookups. Any reference can +be found by searching the index, identifying the containing block, and +searching within that block. + +The index may be organized into a multi-level index, where the 1st level +index block points to additional ref index blocks (2nd level), which may +in turn point to either additional index blocks (e.g. 3rd level) or ref +blocks (leaf level). Disk reads required to access a ref go up with +higher index levels. Multi-level indexes may be required to ensure no +single index block exceeds the file format's max block size of +`16777215` bytes (15.99 MiB). To achieve constant O(1) disk seeks for +lookups the index must be a single level, which is permitted to exceed +the file's configured block size, but not the format's max block size of +15.99 MiB. + +If present, the ref index block(s) appears after the last ref block. + +If there are at least 4 ref blocks, a ref index block should be written +to improve lookup times. Cold reads using the index require 2 disk reads +(read index, read block), and binary searching < 4 blocks also requires +<= 2 reads. Omitting the index block from smaller files saves space. + +If the file is unaligned and contains more than one ref block, the ref +index must be written. + +Index block format: + +.... +'i' +uint24( block_len ) +index_record+ +uint24( restart_offset )+ +uint16( restart_count ) + +padding? +.... + +The index blocks begin with `block_type = 'i'` and a 3-byte `block_len` +which encodes the number of bytes in the block, up to but not including +the optional `padding`. + +The `restart_offset` and `restart_count` fields are identical in format, +meaning and usage as in ref blocks. + +To reduce the number of reads required for random access in very large +files the index block may be larger than other blocks. However, readers +must hold the entire index in memory to benefit from this, so it's a +time-space tradeoff in both file size and reader memory. + +Increasing the file's block size decreases the index size. Alternatively +a multi-level index may be used, keeping index blocks within the file's +block size, but increasing the number of blocks that need to be +accessed. + +index record +++++++++++++ + +An index record describes the last entry in another block. Index records +are written as: + +.... +varint( prefix_length ) +varint( (suffix_length << 3) | 0 ) +suffix +varint( block_position ) +.... + +Index records use prefix compression exactly like `ref_record`. + +Index records store `block_position` after the suffix, specifying the +absolute position in bytes (from the start of the file) of the block +that ends with this reference. Readers can seek to `block_position` to +begin reading the block header. + +Readers must examine the block header at `block_position` to determine +if the next block is another level index block, or the leaf-level ref +block. + +Reading the index ++++++++++++++++++ + +Readers loading the ref index must first read the footer (below) to +obtain `ref_index_position`. If not present, the position will be 0. The +`ref_index_position` is for the 1st level root of the ref index. + +Obj block format +^^^^^^^^^^^^^^^^ + +Object blocks are optional. Writers may choose to omit object blocks, +especially if readers will not use the object name to ref mapping. + +Object blocks use unique, abbreviated 2-32 object name keys, mapping to +ref blocks containing references pointing to that object directly, or as +the peeled value of an annotated tag. Like ref blocks, object blocks use +the file's standard block size. The abbrevation length is available in +the footer as `obj_id_len`. + +To save space in small files, object blocks may be omitted if the ref +index is not present, as brute force search will only need to read a few +ref blocks. When missing, readers should brute force a linear search of +all references to lookup by object name. + +An object block is written as: + +.... +'o' +uint24( block_len ) +obj_record+ +uint24( restart_offset )+ +uint16( restart_count ) + +padding? +.... + +Fields are identical to ref block. Binary search using the restart table +works the same as in reference blocks. + +Because object names are abbreviated by writers to the shortest unique +abbreviation within the reftable, obj key lengths have a variable length. Their +length must be at least 2 bytes. Readers must compare only for common prefix +match within an obj block or obj index. + +obj record +++++++++++ + +An `obj_record` describes a single object abbreviation, and the blocks +containing references using that unique abbreviation: + +.... +varint( prefix_length ) +varint( (suffix_length << 3) | cnt_3 ) +suffix +varint( cnt_large )? +varint( position_delta )* +.... + +Like in reference blocks, abbreviations are prefix compressed within an +obj block. On large reftables with many unique objects, higher block +sizes (64k), and higher restart interval (128), a `prefix_length` of 2 +or 3 and `suffix_length` of 3 may be common in obj records (unique +abbreviation of 5-6 raw bytes, 10-12 hex digits). + +Each record contains `position_count` number of positions for matching +ref blocks. For 1-7 positions the count is stored in `cnt_3`. When +`cnt_3 = 0` the actual count follows in a varint, `cnt_large`. + +The use of `cnt_3` bets most objects are pointed to by only a single +reference, some may be pointed to by a couple of references, and very +few (if any) are pointed to by more than 7 references. + +A special case exists when `cnt_3 = 0` and `cnt_large = 0`: there are no +`position_delta`, but at least one reference starts with this +abbreviation. A reader that needs exact reference names must scan all +references to find which specific references have the desired object. +Writers should use this format when the `position_delta` list would have +overflowed the file's block size due to a high number of references +pointing to the same object. + +The first `position_delta` is the position from the start of the file. +Additional `position_delta` entries are sorted ascending and relative to +the prior entry, e.g. a reader would perform: + +.... +pos = position_delta[0] +prior = pos +for (j = 1; j < position_count; j++) { + pos = prior + position_delta[j] + prior = pos +} +.... + +With a position in hand, a reader must linearly scan the ref block, +starting from the first `ref_record`, testing each reference's object names +(for `value_type = 0x1` or `0x2`) for full equality. Faster searching by +object name within a single ref block is not supported by the reftable format. +Smaller block sizes reduce the number of candidates this step must +consider. + +Obj index +^^^^^^^^^ + +The obj index stores the abbreviation from the last entry for every obj +block in the file, enabling reduced disk seeks for all lookups. It is +formatted exactly the same as the ref index, but refers to obj blocks. + +The obj index should be present if obj blocks are present, as obj blocks +should only be written in larger files. + +Readers loading the obj index must first read the footer (below) to +obtain `obj_index_position`. If not present, the position will be 0. + +Log block format +^^^^^^^^^^^^^^^^ + +Unlike ref and obj blocks, log blocks are always unaligned. + +Log blocks are variable in size, and do not match the `block_size` +specified in the file header or footer. Writers should choose an +appropriate buffer size to prepare a log block for deflation, such as +`2 * block_size`. + +A log block is written as: + +.... +'g' +uint24( block_len ) +zlib_deflate { + log_record+ + uint24( restart_offset )+ + uint16( restart_count ) +} +.... + +Log blocks look similar to ref blocks, except `block_type = 'g'`. + +The 4-byte block header is followed by the deflated block contents using +zlib deflate. The `block_len` in the header is the inflated size +(including 4-byte block header), and should be used by readers to +preallocate the inflation output buffer. A log block's `block_len` may +exceed the file's block size. + +Offsets within the log block (e.g. `restart_offset`) still include the +4-byte header. Readers may prefer prefixing the inflation output buffer +with the 4-byte header. + +Within the deflate container, a variable number of `log_record` describe +reference changes. The log record format is described below. See ref +block format (above) for a description of `restart_offset` and +`restart_count`. + +Because log blocks have no alignment or padding between blocks, readers +must keep track of the bytes consumed by the inflater to know where the +next log block begins. + +log record +++++++++++ + +Log record keys are structured as: + +.... +ref_name '\0' reverse_int64( update_index ) +.... + +where `update_index` is the unique transaction identifier. The +`update_index` field must be unique within the scope of a `ref_name`. +See the update transactions section below for further details. + +The `reverse_int64` function inverses the value so lexicographical +ordering the network byte order encoding sorts the more recent records +with higher `update_index` values first: + +.... +reverse_int64(int64 t) { + return 0xffffffffffffffff - t; +} +.... + +Log records have a similar starting structure to ref and index records, +utilizing the same prefix compression scheme applied to the log record +key described above. + +.... + varint( prefix_length ) + varint( (suffix_length << 3) | log_type ) + suffix + log_data { + old_id + new_id + varint( name_length ) name + varint( email_length ) email + varint( time_seconds ) + sint16( tz_offset ) + varint( message_length ) message + }? +.... + +Log record entries use `log_type` to indicate what follows: + +* `0x0`: deletion; no log data. +* `0x1`: standard git reflog data using `log_data` above. + +The `log_type = 0x0` is mostly useful for `git stash drop`, removing an +entry from the reflog of `refs/stash` in a transaction file (below), +without needing to rewrite larger files. Readers reading a stack of +reflogs must treat this as a deletion. + +For `log_type = 0x1`, the `log_data` section follows +linkgit:git-update-ref[1] logging and includes: + +* two object names (old id, new id) +* varint string of committer's name +* varint string of committer's email +* varint time in seconds since epoch (Jan 1, 1970) +* 2-byte timezone offset in minutes (signed) +* varint string of message + +`tz_offset` is the absolute number of minutes from GMT the committer was +at the time of the update. For example `GMT-0800` is encoded in reftable +as `sint16(-480)` and `GMT+0230` is `sint16(150)`. + +The committer email does not contain `<` or `>`, it's the value normally +found between the `<>` in a git commit object header. + +The `message_length` may be 0, in which case there was no message +supplied for the update. + +Contrary to traditional reflog (which is a file), renames are encoded as +a combination of ref deletion and ref creation. A deletion is a log +record with a zero new_id, and a creation is a log record with a zero old_id. + +Reading the log ++++++++++++++++ + +Readers accessing the log must first read the footer (below) to +determine the `log_position`. The first block of the log begins at +`log_position` bytes since the start of the file. The `log_position` is +not block aligned. + +Importing logs +++++++++++++++ + +When importing from `$GIT_DIR/logs` writers should globally order all +log records roughly by timestamp while preserving file order, and assign +unique, increasing `update_index` values for each log line. Newer log +records get higher `update_index` values. + +Although an import may write only a single reftable file, the reftable +file must span many unique `update_index`, as each log line requires its +own `update_index` to preserve semantics. + +Log index +^^^^^^^^^ + +The log index stores the log key +(`refname \0 reverse_int64(update_index)`) for the last log record of +every log block in the file, supporting bounded-time lookup. + +A log index block must be written if 2 or more log blocks are written to +the file. If present, the log index appears after the last log block. +There is no padding used to align the log index to block alignment. + +Log index format is identical to ref index, except the keys are 9 bytes +longer to include `'\0'` and the 8-byte `reverse_int64(update_index)`. +Records use `block_position` to refer to the start of a log block. + +Reading the index ++++++++++++++++++ + +Readers loading the log index must first read the footer (below) to +obtain `log_index_position`. If not present, the position will be 0. + +Footer +^^^^^^ + +After the last block of the file, a file footer is written. It begins +like the file header, but is extended with additional data. + +.... + HEADER + + uint64( ref_index_position ) + uint64( (obj_position << 5) | obj_id_len ) + uint64( obj_index_position ) + + uint64( log_position ) + uint64( log_index_position ) + + uint32( CRC-32 of above ) +.... + +If a section is missing (e.g. ref index) the corresponding position +field (e.g. `ref_index_position`) will be 0. + +* `obj_position`: byte position for the first obj block. +* `obj_id_len`: number of bytes used to abbreviate object names in +obj blocks. +* `log_position`: byte position for the first log block. +* `ref_index_position`: byte position for the start of the ref index. +* `obj_index_position`: byte position for the start of the obj index. +* `log_index_position`: byte position for the start of the log index. + +The size of the footer is 68 bytes for version 1, and 72 bytes for +version 2. + +Reading the footer +++++++++++++++++++ + +Readers must first read the file start to determine the version +number. Then they seek to `file_length - FOOTER_LENGTH` to access the +footer. A trusted external source (such as `stat(2)`) is necessary to +obtain `file_length`. When reading the footer, readers must verify: + +* 4-byte magic is correct +* 1-byte version number is recognized +* 4-byte CRC-32 matches the other 64 bytes (including magic, and +version) + +Once verified, the other fields of the footer can be accessed. + +Empty tables +++++++++++++ + +A reftable may be empty. In this case, the file starts with a header +and is immediately followed by a footer. + +Binary search +^^^^^^^^^^^^^ + +Binary search within a block is supported by the `restart_offset` fields +at the end of the block. Readers can binary search through the restart +table to locate between which two restart points the sought reference or +key should appear. + +Each record identified by a `restart_offset` stores the complete key in +the `suffix` field of the record, making the compare operation during +binary search straightforward. + +Once a restart point lexicographically before the sought reference has +been identified, readers can linearly scan through the following record +entries to locate the sought record, terminating if the current record +sorts after (and therefore the sought key is not present). + +Restart point selection ++++++++++++++++++++++++ + +Writers determine the restart points at file creation. The process is +arbitrary, but every 16 or 64 records is recommended. Every 16 may be +more suitable for smaller block sizes (4k or 8k), every 64 for larger +block sizes (64k). + +More frequent restart points reduces prefix compression and increases +space consumed by the restart table, both of which increase file size. + +Less frequent restart points makes prefix compression more effective, +decreasing overall file size, with increased penalties for readers +walking through more records after the binary search step. + +A maximum of `65535` restart points per block is supported. + +Considerations +~~~~~~~~~~~~~~ + +Lightweight refs dominate +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The reftable format assumes the vast majority of references are single +object names valued with common prefixes, such as Gerrit Code Review's +`refs/changes/` namespace, GitHub's `refs/pulls/` namespace, or many +lightweight tags in the `refs/tags/` namespace. + +Annotated tags storing the peeled object cost an additional object name per +reference. + +Low overhead +^^^^^^^^^^^^ + +A reftable with very few references (e.g. git.git with 5 heads) is 269 +bytes for reftable, vs. 332 bytes for packed-refs. This supports +reftable scaling down for transaction logs (below). + +Block size +^^^^^^^^^^ + +For a Gerrit Code Review type repository with many change refs, larger +block sizes (64 KiB) and less frequent restart points (every 64) yield +better compression due to more references within the block compressing +against the prior reference. + +Larger block sizes reduce the index size, as the reftable will require +fewer blocks to store the same number of references. + +Minimal disk seeks +^^^^^^^^^^^^^^^^^^ + +Assuming the index block has been loaded into memory, binary searching +for any single reference requires exactly 1 disk seek to load the +containing block. + +Scans and lookups dominate +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Scanning all references and lookup by name (or namespace such as +`refs/heads/`) are the most common activities performed on repositories. +Object names are stored directly with references to optimize this use case. + +Logs are infrequently read +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Logs are infrequently accessed, but can be large. Deflating log blocks +saves disk space, with some increased penalty at read time. + +Logs are stored in an isolated section from refs, reducing the burden on +reference readers that want to ignore logs. Further, historical logs can +be isolated into log-only files. + +Logs are read backwards +^^^^^^^^^^^^^^^^^^^^^^^ + +Logs are frequently accessed backwards (most recent N records for master +to answer `master@{4}`), so log records are grouped by reference, and +sorted descending by update index. + +Repository format +~~~~~~~~~~~~~~~~~ + +Version 1 +^^^^^^^^^ + +A repository must set its `$GIT_DIR/config` to configure reftable: + +.... +[core] + repositoryformatversion = 1 +[extensions] + refStorage = reftable +.... + +Layout +^^^^^^ + +A collection of reftable files are stored in the `$GIT_DIR/reftable/` +directory: + +.... +00000001-00000001.log +00000002-00000002.ref +00000003-00000003.ref +.... + +where reftable files are named by a unique name such as produced by the +function `${min_update_index}-${max_update_index}.ref`. + +Log-only files use the `.log` extension, while ref-only and mixed ref +and log files use `.ref`. extension. + +The stack ordering file is `$GIT_DIR/reftable/tables.list` and lists the +current files, one per line, in order, from oldest (base) to newest +(most recent): + +.... +$ cat .git/reftable/tables.list +00000001-00000001.log +00000002-00000002.ref +00000003-00000003.ref +.... + +Readers must read `$GIT_DIR/reftable/tables.list` to determine which +files are relevant right now, and search through the stack in reverse +order (last reftable is examined first). + +Reftable files not listed in `tables.list` may be new (and about to be +added to the stack by the active writer), or ancient and ready to be +pruned. + +Backward compatibility +^^^^^^^^^^^^^^^^^^^^^^ + +Older clients should continue to recognize the directory as a git +repository so they don't look for an enclosing repository in parent +directories. To this end, a reftable-enabled repository must contain the +following dummy files + +* `.git/HEAD`, a regular file containing `ref: refs/heads/.invalid`. +* `.git/refs/`, a directory +* `.git/refs/heads`, a regular file + +Readers +^^^^^^^ + +Readers can obtain a consistent snapshot of the reference space by +following: + +1. Open and read the `tables.list` file. +2. Open each of the reftable files that it mentions. +3. If any of the files is missing, goto 1. +4. Read from the now-open files as long as necessary. + +Update transactions +^^^^^^^^^^^^^^^^^^^ + +Although reftables are immutable, mutations are supported by writing a +new reftable and atomically appending it to the stack: + +1. Acquire `tables.list.lock`. +2. Read `tables.list` to determine current reftables. +3. Select `update_index` to be most recent file's +`max_update_index + 1`. +4. Prepare temp reftable `tmp_XXXXXX`, including log entries. +5. Rename `tmp_XXXXXX` to `${update_index}-${update_index}.ref`. +6. Copy `tables.list` to `tables.list.lock`, appending file from (5). +7. Rename `tables.list.lock` to `tables.list`. + +During step 4 the new file's `min_update_index` and `max_update_index` +are both set to the `update_index` selected by step 3. All log records +for the transaction use the same `update_index` in their keys. This +enables later correlation of which references were updated by the same +transaction. + +Because a single `tables.list.lock` file is used to manage locking, the +repository is single-threaded for writers. Writers may have to busy-spin +(with backoff) around creating `tables.list.lock`, for up to an +acceptable wait period, aborting if the repository is too busy to +mutate. Application servers wrapped around repositories (e.g. Gerrit +Code Review) can layer their own lock/wait queue to improve fairness to +writers. + +Reference deletions +^^^^^^^^^^^^^^^^^^^ + +Deletion of any reference can be explicitly stored by setting the `type` +to `0x0` and omitting the `value` field of the `ref_record`. This serves +as a tombstone, overriding any assertions about the existence of the +reference from earlier files in the stack. + +Compaction +^^^^^^^^^^ + +A partial stack of reftables can be compacted by merging references +using a straightforward merge join across reftables, selecting the most +recent value for output, and omitting deleted references that do not +appear in remaining, lower reftables. + +A compacted reftable should set its `min_update_index` to the smallest +of the input files' `min_update_index`, and its `max_update_index` +likewise to the largest input `max_update_index`. + +For sake of illustration, assume the stack currently consists of +reftable files (from oldest to newest): A, B, C, and D. The compactor is +going to compact B and C, leaving A and D alone. + +1. Obtain lock `tables.list.lock` and read the `tables.list` file. +2. Obtain locks `B.lock` and `C.lock`. Ownership of these locks +prevents other processes from trying to compact these files. +3. Release `tables.list.lock`. +4. Compact `B` and `C` into a temp file +`${min_update_index}-${max_update_index}_XXXXXX`. +5. Reacquire lock `tables.list.lock`. +6. Verify that `B` and `C` are still in the stack, in that order. This +should always be the case, assuming that other processes are adhering to +the locking protocol. +7. Rename `${min_update_index}-${max_update_index}_XXXXXX` to +`${min_update_index}-${max_update_index}.ref`. +8. Write the new stack to `tables.list.lock`, replacing `B` and `C` +with the file from (4). +9. Rename `tables.list.lock` to `tables.list`. +10. Delete `B` and `C`, perhaps after a short sleep to avoid forcing +readers to backtrack. + +This strategy permits compactions to proceed independently of updates. + +Each reftable (compacted or not) is uniquely identified by its name, so +open reftables can be cached by their name. + +Alternatives considered +~~~~~~~~~~~~~~~~~~~~~~~ + +bzip packed-refs +^^^^^^^^^^^^^^^^ + +`bzip2` can significantly shrink a large packed-refs file (e.g. 62 MiB +compresses to 23 MiB, 37%). However the bzip format does not support +random access to a single reference. Readers must inflate and discard +while performing a linear scan. + +Breaking packed-refs into chunks (individually compressing each chunk) +would reduce the amount of data a reader must inflate, but still leaves +the problem of indexing chunks to support readers efficiently locating +the correct chunk. + +Given the compression achieved by reftable's encoding, it does not seem +necessary to add the complexity of bzip/gzip/zlib. + +Michael Haggerty's alternate format +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Michael Haggerty proposed +link:https://lore.kernel.org/git/CAMy9T_HCnyc1g8XWOOWhe7nN0aEFyyBskV2aOMb_fe%2BwGvEJ7A%40mail.gmail.com/[an +alternate] format to reftable on the Git mailing list. This format uses +smaller chunks, without the restart table, and avoids block alignment +with padding. Reflog entries immediately follow each ref, and are thus +interleaved between refs. + +Performance testing indicates reftable is faster for lookups (51% +faster, 11.2 usec vs. 5.4 usec), although reftable produces a slightly +larger file (+ ~3.2%, 28.3M vs 29.2M): + +[cols=">,>,>,>",options="header",] +|===================================== +|format |size |seek cold |seek hot +|mh-alt |28.3 M |23.4 usec |11.2 usec +|reftable |29.2 M |19.9 usec |5.4 usec +|===================================== + +JGit Ketch RefTree +^^^^^^^^^^^^^^^^^^ + +https://dev.eclipse.org/mhonarc/lists/jgit-dev/msg03073.html[JGit Ketch] +proposed +link:https://lore.kernel.org/git/CAJo%3DhJvnAPNAdDcAAwAvU9C4RVeQdoS3Ev9WTguHx4fD0V_nOg%40mail.gmail.com/[RefTree], +an encoding of references inside Git tree objects stored as part of the +repository's object database. + +The RefTree format adds additional load on the object database storage +layer (more loose objects, more objects in packs), and relies heavily on +the packer's delta compression to save space. Namespaces which are flat +(e.g. thousands of tags in refs/tags) initially create very large loose +objects, and so RefTree does not address the problem of copying many +references to modify a handful. + +Flat namespaces are not efficiently searchable in RefTree, as tree +objects in canonical formatting cannot be binary searched. This fails +the need to handle a large number of references in a single namespace, +such as GitHub's `refs/pulls`, or a project with many tags. + +LMDB +^^^^ + +David Turner proposed +https://lore.kernel.org/git/1455772670-21142-26-git-send-email-dturner@twopensource.com/[using +LMDB], as LMDB is lightweight (64k of runtime code) and GPL-compatible +license. + +A downside of LMDB is its reliance on a single C implementation. This +makes embedding inside JGit (a popular reimplementation of Git) +difficult, and hoisting onto virtual storage (for JGit DFS) virtually +impossible. + +A common format that can be supported by all major Git implementations +(git-core, JGit, libgit2) is strongly preferred. diff --git a/Documentation/user-manual.txt b/Documentation/user-manual.txt index 8336529..fd480b8 100644 --- a/Documentation/user-manual.txt +++ b/Documentation/user-manual.txt @@ -347,7 +347,7 @@ $ git branch -r origin/man origin/master origin/next - origin/pu + origin/seen origin/todo ------------------------------------------------ |