diff options
Diffstat (limited to 'Documentation')
33 files changed, 1319 insertions, 93 deletions
diff --git a/Documentation/Makefile b/Documentation/Makefile index 6232143..fa9e5c0 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -78,6 +78,7 @@ TECH_DOCS += technical/pack-heuristics TECH_DOCS += technical/pack-protocol TECH_DOCS += technical/protocol-capabilities TECH_DOCS += technical/protocol-common +TECH_DOCS += technical/protocol-v2 TECH_DOCS += technical/racy-git TECH_DOCS += technical/send-pack-pipeline TECH_DOCS += technical/shallow diff --git a/Documentation/RelNotes/2.18.0.txt b/Documentation/RelNotes/2.18.0.txt index 31c3f6d..fccc2f3 100644 --- a/Documentation/RelNotes/2.18.0.txt +++ b/Documentation/RelNotes/2.18.0.txt @@ -44,6 +44,44 @@ UI, Workflows & Features * "git mergetools" learned talking to guiffy. + * The scripts in contrib/emacs/ have outlived their usefulness and + have been replaced with a stub that errors out and tells the user + there are replacements. + + * The new "checkout-encoding" attribute can ask Git to convert the + contents to the specified encoding when checking out to the working + tree (and the other way around when checking in). + + * The "git config" command uses separate options e.g. "--int", + "--bool", etc. to specify what type the caller wants the value to + be interpreted as. A new "--type=<typename>" option has been + introduced, which would make it cleaner to define new types. + + * "git config --get" learned the "--default" option, to help the + calling script. Building on top of the above changes, the + "git config" learns "--type=color" type. Taken together, you can + do things like "git config --get foo.color --default blue" and get + the ANSI color sequence for the color given to foo.color variable, + or "blue" if the variable does not exist. + + * "git ls-remote" learned an option to allow sorting its output based + on the refnames being shown. + + * The command line completion (in contrib/) has been taught that "git + stash save" has been deprecated ("git stash push" is the preferred + spelling in the new world) and does not offer it as a possible + completion candidate when "git stash push" can be. + + * "git gc --prune=nonsense" spent long time repacking and then + silently failed when underlying "git prune --expire=nonsense" + failed to parse its command line. This has been corrected. + + * Error messages from "git push" can be painted for more visibility. + + * "git http-fetch" (deprecated) had an optional and experimental + "feature" to fetch only commits and/or trees, which nobody used. + This has been removed. + Performance, Internal Implementation, Development Support etc. @@ -118,6 +156,30 @@ Performance, Internal Implementation, Development Support etc. fast-import.c, which in turn has become the first user of the mem-pool API. + * A build-time option has been added to allow Git to be told to refer + to its associated files relative to the main binary, in the same + way that has been possible on Windows for quite some time, for + Linux, BSDs and Darwin. + + * Precompute and store information necessary for ancestry traversal + in a separate file to optimize graph walking. + + * The effort to pass the repository in-core structure throughout the + API continues. This round deals with the code that implements the + refs/replace/ mechanism. + + * The build procedure "make DEVELOPER=YesPlease" learned to enable a + bit more warning options depending on the compiler used to help + developers more. There also is "make DEVOPTS=tokens" knob + available now, for those who want to help fixing warnings we + usually ignore, for example. + + * A new version of the transport protocol is being worked on. + + * The code to interface to GPG has been restructured somewhat to make + it cleaner to integrate with other types of signature systems later. + + Also contains various documentation updates and code clean-ups. @@ -183,6 +245,53 @@ Fixes since v2.17 attacker's control) buffer overflow. (merge d8579accfa bp/fsmonitor-bufsize-fix later to maint). + * Recent simplification of build procedure forgot a bit of tweak to + the build procedure of contrib/mw-to-git/ + (merge d8698987f3 ab/simplify-perl-makefile later to maint). + + * Moving a submodule that itself has submodule in it with "git mv" + forgot to make necessary adjustment to the nested sub-submodules; + now the codepath learned to recurse into the submodules. + + * "git config --unset a.b", when "a.b" is the last variable in an + otherwise empty section "a", left an empty section "a" behind, and + worse yet, a subsequent "git config a.c value" did not reuse that + empty shell and instead created a new one. These have been + (partially) corrected. + (merge c71d8bb38a js/empty-config-section-fix later to maint). + + * "git worktree remove" learned that "-f" is a shorthand for + "--force" option, just like for "git worktree add". + (merge d228eea514 sb/worktree-remove-opt-force later to maint). + + * The completion script (in contrib/) learned to clear cached list of + command line options upon dot-sourcing it again in a more efficient + way. + (merge 94408dc71c sg/completion-clear-cached later to maint). + + * "git svn" had a minor thinko/typo which has been fixed. + (merge 51db271587 ab/git-svn-get-record-typofix later to maint). + + * During a "rebase -i" session, the code could give older timestamp + to commits created by later "pick" than an earlier "reword", which + has been corrected. + (merge 12f7babd6b js/ident-date-fix later to maint). + + * "git submodule status" did not check the symbolic revision name it + computed for the submodule HEAD is not the NULL, and threw it at + printf routines, which has been corrected. + (merge 0b5e2ea7cf nd/submodule-status-fix later to maint). + + * When fed input that already has In-Reply-To: and/or References: + headers and told to add the same information, "git send-email" + added these headers separately, instead of appending to an existing + one, which is a violation of the RFC. This has been corrected. + (merge 256be1d3f0 sa/send-email-dedup-some-headers later to maint). + + * "git fast-export" had a regression in v2.15.0 era where it skipped + some merge commits in certain cases, which has been corrected. + (merge be011bbe00 ma/fast-export-skip-merge-fix later to maint). + * Other minor doc, test and build updates and code cleanups. (merge 248f66ed8e nd/trace-with-env later to maint). (merge 14ced5562c ys/bisect-object-id-missing-conversion-fix later to maint). @@ -198,3 +307,6 @@ Fixes since v2.17 (merge decf711fc1 ps/test-chmtime-get later to maint). (merge 22d11a6e8e es/worktree-docs later to maint). (merge 92a5dbbc22 tg/use-git-contacts later to maint). + (merge adc887221f tq/t1510 later to maint). + (merge bed21a8ad6 sg/doc-gc-quote-mismatch-fix later to maint). + (merge 73364e4f10 tz/doc-git-urls-reference later to maint). diff --git a/Documentation/config.txt b/Documentation/config.txt index d6bcb5d..84e2891 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -530,6 +530,12 @@ core.autocrlf:: This variable can be set to 'input', in which case no output conversion is performed. +core.checkRoundtripEncoding:: + A comma and/or whitespace separated list of encodings that Git + performs UTF-8 round trip checks on if they are used in an + `working-tree-encoding` attribute (see linkgit:gitattributes[5]). + The default value is `SHIFT-JIS`. + core.symlinks:: If false, symbolic links are checked out as small plain files that contain the link text. linkgit:git-update-index[1] and @@ -898,6 +904,10 @@ core.notesRef:: This setting defaults to "refs/notes/commits", and it can be overridden by the `GIT_NOTES_REF` environment variable. See linkgit:git-notes[1]. +core.commitGraph:: + Enable git commit graph feature. Allows reading from the + commit-graph file. + core.sparseCheckout:: Enable "sparse checkout" feature. See section "Sparse checkout" in linkgit:git-read-tree[1] for more information. @@ -1092,6 +1102,16 @@ clean.requireForce:: A boolean to make git-clean do nothing unless given -f, -i or -n. Defaults to true. +color.advice:: + A boolean to enable/disable color in hints (e.g. when a push + failed, see `advice.*` for a list). May be set to `always`, + `false` (or `never`) or `auto` (or `true`), in which case colors + are used only when the error output goes to a terminal. If + unset, then the value of `color.ui` is used (`auto` by default). + +color.advice.hint:: + Use customized color for hints. + color.branch:: A boolean to enable/disable color in the output of linkgit:git-branch[1]. May be set to `always`, @@ -1194,6 +1214,15 @@ color.pager:: A boolean to enable/disable colored output when the pager is in use (default is true). +color.push:: + A boolean to enable/disable color in push errors. May be set to + `always`, `false` (or `never`) or `auto` (or `true`), in which + case colors are used only when the error output goes to a terminal. + If unset, then the value of `color.ui` is used (`auto` by default). + +color.push.error:: + Use customized color for push errors. + color.showBranch:: A boolean to enable/disable color in the output of linkgit:git-show-branch[1]. May be set to `always`, @@ -1222,6 +1251,15 @@ color.status.<slot>:: status short-format), or `unmerged` (files which have unmerged changes). +color.transport:: + A boolean to enable/disable color when pushes are rejected. May be + set to `always`, `false` (or `never`) or `auto` (or `true`), in which + case colors are used only when the error output goes to a terminal. + If unset, then the value of `color.ui` is used (`auto` by default). + +color.transport.rejected:: + Use customized color when a push was rejected. + color.ui:: This variable determines the default value for variables such as `color.diff` and `color.grep` that control the use of color @@ -1562,6 +1600,18 @@ gc.autoDetach:: Make `git gc --auto` return immediately and run in background if the system supports it. Default is true. +gc.bigPackThreshold:: + If non-zero, all packs larger than this limit are kept when + `git gc` is run. This is very similar to `--keep-base-pack` + except that all packs that meet the threshold are kept, not + just the base pack. Defaults to zero. Common unit suffixes of + 'k', 'm', or 'g' are supported. ++ +Note that if the number of kept packs is more than gc.autoPackLimit, +this configuration variable is ignored, all packs except the base pack +will be repacked. After this the number of packs should go below +gc.autoPackLimit and gc.bigPackThreshold should be respected again. + gc.logExpiry:: If the file gc.log exists, then `git gc --auto` won't run unless that file is more than 'gc.logExpiry' old. Default is @@ -2426,6 +2476,7 @@ pack.window:: pack.depth:: The maximum delta depth used by linkgit:git-pack-objects[1] when no maximum depth is given on the command line. Defaults to 50. + Maximum value is 4095. pack.windowMemory:: The maximum size of memory that is consumed by each thread @@ -2462,7 +2513,8 @@ pack.deltaCacheLimit:: The maximum size of a delta, that is cached in linkgit:git-pack-objects[1]. This cache is used to speed up the writing object phase by not having to recompute the final delta - result once the best match for all objects is found. Defaults to 1000. + result once the best match for all objects is found. + Defaults to 1000. Maximum value is 65535. pack.threads:: Specifies the number of threads to spawn when searching for best diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt index 8631e36..97d3217 100644 --- a/Documentation/fetch-options.txt +++ b/Documentation/fetch-options.txt @@ -188,6 +188,14 @@ endif::git-pull[] is specified. This flag forces progress status even if the standard error stream is not directed to a terminal. +-o <option>:: +--server-option=<option>:: + Transmit the given string to the server when communicating using + protocol version 2. The given string must not contain a NUL or LF + character. + When multiple `--server-option=<option>` are given, they are all + sent to the other side in the order listed on the command line. + -4:: --ipv4:: Use IPv4 addresses only, ignoring IPv6 addresses. diff --git a/Documentation/git-apply.txt b/Documentation/git-apply.txt index 4ebc3d3..c993fbf 100644 --- a/Documentation/git-apply.txt +++ b/Documentation/git-apply.txt @@ -113,8 +113,10 @@ explained for the configuration variable `core.quotePath` (see linkgit:git-config[1]). -p<n>:: - Remove <n> leading slashes from traditional diff paths. The - default is 1. + Remove <n> leading path components (separated by slashes) from + traditional diff paths. E.g., with `-p2`, a patch against + `a/dir/file` will be applied directly to `file`. The default is + 1. -C<n>:: Ensure at least <n> lines of surrounding context match before diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt index 42ca7b5..b844b99 100644 --- a/Documentation/git-clone.txt +++ b/Documentation/git-clone.txt @@ -260,7 +260,7 @@ or `--mirror` is given) <repository>:: The (possibly remote) repository to clone from. See the - <<URLS,URLS>> section below for more information on specifying + <<URLS,GIT URLS>> section below for more information on specifying repositories. <directory>:: diff --git a/Documentation/git-commit-graph.txt b/Documentation/git-commit-graph.txt new file mode 100644 index 0000000..4c97b55 --- /dev/null +++ b/Documentation/git-commit-graph.txt @@ -0,0 +1,94 @@ +git-commit-graph(1) +=================== + +NAME +---- +git-commit-graph - Write and verify Git commit graph files + + +SYNOPSIS +-------- +[verse] +'git commit-graph read' [--object-dir <dir>] +'git commit-graph write' <options> [--object-dir <dir>] + + +DESCRIPTION +----------- + +Manage the serialized commit graph file. + + +OPTIONS +------- +--object-dir:: + Use given directory for the location of packfiles and commit graph + file. This parameter exists to specify the location of an alternate + that only has the objects directory, not a full .git directory. The + commit graph file is expected to be at <dir>/info/commit-graph and + the packfiles are expected to be in <dir>/pack. + + +COMMANDS +-------- +'write':: + +Write a commit graph file based on the commits found in packfiles. ++ +With the `--stdin-packs` option, generate the new commit graph by +walking objects only in the specified pack-indexes. (Cannot be combined +with --stdin-commits.) ++ +With the `--stdin-commits` option, generate the new commit graph by +walking commits starting at the commits specified in stdin as a list +of OIDs in hex, one OID per line. (Cannot be combined with +--stdin-packs.) ++ +With the `--append` option, include all commits that are present in the +existing commit-graph file. + +'read':: + +Read a graph file given by the commit-graph file and output basic +details about the graph file. Used for debugging purposes. + + +EXAMPLES +-------- + +* Write a commit graph file for the packed commits in your local .git folder. ++ +------------------------------------------------ +$ git commit-graph write +------------------------------------------------ + +* Write a graph file, extending the current graph file using commits +* in <pack-index>. ++ +------------------------------------------------ +$ echo <pack-index> | git commit-graph write --stdin-packs +------------------------------------------------ + +* Write a graph file containing all reachable commits. ++ +------------------------------------------------ +$ git show-ref -s | git commit-graph write --stdin-commits +------------------------------------------------ + +* Write a graph file containing all commits in the current +* commit-graph file along with those reachable from HEAD. ++ +------------------------------------------------ +$ git rev-parse HEAD | git commit-graph write --stdin-commits --append +------------------------------------------------ + +* Read basic information from the commit-graph file. ++ +------------------------------------------------ +$ git commit-graph read +------------------------------------------------ + + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt index e09ed5d..18ddc78 100644 --- a/Documentation/git-config.txt +++ b/Documentation/git-config.txt @@ -9,13 +9,13 @@ git-config - Get and set repository or global options SYNOPSIS -------- [verse] -'git config' [<file-option>] [type] [--show-origin] [-z|--null] name [value [value_regex]] -'git config' [<file-option>] [type] --add name value -'git config' [<file-option>] [type] --replace-all name value [value_regex] -'git config' [<file-option>] [type] [--show-origin] [-z|--null] --get name [value_regex] -'git config' [<file-option>] [type] [--show-origin] [-z|--null] --get-all name [value_regex] -'git config' [<file-option>] [type] [--show-origin] [-z|--null] [--name-only] --get-regexp name_regex [value_regex] -'git config' [<file-option>] [type] [-z|--null] --get-urlmatch name URL +'git config' [<file-option>] [--type=<type>] [--show-origin] [-z|--null] name [value [value_regex]] +'git config' [<file-option>] [--type=<type>] --add name value +'git config' [<file-option>] [--type=<type>] --replace-all name value [value_regex] +'git config' [<file-option>] [--type=<type>] [--show-origin] [-z|--null] --get name [value_regex] +'git config' [<file-option>] [--type=<type>] [--show-origin] [-z|--null] --get-all name [value_regex] +'git config' [<file-option>] [--type=<type>] [--show-origin] [-z|--null] [--name-only] --get-regexp name_regex [value_regex] +'git config' [<file-option>] [--type=<type>] [-z|--null] --get-urlmatch name URL 'git config' [<file-option>] --unset name [value_regex] 'git config' [<file-option>] --unset-all name [value_regex] 'git config' [<file-option>] --rename-section old_name new_name @@ -38,12 +38,10 @@ existing values that match the regexp are updated or unset. If you want to handle the lines that do *not* match the regex, just prepend a single exclamation mark in front (see also <<EXAMPLES>>). -The type specifier can be either `--int` or `--bool`, to make -'git config' ensure that the variable(s) are of the given type and -convert the value to the canonical form (simple decimal number for int, -a "true" or "false" string for bool), or `--path`, which does some -path expansion (see `--path` below). If no type specifier is passed, no -checks or transformations are performed on the value. +The `--type=<type>` option instructs 'git config' to ensure that incoming and +outgoing values are canonicalize-able under the given <type>. If no +`--type=<type>` is given, no canonicalization will be performed. Callers may +unset an existing `--type` specifier with `--no-type`. When reading, the values are read from the system, global and repository local configuration files by default, and options @@ -160,30 +158,43 @@ See also <<FILES>>. --list:: List all variables set in config file, along with their values. ---bool:: - 'git config' will ensure that the output is "true" or "false" +--type <type>:: + 'git config' will ensure that any input or output is valid under the given + type constraint(s), and will canonicalize outgoing values in `<type>`'s + canonical form. ++ +Valid `<type>`'s include: ++ +- 'bool': canonicalize values as either "true" or "false". +- 'int': canonicalize values as simple decimal numbers. An optional suffix of + 'k', 'm', or 'g' will cause the value to be multiplied by 1024, 1048576, or + 1073741824 upon input. +- 'bool-or-int': canonicalize according to either 'bool' or 'int', as described + above. +- 'path': canonicalize by adding a leading `~` to the value of `$HOME` and + `~user` to the home directory for the specified user. This specifier has no + effect when setting the value (but you can use `git config section.variable + ~/` from the command line to let your shell do the expansion.) +- 'expiry-date': canonicalize by converting from a fixed or relative date-string + to a timestamp. This specifier has no effect when setting the value. +- 'color': When getting a value, canonicalize by converting to an ANSI color + escape sequence. When setting a value, a sanity-check is performed to ensure + that the given value is canonicalize-able as an ANSI color, but it is written + as-is. ++ +--bool:: --int:: - 'git config' will ensure that the output is a simple - decimal number. An optional value suffix of 'k', 'm', or 'g' - in the config file will cause the value to be multiplied - by 1024, 1048576, or 1073741824 prior to output. - --bool-or-int:: - 'git config' will ensure that the output matches the format of - either --bool or --int, as described above. - --path:: - `git config` will expand a leading `~` to the value of - `$HOME`, and `~user` to the home directory for the - specified user. This option has no effect when setting the - value (but you can use `git config section.variable ~/` - from the command line to let your shell do the expansion). - --expiry-date:: - `git config` will ensure that the output is converted from - a fixed or relative date-string to a timestamp. This option - has no effect when setting the value. + Historical options for selecting a type specifier. Prefer instead `--type`, + (see: above). + +--no-type:: + Un-sets the previously set type specifier (if one was previously set). This + option requests that 'git config' not canonicalize the retrieved variable. + `--no-type` has no effect without `--type=<type>` or `--<type>`. -z:: --null:: @@ -221,6 +232,8 @@ See also <<FILES>>. output it as the ANSI color escape sequence to the standard output. The optional `default` parameter is used instead, if there is no color configured for `name`. ++ +`--type=color [--default=<default>]` is preferred over `--get-color`. -e:: --edit:: @@ -233,6 +246,10 @@ See also <<FILES>>. using `--file`, `--global`, etc) and `on` when searching all config files. +--default <value>:: + When using `--get`, and the requested variable is not found, behave as if + <value> were the value assigned to the that variable. + CONFIGURATION ------------- `pager.config` is only respected when listing configuration, i.e., when diff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt index b634043..1d4d2f8 100644 --- a/Documentation/git-filter-branch.txt +++ b/Documentation/git-filter-branch.txt @@ -288,7 +288,7 @@ git filter-branch --parent-filter \ or even simpler: ----------------------------------------------- -echo "$commit-id $graft-id" >> .git/info/grafts +git replace --graft $commit-id $graft-id git filter-branch $graft-id..HEAD ----------------------------------------------- diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt index 6cbe462..b41e132 100644 --- a/Documentation/git-format-patch.txt +++ b/Documentation/git-format-patch.txt @@ -47,7 +47,7 @@ There are two ways to specify which commits to operate on. The first rule takes precedence in the case of a single <commit>. To apply the second rule, i.e., format everything since the beginning of -history up until <commit>, use the '\--root' option: `git format-patch +history up until <commit>, use the `--root` option: `git format-patch --root <commit>`. If you want to format only <commit> itself, you can do this with `git format-patch -1 <commit>`. diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt index 3126e0d..bb376ac 100644 --- a/Documentation/git-gc.txt +++ b/Documentation/git-gc.txt @@ -9,7 +9,7 @@ git-gc - Cleanup unnecessary files and optimize the local repository SYNOPSIS -------- [verse] -'git gc' [--aggressive] [--auto] [--quiet] [--prune=<date> | --no-prune] [--force] +'git gc' [--aggressive] [--auto] [--quiet] [--prune=<date> | --no-prune] [--force] [--keep-largest-pack] DESCRIPTION ----------- @@ -56,10 +56,16 @@ single pack using `git repack -d -l`. Setting the value of `gc.auto` to 0 disables automatic packing of loose objects. + If the number of packs exceeds the value of `gc.autoPackLimit`, -then existing packs (except those marked with a `.keep` file) +then existing packs (except those marked with a `.keep` file +or over `gc.bigPackThreshold` limit) are consolidated into a single pack by using the `-A` option of -'git repack'. Setting `gc.autoPackLimit` to 0 disables -automatic consolidation of packs. +'git repack'. +If the amount of memory is estimated not enough for `git repack` to +run smoothly and `gc.bigPackThreshold` is not set, the largest +pack will also be excluded (this is the equivalent of running `git gc` +with `--keep-base-pack`). +Setting `gc.autoPackLimit` to 0 disables automatic consolidation of +packs. + If houskeeping is required due to many loose objects or packs, all other housekeeping tasks (e.g. rerere, working trees, reflog...) will @@ -84,6 +90,11 @@ be performed as well. Force `git gc` to run even if there may be another `git gc` instance running on this repository. +--keep-largest-pack:: + All packs except the largest pack and those marked with a + `.keep` files are consolidated into a single pack. When this + option is used, `gc.bigPackThreshold` is ignored. + Configuration ------------- @@ -129,7 +140,7 @@ The optional configuration variable `gc.aggressiveWindow` controls how much time is spent optimizing the delta compression of the objects in the repository when the --aggressive option is specified. The larger the value, the more time is spent optimizing the delta compression. See -the documentation for the --window' option in linkgit:git-repack[1] for +the documentation for the --window option in linkgit:git-repack[1] for more details. This defaults to 250. Similarly, the optional configuration variable `gc.aggressiveDepth` diff --git a/Documentation/git-http-fetch.txt b/Documentation/git-http-fetch.txt index 21a33d2..666b042 100644 --- a/Documentation/git-http-fetch.txt +++ b/Documentation/git-http-fetch.txt @@ -15,8 +15,9 @@ DESCRIPTION ----------- Downloads a remote Git repository via HTTP. -*NOTE*: use of this command without -a is deprecated. The -a -behaviour will become the default in a future release. +This command always gets all objects. Historically, there were three options +`-a`, `-c` and `-t` for choosing which objects to download. They are now +silently ignored. OPTIONS ------- @@ -24,12 +25,8 @@ commit-id:: Either the hash or the filename under [URL]/refs/ to pull. --c:: - Get the commit objects. --t:: - Get trees associated with the commit objects. --a:: - Get all the objects. +-a, -c, -t:: + These options are ignored for historical reasons. -v:: Report what is downloaded. diff --git a/Documentation/git-log.txt b/Documentation/git-log.txt index 5437f8b..90761f1 100644 --- a/Documentation/git-log.txt +++ b/Documentation/git-log.txt @@ -9,7 +9,7 @@ git-log - Show commit logs SYNOPSIS -------- [verse] -'git log' [<options>] [<revision range>] [[\--] <path>...] +'git log' [<options>] [<revision range>] [[--] <path>...] DESCRIPTION ----------- @@ -90,13 +90,13 @@ include::line-range-format.txt[] ways to spell <revision range>, see the 'Specifying Ranges' section of linkgit:gitrevisions[7]. -[\--] <path>...:: +[--] <path>...:: Show only commits that are enough to explain how the files that match the specified paths came to be. See 'History Simplification' below for details and other simplification modes. + -Paths may need to be prefixed with ``\-- '' to separate them from +Paths may need to be prefixed with `--` to separate them from options or the revision range, when confusion arises. include::rev-list-options.txt[] @@ -125,7 +125,7 @@ EXAMPLES `git log --since="2 weeks ago" -- gitk`:: Show the changes during the last two weeks to the file 'gitk'. - The ``--'' is necessary to avoid confusion with the *branch* named + The `--` is necessary to avoid confusion with the *branch* named 'gitk' `git log --name-status release..test`:: diff --git a/Documentation/git-ls-remote.txt b/Documentation/git-ls-remote.txt index 5f2628c..b9fd377 100644 --- a/Documentation/git-ls-remote.txt +++ b/Documentation/git-ls-remote.txt @@ -10,7 +10,7 @@ SYNOPSIS -------- [verse] 'git ls-remote' [--heads] [--tags] [--refs] [--upload-pack=<exec>] - [-q | --quiet] [--exit-code] [--get-url] + [-q | --quiet] [--exit-code] [--get-url] [--sort=<key>] [--symref] [<repository> [<refs>...]] DESCRIPTION @@ -60,6 +60,24 @@ OPTIONS upload-pack only shows the symref HEAD, so it will be the only one shown by ls-remote. +--sort=<key>:: + Sort based on the key given. Prefix `-` to sort in descending order + of the value. Supports "version:refname" or "v:refname" (tag names + are treated as versions). The "version:refname" sort order can also + be affected by the "versionsort.suffix" configuration variable. + See linkgit:git-for-each-ref[1] for more sort options, but be aware + keys like `committerdate` that require access to the objects + themselves will not work for refs whose objects have not yet been + fetched from the remote, and will give a `missing object` error. + +-o <option>:: +--server-option=<option>:: + Transmit the given string to the server when communicating using + protocol version 2. The given string must not contain a NUL or LF + character. + When multiple `--server-option=<option>` are given, they are all + sent to the other side in the order listed on the command line. + <repository>:: The "remote" repository to query. This parameter can be either a URL or the name of a remote (see the GIT URLS and @@ -90,6 +108,10 @@ EXAMPLES c5db5456ae3b0873fc659c19fafdde22313cc441 refs/tags/v0.99.2 7ceca275d047c90c0c7d5afb13ab97efdf51bd6e refs/tags/v0.99.3 +SEE ALSO +-------- +linkgit:git-check-ref-format[1]. + GIT --- Part of the linkgit:git[1] suite diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt index 81bc490..6bfac6b 100644 --- a/Documentation/git-pack-objects.txt +++ b/Documentation/git-pack-objects.txt @@ -12,7 +12,7 @@ SYNOPSIS 'git pack-objects' [-q | --progress | --all-progress] [--all-progress-implied] [--no-reuse-delta] [--delta-base-offset] [--non-empty] [--local] [--incremental] [--window=<n>] [--depth=<n>] - [--revs [--unpacked | --all]] + [--revs [--unpacked | --all]] [--keep-pack=<pack-name>] [--stdout [--filter=<filter-spec>] | base-name] [--shallow] [--keep-true-parents] < object-list @@ -96,7 +96,9 @@ base-name:: it too deep affects the performance on the unpacker side, because delta data needs to be applied that many times to get to the necessary object. - The default value for --window is 10 and --depth is 50. ++ +The default value for --window is 10 and --depth is 50. The maximum +depth is 4095. --window-memory=<n>:: This option provides an additional limit on top of `--window`; @@ -126,6 +128,13 @@ base-name:: has a .keep file to be ignored, even if it would have otherwise been packed. +--keep-pack=<pack-name>:: + This flag causes an object already in the given pack to be + ignored, even if it would have otherwise been + packed. `<pack-name>` is the the pack file name without + leading directory (e.g. `pack-123.pack`). The option could be + specified multiple times to keep multiple packs. + --incremental:: This flag causes an object already in a pack to be ignored even if it would have otherwise been packed. diff --git a/Documentation/git-push.txt b/Documentation/git-push.txt index 5b08302..34410f9 100644 --- a/Documentation/git-push.txt +++ b/Documentation/git-push.txt @@ -300,7 +300,7 @@ origin +master` to force a push to the `master` branch). See the These options are passed to linkgit:git-send-pack[1]. A thin transfer significantly reduces the amount of sent data when the sender and receiver share many of the same objects in common. The default is - \--thin. + `--thin`. -q:: --quiet:: diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt index ae750e9..d90e790 100644 --- a/Documentation/git-repack.txt +++ b/Documentation/git-repack.txt @@ -9,7 +9,7 @@ git-repack - Pack unpacked objects in a repository SYNOPSIS -------- [verse] -'git repack' [-a] [-A] [-d] [-f] [-F] [-l] [-n] [-q] [-b] [--window=<n>] [--depth=<n>] [--threads=<n>] +'git repack' [-a] [-A] [-d] [-f] [-F] [-l] [-n] [-q] [-b] [--window=<n>] [--depth=<n>] [--threads=<n>] [--keep-pack=<pack-name>] DESCRIPTION ----------- @@ -90,7 +90,9 @@ other objects in that pack they already have locally. space. `--depth` limits the maximum delta depth; making it too deep affects the performance on the unpacker side, because delta data needs to be applied that many times to get to the necessary object. - The default value for --window is 10 and --depth is 50. ++ +The default value for --window is 10 and --depth is 50. The maximum +depth is 4095. --threads=<n>:: This option is passed through to `git pack-objects`. @@ -133,6 +135,13 @@ other objects in that pack they already have locally. with `-b` or `repack.writeBitmaps`, as it ensures that the bitmapped packfile has the necessary objects. +--keep-pack=<pack-name>:: + Exclude the given pack from repacking. This is the equivalent + of having `.keep` file on the pack. `<pack-name>` is the the + pack file name without leading directory (e.g. `pack-123.pack`). + The option could be specified multiple times to keep multiple + packs. + --unpack-unreachable=<when>:: When loosening unreachable objects, do not bother loosening any objects older than `<when>`. This can be used to optimize out diff --git a/Documentation/git-replace.txt b/Documentation/git-replace.txt index e5c57ae..246dc99 100644 --- a/Documentation/git-replace.txt +++ b/Documentation/git-replace.txt @@ -11,6 +11,7 @@ SYNOPSIS 'git replace' [-f] <object> <replacement> 'git replace' [-f] --edit <object> 'git replace' [-f] --graft <commit> [<parent>...] +'git replace' [-f] --convert-graft-file 'git replace' -d <object>... 'git replace' [--format=<format>] [-l [<pattern>]] @@ -87,9 +88,13 @@ OPTIONS content as <commit> except that its parents will be [<parent>...] instead of <commit>'s parents. A replacement ref is then created to replace <commit> with the newly created - commit. See contrib/convert-grafts-to-replace-refs.sh for an - example script based on this option that can convert grafts to - replace refs. + commit. Use `--convert-graft-file` to convert a + `$GIT_DIR/info/grafts` file and use replace refs instead. + +--convert-graft-file:: + Creates graft commits for all entries in `$GIT_DIR/info/grafts` + and deletes that file upon success. The purpose is to help users + with transitioning off of the now-deprecated graft file. -l <pattern>:: --list <pattern>:: diff --git a/Documentation/git-shortlog.txt b/Documentation/git-shortlog.txt index 5e35ea1..bc80905 100644 --- a/Documentation/git-shortlog.txt +++ b/Documentation/git-shortlog.txt @@ -8,7 +8,7 @@ git-shortlog - Summarize 'git log' output SYNOPSIS -------- [verse] -'git shortlog' [<options>] [<revision range>] [[\--] <path>...] +'git shortlog' [<options>] [<revision range>] [[--] <path>...] git log --pretty=short | 'git shortlog' [<options>] DESCRIPTION @@ -69,11 +69,11 @@ them. ways to spell <revision range>, see the "Specifying Ranges" section of linkgit:gitrevisions[7]. -[\--] <path>...:: +[--] <path>...:: Consider only commits that are enough to explain how the files that match the specified paths came to be. + -Paths may need to be prefixed with "\-- " to separate them from +Paths may need to be prefixed with `--` to separate them from options or the revision range, when confusion arises. MAPPING AUTHORS diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index 71c5618..630999f 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -213,8 +213,8 @@ sync [--recursive] [--] [<path>...]:: submodule URLs change upstream and you need to update your local repositories accordingly. + -"git submodule sync" synchronizes all submodules while -"git submodule sync \-- A" synchronizes submodule "A" only. +`git submodule sync` synchronizes all submodules while +`git submodule sync -- A` synchronizes submodule "A" only. + If `--recursive` is specified, this command will recurse into the registered submodules, and sync any nested submodules within. diff --git a/Documentation/git-worktree.txt b/Documentation/git-worktree.txt index 2755ca9..afc6576 100644 --- a/Documentation/git-worktree.txt +++ b/Documentation/git-worktree.txt @@ -14,7 +14,7 @@ SYNOPSIS 'git worktree lock' [--reason <string>] <worktree> 'git worktree move' <worktree> <new-path> 'git worktree prune' [-n] [-v] [--expire <expire>] -'git worktree remove' [--force] <worktree> +'git worktree remove' [-f] <worktree> 'git worktree unlock' <worktree> DESCRIPTION @@ -61,8 +61,13 @@ $ git worktree add --track -b <branch> <path> <remote>/<branch> ------------ + If `<commit-ish>` is omitted and neither `-b` nor `-B` nor `--detach` used, -then, as a convenience, a new branch based at HEAD is created automatically, -as if `-b $(basename <path>)` was specified. +then, as a convenience, the new worktree is associated with a branch +(call it `<branch>`) named after `$(basename <path>)`. If `<branch>` +doesn't exist, a new branch based on HEAD is automatically created as +if `-b <branch>` was given. If `<branch>` does exist, it will be +checked out in the new worktree, if it's not checked out anywhere +else, otherwise the command will refuse to create the worktree (unless +`--force` is used). list:: diff --git a/Documentation/git.txt b/Documentation/git.txt index 4767860..c662f41 100644 --- a/Documentation/git.txt +++ b/Documentation/git.txt @@ -11,7 +11,7 @@ SYNOPSIS [verse] 'git' [--version] [--help] [-C <path>] [-c <name>=<value>] [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path] - [-p|--paginate|--no-pager] [--no-replace-objects] [--bare] + [-p|--paginate|-P|--no-pager] [--no-replace-objects] [--bare] [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>] [--super-prefix=<path>] <command> [<args>] @@ -103,6 +103,7 @@ foo.bar= ...`) sets `foo.bar` to the empty string which `git config configuration options (see the "Configuration Mechanism" section below). +-P:: --no-pager:: Do not pipe Git output into a pager. diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt index 1094fe2..ee210be 100644 --- a/Documentation/gitattributes.txt +++ b/Documentation/gitattributes.txt @@ -279,6 +279,94 @@ few exceptions. Even though... catch potential problems early, safety triggers. +`working-tree-encoding` +^^^^^^^^^^^^^^^^^^^^^^^ + +Git recognizes files encoded in ASCII or one of its supersets (e.g. +UTF-8, ISO-8859-1, ...) as text files. Files encoded in certain other +encodings (e.g. UTF-16) are interpreted as binary and consequently +built-in Git text processing tools (e.g. 'git diff') as well as most Git +web front ends do not visualize the contents of these files by default. + +In these cases you can tell Git the encoding of a file in the working +directory with the `working-tree-encoding` attribute. If a file with this +attribute is added to Git, then Git reencodes the content from the +specified encoding to UTF-8. Finally, Git stores the UTF-8 encoded +content in its internal data structure (called "the index"). On checkout +the content is reencoded back to the specified encoding. + +Please note that using the `working-tree-encoding` attribute may have a +number of pitfalls: + +- Alternative Git implementations (e.g. JGit or libgit2) and older Git + versions (as of March 2018) do not support the `working-tree-encoding` + attribute. If you decide to use the `working-tree-encoding` attribute + in your repository, then it is strongly recommended to ensure that all + clients working with the repository support it. + + For example, Microsoft Visual Studio resources files (`*.rc`) or + PowerShell script files (`*.ps1`) are sometimes encoded in UTF-16. + If you declare `*.ps1` as files as UTF-16 and you add `foo.ps1` with + a `working-tree-encoding` enabled Git client, then `foo.ps1` will be + stored as UTF-8 internally. A client without `working-tree-encoding` + support will checkout `foo.ps1` as UTF-8 encoded file. This will + typically cause trouble for the users of this file. + + If a Git client, that does not support the `working-tree-encoding` + attribute, adds a new file `bar.ps1`, then `bar.ps1` will be + stored "as-is" internally (in this example probably as UTF-16). + A client with `working-tree-encoding` support will interpret the + internal contents as UTF-8 and try to convert it to UTF-16 on checkout. + That operation will fail and cause an error. + +- Reencoding content to non-UTF encodings can cause errors as the + conversion might not be UTF-8 round trip safe. If you suspect your + encoding to not be round trip safe, then add it to + `core.checkRoundtripEncoding` to make Git check the round trip + encoding (see linkgit:git-config[1]). SHIFT-JIS (Japanese character + set) is known to have round trip issues with UTF-8 and is checked by + default. + +- Reencoding content requires resources that might slow down certain + Git operations (e.g 'git checkout' or 'git add'). + +Use the `working-tree-encoding` attribute only if you cannot store a file +in UTF-8 encoding and if you want Git to be able to process the content +as text. + +As an example, use the following attributes if your '*.ps1' files are +UTF-16 encoded with byte order mark (BOM) and you want Git to perform +automatic line ending conversion based on your platform. + +------------------------ +*.ps1 text working-tree-encoding=UTF-16 +------------------------ + +Use the following attributes if your '*.ps1' files are UTF-16 little +endian encoded without BOM and you want Git to use Windows line endings +in the working directory. Please note, it is highly recommended to +explicitly define the line endings with `eol` if the `working-tree-encoding` +attribute is used to avoid ambiguity. + +------------------------ +*.ps1 text working-tree-encoding=UTF-16LE eol=CRLF +------------------------ + +You can get a list of all available encodings on your platform with the +following command: + +------------------------ +iconv --list +------------------------ + +If you do not know the encoding of a file, then you can use the `file` +command to guess the encoding: + +------------------------ +file foo.ps1 +------------------------ + + `ident` ^^^^^^^ diff --git a/Documentation/gitk.txt b/Documentation/gitk.txt index ca96c28..244cd01 100644 --- a/Documentation/gitk.txt +++ b/Documentation/gitk.txt @@ -8,7 +8,7 @@ gitk - The Git repository browser SYNOPSIS -------- [verse] -'gitk' [<options>] [<revision range>] [\--] [<path>...] +'gitk' [<options>] [<revision range>] [--] [<path>...] DESCRIPTION ----------- diff --git a/Documentation/gitremote-helpers.txt b/Documentation/gitremote-helpers.txt index 4b8c93e..9d1459a 100644 --- a/Documentation/gitremote-helpers.txt +++ b/Documentation/gitremote-helpers.txt @@ -102,6 +102,14 @@ Capabilities for Pushing + Supported commands: 'connect'. +'stateless-connect':: + Experimental; for internal use only. + Can attempt to connect to a remote server for communication + using git's wire-protocol version 2. See the documentation + for the stateless-connect command for more information. ++ +Supported commands: 'stateless-connect'. + 'push':: Can discover remote refs and push local commits and the history leading up to them to new or existing remote refs. @@ -136,6 +144,14 @@ Capabilities for Fetching + Supported commands: 'connect'. +'stateless-connect':: + Experimental; for internal use only. + Can attempt to connect to a remote server for communication + using git's wire-protocol version 2. See the documentation + for the stateless-connect command for more information. ++ +Supported commands: 'stateless-connect'. + 'fetch':: Can discover remote refs and transfer objects reachable from them to the local object store. @@ -375,6 +391,22 @@ Supported if the helper has the "export" capability. + Supported if the helper has the "connect" capability. +'stateless-connect' <service>:: + Experimental; for internal use only. + Connects to the given remote service for communication using + git's wire-protocol version 2. Valid replies to this command + are empty line (connection established), 'fallback' (no smart + transport support, fall back to dumb transports) and just + exiting with error message printed (can't connect, don't bother + trying to fall back). After line feed terminating the positive + (empty) response, the output of the service starts. Messages + (both request and response) must consist of zero or more + PKT-LINEs, terminating in a flush packet. The client must not + expect the server to store any state in between request-response + pairs. After the connection ends, the remote helper exits. ++ +Supported if the helper has the "stateless-connect" capability. + If a fatal error occurs, the program writes the error message to stderr and exits. The caller should expect that a suitable error message has been printed if the child closes the connection without diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt index 6b8888d..6c2d23d 100644 --- a/Documentation/glossary-content.txt +++ b/Documentation/glossary-content.txt @@ -463,7 +463,7 @@ exclude;; [[def_push]]push:: Pushing a <<def_branch,branch>> means to get the branch's <<def_head_ref,head ref>> from a remote <<def_repository,repository>>, - find out if it is a direct ancestor to the branch's local + find out if it is an ancestor to the branch's local head ref, and in that case, putting all objects, which are <<def_reachable,reachable>> from the local head ref, and which are missing from the remote diff --git a/Documentation/technical/api-config.txt b/Documentation/technical/api-config.txt index 9a778b0..fa39ac9 100644 --- a/Documentation/technical/api-config.txt +++ b/Documentation/technical/api-config.txt @@ -47,21 +47,23 @@ will first feed the user-wide one to the callback, and then the repo-specific one; by overwriting, the higher-priority repo-specific value is left at the end). -The `git_config_with_options` function lets the caller examine config +The `config_with_options` function lets the caller examine config while adjusting some of the default behavior of `git_config`. It should almost never be used by "regular" Git code that is looking up configuration variables. It is intended for advanced callers like `git-config`, which are intentionally tweaking the normal config-lookup process. It takes two extra parameters: -`filename`:: -If this parameter is non-NULL, it specifies the name of a file to -parse for configuration, rather than looking in the usual files. Regular -`git_config` defaults to `NULL`. +`config_source`:: +If this parameter is non-NULL, it specifies the source to parse for +configuration, rather than looking in the usual files. See `struct +git_config_source` in `config.h` for details. Regular `git_config` defaults +to `NULL`. -`respect_includes`:: -Specify whether include directives should be followed in parsed files. -Regular `git_config` defaults to `1`. +`opts`:: +Specify options to adjust the behavior of parsing config files. See `struct +config_options` in `config.h` for details. As an example: regular `git_config` +sets `opts.respect_includes` to `1` by default. Reading Specific Files ---------------------- diff --git a/Documentation/technical/api-submodule-config.txt b/Documentation/technical/api-submodule-config.txt index ee907c4..fb06089 100644 --- a/Documentation/technical/api-submodule-config.txt +++ b/Documentation/technical/api-submodule-config.txt @@ -38,7 +38,7 @@ Data Structures Functions --------- -`void submodule_free()`:: +`void submodule_free(struct repository *r)`:: Use these to free the internally cached values. diff --git a/Documentation/technical/commit-graph-format.txt b/Documentation/technical/commit-graph-format.txt new file mode 100644 index 0000000..ad6af81 --- /dev/null +++ b/Documentation/technical/commit-graph-format.txt @@ -0,0 +1,97 @@ +Git commit graph format +======================= + +The Git commit graph stores a list of commit OIDs and some associated +metadata, including: + +- The generation number of the commit. Commits with no parents have + generation number 1; commits with parents have generation number + one more than the maximum generation number of its parents. We + reserve zero as special, and can be used to mark a generation + number invalid or as "not computed". + +- The root tree OID. + +- The commit date. + +- The parents of the commit, stored using positional references within + the graph file. + +These positional references are stored as unsigned 32-bit integers +corresponding to the array position withing the list of commit OIDs. We +use the most-significant bit for special purposes, so we can store at most +(1 << 31) - 1 (around 2 billion) commits. + +== Commit graph files have the following format: + +In order to allow extensions that add extra data to the graph, we organize +the body into "chunks" and provide a binary lookup table at the beginning +of the body. The header includes certain values, such as number of chunks +and hash type. + +All 4-byte numbers are in network order. + +HEADER: + + 4-byte signature: + The signature is: {'C', 'G', 'P', 'H'} + + 1-byte version number: + Currently, the only valid version is 1. + + 1-byte Hash Version (1 = SHA-1) + We infer the hash length (H) from this value. + + 1-byte number (C) of "chunks" + + 1-byte (reserved for later use) + Current clients should ignore this value. + +CHUNK LOOKUP: + + (C + 1) * 12 bytes listing the table of contents for the chunks: + First 4 bytes describe the chunk id. Value 0 is a terminating label. + Other 8 bytes provide the byte-offset in current file for chunk to + start. (Chunks are ordered contiguously in the file, so you can infer + the length using the next chunk position if necessary.) Each chunk + ID appears at most once. + + The remaining data in the body is described one chunk at a time, and + these chunks may be given in any order. Chunks are required unless + otherwise specified. + +CHUNK DATA: + + OID Fanout (ID: {'O', 'I', 'D', 'F'}) (256 * 4 bytes) + The ith entry, F[i], stores the number of OIDs with first + byte at most i. Thus F[255] stores the total + number of commits (N). + + OID Lookup (ID: {'O', 'I', 'D', 'L'}) (N * H bytes) + The OIDs for all commits in the graph, sorted in ascending order. + + Commit Data (ID: {'C', 'G', 'E', 'T' }) (N * (H + 16) bytes) + * The first H bytes are for the OID of the root tree. + * The next 8 bytes are for the positions of the first two parents + of the ith commit. Stores value 0xffffffff if no parent in that + position. If there are more than two parents, the second value + has its most-significant bit on and the other bits store an array + position into the Large Edge List chunk. + * The next 8 bytes store the generation number of the commit and + the commit time in seconds since EPOCH. The generation number + uses the higher 30 bits of the first 4 bytes, while the commit + time uses the 32 bits of the second 4 bytes, along with the lowest + 2 bits of the lowest byte, storing the 33rd and 34th bit of the + commit time. + + Large Edge List (ID: {'E', 'D', 'G', 'E'}) [Optional] + This list of 4-byte values store the second through nth parents for + all octopus merges. The second parent value in the commit data stores + an array position within this list along with the most-significant bit + on. Starting at that array position, iterate through this list of commit + positions for the parents until reaching a value with the most-significant + bit on. The other bits correspond to the position of the last parent. + +TRAILER: + + H-byte HASH-checksum of all of the above. diff --git a/Documentation/technical/commit-graph.txt b/Documentation/technical/commit-graph.txt new file mode 100644 index 0000000..0550c6d --- /dev/null +++ b/Documentation/technical/commit-graph.txt @@ -0,0 +1,163 @@ +Git Commit Graph Design Notes +============================= + +Git walks the commit graph for many reasons, including: + +1. Listing and filtering commit history. +2. Computing merge bases. + +These operations can become slow as the commit count grows. The merge +base calculation shows up in many user-facing commands, such as 'merge-base' +or 'status' and can take minutes to compute depending on history shape. + +There are two main costs here: + +1. Decompressing and parsing commits. +2. Walking the entire graph to satisfy topological order constraints. + +The commit graph file is a supplemental data structure that accelerates +commit graph walks. If a user downgrades or disables the 'core.commitGraph' +config setting, then the existing ODB is sufficient. The file is stored +as "commit-graph" either in the .git/objects/info directory or in the info +directory of an alternate. + +The commit graph file stores the commit graph structure along with some +extra metadata to speed up graph walks. By listing commit OIDs in lexi- +cographic order, we can identify an integer position for each commit and +refer to the parents of a commit using those integer positions. We use +binary search to find initial commits and then use the integer positions +for fast lookups during the walk. + +A consumer may load the following info for a commit from the graph: + +1. The commit OID. +2. The list of parents, along with their integer position. +3. The commit date. +4. The root tree OID. +5. The generation number (see definition below). + +Values 1-4 satisfy the requirements of parse_commit_gently(). + +Define the "generation number" of a commit recursively as follows: + + * A commit with no parents (a root commit) has generation number one. + + * A commit with at least one parent has generation number one more than + the largest generation number among its parents. + +Equivalently, the generation number of a commit A is one more than the +length of a longest path from A to a root commit. The recursive definition +is easier to use for computation and observing the following property: + + If A and B are commits with generation numbers N and M, respectively, + and N <= M, then A cannot reach B. That is, we know without searching + that B is not an ancestor of A because it is further from a root commit + than A. + + Conversely, when checking if A is an ancestor of B, then we only need + to walk commits until all commits on the walk boundary have generation + number at most N. If we walk commits using a priority queue seeded by + generation numbers, then we always expand the boundary commit with highest + generation number and can easily detect the stopping condition. + +This property can be used to significantly reduce the time it takes to +walk commits and determine topological relationships. Without generation +numbers, the general heuristic is the following: + + If A and B are commits with commit time X and Y, respectively, and + X < Y, then A _probably_ cannot reach B. + +This heuristic is currently used whenever the computation is allowed to +violate topological relationships due to clock skew (such as "git log" +with default order), but is not used when the topological order is +required (such as merge base calculations, "git log --graph"). + +In practice, we expect some commits to be created recently and not stored +in the commit graph. We can treat these commits as having "infinite" +generation number and walk until reaching commits with known generation +number. + +Design Details +-------------- + +- The commit graph file is stored in a file named 'commit-graph' in the + .git/objects/info directory. This could be stored in the info directory + of an alternate. + +- The core.commitGraph config setting must be on to consume graph files. + +- The file format includes parameters for the object ID hash function, + so a future change of hash algorithm does not require a change in format. + +Future Work +----------- + +- The commit graph feature currently does not honor commit grafts. This can + be remedied by duplicating or refactoring the current graft logic. + +- The 'commit-graph' subcommand does not have a "verify" mode that is + necessary for integration with fsck. + +- The file format includes room for precomputed generation numbers. These + are not currently computed, so all generation numbers will be marked as + 0 (or "uncomputed"). A later patch will include this calculation. + +- After computing and storing generation numbers, we must make graph + walks aware of generation numbers to gain the performance benefits they + enable. This will mostly be accomplished by swapping a commit-date-ordered + priority queue with one ordered by generation number. The following + operations are important candidates: + + - paint_down_to_common() + - 'log --topo-order' + +- Currently, parse_commit_gently() requires filling in the root tree + object for a commit. This passes through lookup_tree() and consequently + lookup_object(). Also, it calls lookup_commit() when loading the parents. + These method calls check the ODB for object existence, even if the + consumer does not need the content. For example, we do not need the + tree contents when computing merge bases. Now that commit parsing is + removed from the computation time, these lookup operations are the + slowest operations keeping graph walks from being fast. Consider + loading these objects without verifying their existence in the ODB and + only loading them fully when consumers need them. Consider a method + such as "ensure_tree_loaded(commit)" that fully loads a tree before + using commit->tree. + +- The current design uses the 'commit-graph' subcommand to generate the graph. + When this feature stabilizes enough to recommend to most users, we should + add automatic graph writes to common operations that create many commits. + For example, one could compute a graph on 'clone', 'fetch', or 'repack' + commands. + +- A server could provide a commit graph file as part of the network protocol + to avoid extra calculations by clients. This feature is only of benefit if + the user is willing to trust the file, because verifying the file is correct + is as hard as computing it from scratch. + +Related Links +------------- +[0] https://bugs.chromium.org/p/git/issues/detail?id=8 + Chromium work item for: Serialized Commit Graph + +[1] https://public-inbox.org/git/20110713070517.GC18566@sigill.intra.peff.net/ + An abandoned patch that introduced generation numbers. + +[2] https://public-inbox.org/git/20170908033403.q7e6dj7benasrjes@sigill.intra.peff.net/ + Discussion about generation numbers on commits and how they interact + with fsck. + +[3] https://public-inbox.org/git/20170908034739.4op3w4f2ma5s65ku@sigill.intra.peff.net/ + More discussion about generation numbers and not storing them inside + commit objects. A valuable quote: + + "I think we should be moving more in the direction of keeping + repo-local caches for optimizations. Reachability bitmaps have been + a big performance win. I think we should be doing the same with our + properties of commits. Not just generation numbers, but making it + cheap to access the graph structure without zlib-inflating whole + commit objects (i.e., packv4 or something like the "metapacks" I + proposed a few years ago)." + +[4] https://public-inbox.org/git/20180108154822.54829-1-git@jeffhostetler.com/T/#u + A patch to remove the ahead-behind calculation from 'status'. diff --git a/Documentation/technical/pack-format.txt b/Documentation/technical/pack-format.txt index 8e5bf60..70a99fd 100644 --- a/Documentation/technical/pack-format.txt +++ b/Documentation/technical/pack-format.txt @@ -36,6 +36,98 @@ Git pack format - The trailer records 20-byte SHA-1 checksum of all of the above. +=== Object types + +Valid object types are: + +- OBJ_COMMIT (1) +- OBJ_TREE (2) +- OBJ_BLOB (3) +- OBJ_TAG (4) +- OBJ_OFS_DELTA (6) +- OBJ_REF_DELTA (7) + +Type 5 is reserved for future expansion. Type 0 is invalid. + +=== Deltified representation + +Conceptually there are only four object types: commit, tree, tag and +blob. However to save space, an object could be stored as a "delta" of +another "base" object. These representations are assigned new types +ofs-delta and ref-delta, which is only valid in a pack file. + +Both ofs-delta and ref-delta store the "delta" to be applied to +another object (called 'base object') to reconstruct the object. The +difference between them is, ref-delta directly encodes 20-byte base +object name. If the base object is in the same pack, ofs-delta encodes +the offset of the base object in the pack instead. + +The base object could also be deltified if it's in the same pack. +Ref-delta can also refer to an object outside the pack (i.e. the +so-called "thin pack"). When stored on disk however, the pack should +be self contained to avoid cyclic dependency. + +The delta data is a sequence of instructions to reconstruct an object +from the base object. If the base object is deltified, it must be +converted to canonical form first. Each instruction appends more and +more data to the target object until it's complete. There are two +supported instructions so far: one for copy a byte range from the +source object and one for inserting new data embedded in the +instruction itself. + +Each instruction has variable length. Instruction type is determined +by the seventh bit of the first octet. The following diagrams follow +the convention in RFC 1951 (Deflate compressed data format). + +==== Instruction to copy from base object + + +----------+---------+---------+---------+---------+-------+-------+-------+ + | 1xxxxxxx | offset1 | offset2 | offset3 | offset4 | size1 | size2 | size3 | + +----------+---------+---------+---------+---------+-------+-------+-------+ + +This is the instruction format to copy a byte range from the source +object. It encodes the offset to copy from and the number of bytes to +copy. Offset and size are in little-endian order. + +All offset and size bytes are optional. This is to reduce the +instruction size when encoding small offsets or sizes. The first seven +bits in the first octet determines which of the next seven octets is +present. If bit zero is set, offset1 is present. If bit one is set +offset2 is present and so on. + +Note that a more compact instruction does not change offset and size +encoding. For example, if only offset2 is omitted like below, offset3 +still contains bits 16-23. It does not become offset2 and contains +bits 8-15 even if it's right next to offset1. + + +----------+---------+---------+ + | 10000101 | offset1 | offset3 | + +----------+---------+---------+ + +In its most compact form, this instruction only takes up one byte +(0x80) with both offset and size omitted, which will have default +values zero. There is another exception: size zero is automatically +converted to 0x10000. + +==== Instruction to add new data + + +----------+============+ + | 0xxxxxxx | data | + +----------+============+ + +This is the instruction to construct target object without the base +object. The following data is appended to the target object. The first +seven bits of the first octet determines the size of data in +bytes. The size must be non-zero. + +==== Reserved instruction + + +----------+============ + | 00000000 | + +----------+============ + +This is the instruction reserved for future expansion. + == Original (version 1) pack-*.idx files have the following format: - The header consists of 256 4-byte network byte order diff --git a/Documentation/technical/protocol-v2.txt b/Documentation/technical/protocol-v2.txt new file mode 100644 index 0000000..d7b6f38 --- /dev/null +++ b/Documentation/technical/protocol-v2.txt @@ -0,0 +1,405 @@ + Git Wire Protocol, Version 2 +============================== + +This document presents a specification for a version 2 of Git's wire +protocol. Protocol v2 will improve upon v1 in the following ways: + + * Instead of multiple service names, multiple commands will be + supported by a single service + * Easily extendable as capabilities are moved into their own section + of the protocol, no longer being hidden behind a NUL byte and + limited by the size of a pkt-line + * Separate out other information hidden behind NUL bytes (e.g. agent + string as a capability and symrefs can be requested using 'ls-refs') + * Reference advertisement will be omitted unless explicitly requested + * ls-refs command to explicitly request some refs + * Designed with http and stateless-rpc in mind. With clear flush + semantics the http remote helper can simply act as a proxy + +In protocol v2 communication is command oriented. When first contacting a +server a list of capabilities will advertised. Some of these capabilities +will be commands which a client can request be executed. Once a command +has completed, a client can reuse the connection and request that other +commands be executed. + + Packet-Line Framing +--------------------- + +All communication is done using packet-line framing, just as in v1. See +`Documentation/technical/pack-protocol.txt` and +`Documentation/technical/protocol-common.txt` for more information. + +In protocol v2 these special packets will have the following semantics: + + * '0000' Flush Packet (flush-pkt) - indicates the end of a message + * '0001' Delimiter Packet (delim-pkt) - separates sections of a message + + Initial Client Request +------------------------ + +In general a client can request to speak protocol v2 by sending +`version=2` through the respective side-channel for the transport being +used which inevitably sets `GIT_PROTOCOL`. More information can be +found in `pack-protocol.txt` and `http-protocol.txt`. In all cases the +response from the server is the capability advertisement. + + Git Transport +~~~~~~~~~~~~~~~ + +When using the git:// transport, you can request to use protocol v2 by +sending "version=2" as an extra parameter: + + 003egit-upload-pack /project.git\0host=myserver.com\0\0version=2\0 + + SSH and File Transport +~~~~~~~~~~~~~~~~~~~~~~~~ + +When using either the ssh:// or file:// transport, the GIT_PROTOCOL +environment variable must be set explicitly to include "version=2". + + HTTP Transport +~~~~~~~~~~~~~~~~ + +When using the http:// or https:// transport a client makes a "smart" +info/refs request as described in `http-protocol.txt` and requests that +v2 be used by supplying "version=2" in the `Git-Protocol` header. + + C: Git-Protocol: version=2 + C: + C: GET $GIT_URL/info/refs?service=git-upload-pack HTTP/1.0 + +A v2 server would reply: + + S: 200 OK + S: <Some headers> + S: ... + S: + S: 000eversion 2\n + S: <capability-advertisement> + +Subsequent requests are then made directly to the service +`$GIT_URL/git-upload-pack`. (This works the same for git-receive-pack). + + Capability Advertisement +-------------------------- + +A server which decides to communicate (based on a request from a client) +using protocol version 2, notifies the client by sending a version string +in its initial response followed by an advertisement of its capabilities. +Each capability is a key with an optional value. Clients must ignore all +unknown keys. Semantics of unknown values are left to the definition of +each key. Some capabilities will describe commands which can be requested +to be executed by the client. + + capability-advertisement = protocol-version + capability-list + flush-pkt + + protocol-version = PKT-LINE("version 2" LF) + capability-list = *capability + capability = PKT-LINE(key[=value] LF) + + key = 1*(ALPHA | DIGIT | "-_") + value = 1*(ALPHA | DIGIT | " -_.,?\/{}[]()<>!@#$%^&*+=:;") + + Command Request +----------------- + +After receiving the capability advertisement, a client can then issue a +request to select the command it wants with any particular capabilities +or arguments. There is then an optional section where the client can +provide any command specific parameters or queries. Only a single +command can be requested at a time. + + request = empty-request | command-request + empty-request = flush-pkt + command-request = command + capability-list + [command-args] + flush-pkt + command = PKT-LINE("command=" key LF) + command-args = delim-pkt + *command-specific-arg + + command-specific-args are packet line framed arguments defined by + each individual command. + +The server will then check to ensure that the client's request is +comprised of a valid command as well as valid capabilities which were +advertised. If the request is valid the server will then execute the +command. A server MUST wait till it has received the client's entire +request before issuing a response. The format of the response is +determined by the command being executed, but in all cases a flush-pkt +indicates the end of the response. + +When a command has finished, and the client has received the entire +response from the server, a client can either request that another +command be executed or can terminate the connection. A client may +optionally send an empty request consisting of just a flush-pkt to +indicate that no more requests will be made. + + Capabilities +-------------- + +There are two different types of capabilities: normal capabilities, +which can be used to to convey information or alter the behavior of a +request, and commands, which are the core actions that a client wants to +perform (fetch, push, etc). + +Protocol version 2 is stateless by default. This means that all commands +must only last a single round and be stateless from the perspective of the +server side, unless the client has requested a capability indicating that +state should be maintained by the server. Clients MUST NOT require state +management on the server side in order to function correctly. This +permits simple round-robin load-balancing on the server side, without +needing to worry about state management. + + agent +~~~~~~~ + +The server can advertise the `agent` capability with a value `X` (in the +form `agent=X`) to notify the client that the server is running version +`X`. The client may optionally send its own agent string by including +the `agent` capability with a value `Y` (in the form `agent=Y`) in its +request to the server (but it MUST NOT do so if the server did not +advertise the agent capability). The `X` and `Y` strings may contain any +printable ASCII characters except space (i.e., the byte range 32 < x < +127), and are typically of the form "package/version" (e.g., +"git/1.8.3.1"). The agent strings are purely informative for statistics +and debugging purposes, and MUST NOT be used to programmatically assume +the presence or absence of particular features. + + ls-refs +~~~~~~~~~ + +`ls-refs` is the command used to request a reference advertisement in v2. +Unlike the current reference advertisement, ls-refs takes in arguments +which can be used to limit the refs sent from the server. + +Additional features not supported in the base command will be advertised +as the value of the command in the capability advertisement in the form +of a space separated list of features: "<command>=<feature 1> <feature 2>" + +ls-refs takes in the following arguments: + + symrefs + In addition to the object pointed by it, show the underlying ref + pointed by it when showing a symbolic ref. + peel + Show peeled tags. + ref-prefix <prefix> + When specified, only references having a prefix matching one of + the provided prefixes are displayed. + +The output of ls-refs is as follows: + + output = *ref + flush-pkt + ref = PKT-LINE(obj-id SP refname *(SP ref-attribute) LF) + ref-attribute = (symref | peeled) + symref = "symref-target:" symref-target + peeled = "peeled:" obj-id + + fetch +~~~~~~~ + +`fetch` is the command used to fetch a packfile in v2. It can be looked +at as a modified version of the v1 fetch where the ref-advertisement is +stripped out (since the `ls-refs` command fills that role) and the +message format is tweaked to eliminate redundancies and permit easy +addition of future extensions. + +Additional features not supported in the base command will be advertised +as the value of the command in the capability advertisement in the form +of a space separated list of features: "<command>=<feature 1> <feature 2>" + +A `fetch` request can take the following arguments: + + want <oid> + Indicates to the server an object which the client wants to + retrieve. Wants can be anything and are not limited to + advertised objects. + + have <oid> + Indicates to the server an object which the client has locally. + This allows the server to make a packfile which only contains + the objects that the client needs. Multiple 'have' lines can be + supplied. + + done + Indicates to the server that negotiation should terminate (or + not even begin if performing a clone) and that the server should + use the information supplied in the request to construct the + packfile. + + thin-pack + Request that a thin pack be sent, which is a pack with deltas + which reference base objects not contained within the pack (but + are known to exist at the receiving end). This can reduce the + network traffic significantly, but it requires the receiving end + to know how to "thicken" these packs by adding the missing bases + to the pack. + + no-progress + Request that progress information that would normally be sent on + side-band channel 2, during the packfile transfer, should not be + sent. However, the side-band channel 3 is still used for error + responses. + + include-tag + Request that annotated tags should be sent if the objects they + point to are being sent. + + ofs-delta + Indicate that the client understands PACKv2 with delta referring + to its base by position in pack rather than by an oid. That is, + they can read OBJ_OFS_DELTA (ake type 6) in a packfile. + +If the 'shallow' feature is advertised the following arguments can be +included in the clients request as well as the potential addition of the +'shallow-info' section in the server's response as explained below. + + shallow <oid> + A client must notify the server of all commits for which it only + has shallow copies (meaning that it doesn't have the parents of + a commit) by supplying a 'shallow <oid>' line for each such + object so that the server is aware of the limitations of the + client's history. This is so that the server is aware that the + client may not have all objects reachable from such commits. + + deepen <depth> + Requests that the fetch/clone should be shallow having a commit + depth of <depth> relative to the remote side. + + deepen-relative + Requests that the semantics of the "deepen" command be changed + to indicate that the depth requested is relative to the client's + current shallow boundary, instead of relative to the requested + commits. + + deepen-since <timestamp> + Requests that the shallow clone/fetch should be cut at a + specific time, instead of depth. Internally it's equivalent to + doing "git rev-list --max-age=<timestamp>". Cannot be used with + "deepen". + + deepen-not <rev> + Requests that the shallow clone/fetch should be cut at a + specific revision specified by '<rev>', instead of a depth. + Internally it's equivalent of doing "git rev-list --not <rev>". + Cannot be used with "deepen", but can be used with + "deepen-since". + +The response of `fetch` is broken into a number of sections separated by +delimiter packets (0001), with each section beginning with its section +header. + + output = *section + section = (acknowledgments | shallow-info | packfile) + (flush-pkt | delim-pkt) + + acknowledgments = PKT-LINE("acknowledgments" LF) + (nak | *ack) + (ready) + ready = PKT-LINE("ready" LF) + nak = PKT-LINE("NAK" LF) + ack = PKT-LINE("ACK" SP obj-id LF) + + shallow-info = PKT-LINE("shallow-info" LF) + *PKT-LINE((shallow | unshallow) LF) + shallow = "shallow" SP obj-id + unshallow = "unshallow" SP obj-id + + packfile = PKT-LINE("packfile" LF) + *PKT-LINE(%x01-03 *%x00-ff) + + acknowledgments section + * If the client determines that it is finished with negotiations + by sending a "done" line, the acknowledgments sections MUST be + omitted from the server's response. + + * Always begins with the section header "acknowledgments" + + * The server will respond with "NAK" if none of the object ids sent + as have lines were common. + + * The server will respond with "ACK obj-id" for all of the + object ids sent as have lines which are common. + + * A response cannot have both "ACK" lines as well as a "NAK" + line. + + * The server will respond with a "ready" line indicating that + the server has found an acceptable common base and is ready to + make and send a packfile (which will be found in the packfile + section of the same response) + + * If the server has found a suitable cut point and has decided + to send a "ready" line, then the server can decide to (as an + optimization) omit any "ACK" lines it would have sent during + its response. This is because the server will have already + determined the objects it plans to send to the client and no + further negotiation is needed. + + shallow-info section + * If the client has requested a shallow fetch/clone, a shallow + client requests a fetch or the server is shallow then the + server's response may include a shallow-info section. The + shallow-info section will be included if (due to one of the + above conditions) the server needs to inform the client of any + shallow boundaries or adjustments to the clients already + existing shallow boundaries. + + * Always begins with the section header "shallow-info" + + * If a positive depth is requested, the server will compute the + set of commits which are no deeper than the desired depth. + + * The server sends a "shallow obj-id" line for each commit whose + parents will not be sent in the following packfile. + + * The server sends an "unshallow obj-id" line for each commit + which the client has indicated is shallow, but is no longer + shallow as a result of the fetch (due to its parents being + sent in the following packfile). + + * The server MUST NOT send any "unshallow" lines for anything + which the client has not indicated was shallow as a part of + its request. + + * This section is only included if a packfile section is also + included in the response. + + packfile section + * This section is only included if the client has sent 'want' + lines in its request and either requested that no more + negotiation be done by sending 'done' or if the server has + decided it has found a sufficient cut point to produce a + packfile. + + * Always begins with the section header "packfile" + + * The transmission of the packfile begins immediately after the + section header + + * The data transfer of the packfile is always multiplexed, using + the same semantics of the 'side-band-64k' capability from + protocol version 1. This means that each packet, during the + packfile data stream, is made up of a leading 4-byte pkt-line + length (typical of the pkt-line format), followed by a 1-byte + stream code, followed by the actual data. + + The stream code can be one of: + 1 - pack data + 2 - progress messages + 3 - fatal error message just before stream aborts + + server-option +~~~~~~~~~~~~~~~ + +If advertised, indicates that any number of server specific options can be +included in a request. This is done by sending each option as a +"server-option=<option>" capability line in the capability-list section of +a request. + +The provided options must not contain a NUL or LF character. diff --git a/Documentation/technical/shallow.txt b/Documentation/technical/shallow.txt index 5183b15..01dedfe 100644 --- a/Documentation/technical/shallow.txt +++ b/Documentation/technical/shallow.txt @@ -8,20 +8,22 @@ repo, and therefore grafts are introduced pretending that these commits have no parents. ********************************************************* -The basic idea is to write the SHA-1s of shallow commits into -$GIT_DIR/shallow, and handle its contents like the contents -of $GIT_DIR/info/grafts (with the difference that shallow -cannot contain parent information). - -This information is stored in a new file instead of grafts, or -even the config, since the user should not touch that file -at all (even throughout development of the shallow clone, it -was never manually edited!). +$GIT_DIR/shallow lists commit object names and tells Git to +pretend as if they are root commits (e.g. "git log" traversal +stops after showing them; "git fsck" does not complain saying +the commits listed on their "parent" lines do not exist). Each line contains exactly one SHA-1. When read, a commit_graft will be constructed, which has nr_parent < 0 to make it easier to discern from user provided grafts. +Note that the shallow feature could not be changed easily to +use replace refs: a commit containing a `mergetag` is not allowed +to be replaced, not even by a root commit. Such a commit can be +made shallow, though. Also, having a `shallow` file explicitly +listing all the commits made shallow makes it a *lot* easier to +do shallow-specific things such as to deepen the history. + Since fsck-objects relies on the library to read the objects, it honours shallow commits automatically. |