path: root/Documentation/technical
AgeCommit message (Collapse)Author
2014-02-10pack-protocol.txt: clarify 'obj-id' in the last ACK after 'done'Nguyễn Thái Ngọc Duy
It's introduced in 1bd8c8f (git-upload-pack: Support the multi_ack protocol - 2005-10-28) but probably better documented in the commit message of 78affc4 (Add multi_ack_detailed capability to fetch-pack/upload-pack - 2009-10-30). Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2014-01-27http-protocol.txt: don't use uppercase for variable names in "The ↵Thomas Ackermann
Negotiation Algorithm" Signed-off-by: Thomas Ackermann <> Signed-off-by: Junio C Hamano <>
2014-01-27create HTML for http-protocol.txtThomas Ackermann
./Documentation/technical/http-protocol.txt was missing from TECH_DOCS in Makefile. Add it and also improve HTML formatting while still retaining good readability of the ASCII text: - Use monospace font instead of italicized or roman font for machine output and source text - Use roman font for things which should be body text - Use double quotes consistently for "want" and "have" commands - Use uppercase "C" / "S" consistently for "client" / "server"; also use "C:" / "S:" instead of "(C)" / "(S)" for consistency and to avoid having formatted "(C)" as copyright symbol in HTML - Use only spaces and not a combination of tabs and spaces for whitespace Signed-off-by: Thomas Ackermann <> Signed-off-by: Junio C Hamano <>
2014-01-17Merge branch 'nd/shallow-clone'Junio C Hamano
Fetching from a shallow-cloned repository used to be forbidden, primarily because the codepaths involved were not carefully vetted and we did not bother supporting such usage. This attempts to allow object transfer out of a shallow-cloned repository in a controlled way (i.e. the receiver become a shallow repository with truncated history). * nd/shallow-clone: (31 commits) t5537: fix incorrect expectation in test case 10 shallow: remove unused code send-pack.c: mark a file-local function static git-clone.txt: remove shallow clone limitations prune: clean .git/shallow after pruning objects clone: use git protocol for cloning shallow repo locally send-pack: support pushing from a shallow clone via http receive-pack: support pushing to a shallow clone via http smart-http: support shallow fetch/clone remote-curl: pass ref SHA-1 to fetch-pack as well send-pack: support pushing to a shallow clone receive-pack: allow pushes that update .git/shallow connected.c: add new variant that runs with --shallow-file add GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses receive/send-pack: support pushing from a shallow clone receive-pack: reorder some code in unpack() fetch: add --update-shallow to accept refs that update .git/shallow upload-pack: make sure deepening preserves shallow roots fetch: support fetching from a shallow repository clone: support remote shallow repository ...
2014-01-13Sync with C Hamano
* maint: Git pack-heuristics.txt: mark up the file header properly
2014-01-13pack-heuristics.txt: mark up the file header properlyThomas Ackermann
AsciiDoc wants these header-lines left-aligned. Signed-off-by: Thomas Ackermann <> Signed-off-by: Junio C Hamano <>
2014-01-06builtin/help.c: speed up is_git_command() by checking for builtin commands firstSebastian Schuberth
Since 2dce956 is_git_command() is a bit slow as it does file I/O in the call to list_commands_in_dir(). Avoid the file I/O by adding an early check for the builtin commands. Signed-off-by: Sebastian Schuberth <> Signed-off-by: Junio C Hamano <>
2014-01-06git.c: consistently use the term "builtin" instead of "internal command"Sebastian Schuberth
Signed-off-by: Sebastian Schuberth <> Signed-off-by: Junio C Hamano <>
2013-12-30pack-bitmap: implement optional name_hash cacheVicent Marti
When we use pack bitmaps rather than walking the object graph, we end up with the list of objects to include in the packfile, but we do not know the path at which any tree or blob objects would be found. In a recently packed repository, this is fine. A fetch would use the paths only as a heuristic in the delta compression phase, and a fully packed repository should not need to do much delta compression. As time passes, though, we may acquire more objects on top of our large bitmapped pack. If clients fetch frequently, then they never even look at the bitmapped history, and all works as usual. However, a client who has not fetched since the last bitmap repack will have "have" tips in the bitmapped history, but "want" newer objects. The bitmaps themselves degrade gracefully in this circumstance. We manually walk the more recent bits of history, and then use bitmaps when we hit them. But we would also like to perform delta compression between the newer objects and the bitmapped objects (both to delta against what we know the user already has, but also between "new" and "old" objects that the user is fetching). The lack of pathnames makes our delta heuristics much less effective. This patch adds an optional cache of the 32-bit name_hash values to the end of the bitmap file. If present, a reader can use it to match bitmapped and non-bitmapped names during delta compression. Here are perf results for p5310: Test origin/master HEAD^ HEAD ------------------------------------------------------------------------------------------------- 5310.2: repack to disk 36.81(37.82+1.43) 47.70(48.74+1.41) +29.6% 47.75(48.70+1.51) +29.7% 5310.3: simulated clone 30.78(29.70+2.14) 1.08(0.97+0.10) -96.5% 1.07(0.94+0.12) -96.5% 5310.4: simulated fetch 3.16(6.10+0.08) 3.54(10.65+0.06) +12.0% 1.70(3.07+0.06) -46.2% 5310.6: partial bitmap 36.76(43.19+1.81) 6.71(11.25+0.76) -81.7% 4.08(6.26+0.46) -88.9% You can see that the time spent on an incremental fetch goes down, as our delta heuristics are able to do their work. And we save time on the partial bitmap clone for the same reason. Signed-off-by: Vicent Marti <> Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2013-12-30documentation: add documentation for the bitmap formatVicent Marti
This is the technical documentation for the JGit-compatible Bitmap v1 on-disk format. Signed-off-by: Vicent Marti <> Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2013-12-17Merge branch 'mi/typofixes' into maintJunio C Hamano
* mi/typofixes: contrib: typofixes Documentation/technical/http-protocol.txt: typofixes typofixes: fix misspelt comments
2013-12-12Merge branch 'cn/thin-push-capability'Junio C Hamano
Allow receive-pack to insist on receiving a fat pack from "git push" clients. * cn/thin-push-capability: send-pack: don't send a thin pack to a server which doesn't support it
2013-12-12Merge branch 'mh/fetch-tags-in-addition-to-normal-refs'Junio C Hamano
The "--tags" option to "git fetch" used to be literally a synonym to a "refs/tags/*:refs/tags/*" refspec, which meant that (1) as an explicit refspec given from the command line, it silenced the lazy "git fetch" default that is configured, and (2) also as an explicit refspec given from the command line, it interacted with "--prune" to remove any tag that the remote we are fetching from does not have. This demotes it to an option; with it, we fetch all tags in addition to what would be fetched without the option, and it does not interact with the decision "--prune" makes to see what remote-tracking refs the local has are missing the remote counterpart. * mh/fetch-tags-in-addition-to-normal-refs: (23 commits) fetch: improve the error messages emitted for conflicting refspecs handle_duplicate(): mark error message for translation ref_remote_duplicates(): extract a function handle_duplicate() ref_remove_duplicates(): simplify loop logic t5536: new test of refspec conflicts when fetching ref_remove_duplicates(): avoid redundant bisection git-fetch.txt: improve description of tag auto-following fetch-options.txt: simplify ifdef/ifndef/endif usage fetch, remote: properly convey --no-prune options to subprocesses builtin/remote.c:update(): use struct argv_array builtin/remote.c: reorder function definitions query_refspecs(): move some constants out of the loop fetch --prune: prune only based on explicit refspecs fetch --tags: fetch tags *in addition to* other stuff fetch: only opportunistically update references based on command line get_expanded_map(): avoid memory leak get_expanded_map(): add docstring builtin/fetch.c: reorder function definitions get_ref_map(): rename local variables api-remote.txt: correct section "struct refspec" ...
2013-12-11receive/send-pack: support pushing from a shallow cloneNguyễn Thái Ngọc Duy
Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2013-12-11make the sender advertise shallow commits to the receiverNguyễn Thái Ngọc Duy
If either receive-pack or upload-pack is called on a shallow repository, shallow commits (*) will be sent after the ref advertisement (but before the packet flush), so that the receiver has the full "shape" of the sender's commit graph. This will be needed for the receiver to update its .git/shallow if necessary. This breaks the protocol for all clients trying to push to a shallow repo, or fetch from one. Which is basically the same end result as today's "is_repository_shallow() && die()" in receive-pack and upload-pack. New clients will be made aware of shallow upstream and can make use of this information. The sender must send all shallow commits that are sent in the following pack. It may send more shallow commits than necessary. upload-pack for example may choose to advertise no shallow commits if it knows in advance that the pack it's going to send contains no shallow commits. But upload-pack is the server, so we choose the cheaper way, send full .git/shallow and let the client deal with it. Smart HTTP is not affected by this patch. Shallow support on smart-http comes later separately. (*) A shallow commit is a commit that terminates the revision walker. It is usually put in .git/shallow in order to keep the revision walker from going out of bound because there is no guarantee that objects behind this commit is available. Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2013-12-05Merge branch 'nv/parseopt-opt-arg'Junio C Hamano
Enhance "rev-parse --parseopt" mode to help parsing options with an optional parameter. * nv/parseopt-opt-arg: rev-parse --parseopt: add the --stuck-long mode Use the word 'stuck' instead of 'sticked'
2013-12-03Sync with C Hamano
* maint: Git ref-iteration doc: add_submodule_odb() returns 0 for success
2013-12-03ref-iteration doc: add_submodule_odb() returns 0 for successNick Townsend
The usage sample of add_submodule_odb() function in the Submodules section expects non-zero return value for success, but the function actually reports success with zero. Helped-by: René Scharfe <> Reviewed-by: Heiko Voigt <> Signed-off-by: Nick Townsend <> Signed-off-by: Junio C Hamano <>
2013-11-25send-pack: don't send a thin pack to a server which doesn't support itCarlos Martín Nieto
Up to now git has assumed that all servers are able to fix thin packs. This is however not always the case. Document the 'no-thin' capability and prevent send-pack from generating a thin pack if the server advertises it. Signed-off-by: Carlos Martín Nieto <> Helped-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2013-11-18remove old hash.[ch] implementationKarsten Blees
Signed-off-by: Karsten Blees <> Signed-off-by: Junio C Hamano <>
2013-11-18add a hashtable implementation that supports O(1) removalKarsten Blees
The existing hashtable implementation (in hash.[ch]) uses open addressing (i.e. resolve hash collisions by distributing entries across the table). Thus, removal is difficult to implement with less than O(n) complexity. Resolving collisions of entries with identical hashes (e.g. via chaining) is left to the client code. Add a hashtable implementation that supports O(1) removal and is slightly easier to use due to builtin entry chaining. Supports all basic operations init, free, get, add, remove and iteration. Also includes ready-to-use hash functions based on the public domain FNV-1 algorithm ( The per-entry data structure (hashmap_entry) is piggybacked in front of the client's data structure to save memory. See test-hashmap.c for usage examples. The hashtable is resized by a factor of four when 80% full. With these settings, average memory consumption is about 2/3 of hash.[ch], and insertion is about twice as fast due to less frequent resizing. Lookups are also slightly faster, because entries are strictly confined to their bucket (i.e. no data of other buckets needs to be traversed). Signed-off-by: Karsten Blees <> Signed-off-by: Junio C Hamano <>
2013-11-12Documentation/technical/http-protocol.txt: typofixesMasanari Iida
Signed-off-by: Masanari Iida <> Signed-off-by: Junio C Hamano <>
2013-10-31Use the word 'stuck' instead of 'sticked'Nicolas Vigier
The past participle of 'stick' is 'stuck'. Signed-off-by: Nicolas Vigier <> Signed-off-by: Junio C Hamano <>
2013-10-24api-remote.txt: correct section "struct refspec"Michael Haggerty
* Replace reference to function parse_ref_spec() with references to functions parse_fetch_refspec() and parse_push_refspec(). * Correct description of src and dst: they *do* include the '*' characters. Signed-off-by: Michael Haggerty <> Signed-off-by: Junio C Hamano <>
2013-10-23Merge branch 'mg/more-textconv'Junio C Hamano
Make "git grep" and "git show" pay attention to --textconv when dealing with blob objects. * mg/more-textconv: grep: honor --textconv for the case rev:path grep: allow to use textconv filters t7008: demonstrate behavior of grep with textconv cat-file: do not die on --textconv without textconv filters show: honor --textconv for blobs diff_opt: track whether flags have been set explicitly t4030: demonstrate behavior of show with textconv
2013-09-17Merge branch 'ss/doclinks'Junio C Hamano
When we converted many documents that were traditionally text-only to be formatted to AsciiDoc, we did not update links that point at them to refer to the formatted HTML files. * ss/doclinks: Documentation: make AsciiDoc links always point to HTML files
2013-09-12Merge branch 'mn/doc-pack-heu-remove-dead-pastebin'Junio C Hamano
* mn/doc-pack-heu-remove-dead-pastebin: remove dead pastebin link from pack-heuristics document
2013-09-09Merge branch 'jl/submodule-mv'Junio C Hamano
"git mv A B" when moving a submodule A does "the right thing", inclusing relocating its working tree and adjusting the paths in the .gitmodules file. * jl/submodule-mv: (53 commits) rm: delete .gitmodules entry of submodules removed from the work tree mv: update the path entry in .gitmodules for moved submodules submodule.c: add .gitmodules staging helper functions mv: move submodules using a gitfile mv: move submodules together with their work trees rm: do not set a variable twice without intermediate reading. t6131 - skip tests if on case-insensitive file system parse_pathspec: accept :(icase)path syntax pathspec: support :(glob) syntax pathspec: make --literal-pathspecs disable pathspec magic pathspec: support :(literal) syntax for noglob pathspec kill limit_pathspec_to_literal() as it's only used by parse_pathspec() parse_pathspec: preserve prefix length via PATHSPEC_PREFIX_ORIGIN parse_pathspec: make sure the prefix part is wildcard-free rename field "raw" to "_raw" in struct pathspec tree-diff: remove the use of pathspec's raw[] in follow-rename codepath remove match_pathspec() in favor of match_pathspec_depth() remove init_pathspec() in favor of parse_pathspec() remove diff_tree_{setup,release}_paths convert common_prefix() to use struct pathspec ...
2013-09-06Documentation: make AsciiDoc links always point to HTML filesSebastian Schuberth
AsciiDoc's "link" is supposed to create hyperlinks for HTML output, so prefer a "link" to point to an HTML file instead of a text file if an HTML version of the file is being generated. For RelNotes, keep pointing to text files as no equivalent HTML files are generated. If appropriate, also update the link description to not contain the linked file's extension. Signed-off-by: Sebastian Schuberth <> Signed-off-by: Junio C Hamano <>
2013-08-23remove dead pastebin link from pack-heuristics documentMichal Nazarewicz
Signed-off-by: Michal Nazarewicz <> Acked-by: Jon Loeliger <> Signed-off-by: Junio C Hamano <>
2013-08-21Document the HTTP transport protocolsShawn O. Pearce
Signed-off-by: Shawn O. Pearce <> Revised-by: Tay Ray Chuan <> Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2013-08-01Merge branch 'ob/typofixes'Junio C Hamano
* ob/typofixes: many small typofixes
2013-07-30Merge branch 'jk/capabilities-doc'Junio C Hamano
* jk/capabilities-doc: document 'allow-tip-sha1-in-want' capability document 'quiet' receive-pack capability document 'agent' protocol capability docs: note that receive-pack knows side-band-64k capability docs: fix 'report-status' protocol capability thinko
2013-07-29many small typofixesOndřej Bílka
Signed-off-by: Ondřej Bílka <> Reviewed-by: Marc Branchaud <> Signed-off-by: Junio C Hamano <>
2013-07-26document 'allow-tip-sha1-in-want' capabilityNguyễn Thái Ngọc Duy
See 390eb36 (upload-pack: optionally allow fetching from the tips of hidden refs - 2013-01-28) for more information. Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2013-07-24document 'quiet' receive-pack capabilityJeff King
This was added in c207e34 (fix push --quiet: add 'quiet' capability to receive-pack, 2012-01-08) but never documented. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2013-07-24document 'agent' protocol capabilityJeff King
This was added in ff5effd (include agent identifier in capability string, 2012-08-03), but neither the syntax nor the semantics were ever documented outside of the commit message. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2013-07-24docs: note that receive-pack knows side-band-64k capabilityJeff King
The protocol-capabilities documentation notes that any capabilities not explicitly mentioned for receive-pack work only for upload-pack. Receive-pack has advertised and understood side-band-64k since 38a81b4 (receive-pack: Wrap status reports inside side-band-64k, 2010-02-05), but we do not mention it explicitly. Let's do so. Note that receive-pack does not understand side-band, which was obsolete by that point. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2013-07-24docs: fix 'report-status' protocol capability thinkoJeff King
The report-status capability is understood by receive-pack, not upload-pack. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2013-07-22typofix: documentationOndřej Bílka
Signed-off-by: Ondřej Bílka <> Signed-off-by: Junio C Hamano <>
2013-07-15guard against new pathspec magic in pathspec matching codeNguyễn Thái Ngọc Duy
GUARD_PATHSPEC() marks pathspec-sensitive code, basically all those that touch anything in 'struct pathspec' except fields "nr" and "original". GUARD_PATHSPEC() is not supposed to fail. It's mainly to help the designers catch unsupported codepaths. Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2013-07-15add parse_pathspec() that converts cmdline args to struct pathspecNguyễn Thái Ngọc Duy
Currently to fill a struct pathspec, we do: const char **paths; paths = get_pathspec(prefix, argv); ... init_pathspec(&pathspec, paths); "paths" can only carry bare strings, which loses information from command line arguments such as pathspec magic or the prefix part's length for each argument. parse_pathspec() is introduced to combine the two calls into one. The plan is gradually replace all get_pathspec() and init_pathspec() with parse_pathspec(). get_pathspec() now becomes a thin wrapper of parse_pathspec(). parse_pathspec() allows the caller to reject the pathspec magics that it does not support. When a new pathspec magic is introduced, we can enable it per command after making sure that all underlying code has no problem with the new magic. "flags" parameter is currently unused. But it would allow callers to pass certain instructions to parse_pathspec, for example forcing literal pathspec when no magic is used. With the introduction of parse_pathspec, there are now two functions that can initialize struct pathspec: init_pathspec and parse_pathspec. Any semantic changes in struct pathspec must be reflected in both functions. init_pathspec() will be phased out in favor of parse_pathspec(). Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2013-07-01Merge branch 'wk/doc-in-linux-3.x-era'Junio C Hamano
Update documentation to match more recent realities. * wk/doc-in-linux-3.x-era: Documentation: Update 'linux-2.6.git' -> 'linux.git' Documentation: Update the NFS remote examples to use the staging repo doc/clone: Pick more compelling paths for the --reference example doc/clone: Remove the '--bare -l -s' example
2013-06-26Merge branch 'ph/builtin-srcs-are-in-subdir-these-days'Junio C Hamano
* ph/builtin-srcs-are-in-subdir-these-days: fix "builtin-*" references to be "builtin/*"
2013-06-23Documentation: Update 'linux-2.6.git' -> 'linux.git'W. Trevor King
The 3.x tree has been out for a while now. The -2.6 repository name survived the initial release [1], but now only lists 'linux.git' (for aegl as well as torvalds) [2]. [1]: On 2011-05-30 01:47:57 GMT, Linus Torvalds wrote: > ... yes, that means that my git tree is still called > "linux-2.6.git" on [2]: Signed-off-by: W. Trevor King <> Signed-off-by: Junio C Hamano <>
2013-06-18fix "builtin-*" references to be "builtin/*"Phil Hord
Documentation and some comments still refer to files in builtin/ as 'builtin-*.[cho]'. Update these to show the correct location. Signed-off-by: Phil Hord <> Reviewed-by: Jonathan Nieder <> Assisted-by: Junio C Hamano <> Signed-off-by: Junio C Hamano <>
2013-05-29Merge branch 'jc/prune-all'Junio C Hamano
We used the approxidate() parser for "--expire=<timestamp>" options of various commands, but it is better to treat --expire=all and --expire=now a bit more specially than using the current timestamp. Update "git gc" and "git reflog" with a new parsing function for expiry dates. * jc/prune-all: prune: introduce OPT_EXPIRY_DATE() and use it api-parse-options.txt: document "no-" for non-boolean options git-gc.txt, git-reflog.txt: document new expiry options date.c: add parse_expiry_date()
2013-05-10diff_opt: track whether flags have been set explicitlyJunio C Hamano
The diff_opt infrastructure sets flags based on defaults and command line options. It is impossible to tell whether a flag has been set as a default or on explicit request. Update the structure so that this detection is possible: * Add an extra "opt->touched_flags" that keeps track of all the fields that have been touched by DIFF_OPT_SET and DIFF_OPT_CLR. * You may continue setting the default values to the flags, like commands in the "log" family do in cmd_log_init_defaults(), but after you finished setting the defaults, you clear the touched_flags field; * And then you let the usual callchain call diff_opt_parse(), allowing the opt->flags be set or unset, while keeping track of which bits the user touched; * There is an optional callback "opt->set_default" that is called at the very beginning to let you inspect touched_flags and update opt->flags appropriately, before the remainder of the diffcore machinery is set up, taking the opt->flags value into account. Signed-off-by: Michael J Gruber <> Signed-off-by: Junio C Hamano <>
2013-04-29upload-pack: ignore 'shallow' lines with unknown obj-idsMichael Heemskerk
When the client sends a 'shallow' line for an object that the server does not have, the server currently dies with the error: "did not find object for shallow <obj-id>". The client may have truncated the history at the commit by fetching shallowly from a different server, or the commit may have been garbage collected by the server. In either case, this unknown commit is not relevant for calculating the pack that is to be sent and can be safely ignored, and it is not used when recomputing where the updated history of the client is cauterised. The documentation in technical/pack-protocol.txt has been updated to remove the restriction that "Clients MUST NOT mention an obj-id which it does not know exists on the server". This requirement is not realistic because clients cannot know whether an object has been garbage collected by the server. Signed-off-by: Michael Heemskerk <> Reviewed-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2013-04-25prune: introduce OPT_EXPIRY_DATE() and use itJunio C Hamano
Earlier we added support for --expire=all (or --expire=now) that considers all crufts, regardless of their age, as eligible for garbage collection by turning command argument parsers that use approxidate() to use parse_expiry_date(), but "git prune" used a built-in parse-options facility OPT_DATE() and did not benefit from the new function. Signed-off-by: Junio C Hamano <>