path: root/Documentation/technical
AgeCommit message (Collapse)Author
2018-06-28commit-graph: fix documentation inconsistenciesDerrick Stolee
The commit-graph feature shipped in Git 2.18 has some inconsistencies in the constants used by the implementation and specified by the format document. The commit data chunk uses the key "CDAT" in the file format, but was previously documented to say "CGET". The commit data chunk stores commit parents using two 32-bit fields that typically store the integer position of the parent in the list of commit ids within the commit-graph file. When a parent does not exist, we had documented the value 0xffffffff, but implemented the value 0x70000000. This swap is easy to correct in the documentation, but unfortunately reduces the number of commits that we can store in the commit-graph. Update that estimate, too. Reported-by: Grant Welch <> Signed-off-by: Derrick Stolee <> Signed-off-by: Junio C Hamano <>
2018-04-02graph: add commit graph design documentDerrick Stolee
Add Documentation/technical/commit-graph.txt with details of the planned commit graph feature, including future plans. Signed-off-by: Derrick Stolee <> Signed-off-by: Junio C Hamano <>
2018-04-02commit-graph: add format documentDerrick Stolee
Add document specifying the binary format for commit graphs. This format allows for: * New versions. * New hash functions and hash lengths. * Optional extensions. Basic header information is followed by a binary table of contents into "chunks" that include: * An ordered list of commit object IDs. * A 256-entry fanout into that list of OIDs. * A list of metadata for the commits. * A list of "large edges" to enable octopus merges. The format automatically includes two parent positions for every commit. This favors speed over space, since using only one position per commit would cause an extra level of indirection for every merge commit. (Octopus merges suffer from this indirection, but they are very rare.) Signed-off-by: Derrick Stolee <> Signed-off-by: Junio C Hamano <>
2018-02-15Merge branch 'po/object-id'Junio C Hamano
Conversion from uchar[20] to struct object_id continues. * po/object-id: sha1_file: rename hash_sha1_file_literally sha1_file: convert write_loose_object to object_id sha1_file: convert force_object_loose to object_id sha1_file: convert write_sha1_file to object_id notes: convert write_notes_tree to object_id notes: convert combine_notes_* to object_id commit: convert commit_tree* to object_id match-trees: convert splice_tree to object_id cache: clear whole hash buffer with oidclr sha1_file: convert hash_sha1_file to object_id dir: convert struct sha1_stat to use object_id sha1_file: convert pretend_sha1_file to object_id
2018-02-13Merge branch 'jt/long-running-process-doc'Junio C Hamano
Doc updates. * jt/long-running-process-doc: Docs: split out long-running subprocess handshake
2018-02-13Merge branch 'jh/partial-clone'Junio C Hamano
The machinery to clone & fetch, which in turn involves packing and unpacking objects, have been told how to omit certain objects using the filtering mechanism introduced by the jh/object-filtering topic, and also mark the resulting pack as a promisor pack to tolerate missing objects, taking advantage of the mechanism introduced by the jh/fsck-promisors topic. * jh/partial-clone: t5616: test bulk prefetch after partial fetch fetch: inherit filter-spec from partial clone t5616: end-to-end tests for partial clone fetch-pack: restore save_commit_buffer after use unpack-trees: batch fetching of missing blobs clone: partial clone partial-clone: define partial clone settings in config fetch: support filters fetch: refactor calculation of remote list fetch-pack: test support excluding large blobs fetch-pack: add --no-filter fetch-pack, index-pack, transport: partial clone upload-pack: add object filtering for partial clone
2018-02-13Merge branch 'jh/fsck-promisors'Junio C Hamano
In preparation for implementing narrow/partial clone, the machinery for checking object connectivity used by gc and fsck has been taught that a missing object is OK when it is referenced by a packfile specially marked as coming from trusted repository that promises to make them available on-demand and lazily. * jh/fsck-promisors: gc: do not repack promisor packfiles rev-list: support termination at promisor objects sha1_file: support lazily fetching missing objects introduce fetch-object: fetch one promisor object index-pack: refactor writing of .keep files fsck: support promisor objects as CLI argument fsck: support referenced promisor objects fsck: support refs pointing to promisor objects fsck: introduce partialclone extension extension.partialclone: introduce partial clone extension
2018-01-30sha1_file: convert pretend_sha1_file to object_idPatryk Obara
Convert the declaration and definition of pretend_sha1_file to use struct object_id and adjust all usages of this function. Rename it to pretend_object_file. Signed-off-by: Patryk Obara <> Signed-off-by: Junio C Hamano <>
2018-01-25Docs: split out long-running subprocess handshakeJonathan Tan
Separating out the implementation of the handshake when starting a long-running subprocess (for example, as is done for a clean/smudge filter) was done in commit fa64a2fdbeed ("sub-process: refactor handshake to common function", 2017-07-26), but its documentation still resides in gitattributes. Split out the documentation as well. Signed-off-by: Jonathan Tan <> Signed-off-by: Junio C Hamano <>
2017-12-28Merge branch 'jh/partial-clone-doc'Junio C Hamano
* jh/partial-clone-doc: partial-clone: design doc
2017-12-14partial-clone: design docJeff Hostetler
Design document for partial clone feature. Signed-off-by: Jeff Hostetler <> Signed-off-by: Junio C Hamano <>
2017-12-08upload-pack: add object filtering for partial cloneJeff Hostetler
Teach upload-pack to negotiate object filtering over the protocol and to send filter parameters to pack-objects. This is intended for partial clone and fetch. The idea to make upload-pack configurable using uploadpack.allowFilter comes from Jonathan Tan's work in [1]. [1] Signed-off-by: Jeff Hostetler <> Signed-off-by: Junio C Hamano <>
2017-12-08decorate: clean up and document APIJonathan Tan
Improve the names of the identifiers in decorate.h, document them, and add an example of how to use these functions. The example is compiled and run as part of the test suite. Signed-off-by: Jonathan Tan <> Signed-off-by: Junio C Hamano <>
2017-12-06Merge branch 'bw/protocol-v1'Junio C Hamano
A new mechanism to upgrade the wire protocol in place is proposed and demonstrated that it works with the older versions of Git without harming them. * bw/protocol-v1: Documentation: document Extra Parameters ssh: introduce a 'simple' ssh variant i5700: add interop test for protocol transition http: tell server that the client understands v1 connect: tell server that the client understands v1 connect: teach client to recognize v1 server response upload-pack, receive-pack: introduce protocol version 1 daemon: recognize hidden request arguments protocol: introduce protocol extension mechanisms pkt-line: add packet_write function connect: in ref advertisement, shallows are last
2017-12-05extension.partialclone: introduce partial clone extensionJonathan Tan
Introduce new repository extension option: `extensions.partialclone` See the update to Documentation/technical/repository-version.txt in this patch for more information. Signed-off-by: Jonathan Tan <> Signed-off-by: Junio C Hamano <>
2017-11-21Merge branch 'bp/fsmonitor'Junio C Hamano
We learned to talk to watchman to speed up "git status" and other operations that need to see which paths have been modified. * bp/fsmonitor: fsmonitor: preserve utf8 filenames in fsmonitor-watchman log fsmonitor: read entirety of watchman output fsmonitor: MINGW support for watchman integration fsmonitor: add a performance test fsmonitor: add a sample integration script for Watchman fsmonitor: add test cases for fsmonitor extension split-index: disable the fsmonitor extension when running the split index test fsmonitor: add a test tool to dump the index extension update-index: add fsmonitor support to update-index ls-files: Add support in ls-files to display the fsmonitor valid bit fsmonitor: add documentation for the fsmonitor extension. fsmonitor: teach git to optionally utilize a file system monitor to speed up detecting new or changed files. update-index: add a new --force-write-index option preload-index: add override to enable testing preload-index bswap: add 64 bit endianness helper get_be64
2017-10-31status: document options to show matching ignored filesJameson Miller
Signed-off-by: Jameson Miller <> Signed-off-by: Junio C Hamano <>
2017-10-17Documentation: document Extra ParametersJonathan Tan
Document the server support for Extra Parameters, additional information that the client can send in its first message to the server during a Git client-server interaction. Signed-off-by: Jonathan Tan <> Signed-off-by: Brandon Williams <> Signed-off-by: Junio C Hamano <>
2017-10-11Merge branch 'hn/string-list-doc'Junio C Hamano
Docfix. * hn/string-list-doc: api-argv-array.txt: remove broken link to string-list API
2017-10-11Merge branch 'jr/hash-migration-plan-doc'Junio C Hamano
Lay out plans for weaning us off of SHA-1. * jr/hash-migration-plan-doc: technical doc: add a design doc for hash function transition
2017-10-06api-argv-array.txt: remove broken link to string-list APITodd Zullinger
In 4f665f2cf3 (string-list.h: move documentation from Documentation/api/ into header, 2017-09-26) the string-list API documentation was moved to string-list.h. The argv-array API documentation may follow a similar course in the future. Until then, prevent the broken link from making it to the end-user documentation. Signed-off-by: Todd Zullinger <> Reviewed-by: Jonathan Nieder <> Signed-off-by: Junio C Hamano <>
2017-10-01fsmonitor: add documentation for the fsmonitor extension.Ben Peart
This includes the core.fsmonitor setting, the fsmonitor integration hook, and the fsmonitor index extension. Also add documentation for the new fsmonitor options to ls-files and update-index. Signed-off-by: Ben Peart <> Signed-off-by: Junio C Hamano <>
2017-09-28technical doc: add a design doc for hash function transitionJonathan Nieder
This document describes what a transition to a new hash function for Git would look like. Add it to Documentation/technical/ as the plan of record so that future changes can be recorded as patches. Also-by: Brandon Williams <> Also-by: Jonathan Tan <> Also-by: Stefan Beller <> Signed-off-by: Jonathan Nieder <> Signed-off-by: Junio C Hamano <>
2017-09-27string-list.h: move documentation from Documentation/api/ into headerHan-Wen Nienhuys
This mirrors commit 'bdfdaa497 ("strbuf.h: integrate api-strbuf.txt documentation, 2015-01-16") which did the same for strbuf.h: * API documentation uses /** */ to set it apart from other comments. * Function names were stripped from the comments. * Ordering of the header was adjusted to follow the one from the text file. * Edited some existing comments from string-list.h for consistency. Signed-off-by: Han-Wen Nienhuys <> Signed-off-by: Junio C Hamano <>
2017-09-19Merge branch 'ma/remove-config-maybe-bool'Junio C Hamano
Finishing touches to a recent topic. * ma/remove-config-maybe-bool: config: remove git_config_maybe_bool
2017-09-19Merge branch 'nd/prune-in-worktree'Junio C Hamano
"git gc" and friends when multiple worktrees are used off of a single repository did not consider the index and per-worktree refs of other worktrees as the root for reachability traversal, making objects that are in use only in other worktrees to be subject to garbage collection. * nd/prune-in-worktree: refs.c: reindent get_submodule_ref_store() refs.c: remove fallback-to-main-store code get_submodule_ref_store() rev-list: expose and document --single-worktree revision.c: --reflog add HEAD reflog from all worktrees files-backend: make reflog iterator go through per-worktree reflog revision.c: --all adds HEAD from all worktrees refs: remove dead for_each_*_submodule() refs.c: move for_each_remote_ref_submodule() to submodule.c revision.c: use refs_for_each*() instead of for_each_*_submodule() refs: add refs_head_ref() refs: move submodule slash stripping code to get_submodule_ref_store refs.c: refactor get_submodule_ref_store(), share common free block revision.c: --indexed-objects add objects from all worktrees revision.c: refactor add_index_objects_to_pending() refs.c: use is_dir_sep() in resolve_gitlink_ref() revision.h: new flag in struct rev_info wrt. worktree-related refs
2017-09-10Merge branch 'ma/up-to-date'Junio C Hamano
Message and doc updates. * ma/up-to-date: treewide: correct several "up-to-date" to "up to date" Documentation/user-manual: update outdated example output
2017-09-07config: remove git_config_maybe_boolMartin Ågren
The function was deprecated in commit 89576613 ("treewide: deprecate git_config_maybe_bool, use git_parse_maybe_bool", 2017-08-07) and has no users. Signed-off-by: Martin Ågren <> Signed-off-by: Junio C Hamano <>
2017-08-24refs: remove dead for_each_*_submodule()Nguyễn Thái Ngọc Duy
These are used in revision.c. After the last patch they are replaced with the refs_ version. Delete them. Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2017-08-24Merge branch 'rs/object-id'Junio C Hamano
Conversion from uchar[20] to struct object_id continues. * rs/object-id: tree-walk: convert fill_tree_descriptor() to object_id
2017-08-23treewide: correct several "up-to-date" to "up to date"Martin Ågren
Follow the Oxford style, which says to use "up-to-date" before the noun, but "up to date" after it. Don't change plumbing (specifically send-pack.c, but transport.c (git push) also has the same string). This was produced by grepping for "up-to-date" and "up to date". It turned out we only had to edit in one direction, removing the hyphens. Fix a typo in Documentation/git-diff-index.txt while we're there. Reported-by: Jeffrey Manian <> Reported-by: STEVEN WHITE <> Signed-off-by: Martin Ågren <> Signed-off-by: Junio C Hamano <>
2017-08-22Merge branch 'ma/parse-maybe-bool'Junio C Hamano
Code clean-up. * ma/parse-maybe-bool: parse_decoration_style: drop unused argument `var` treewide: deprecate git_config_maybe_bool, use git_parse_maybe_bool config: make git_{config,parse}_maybe_bool equivalent config: introduce git_parse_maybe_bool_text t5334: document that git push --signed=1 does not work Doc/git-{push,send-pack}: correct --sign= to --signed=
2017-08-14tree-walk: convert fill_tree_descriptor() to object_idRené Scharfe
All callers of fill_tree_descriptor() have been converted to object_id already, so convert that function as well. As a nice side-effect we get rid of NULL checks in tree-diff.c, as fill_tree_descriptor() already does them for us. Helped-by: Johannes Sixt <> Signed-off-by: Rene Scharfe <> Reviewed-by: brian m. carlson <> Signed-off-by: Junio C Hamano <>
2017-08-11Merge branch 'ma/pager-per-subcommand-action'Junio C Hamano
The "tag.pager" configuration variable was useless for those who actually create tag objects, as it interfered with the use of an editor. A new mechanism has been introduced for commands to enable pager depending on what operation is being carried out to fix this, and then "git tag -l" is made to run pager by default. * ma/pager-per-subcommand-action: git.c: ignore pager.* when launching builtin as dashed external tag: change default of `pager.tag` to "on" tag: respect `pager.tag` in list-mode only t7006: add tests for how git tag paginates git.c: provide setup_auto_pager() git.c: let builtins opt for handling `` themselves builtin.h: take over documentation from api-builtin.txt
2017-08-07treewide: deprecate git_config_maybe_bool, use git_parse_maybe_boolMartin Ågren
The only difference between these is that the former takes an argument `name` which it ignores completely. Still, the callers are quite careful to provide reasonable values for it. Once in-flight topics have landed, we should be able to remove git_config_maybe_bool. In the meantime, document it as deprecated in the technical documentation. While at it, document git_parse_maybe_bool. Signed-off-by: Martin Ågren <> Signed-off-by: Junio C Hamano <>
2017-08-03builtin.h: take over documentation from api-builtin.txtMartin Ågren
Delete Documentation/technical/api-builtin.txt and move its content into builtin.h. Format it as a comment. Remove a '+' which was needed when the information was formatted for AsciiDoc. Similarly, change "::" to ":". Document SUPPORT_SUPER_PREFIX, thereby bringing the documentation up to date with the available flags. While at it, correct '3 more things to do' to '4 more things to do'. Signed-off-by: Martin Ågren <> Signed-off-by: Junio C Hamano <>
2017-07-26Documentation: migrate sub-process docs to headerJonathan Tan
Move the documentation for the sub-process API from a separate txt file to its header file. Signed-off-by: Jonathan Tan <> Signed-off-by: Junio C Hamano <>
2017-06-30hashmap: migrate documentation from Documentation/technical into headerStefan Beller
While at it, clarify the use of `key`, `keydata`, `entry_or_key` as well as documenting the new data pointer for the compare function. Rework the example. Signed-off-by: Stefan Beller <> Signed-off-by: Junio C Hamano <>
2017-06-02Merge branch 'sl/clean-d-ignored-fix'Junio C Hamano
"git clean -d" used to clean directories that has ignored files, even though the command should not lose ignored ones without "-x". "git status --ignored" did not list ignored and untracked files without "-uall". These have been corrected. * sl/clean-d-ignored-fix: clean: teach clean -d to preserve ignored paths dir: expose cmp_name() and check_contains() dir: hide untracked contents of untracked dirs dir: recurse into untracked dirs for ignored files t7061: status --ignored should search untracked dirs t7300: clean -d should skip dirs with ignored files
2017-05-30Merge branch 'bp/sub-process-convert-filter'Junio C Hamano
Code from "conversion using external process" codepath has been extracted to a separate sub-process.[ch] module. * bp/sub-process-convert-filter: convert: update subprocess_read_status() to not die on EOF sub-process: move sub-process functions into separate files convert: rename reusable sub-process functions convert: update generic functions to only use generic data structures convert: separate generic structures and variables from the filter specific ones convert: split start_multi_file_filter() into two separate functions pkt-line: annotate packet_writel with LAST_ARG_MUST_BE_NULL convert: move packet_write_line() into pkt-line as packet_writel() pkt-line: add packet_read_line_gently() pkt-line: fix packet_read_line() to handle len < 0 errors convert: remove erroneous tests for errno == EPIPE
2017-05-23Merge branch 'jt/push-options-doc'Junio C Hamano
The receive-pack program now makes sure that the push certificate records the same set of push options used for pushing. * jt/push-options-doc: receive-pack: verify push options in cert docs: correct receive.advertisePushOptions default
2017-05-22dir: hide untracked contents of untracked dirsSamuel Lijin
When we taught read_directory_recursive() to recurse into untracked directories in search of ignored files given DIR_SHOW_IGNORED_TOO, that had the side effect of teaching it to collect the untracked contents of untracked directories. It doesn't always make sense to return these, though (we do need them for `clean -d`), so we introduce a flag (DIR_KEEP_UNTRACKED_CONTENTS) to control whether or not read_directory() strips dir->entries of the untracked contents of untracked dirs. We also introduce check_contains() to check if one dir_entry corresponds to a path which contains the path corresponding to another dir_entry. This also fixes known breakages in t7061, since status --ignored now searches untracked directories for ignored files. Signed-off-by: Samuel Lijin <> Signed-off-by: Junio C Hamano <>
2017-05-16Merge branch 'js/larger-timestamps'Junio C Hamano
Some platforms have ulong that is smaller than time_t, and our historical use of ulong for timestamp would mean they cannot represent some timestamp that the platform allows. Invent a separate and dedicated timestamp_t (so that we can distingiuish timestamps and a vanilla ulongs, which along is already a good move), and then declare uintmax_t is the type to be used as the timestamp_t. * js/larger-timestamps: archive-tar: fix a sparse 'constant too large' warning use uintmax_t for timestamps date.c: abort if the system time cannot handle one of our timestamps timestamp_t: a new data type for timestamps PRItime: introduce a new "printf format" for timestamps parse_timestamp(): specify explicitly where we parse timestamps t0006 & t5000: skip "far in the future" test when time_t is too limited t0006 & t5000: prepare for 64-bit timestamps ref-filter: avoid using `unsigned long` for catch-all data type
2017-05-15sub-process: move sub-process functions into separate filesBen Peart
Move the sub-proces functions into sub-process.h/c. Add documentation for the new module in Documentation/technical/api-sub-process.txt Signed-off-by: Ben Peart <> Signed-off-by: Junio C Hamano <>
2017-05-10receive-pack: verify push options in certJonathan Tan
In commit f6a4e61 ("push: accept push options", 2016-07-14), send-pack was taught to include push options both within the signed cert (if the push is a signed push) and outside the signed cert; however, receive-pack ignores push options within the cert, only handling push options outside the cert. Teach receive-pack, in the case that push options are provided for a signed push, to verify that the push options both within the cert and outside the cert are consistent. This sets in stone the requirement that send-pack redundantly send its push options in 2 places, but I think that this is better than the alternatives. Sending push options only within the cert is backwards-incompatible with existing Git servers (which read push options only from outside the cert), and sending push options only outside the cert means that the push options are not signed for. Signed-off-by: Jonathan Tan <> Signed-off-by: Junio C Hamano <>
2017-04-27timestamp_t: a new data type for timestampsJohannes Schindelin
Git's source code assumes that unsigned long is at least as precise as time_t. Which is incorrect, and causes a lot of problems, in particular where unsigned long is only 32-bit (notably on Windows, even in 64-bit versions). So let's just use a more appropriate data type instead. In preparation for this, we introduce the new `timestamp_t` data type. By necessity, this is a very, very large patch, as it has to replace all timestamps' data type in one go. As we will use a data type that is not necessarily identical to `time_t`, we need to be very careful to use `time_t` whenever we interact with the system functions, and `timestamp_t` everywhere else. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2017-04-24Merge branch 'jt/fetch-pack-error-reporting'Junio C Hamano
"git fetch-pack" was not prepared to accept ERR packet that the upload-pack can send with a human-readable error message. It showed the packet contents with ERR prefix, so there was no data loss, but it was redundant to say "ERR" in an error message. * jt/fetch-pack-error-reporting: fetch-pack: show clearer error message upon ERR
2017-04-20Merge branch 'bc/object-id'Junio C Hamano
Conversion from unsigned char [40] to struct object_id continues. * bc/object-id: Documentation: update and rename api-sha1-array.txt Rename sha1_array to oid_array Convert sha1_array_for_each_unique and for_each_abbrev to object_id Convert sha1_array_lookup to take struct object_id Convert remaining callers of sha1_array_lookup to object_id Make sha1_array_append take a struct object_id * sha1-array: convert internal storage for struct sha1_array to object_id builtin/pull: convert to struct object_id submodule: convert check_for_new_submodule_commits to object_id sha1_name: convert disambiguate_hint_fn to take object_id sha1_name: convert struct disambiguate_state to object_id test-sha1-array: convert most code to struct object_id parse-options-cb: convert sha1_array_append caller to struct object_id fsck: convert init_skiplist to struct object_id builtin/receive-pack: convert portions to struct object_id builtin/pull: convert portions to struct object_id builtin/diff: convert to struct object_id Convert GIT_SHA1_RAWSZ used for allocation to GIT_MAX_RAWSZ Convert GIT_SHA1_HEXSZ used for allocation to GIT_MAX_HEXSZ Define new hash-size constants for allocating memory
2017-04-18fetch-pack: show clearer error message upon ERRJonathan Tan
Currently, fetch-pack prints a confusing error message ("expected ACK/NAK") when the server it's communicating with sends a pkt-line starting with "ERR". Replace it with a less confusing error message. Also update the documentation describing the fetch-pack/upload-pack protocol (pack-protocol.txt) to indicate that "ERR" can be sent in the place of "ACK" or "NAK". In practice, this has been done for quite some time by other Git implementations (e.g. JGit sends "want $id not valid") and by Git itself (since commit bdb31ea: "upload-pack: report "not our ref" to client", 2017-02-23) whenever a "want" line references an object that it does not have. (This is uncommon, but can happen if a repository is garbage-collected during a negotiation.) Signed-off-by: Jonathan Tan <> Reviewed-by: Jonathan Nieder <> Signed-off-by: Junio C Hamano <>
2017-03-31Documentation: update and rename api-sha1-array.txtbrian m. carlson
Since the structure and functions have changed names, update the code examples and the documentation. Rename the file to match the new name of the API. Signed-off-by: brian m. carlson <> Signed-off-by: Junio C Hamano <>