From 8c7d05171e8c7588b3f87b66b1428ac298d72ba1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nguy=E1=BB=85n=20Th=C3=A1i=20Ng=E1=BB=8Dc=20Duy?= Date: Mon, 6 Sep 2010 20:37:10 +1000 Subject: doc: technical details about the index file format MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This bases on the original work by Robin Rosenberg. Signed-off-by: Robin Rosenberg Signed-off-by: Nguyễn Thái Ngọc Duy Signed-off-by: Junio C Hamano diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt new file mode 100644 index 0000000..89e410a --- /dev/null +++ b/Documentation/technical/index-format.txt @@ -0,0 +1,165 @@ +GIT index format +================ + += The git index file has the following format + + All binary numbers are in network byte order. Version 2 is described + here unless stated otherwise. + + - A 12-byte header consisting of + + 4-byte signature: + The signature is { 'D', 'I', 'R', 'C' } + + 4-byte version number: + The current supported versions are 2 and 3. + + 32-bit number of index entries. + + - A number of sorted index entries + + - Extensions + + Extensions are identified by signature. Optional extensions can + be ignored if GIT does not understand them. + + GIT currently supports tree cache and resolve undo extensions. + + 4-byte extension signature. If the first byte is 'A'..'Z' the + extension is optional and can be ignored. + + 32-bit size of the extension + + Extension data + + - 160-bit SHA-1 over the content of the index file before this + checksum. + +== Index entry + + Index entries are sorted in ascending order on the name field, + interpreted as a string of unsigned bytes. Entries with the same + name are sorted by their stage field. + + 32-bit ctime seconds, the last time a file's metadata changed + this is stat(2) data + + 32-bit ctime nanosecond fractions + this is stat(2) data + + 32-bit mtime seconds, the last time a file's data changed + this is stat(2) data + + 32-bit mtime nanosecond fractions + this is stat(2) data + + 32-bit dev + this is stat(2) data + + 32-bit ino + this is stat(2) data + + 32-bit mode, split into (high to low bits) + + 4-bit object type + valid values in binary are 1000 (blob), 1010 (symbolic link) + and 1110 (gitlink) + + 3-bit unused + + 9-bit unix permission (only 0755 and 0644 are valid) + + 32-bit uid + this is stat(2) data + + 32-bit gid + this is stat(2) data + + 32-bit file size + This is the on-disk size from stat(2) + + 160-bit SHA-1 for the represented object + + A 16-bit field split into (high to low bits) + + 1-bit assume-valid flag + + 1-bit extended flag (must be zero in version 2) + + 2-bit stage (during merge) + + 12-bit name length if the length is less than 0x0FFF + + (Version 3) A 16-bit field, only applicable if the "extended flag" + above is 1, split into (high to low bits). + + 1-bit reserved for future + + 1-bit skip-worktree flag (used by sparse checkout) + + 1-bit intent-to-add flag (used by "git add -N") + + 13-bit unused, must be zero + + Entry path name (variable length) relative to top level directory + (without leading slash). '/' is used as path separator. The special + paths ".", ".." and ".git" (without quotes) are disallowed. + Trailing slash is also disallowed. + + The exact encoding is undefined, but the '.' and '/' characters + are encoded in 7-bit ASCII and the encoding cannot contain a nul + byte. Generally a superset of ASCII. + + 1-8 nul bytes as necessary to pad the entry to a multiple of eight bytes + while keeping the name NUL-terminated. + +== Extensions + +=== Tree cache + + Tree cache extension contains pre-computed hashes for trees that can + be derived from the index. It helps speed up tree object generation + from index for a new commit. + + When a path is updated in index, the path must be invalidated and + removed from tree cache. + + - Extension tag { 'T', 'R', 'E', 'E' } + + - 32-bit size + + - A number of entries + + NUL-terminated tree name + + Blank-terminated ASCII decimal number of entries in this tree + + Newline-terminated position of this tree in the parent tree. 0 for + the root tree + + 160-bit SHA-1 for this tree and it's children + +=== Resolve undo + + A conflict is represented in index as a set of higher stage entries. + When a conflict is resolved (e.g. with "git add path"), these higher + stage entries will be removed and a stage-0 entry with proper + resoluton is added. + + Resolve undo extension saves these higher stage entries so that + conflicts can be recreated (e.g. with "git checkout -m"), in case + users want to redo a conflict resolution from scratch. + + - Extension tag { 'R', 'E', 'U', 'C' } + + - 32-bit size + + - A number of conflict entries + + NUL-terminated conflict path + + Three NUL-terminated ASCII octal numbers, entry mode of entries in + stage 1 to 3. + + At most three 160-bit SHA-1s of the entry in three stages from 1 + to 3. SHA-1 is not saved for any stage with entry mode zero. -- cgit v0.10.2-6-g49f6 From 23fcc98f7f4880829ab47210278ff9ff3c7d5f59 Mon Sep 17 00:00:00 2001 From: Junio C Hamano Date: Tue, 1 Mar 2011 17:51:01 -0800 Subject: doc: technical details about the index file format * Clarify "string of unsigned bytes"; * Blob has two variants (regular file vs symlink), not (blob vs symlink); * Clarify permission mode bits; * Clarify ce_namelen() "too long to fit in the length field" case; * Clarify "." etc are forbidden as path components; * Match the description with the internal wording "cache-tree"; * All types of extension begin with signature and length as explained in the first part. Don't repeat the "length" part in the description of each extension (can be mistaken as if there is a separate 32-bit size field inside the extension), but state what the signature for each extension is. * Don't say "Extension tag", as we have said "Extension signature" in the first part---be consistent; * Clarify the invalidation of cache-tree entries; * Correct description on subtree_nr field in the cache-tree; * Clarify the order of entries in cache-tree; Signed-off-by: Junio C Hamano diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt index 89e410a..7b233ca 100644 --- a/Documentation/technical/index-format.txt +++ b/Documentation/technical/index-format.txt @@ -9,21 +9,21 @@ GIT index format - A 12-byte header consisting of 4-byte signature: - The signature is { 'D', 'I', 'R', 'C' } + The signature is { 'D', 'I', 'R', 'C' } (stands for "dircache") 4-byte version number: The current supported versions are 2 and 3. 32-bit number of index entries. - - A number of sorted index entries + - A number of sorted index entries (see below). - Extensions Extensions are identified by signature. Optional extensions can be ignored if GIT does not understand them. - GIT currently supports tree cache and resolve undo extensions. + GIT currently supports cached tree and resolve undo extensions. 4-byte extension signature. If the first byte is 'A'..'Z' the extension is optional and can be ignored. @@ -38,8 +38,9 @@ GIT index format == Index entry Index entries are sorted in ascending order on the name field, - interpreted as a string of unsigned bytes. Entries with the same - name are sorted by their stage field. + interpreted as a string of unsigned bytes (i.e. memcmp() order, no + localization, no special casing of directory separator '/'). Entries + with the same name are sorted by their stage field. 32-bit ctime seconds, the last time a file's metadata changed this is stat(2) data @@ -62,12 +63,13 @@ GIT index format 32-bit mode, split into (high to low bits) 4-bit object type - valid values in binary are 1000 (blob), 1010 (symbolic link) + valid values in binary are 1000 (regular file), 1010 (symbolic link) and 1110 (gitlink) 3-bit unused - 9-bit unix permission (only 0755 and 0644 are valid) + 9-bit unix permission. Only 0755 and 0644 are valid for regular files. + Symbolic links and gitlinks have value 0 in this field. 32-bit uid this is stat(2) data @@ -76,11 +78,11 @@ GIT index format this is stat(2) data 32-bit file size - This is the on-disk size from stat(2) + This is the on-disk size from stat(2), truncated to 32-bit. 160-bit SHA-1 for the represented object - A 16-bit field split into (high to low bits) + A 16-bit 'flags' field split into (high to low bits) 1-bit assume-valid flag @@ -88,7 +90,8 @@ GIT index format 2-bit stage (during merge) - 12-bit name length if the length is less than 0x0FFF + 12-bit name length if the length is less than 0xFFF; otherwise 0xFFF + is stored in this field. (Version 3) A 16-bit field, only applicable if the "extended flag" above is 1, split into (high to low bits). @@ -103,63 +106,80 @@ GIT index format Entry path name (variable length) relative to top level directory (without leading slash). '/' is used as path separator. The special - paths ".", ".." and ".git" (without quotes) are disallowed. + path components ".", ".." and ".git" (without quotes) are disallowed. Trailing slash is also disallowed. The exact encoding is undefined, but the '.' and '/' characters - are encoded in 7-bit ASCII and the encoding cannot contain a nul - byte. Generally a superset of ASCII. + are encoded in 7-bit ASCII and the encoding cannot contain a NUL + byte (iow, this is a UNIX pathname). 1-8 nul bytes as necessary to pad the entry to a multiple of eight bytes while keeping the name NUL-terminated. == Extensions -=== Tree cache +=== Cached tree - Tree cache extension contains pre-computed hashes for trees that can + Cached tree extension contains pre-computed hashes for trees that can be derived from the index. It helps speed up tree object generation from index for a new commit. When a path is updated in index, the path must be invalidated and removed from tree cache. - - Extension tag { 'T', 'R', 'E', 'E' } + The signature for this extension is { 'T', 'R', 'E', 'E' }. - - 32-bit size + A series of entries fill the entire extension; each of which + consists of: - - A number of entries + - NUL-terminated path component (relative to its parent directory); - NUL-terminated tree name + - ASCII decimal number of entries in the index that is covered by the + tree this entry represents (entry_count); - Blank-terminated ASCII decimal number of entries in this tree + - A space (ASCII 32); - Newline-terminated position of this tree in the parent tree. 0 for - the root tree + - ASCII decimal number that represents the number of subtrees this + tree has; - 160-bit SHA-1 for this tree and it's children + - A newline (ASCII 10); and + + - 160-bit object name for the object that would result from writing + this span of index as a tree. + + An entry can be in an invalidated state and is represented by having -1 + in the entry_count field. + + The entries are written out in the top-down, depth-first order. The + first entry represents the root level of the repository, followed by the + first subtree---let's call this A---of the root level (with its name + relative to the root level), followed by the first subtree of A (with + its name relative to A), ... === Resolve undo - A conflict is represented in index as a set of higher stage entries. + A conflict is represented in the index as a set of higher stage entries. When a conflict is resolved (e.g. with "git add path"), these higher - stage entries will be removed and a stage-0 entry with proper - resoluton is added. + stage entries will be removed and a stage-0 entry with proper resoluton + is added. - Resolve undo extension saves these higher stage entries so that - conflicts can be recreated (e.g. with "git checkout -m"), in case - users want to redo a conflict resolution from scratch. + When these higher stage entries are removed, they are saved in the + resolve undo extension, so that conflicts can be recreated (e.g. with + "git checkout -m"), in case users want to redo a conflict resolution + from scratch. - - Extension tag { 'R', 'E', 'U', 'C' } + The signature for this extension is { 'R', 'E', 'U', 'C' }. - - 32-bit size + A series of entries fill the entire extension; each of which + consists of: - - A number of conflict entries + - NUL-terminated pathname the entry describes (relative to the root of + the repository, i.e. full pathname); - NUL-terminated conflict path + - Three NUL-terminated ASCII octal numbers, entry mode of entries in + stage 1 to 3 (a missing stage is represented by "0" in this field); + and - Three NUL-terminated ASCII octal numbers, entry mode of entries in - stage 1 to 3. + - At most three 160-bit object names of the entry in stages from 1 to 3 + (nothing is written for a missing stage). - At most three 160-bit SHA-1s of the entry in three stages from 1 - to 3. SHA-1 is not saved for any stage with entry mode zero. -- cgit v0.10.2-6-g49f6