path: root/Documentation/technical/index-format.txt
diff options
authorBen Peart <>2018-10-10 15:59:37 (GMT)
committerJunio C Hamano <>2018-10-11 06:32:48 (GMT)
commit3255089ada6fc8f18d41dfc37cf66d7af994603d (patch)
treed275c6ff552357cbae9ef5772ea7f2bb2c4fdb1f /Documentation/technical/index-format.txt
parentabb4bb83845dc012ffe1c04750d1a09edd598a82 (diff)
ieot: add Index Entry Offset Table (IEOT) extension
This patch enables addressing the CPU cost of loading the index by adding additional data to the index that will allow us to efficiently multi- thread the loading and conversion of cache entries. It accomplishes this by adding an (optional) index extension that is a table of offsets to blocks of cache entries in the index file. To make this work for V4 indexes, when writing the cache entries, it periodically "resets" the prefix-compression by encoding the current entry as if the path name for the previous entry is completely different and saves the offset of that entry in the IEOT. Basically, with V4 indexes, it generates offsets into blocks of prefix-compressed entries. Signed-off-by: Ben Peart <> Signed-off-by: Junio C Hamano <>
Diffstat (limited to 'Documentation/technical/index-format.txt')
1 files changed, 18 insertions, 0 deletions
diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt
index 6bc2d90..7c4d67a 100644
--- a/Documentation/technical/index-format.txt
+++ b/Documentation/technical/index-format.txt
@@ -337,3 +337,21 @@ The remaining data of each directory block is grouped by type:
SHA-1("TREE" + <binary representation of N> +
"REUC" + <binary representation of M>)
+== Index Entry Offset Table
+ The Index Entry Offset Table (IEOT) is used to help address the CPU
+ cost of loading the index by enabling multi-threading the process of
+ converting cache entries from the on-disk format to the in-memory format.
+ The signature for this extension is { 'I', 'E', 'O', 'T' }.
+ The extension consists of:
+ - 32-bit version (currently 1)
+ - A number of index offset entries each consisting of:
+ - 32-bit offset from the begining of the file to the first cache entry
+ in this block of entries.
+ - 32-bit count of cache entries in this block