path: root/builtin-unpack-objects.c
AgeCommit message (Collapse)Author
2008-03-04Revert "unpack-objects: prevent writing of inconsistent objects"Junio C Hamano
This reverts commit d5ef408b9afb5b4417f4e7e1593a96302d666650.
2008-02-29unpack-objects: prevent writing of inconsistent objectsMartin Koegler
This patch introduces a strict mode, which ensures that: - no malformed object will be written - no object with broken links will be written The patch ensures this by delaying the write of all non blob object. These object are written, after all objects they link to are written. An error can only result in unreferenced objects. Signed-off-by: Martin Koegler <> Signed-off-by: Junio C Hamano <>
2008-02-29unpack-object: cache for non written objectsMartin Koegler
Preventing objects with broken links entering the repository means, that write of some objects must be delayed. This patch adds a cache to keep the object data in memory. The delta resolving code must also search in the cache. Signed-off-by: Martin Koegler <> Signed-off-by: Junio C Hamano <>
2007-10-30relax usage of the progress APINicolas Pitre
Since it is now OK to pass a null pointer to display_progress() and stop_progress() resulting in a no-op, then we can simplify the code and remove a bunch of lines by not making those calls conditional all the time. Signed-off-by: Nicolas Pitre <> Signed-off-by: Junio C Hamano <>
2007-10-30make struct progress an opaque typeNicolas Pitre
This allows for better management of progress "object" existence, as well as making the progress display implementation more independent from its callers. Signed-off-by: Nicolas Pitre <> Signed-off-by: Junio C Hamano <>
2007-10-17more compact progress displayNicolas Pitre
Each progress can be on a single line instead of two. [sp: Changed "Checking files out" to "Checking out files" at Johannes Sixt's suggestion as it better explains the action that is taking place] Signed-off-by: Nicolas Pitre <> Signed-off-by: Shawn O. Pearce <>
2007-05-16Ensure return value from xread() is always stored into an ssize_tJohan Herland
This patch fixes all calls to xread() where the return value is not stored into an ssize_t. The patch should not have any effect whatsoever, other than putting better/more appropriate type names on variables. Signed-off-by: Johan Herland <> Signed-off-by: Junio C Hamano <>
2007-04-23make progress "title" part of the common progress interfaceNicolas Pitre
If the progress bar ends up in a box, better provide a title for it too. Signed-off-by: Nicolas Pitre <> Signed-off-by: Junio C Hamano <>
2007-04-23common progress display supportNicolas Pitre
Instead of having this code duplicated in multiple places, let's have a common interface for progress display. If someday someone wishes to display a cheezy progress bar instead then only one file will have to be changed. Note: I left merge-recursive.c out since it has a strange notion of progress as it apparently increase the expected total number as it goes. Someone with more intimate knowledge of what that is supposed to mean might look at converting it to the common progress interface. Signed-off-by: Nicolas Pitre <> Signed-off-by: Junio C Hamano <>
2007-04-10add overflow tests on pack offset variablesNicolas Pitre
Change a few size and offset variables to more appropriate type, then add overflow tests on those offsets. This prevents any bad data to be generated/processed if off_t happens to not be large enough to handle some big packs. Better be safe than sorry. Signed-off-by: Nicolas Pitre <> Signed-off-by: Junio C Hamano <>
2007-04-10make overflow test on delta base offset work regardless of variable sizeNicolas Pitre
This patch introduces the MSB() macro to obtain the desired number of most significant bits from a given variable independently of the variable type. It is then used to better implement the overflow test on the OBJ_OFS_DELTA base offset variable with the property of always working correctly regardless of the type/size of that variable. Signed-off-by: Nicolas Pitre <> Signed-off-by: Junio C Hamano <>
2007-02-27convert object type handling from a string to a numberNicolas Pitre
We currently have two parallel notation for dealing with object types in the code: a string and a numerical value. One of them is obviously redundent, and the most used one requires more stack space and a bunch of strcmp() all over the place. This is an initial step for the removal of the version using a char array found in object reading code paths. The patch is unfortunately large but there is no sane way to split it in smaller parts without breaking the system. Signed-off-by: Nicolas Pitre <> Signed-off-by: Junio C Hamano <>
2007-02-21Mechanical conversion to use prefixcmp()Junio C Hamano
This mechanically converts strncmp() to use prefixcmp(), but only when the parameters match specific patterns, so that they can be verified easily. Leftover from this will be fixed in a separate step, including idiotic conversions like if (!strncmp("foo", arg, 3)) => if (!(-prefixcmp(arg, "foo"))) This was done by using this script in px.perl #!/usr/bin/perl -i.bak -p if (/strncmp\(([^,]+), "([^\\"]*)", (\d+)\)/ && (length($2) == $3)) { s|strncmp\(([^,]+), "([^\\"]*)", (\d+)\)|prefixcmp($1, "$2")|; } if (/strncmp\("([^\\"]*)", ([^,]+), (\d+)\)/ && (length($1) == $3)) { s|strncmp\("([^\\"]*)", ([^,]+), (\d+)\)|(-prefixcmp($2, "$1"))|; } and running: $ git grep -l strncmp -- '*.c' | xargs perl px.perl Signed-off-by: Junio C Hamano <>
2006-12-20simplify inclusion of system header files.Junio C Hamano
This is a mechanical clean-up of the way *.c files include system header files. (1) sources under compat/, platform sha-1 implementations, and xdelta code are exempt from the following rules; (2) the first #include must be "git-compat-util.h" or one of our own header file that includes it first (e.g. config.h, builtin.h, pkt-line.h); (3) system headers that are included in "git-compat-util.h" need not be included in individual C source files. (4) "git-compat-util.h" does not have to include subsystem specific header files (e.g. expat.h). Signed-off-by: Junio C Hamano <>
2006-11-03Allow pack header preprocessing before unpack-objects/index-pack.Nicolas Pitre
Some applications which invoke unpack-objects or index-pack --stdin may want to examine the pack header to determine the number of objects contained in the pack and use that value to determine which executable to invoke to handle the rest of the pack stream. However if the caller consumes the pack header from the input stream then its no longer available for unpack-objects or index-pack --stdin, both of which need the version and object count to process the stream. This change introduces --pack_header=ver,cnt as a command line option that the caller can supply to indicate it has already consumed the pack header and what version and object count were found in that header. As this option is only meant for low level applications such as receive-pack we are not documenting it at this time. Signed-off-by: Nicolas Pitre <> Signed-off-by: Junio C Hamano <>
2006-10-31Use memmove instead of memcpy for overlapping areasEdgar Toernig
Signed-off-by: Junio C Hamano <>
2006-09-27teach git-unpack-objects about deltas with offset to baseNicolas Pitre
For delta resolution to be possible, a list of sha1/offset tuple must be constructed in memory in order to load the appropriate base object. Signed-off-by: Nicolas Pitre <> Signed-off-by: Junio C Hamano <>
2006-09-27introduce delta objects with offset to baseNicolas Pitre
This adds a new object, namely OBJ_OFS_DELTA, renames OBJ_DELTA to OBJ_REF_DELTA to better make the distinction between those two delta objects, and adds support for the handling of those new delta objects in sha1_file.c only. The OBJ_OFS_DELTA contains a relative offset from the delta object's position in a pack instead of the 20-byte SHA1 reference to identify the base object. Since the base is likely to be not so far away, the relative offset is more likely to have a smaller encoding on average than an absolute offset. And for those delta objects the base must always be stored first because there is no way to know the distance of later objects when streaming a pack. Hence this relative offset is always meant to be negative. The offset encoding is slightly denser than the one used for object size -- credits to <> (whoever this is) for bringing it to my attention. This allows for pack size reduction between 3.2% (Linux-2.6) to over 5% (linux-historic). Runtime pack access should be faster too since delta replay does skip a search in the pack index for each delta in a chain. Signed-off-by: Nicolas Pitre <> Signed-off-by: Junio C Hamano <>
2006-09-13unpack-objects -r: call it "recover".Junio C Hamano
The code called this operation "desperate" but the option flag is -r and the word "recover" describes what it does better. Signed-off-by: Junio C Hamano <>
2006-09-04unpack-objects desperately salvages objects from a corrupt packJunio C Hamano
The command unpack-objects dies upon the first error. This is probably considered a feature -- if a pack is corrupt, instead of trying to extract from it and possibly risking to contaminate a good repository with objects whose validity is dubious, we should seek a good copy of the pack and retry. However, we may not have any good copy anywhere. This implements the last resort effort to extract what are salvageable from such a corrupt pack. This flag might have helped Sergio when recovering from a corrupt pack. In my test, it managed to salvage 247 objects out of a pack that had 251 objects but without it the command stopped after extracting 73 objects. Signed-off-by: Junio C Hamano <>
2006-08-29unpack-objects: remove unused variable "eof"Johannes Schindelin
Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2006-08-23Convert memcpy(a,b,20) to hashcpy(a,b).Shawn Pearce
This abstracts away the size of the hash values when copying them from memory location to memory location, much as the introduction of hashcmp abstracted away hash value comparsion. A few call sites were using char* rather than unsigned char* so I added the cast rather than open hashcpy to be void*. This is a reasonable tradeoff as most call sites already use unsigned char* and the existing hashcmp is also declared to be unsigned char*. [jc: Splitted the patch to "master" part, to be followed by a patch for merge-recursive.c which is not in "master" yet. Fixed the cast in the latter hunk to combine-diff.c which was wrong in the original. Also converted ones left-over in combine-diff.c, diff-lib.c and upload-pack.c ] Signed-off-by: Shawn O. Pearce <> Signed-off-by: Junio C Hamano <>
2006-08-17Do not use memcmp(sha1_1, sha1_2, 20) with hardcoded length.David Rientjes
Introduces global inline: hashcmp(const unsigned char *sha1, const unsigned char *sha2) Uses memcmp for comparison and returns the result based on the length of the hash name (a future runtime decision). Acked-by: Alex Riesen <> Signed-off-by: David Rientjes <> Signed-off-by: Junio C Hamano <>
2006-08-13Merge branch 'jc/pack-objects'Junio C Hamano
2006-08-04Make git-unpack-objects a builtinMatthias Kestenholz
Signed-off-by: Matthias Kestenholz <> Signed-off-by: Junio C Hamano <>