summaryrefslogtreecommitdiff
path: root/builtin/prune.c
AgeCommit message (Collapse)Author
2015-10-26Merge branch 'jk/repository-extension'Junio C Hamano
Prepare for Git on-disk repository representation to undergo backward incompatible changes by introducing a new repository format version "1", with an extension mechanism. * jk/repository-extension: introduce "preciousObjects" repository extension introduce "extensions" form of core.repositoryformatversion
2015-07-13Merge branch 'nd/multiple-work-trees'Junio C Hamano
"git checkout [<tree-ish>] <paths>" spent unnecessary cycles checking if the current branch was checked out elsewhere, when we know we are not switching the branches ourselves. * nd/multiple-work-trees: worktree: new place for "git prune --worktrees" checkout: don't check worktrees when not necessary
2015-06-29worktree: new place for "git prune --worktrees"Nguyễn Thái Ngọc Duy
Commit 23af91d (prune: strategies for linked checkouts - 2014-11-30) adds "--worktrees" to "git prune" without realizing that "git prune" is for object database only. This patch moves the same functionality to a new command "git worktree". Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
2015-06-25introduce "preciousObjects" repository extensionJeff King
If this extension is used in a repository, then no operations should run which may drop objects from the object storage. This can be useful if you are sharing that storage with other repositories whose refs you cannot see. For instance, if you do: $ git clone -s parent child $ git -C parent config extensions.preciousObjects true $ git -C parent config core.repositoryformatversion 1 you now have additional safety when running git in the parent repository. Prunes and repacks will bail with an error, and `git gc` will skip those operations (it will continue to pack refs and do other non-object operations). Older versions of git, when run in the repository, will fail on every operation. Note that we do not set the preciousObjects extension by default when doing a "clone -s", as doing so breaks backwards compatibility. It is a decision the user should make explicitly. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-11Merge branch 'nd/multiple-work-trees'Junio C Hamano
A replacement for contrib/workdir/git-new-workdir that does not rely on symbolic links and make sharing of objects and refs safer by making the borrowee and borrowers aware of each other. * nd/multiple-work-trees: (41 commits) prune --worktrees: fix expire vs worktree existence condition t1501: fix test with split index t2026: fix broken &&-chain t2026 needs procondition SANITY git-checkout.txt: a note about multiple checkout support for submodules checkout: add --ignore-other-wortrees checkout: pass whole struct to parse_branchname_arg instead of individual flags git-common-dir: make "modules/" per-working-directory directory checkout: do not fail if target is an empty directory t2025: add a test to make sure grafts is working from a linked checkout checkout: don't require a work tree when checking out into a new one git_path(): keep "info/sparse-checkout" per work-tree count-objects: report unused files in $GIT_DIR/worktrees/... gc: support prune --worktrees gc: factor out gc.pruneexpire parsing code gc: style change -- no SP before closing parenthesis checkout: clean up half-prepared directories in --to mode checkout: reject if the branch is already checked out elsewhere prune: strategies for linked checkouts checkout: support checking out into a new working directory ...
2015-03-31prune --worktrees: fix expire vs worktree existence conditionMax Kirillov
`git prune --worktrees` was pruning worktrees which were non-existent OR expired, while it rather should prune those which are orphaned AND expired, as git-checkout documentation describes. Fix it. Add test 'not prune proper checkouts', which uses valid but expired worktree. Modify test 'not prune recent checkouts' to remove the worktree before pruning - link in worktrees still must survive. In older form it is useless because would pass always when the other test passes. Signed-off-by: Max Kirillov <max@max630.net> Acked-by: Duy Nguyen <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-20prune: turn on ref_paranoia flagJeff King
Prune should know about broken objects at the tips of refs, so that we can feed them to our traversal rather than ignoring them. It's better for us to abort the operation on the broken object than it is to start deleting objects with an incomplete view of the reachability namespace. Note that for missing objects, aborting is the best we can do. For a badly-named ref, we technically could use its sha1 as a reachability tip. However, the iteration code just feeds us a null sha1, so there would be a reasonable amount of code involved to pass down our wishes. It's not really worth trying to do better, because this is a case that should happen extremely rarely, and the message we provide: fatal: unable to parse object: refs/heads/bogus:name is probably enough to point the user in the right direction. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-12-01prune: strategies for linked checkoutsNguyễn Thái Ngọc Duy
(alias R=$GIT_COMMON_DIR/worktrees/<id>) - linked checkouts are supposed to keep its location in $R/gitdir up to date. The use case is auto fixup after a manual checkout move. - linked checkouts are supposed to update mtime of $R/gitdir. If $R/gitdir's mtime is older than a limit, and it points to nowhere, worktrees/<id> is to be pruned. - If $R/locked exists, worktrees/<id> is not supposed to be pruned. If $R/locked exists and $R/gitdir's mtime is older than a really long limit, warn about old unused repo. - "git checkout --to" is supposed to make a hard link named $R/link pointing to the .git file on supported file systems to help detect the user manually deleting the checkout. If $R/link exists and its link count is greated than 1, the repo is kept. Helped-by: Marc Branchaud <marcnarc@xiplink.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Marc Branchaud <marcnarc@xiplink.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-16prune: keep objects reachable from recent objectsJeff King
Our current strategy with prune is that an object falls into one of three categories: 1. Reachable (from ref tips, reflogs, index, etc). 2. Not reachable, but recent (based on the --expire time). 3. Not reachable and not recent. We keep objects from (1) and (2), but prune objects in (3). The point of (2) is that these objects may be part of an in-progress operation that has not yet updated any refs. However, it is not always the case that objects for an in-progress operation will have a recent mtime. For example, the object database may have an old copy of a blob (from an abandoned operation, a branch that was deleted, etc). If we create a new tree that points to it, a simultaneous prune will leave our tree, but delete the blob. Referencing that tree with a commit will then work (we check that the tree is in the object database, but not that all of its referred objects are), as will mentioning the commit in a ref. But the resulting repo is corrupt; we are missing the blob reachable from a ref. One way to solve this is to be more thorough when referencing a sha1: make sure that not only do we have that sha1, but that we have objects it refers to, and so forth recursively. The problem is that this is very expensive. Creating a parent link would require traversing the entire object graph! Instead, this patch pushes the extra work onto prune, which runs less frequently (and has to look at the whole object graph anyway). It creates a new category of objects: objects which are not recent, but which are reachable from a recent object. We do not prune these objects, just like the reachable and recent ones. This lets us avoid the recursive check above, because if we have an object, even if it is unreachable, we should have its referent. We can make a simple inductive argument that with this patch, this property holds (that there are no objects with missing referents in the repository): 0. When we have no objects, we have nothing to refer or be referred to, so the property holds. 1. If we add objects to the repository, their direct referents must generally exist (e.g., if you create a tree, the blobs it references must exist; if you create a commit to point at the tree, the tree must exist). This is already the case before this patch. And it is not 100% foolproof (you can make bogus objects using `git hash-object`, for example), but it should be the case for normal usage. Therefore for any sequence of object additions, the property will continue to hold. 2. If we remove objects from the repository, then we will not remove a child object (like a blob) if an object that refers to it is being kept. That is the part implemented by this patch. Note, however, that our reachability check and the actual pruning are not atomic. So it _is_ still possible to violate the property (e.g., an object becomes referenced just as we are deleting it). This patch is shooting for eliminating problems where the mtimes of dependent objects differ by hours or days, and one is dropped without the other. It does nothing to help with short races. Naively, the simplest way to implement this would be to add all recent objects as tips to the reachability traversal. However, this does not perform well. In a recently-packed repository, all reachable objects will also be recent, and therefore we have to look at each object twice. This patch instead performs the reachability traversal, then follows up with a second traversal for recent objects, skipping any that have already been marked. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-16prune: factor out loose-object directory traversalJeff King
Prune has to walk $GIT_DIR/objects/?? in order to find the set of loose objects to prune. Other parts of the code (e.g., count-objects) want to do the same. Let's factor it out into a reusable for_each-style function. Note that this is not quite a straight code movement. The original code had strange behavior when it found a file of the form "[0-9a-f]{2}/.{38}" that did _not_ contain all hex digits. It executed a "break" from the loop, meaning that we stopped pruning in that directory (but still pruned other directories!). This was probably a bug; we do not want to process the file as an object, but we should keep going otherwise (and that is how the new code handles it). We are also a little more careful with loose object directories which fail to open. The original code silently ignored any failures, but the new code will complain about any problems besides ENOENT. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-14Merge branch 'mh/replace-refs-variable-rename'Junio C Hamano
* mh/replace-refs-variable-rename: Document some functions defined in object.c Add docstrings for lookup_replace_object() and do_lookup_replace_object() rename read_replace_refs to check_replace_refs
2014-02-24i18n: mark all progress lines for translationNguyễn Thái Ngọc Duy
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-02-20rename read_replace_refs to check_replace_refsMichael Haggerty
The semantics of this flag was changed in commit e1111cef23 inline lookup_replace_object() calls but wasn't renamed at the time to minimize code churn. Rename it now, and add a comment explaining its use. Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-17Merge branch 'nd/shallow-clone'Junio C Hamano
Fetching from a shallow-cloned repository used to be forbidden, primarily because the codepaths involved were not carefully vetted and we did not bother supporting such usage. This attempts to allow object transfer out of a shallow-cloned repository in a controlled way (i.e. the receiver become a shallow repository with truncated history). * nd/shallow-clone: (31 commits) t5537: fix incorrect expectation in test case 10 shallow: remove unused code send-pack.c: mark a file-local function static git-clone.txt: remove shallow clone limitations prune: clean .git/shallow after pruning objects clone: use git protocol for cloning shallow repo locally send-pack: support pushing from a shallow clone via http receive-pack: support pushing to a shallow clone via http smart-http: support shallow fetch/clone remote-curl: pass ref SHA-1 to fetch-pack as well send-pack: support pushing to a shallow clone receive-pack: allow pushes that update .git/shallow connected.c: add new variant that runs with --shallow-file add GIT_SHALLOW_FILE to propagate --shallow-file to subprocesses receive/send-pack: support pushing from a shallow clone receive-pack: reorder some code in unpack() fetch: add --update-shallow to accept refs that update .git/shallow upload-pack: make sure deepening preserves shallow roots fetch: support fetching from a shallow repository clone: support remote shallow repository ...
2014-01-10Merge branch 'mh/path-max'Junio C Hamano
A few places where we relied on a fixed length buffer to hold pathnames in these two programs have been converted to use strbuf. * mh/path-max: builtin/prune.c: use strbuf to avoid having to worry about PATH_MAX prune-packed: use strbuf to avoid having to worry about PATH_MAX
2013-12-18builtin/prune.c: use strbuf to avoid having to worry about PATH_MAXJeff King
While at it, rename prune_tmp_object(), which used to be a helper to remove temporary files that were created to become loose object files, to prune_tmp_file(), as the function is also used to remove any random cruft whose name begins with tmp_ directly in .git/object or .git/object/pack directories these days. Noticed-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-11prune: clean .git/shallow after pruning objectsNguyễn Thái Ngọc Duy
This patch teaches "prune" to remove shallow roots that are no longer reachable from any refs (e.g. when the relevant refs are removed). Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-05replace {pre,suf}fixcmp() with {starts,ends}_with()Christian Couder
Leaving only the function definitions and declarations so that any new topic in flight can still make use of the old functions, replace existing uses of the prefixcmp() and suffixcmp() with new API functions. The change can be recreated by mechanically applying this: $ git grep -l -e prefixcmp -e suffixcmp -- \*.c | grep -v strbuf\\.c | xargs perl -pi -e ' s|!prefixcmp\(|starts_with\(|g; s|prefixcmp\(|!starts_with\(|g; s|!suffixcmp\(|ends_with\(|g; s|suffixcmp\(|!ends_with\(|g; ' on the result of preparatory changes in this series. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-06Merge branch 'nd/prune-packed-dryrun-verbose'Junio C Hamano
* nd/prune-packed-dryrun-verbose: prune-packed: avoid implying "1" is DRY_RUN in prune_packed_objects()
2013-05-28prune-packed: avoid implying "1" is DRY_RUN in prune_packed_objects()Nguyễn Thái Ngọc Duy
Commit b60daf0 (Make git-prune-packed a bit more chatty. - 2007-01-12) changes the meaning of prune_packed_objects()'s argument, from "dry run or not dry run" to a bitmap. It however forgot to update prune_packed_objects() caller in builtin/prune.c to use new DRY_RUN macro. It's fine (for a long time!) but there is a risk that someday someone may change the value of DRY_RUN to something else and builtin/prune.c suddenly breaks. Avoid that possibility. While at there, change "opts == VERBOSE" to "opts & VERBOSE" as there is no obvious reason why we only be chatty when DRY_RUN is not set. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-25prune: introduce OPT_EXPIRY_DATE() and use itJunio C Hamano
Earlier we added support for --expire=all (or --expire=now) that considers all crufts, regardless of their age, as eligible for garbage collection by turning command argument parsers that use approxidate() to use parse_expiry_date(), but "git prune" used a built-in parse-options facility OPT_DATE() and did not benefit from the new function. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-03Merge branch 'jk/fully-peeled-packed-ref' into maint-1.8.1Junio C Hamano
* jk/fully-peeled-packed-ref: pack-refs: add fully-peeled trait pack-refs: write peeled entry for non-tags use parse_object_or_die instead of die("bad object") avoid segfaults on parse_object failure
2013-03-25Merge branch 'jk/fully-peeled-packed-ref'Junio C Hamano
Not that we do not actively encourage having annotated tags outside refs/tags/ hierarchy, but they were not advertised correctly to the ls-remote and fetch with recent version of Git. * jk/fully-peeled-packed-ref: pack-refs: add fully-peeled trait pack-refs: write peeled entry for non-tags use parse_object_or_die instead of die("bad object") avoid segfaults on parse_object failure
2013-03-17use parse_object_or_die instead of die("bad object")Jeff King
Some call-sites do: o = parse_object(sha1); if (!o) die("bad object %s", some_name); We can now handle that as a one-liner, and get more consistent output. In the third case of this patch, it looks like we are losing information, as the existing message also outputs the sha1 hex; however, parse_object will already have written a more specific complaint about the sha1, so there is no point in repeating it here. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-09-14Merge branch 'rj/path-cleanup'Junio C Hamano
* rj/path-cleanup: Call mkpathdup() rather than xstrdup(mkpath(...)) Call git_pathdup() rather than xstrdup(git_path("...")) path.c: Use vsnpath() in the implementation of git_path() path.c: Don't discard the return value of vsnpath() path.c: Remove the 'git_' prefix from a file scope function
2012-09-07Merge branch 'nd/i18n-parseopt-help'Junio C Hamano
A lot of i18n mark-up for the help text from "git <cmd> -h". * nd/i18n-parseopt-help: (66 commits) Use imperative form in help usage to describe an action Reduce translations by using same terminologies i18n: write-tree: mark parseopt strings for translation i18n: verify-tag: mark parseopt strings for translation i18n: verify-pack: mark parseopt strings for translation i18n: update-server-info: mark parseopt strings for translation i18n: update-ref: mark parseopt strings for translation i18n: update-index: mark parseopt strings for translation i18n: tag: mark parseopt strings for translation i18n: symbolic-ref: mark parseopt strings for translation i18n: show-ref: mark parseopt strings for translation i18n: show-branch: mark parseopt strings for translation i18n: shortlog: mark parseopt strings for translation i18n: rm: mark parseopt strings for translation i18n: revert, cherry-pick: mark parseopt strings for translation i18n: rev-parse: mark parseopt strings for translation i18n: reset: mark parseopt strings for translation i18n: rerere: mark parseopt strings for translation i18n: status: mark parseopt strings for translation i18n: replace: mark parseopt strings for translation ...
2012-09-04Call mkpathdup() rather than xstrdup(mkpath(...))Ramsay Jones
In addition to updating the xstrdup(mkpath(...)) call sites with mkpathdup(), we also fix a memory leak (in merge_3way()) caused by neglecting to free the memory allocated to the 'base_name' variable. Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-08-20i18n: prune: mark parseopt strings for translationNguyễn Thái Ngọc Duy
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-08-07prune.c: only print informational message in show_only or verbose modeBrandon Casey
"git prune" reports removal of loose object files that are no longer necessary only under the "-v" option, but unconditionally reports removal of temporary files that are no longer needed. The original thinking was that the presence of a leftover temporary file should be an unusual occurrence that may indicate an earlier failure of some sort, and the user may want to be reminded of it. Removing an unnecessary loose object file, on the other hand, is just part of the normal operation. That is why the former is always printed out and the latter only when -v is used. But neither report is particularly useful. Hide both of these behind the "-v" option for consistency. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-03-07fix deletion of .git/objects sub-directories in git-prune/repackKarsten Blees
Both git-prune and git-repack (and thus, git-gc) try to rmdir while holding a DIR* handle on the directory. This can leave dangling empty directories in the .git/objects on platforms where directory cannot be removed while they are open. First call closedir() and then rmdir(); that is more logical ordering. Reported-by: John Chen <john0312@gmail.com> Reported-by: Stefan Naewe <stefan.naewe@gmail.com> Signed-off-by: Karsten Blees <blees@dcon.de> Improved-and-Acked-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-11-08prune: handle --progress/no-progressJeff King
And have "git gc" pass no-progress when quiet. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-11-08prune: show progress while marking reachable objectsNguyễn Thái Ngọc Duy
prune already shows progress meter while pruning. The marking part may take a few seconds or more, depending on repository size. Show progress meter during this time too. Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-15add description parameter to OPT__DRY_RUNRené Scharfe
Allows better help text to be defined than "dry run". Also make use of the macro in places that already had a different description. No object code changes intended. Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-11-15add description parameter to OPT__VERBOSERené Scharfe
Allows better help text to be defined than "be verbose". Also make use of the macro in places that already had a different description. No object code changes intended. Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-09prune: allow --dry-run for -n and --verbose for -vRené Scharfe
For consistency with other git commands, let git prune accept the long options --dry-run and --verbose for the respective short ones -n and -v. Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-03-10Merge branch 'lt/deepen-builtin-source'Junio C Hamano
* lt/deepen-builtin-source: Move 'builtin-*' into a 'builtin/' subdirectory Conflicts: Makefile
2010-02-22Move 'builtin-*' into a 'builtin/' subdirectoryLinus Torvalds
This shrinks the top-level directory a bit, and makes it much more pleasant to use auto-completion on the thing. Instead of [torvalds@nehalem git]$ em buil<tab> Display all 180 possibilities? (y or n) [torvalds@nehalem git]$ em builtin-sh builtin-shortlog.c builtin-show-branch.c builtin-show-ref.c builtin-shortlog.o builtin-show-branch.o builtin-show-ref.o [torvalds@nehalem git]$ em builtin-shor<tab> builtin-shortlog.c builtin-shortlog.o [torvalds@nehalem git]$ em builtin-shortlog.c you get [torvalds@nehalem git]$ em buil<tab> [type] builtin/ builtin.h [torvalds@nehalem git]$ em builtin [auto-completes to] [torvalds@nehalem git]$ em builtin/sh<tab> [type] shortlog.c shortlog.o show-branch.c show-branch.o show-ref.c show-ref.o [torvalds@nehalem git]$ em builtin/sho [auto-completes to] [torvalds@nehalem git]$ em builtin/shor<tab> [type] shortlog.c shortlog.o [torvalds@nehalem git]$ em builtin/shortlog.c which doesn't seem all that different, but not having that annoying break in "Display all 180 possibilities?" is quite a relief. NOTE! If you do this in a clean tree (no object files etc), or using an editor that has auto-completion rules that ignores '*.o' files, you won't see that annoying 'Display all 180 possibilities?' message - it will just show the choices instead. I think bash has some cut-off around 100 choices or something. So the reason I see this is that I'm using an odd editory, and thus don't have the rules to cut down on auto-completion. But you can simulate that by using 'ls' instead, or something similar. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>