path: root/lockfile.h
AgeCommit message (Collapse)Author
2020-04-27lockfile.c: introduce 'hold_lock_file_for_update_mode'Taylor Blau
We use 'hold_lock_file_for_update' (and the '_timeout') variant to acquire a lock when updating references, the commit-graph file, and so on. In particular, the commit-graph machinery uses this to acquire a temporary file that is used to write a non-split commit-graph. In a subsequent commit, an issue in the commit-graph machinery produces graph files that have a different permission based on whether or not they are part of a multi-layer graph will be addressed. To do so, the commit-graph machinery will need a version of 'hold_lock_file_for_update' that takes the permission bits from the caller. Introduce such a function in this patch for both the 'hold_lock_file_for_update' and 'hold_lock_file_for_update_timeout' functions, and leave the existing functions alone by inlining their definitions in terms of the new mode variants. Note that, like in the previous commit, 'hold_lock_file_for_update_mode' is not guarenteed to set the given mode, since it may be modified by both the umask and 'core.sharedRepository'. Note also that even though the commit-graph machinery only calls 'hold_lock_file_for_update', that this is defined in terms of 'hold_lock_file_for_update_timeout', and so both need an additional mode parameter here. Signed-off-by: Taylor Blau <> Signed-off-by: Junio C Hamano <>
2019-05-05*.[ch]: manually align parameter listsDenton Liu
In previous patches, extern was mechanically removed from function declarations without care to formatting, causing parameter lists to be misaligned. Manually format changed sections such that the parameter lists should be realigned. Viewing this patch with 'git diff -w' should produce no output. Signed-off-by: Denton Liu <> Signed-off-by: Junio C Hamano <>
2019-05-05*.[ch]: remove extern from function declarations using spatchDenton Liu
There has been a push to remove extern from function declarations. Remove some instances of "extern" for function declarations which are caught by Coccinelle. Note that Coccinelle has some difficulty with processing functions with `__attribute__` or varargs so some `extern` declarations are left behind to be dealt with in a future patch. This was the Coccinelle patch used: @@ type T; identifier f; @@ - extern T f(...); and it was run with: $ git ls-files \*.{c,h} | grep -v ^compat/ | xargs spatch --sp-file contrib/coccinelle/noextern.cocci --in-place Files under `compat/` are intentionally excluded as some are directly copied from external sources and we should avoid churning them as much as possible. Signed-off-by: Denton Liu <> Signed-off-by: Junio C Hamano <>
2018-09-05reopen_tempfile(): truncate opened fileJeff King
We provide a reopen_tempfile() function, which is in turn used by reopen_lockfile(). The idea is that a caller may want to rewrite the tempfile without letting go of the lock. And that's what our one caller does: after running add--interactive, "commit -p" will update the cache-tree extension of the index and write out the result, all while holding the lock. However, because we open the file with only the O_WRONLY flag, the existing index content is left in place, and we overwrite it starting at position 0. If the new index after updating the cache-tree is smaller than the original, those final bytes are not overwritten and remain in the file. This results in a corrupt index, since those cruft bytes are interpreted as part of the trailing hash (or even as an extension, if there are enough bytes). This bug actually pre-dates reopen_tempfile(); the original code from 9c4d6c0297 (cache-tree: Write updated cache-tree after commit, 2014-07-13) has the same bug, and those lines were eventually refactored into the tempfile module. Nobody noticed until now for two reasons: - the bug can only be triggered in interactive mode ("commit -p" or "commit -i") - the size of the index must shrink after updating the cache-tree, which implies a non-trivial deletion. Notice that the included test actually has to create a 2-deep hierarchy. A single level is not enough to actually cause shrinkage. The fix is to truncate the file before writing out the second index. We can do that at the caller by using ftruncate(). But we shouldn't have to do that. There is no other place in Git where we want to open a file and overwrite bytes, making reopen_tempfile() a confusing and error-prone interface. Let's pass O_TRUNC there, which gives callers the same state they had after initially opening the file or lock. It's possible that we could later add a caller that wants something else (e.g., to open with O_APPEND). But this is the only caller we've had in the history of the codebase. Let's punt on doing anything more clever until another one comes along. Reported-by: Luc Van Oostenryck <> Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2017-10-06lockfile: fix documentation on `close_lock_file_gently()`Martin Ågren
Commit 83a3069a3 (lockfile: do not rollback lock on failed close, 2017-09-05) forgot to update the documentation by the function definition to reflect that the lock is not rolled back in case closing fails. Signed-off-by: Martin Ågren <> Signed-off-by: Junio C Hamano <>
2017-09-06lockfile: update lifetime requirements in documentationJeff King
Now that the tempfile system we rely on has loosened the lifetime requirements for storage, we can adjust our documentation to match. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2017-09-06tempfile: auto-allocate tempfiles on heapJeff King
The previous commit taught the tempfile code to give up ownership over tempfiles that have been renamed or deleted. That makes it possible to use a stack variable like this: struct tempfile t; create_tempfile(&t, ...); ... if (!err) rename_tempfile(&t, ...); else delete_tempfile(&t); But doing it this way has a high potential for creating memory errors. The tempfile we pass to create_tempfile() ends up on a global linked list, and it's not safe for it to go out of scope until we've called one of those two deactivation functions. Imagine that we add an early return from the function that forgets to call delete_tempfile(). With a static or heap tempfile variable, the worst case is that the tempfile hangs around until the program exits (and some functions like setup_shallow_temporary rely on this intentionally, creating a tempfile and then leaving it for later cleanup). But with a stack variable as above, this is a serious memory error: the variable goes out of scope and may be filled with garbage by the time the tempfile code looks at it. Let's see if we can make it harder to get this wrong. Since many callers need to allocate arbitrary numbers of tempfiles, we can't rely on static storage as a general solution. So we need to turn to the heap. We could just ask all callers to pass us a heap variable, but that puts the burden on them to call free() at the right time. Instead, let's have the tempfile code handle the heap allocation _and_ the deallocation (when the tempfile is deactivated and removed from the list). This changes the return value of all of the creation functions. For the cleanup functions (delete and rename), we'll add one extra bit of safety: instead of taking a tempfile pointer, we'll take a pointer-to-pointer and set it to NULL after freeing the object. This makes it safe to double-call functions like delete_tempfile(), as the second call treats the NULL input as a noop. Several callsites follow this pattern. The resulting patch does have a fair bit of noise, as each caller needs to be converted to handle: 1. Storing a pointer instead of the struct itself. 2. Passing the pointer instead of taking the struct address. 3. Handling a "struct tempfile *" return instead of a file descriptor. We could play games to make this less noisy. For example, by defining the tempfile like this: struct tempfile { struct heap_allocated_part_of_tempfile { int fd; ...etc } *actual_data; } Callers would continue to have a "struct tempfile", and it would be "active" only when the inner pointer was non-NULL. But that just makes things more awkward in the long run. There aren't that many callers, so we can simply bite the bullet and adjust all of them. And the compiler makes it easy for us to find them all. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2017-09-06lockfile: do not rollback lock on failed closeJeff King
Since the lockfile code is based on the tempfile code, it has some of the same problems, including that close_lock_file() erases the tempfile's filename buf, making it hard for the caller to write a good error message. In practice this comes up less for lockfiles than for straight tempfiles, since we usually just report the refname. But there is at least one buggy case in write_ref_to_lockfile(). Besides, given the coupling between the lockfile and tempfile modules, it's less confusing if their close() functions have the same semantics. Just as the previous commit did for close_tempfile(), let's teach close_lock_file() and its wrapper close_ref() not to rollback on error. And just as before, we'll give them new "gently" names to catch any new callers that are added. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2017-09-06tempfile: do not delete tempfile on failed closeJeff King
When close_tempfile() fails, we delete the tempfile and reset the fields of the tempfile struct. This makes it easier for callers to return without cleaning up, but it also makes this common pattern: if (close_tempfile(tempfile)) return error_errno("error closing %s", tempfile->filename.buf); wrong, because the "filename" field has been reset after the failed close. And it's not easy to fix, as in many cases we don't have another copy of the filename (e.g., if it was created via one of the mks_tempfile functions, and we just have the original template string). Let's drop the feature that a failed close automatically deletes the file. This puts the burden on the caller to do the deletion themselves, but this isn't that big a deal. Callers which do: if (write(...) || close_tempfile(...)) { delete_tempfile(...); return -1; } already had to call delete when the write() failed, and so aren't affected. Likewise, any caller which just calls die() in the error path is OK; we'll delete the tempfile during the atexit handler. Because this patch changes the semantics of close_tempfile() without changing its signature, all callers need to be manually checked and converted to the new scheme. This patch covers all in-tree callers, but there may be others for not-yet-merged topics. To catch these, we rename the function to close_tempfile_gently(), which will attract compile-time attention to new callers. (Technically the original could be considered "gentle" already in that it didn't die() on errors, but this one is even more so). Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2017-05-23lockfile: add a new method, is_lock_file_locked()Michael Haggerty
It will soon prove useful. Signed-off-by: Michael Haggerty <> Signed-off-by: Junio C Hamano <>
2016-12-27lockfile: move REPORT_ON_ERROR bit elsewhereJunio C Hamano
There was LOCK_NO_DEREF defined as 2 = 1<<1 with the same value, which was missed due to a huge comment block. Deconflict by moving the new one to 4 = 1<<2 for now. Signed-off-by: Junio C Hamano <>
2016-12-07lockfile: LOCK_REPORT_ON_ERRORJunio C Hamano
The "libify sequencer" topic stopped passing the die_on_error option to hold_locked_index(), and this lost an error message from "git merge --ff-only $commit" when there are competing updates in progress. The command still exits with a non-zero status, but that is not of much help for an interactive user. The last thing the command says is "Updating $from..$to". We used to follow it with a big error message that makes it clear that "merge --ff-only" did not succeed. What is sad is that we should have noticed this regression while reviewing the change. It was clear that the update to the checkout_fast_forward() function made a failing hold_locked_index() silent, but the only caller of the checkout_fast_forward() function had this comment: if (checkout_fast_forward(from, to, 1)) - exit(128); /* the callee should have complained already */ + return -1; /* the callee should have complained already */ which clearly contradicted the assumption X-<. Add a new option LOCK_REPORT_ON_ERROR that can be passed instead of LOCK_DIE_ON_ERROR to the hold_lock*() family of functions and teach checkout_fast_forward() to use it to fix this regression. After going thourgh all calls to hold_lock*() family of functions that used to pass LOCK_DIE_ON_ERROR but were modified to pass 0 in the "libify sequencer" topic "git show --first-parent 2a4062a4a8", it appears that this is the only one that has become silent. Many others used to give detailed report that talked about "there may be competing Git process running" but with the series merged they now only give a single liner "Unable to lock ...", some of which may have to be tweaked further, but at least they say something, unlike the one this patch fixes. Reported-by: Robbie Iannucci <> Signed-off-by: Junio C Hamano <>
2016-08-23mingw: ensure temporary file handles are not inherited by child processesBen Wijen
When the index is locked and child processes inherit the handle to said lock and the parent process wants to remove the lock before the child process exits, on Windows there is a problem: it won't work because files cannot be deleted if a process holds a handle on them. The symptom: Rename from 'xxx/.git/index.lock' to 'xxx/.git/index' failed. Should I try again? (y/n) Spawning child processes with bInheritHandles==FALSE would not work because no file handles would be inherited, not even the hStdXxx handles in STARTUPINFO (stdin/stdout/stderr). Opening every file with O_NOINHERIT does not work, either, as e.g. git-upload-pack expects inherited file handles. This leaves us with the only way out: creating temp files with the O_NOINHERIT flag. This flag is Windows-specific, however. For our purposes, it is equivalent to O_CLOEXEC (which does not exist on Windows), so let's just open temporary files with the O_CLOEXEC flag and map that flag to O_NOINHERIT on Windows. As Eric Wong pointed out, we need to be careful to handle the case where the Linux headers used to compile Git support O_CLOEXEC but the Linux kernel used to run Git does not: it returns an EINVAL. This fixes the test that we just introduced to demonstrate the problem. Signed-off-by: Ben Wijen <> Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2015-08-28lockfile: remove function "hold_lock_file_for_append"Ralf Thielow
With 77b9b1d (add_to_alternates_file: don't add duplicate entries, 2015-08-10) the last caller of function "hold_lock_file_for_append" has been removed, so we can remove the function as well. Signed-off-by: Ralf Thielow <> Acked-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2015-08-10tempfile: a new module for handling temporary filesMichael Haggerty
A lot of work went into defining the state diagram for lockfiles and ensuring correct, race-resistant cleanup in all circumstances. Most of that infrastructure can be applied directly to *any* temporary file. So extract a new "tempfile" module from the "lockfile" module. Reimplement lockfile on top of tempfile. Subsequent commits will add more users of the new module. Signed-off-by: Michael Haggerty <> Signed-off-by: Junio C Hamano <>
2015-08-10lockfile: add accessor get_lock_file_path()Michael Haggerty
Signed-off-by: Michael Haggerty <> Signed-off-by: Junio C Hamano <>
2015-08-10lockfile: add accessors get_lock_file_fd() and get_lock_file_fp()Michael Haggerty
We are about to move those members, so change client code to read them through accessor functions. Signed-off-by: Michael Haggerty <> Signed-off-by: Junio C Hamano <>
2015-08-10lockfile: move documentation to lockfile.h and lockfile.cMichael Haggerty
Rearrange/rewrite it somewhat to fit its new environment. Signed-off-by: Michael Haggerty <> Signed-off-by: Junio C Hamano <>
2015-05-14lockfile: allow file locking to be retried with a timeoutMichael Haggerty
Currently, there is only one attempt to lock a file. If it fails, the whole operation fails. But it might sometimes be advantageous to try acquiring a file lock a few times before giving up. So add a new function, hold_lock_file_for_update_timeout(), that allows a timeout to be specified. Make hold_lock_file_for_update() a thin wrapper around the new function. If timeout_ms is positive, then retry for at least that many milliseconds to acquire the lock. On each failed attempt, use select() to wait for a backoff time that increases quadratically (capped at 1 second) and has a random component to prevent two processes from getting synchronized. If timeout_ms is negative, retry indefinitely. In a moment we will switch to using the new function when locking packed-refs. Signed-off-by: Michael Haggerty <> Signed-off-by: Junio C Hamano <>
2014-10-15lockfile: remove unable_to_lock_errorJonathan Nieder
The former caller uses unable_to_lock_message now. Signed-off-by: Jonathan Nieder <> Reviewed-by: Ronnie Sahlberg <> Signed-off-by: Junio C Hamano <>
2014-10-01fdopen_lock_file(): access a lockfile using stdioMichael Haggerty
Add a new function, fdopen_lock_file(), which returns a FILE pointer open to the lockfile. If a stream is open on a lock_file object, it is closed using fclose() on commit, rollback, or close_lock_file(). This change will allow callers to use stdio to write to a lockfile without having to muck around in the internal representation of the lock_file object (callers will be rewritten in upcoming commits). Signed-off-by: Michael Haggerty <> Signed-off-by: Junio C Hamano <>
2014-10-01lockfile.h: extract new header file for the functions in lockfile.cMichael Haggerty
Move the interface declaration for the functions in lockfile.c from cache.h to a new file, lockfile.h. Add #includes where necessary (and remove some redundant includes of cache.h by files that already include builtin.h). Move the documentation of the lock_file state diagram from lockfile.c to the new header file. Signed-off-by: Michael Haggerty <> Signed-off-by: Junio C Hamano <>