summaryrefslogtreecommitdiff
path: root/Documentation
AgeCommit message (Collapse)Author
2017-01-19clear_delta_base_cache(): don't modify hashmap while iteratingJeff King
On Thu, Jan 19, 2017 at 03:03:46PM +0100, Ulrich Spörlein wrote: > > I suspect the patch below may fix things for you. It works around it by > > walking over the lru list (either is fine, as they both contain all > > entries, and since we're clearing everything, we don't care about the > > order). > > Confirmed. With the patch applied, I can import the whole 55G in one go > without any crashes or aborts. Thanks much! Thanks. Here it is rolled up with a commit message. -- >8 -- Subject: clear_delta_base_cache(): don't modify hashmap while iterating Removing entries while iterating causes fast-import to access an already-freed `struct packed_git`, leading to various confusing errors. What happens is that clear_delta_base_cache() drops the whole contents of the cache by iterating over the hashmap, calling release_delta_base_cache() on each entry. That function removes the item from the hashmap. The hashmap code may then shrink the table, but the hashmap_iter struct retains an offset from the old table. As a result, the next call to hashmap_iter_next() may claim that the iteration is done, even though some items haven't been visited. The only caller of clear_delta_base_cache() is fast-import, which wants to clear the cache because it is discarding the packed_git struct for its temporary pack. So by failing to remove all of the entries, we still have references to the freed packed_git. To make things even more confusing, this doesn't seem to trigger with the test suite, because it depends on complexities like the size of the hash table, which entries got cleared, whether we try to access them before they're evicted from the cache, etc. So I've been able to identify the problem with large imports like freebsd's svn import, or a fast-export of linux.git. But nothing that would be reasonable to run as part of the normal test suite. We can fix this easily by iterating over the lru linked list instead of the hashmap. They both contain the same entries, and we can use the "safe" variant of the list iterator, which exists for exactly this case. Let's also add a warning to the hashmap API documentation to reduce the chances of getting bit by this again. Reported-by: Ulrich Spörlein <uqs@freebsd.org> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-17Almost ready for 2.11.1Junio C Hamano
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-17Merge branch 'mm/gc-safety-doc' into maintJunio C Hamano
Doc update. * mm/gc-safety-doc: git-gc.txt: expand discussion of races with other processes
2017-01-17Merge branch 'mm/push-social-engineering-attack-doc' into maintJunio C Hamano
Doc update on fetching and pushing. * mm/push-social-engineering-attack-doc: doc: mention transfer data leaks in more places
2017-01-17Merge branch 'ls/filter-process' into maintJunio C Hamano
Doc update. * ls/filter-process: t0021: fix flaky test docs: warn about possible '=' in clean/smudge filter process values
2017-01-17Merge branch 'kh/tutorial-grammofix' into maintJunio C Hamano
* kh/tutorial-grammofix: doc: omit needless "for" doc: make the intent of sentence clearer doc: add verb in front of command to run doc: add articles (grammar)
2017-01-17Merge branch 'lr/doc-fix-cet' into maintJunio C Hamano
* lr/doc-fix-cet: date-formats.txt: Typo fix
2017-01-17Merge branch 'dt/smart-http-detect-server-going-away' into maintJunio C Hamano
When the http server gives an incomplete response to a smart-http rpc call, it could lead to client waiting for a full response that will never come. Teach the client side to notice this condition and abort the transfer. An improvement counterproposal has failed. cf. <20161114194049.mktpsvgdhex2f4zv@sigill.intra.peff.net> * dt/smart-http-detect-server-going-away: upload-pack: optionally allow fetching any sha1 remote-curl: don't hang when a server dies before any output
2017-01-17Merge branch 'jk/quote-env-path-list-component' into maintJunio C Hamano
A recent update to receive-pack to make it easier to drop garbage objects made it clear that GIT_ALTERNATE_OBJECT_DIRECTORIES cannot have a pathname with a colon in it (no surprise!), and this in turn made it impossible to push into a repository at such a path. This has been fixed by introducing a quoting mechanism used when appending such a path to the colon-separated list. * jk/quote-env-path-list-component: t5615-alternate-env: double-quotes in file names do not work on Windows t5547-push-quarantine: run the path separator test on Windows, too tmp-objdir: quote paths we add to alternates alternates: accept double-quoted paths
2017-01-17Merge branch 'ak/commit-only-allow-empty' into maintJunio C Hamano
"git commit --allow-empty --only" (no pathspec) with dirty index ought to be an acceptable way to create a new commit that does not change any paths, but it was forbidden, perhaps because nobody needed it so far. * ak/commit-only-allow-empty: commit: remove 'Clever' message for --only --amend commit: make --only --allow-empty work without paths
2017-01-17Merge branch 'jk/http-walker-limit-redirect' into maintJunio C Hamano
Update the error messages from the dumb-http client when it fails to obtain loose objects; we used to give sensible error message only upon 404 but we now forbid unexpected redirects that needs to be reported with something sensible. * jk/http-walker-limit-redirect: http-walker: complain about non-404 loose object errors http: treat http-alternates like redirects http: make redirects more obvious remote-curl: rename shadowed options variable http: always update the base URL for redirects http: simplify update_url_from_redirect
2016-12-12alternates: accept double-quoted pathsJeff King
We read lists of alternates from objects/info/alternates files (delimited by newline), as well as from the GIT_ALTERNATE_OBJECT_DIRECTORIES environment variable (delimited by colon or semi-colon, depending on the platform). There's no mechanism for quoting the delimiters, so it's impossible to specify an alternate path that contains a colon in the environment, or one that contains a newline in a file. We've lived with that restriction for ages because both alternates and filenames with colons are relatively rare, and it's only a problem when the two meet. But since 722ff7f87 (receive-pack: quarantine objects until pre-receive accepts, 2016-10-03), which builds on the alternates system, every push causes the receiver to set GIT_ALTERNATE_OBJECT_DIRECTORIES internally. It would be convenient to have some way to quote the delimiter so that we can represent arbitrary paths. The simplest thing would be an escape character before a quoted delimiter (e.g., "\:" as a literal colon). But that creates a backwards compatibility problem: any path which uses that escape character is now broken, and we've just shifted the problem. We could choose an unlikely escape character (e.g., something from the non-printable ASCII range), but that's awkward to use. Instead, let's treat names as unquoted unless they begin with a double-quote, in which case they are interpreted via our usual C-stylke quoting rules. This also breaks backwards-compatibility, but in a smaller way: it only matters if your file has a double-quote as the very _first_ character in the path (whereas an escape character is a problem anywhere in the path). It's also consistent with many other parts of git, which accept either a bare pathname or a double-quoted one, and the sender can choose to quote or not as required. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-12date-formats.txt: Typo fixLuis Ressel
Last time I checked, I was living in the UTC+01:00 time zone. UTC+02:00 would be Central European _Summer_ Time. Signed-off-by: Luis Ressel <aranea@aixah.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-09doc: omit needless "for"Kristoffer Haugsbakk
What was intended was perhaps "... plumbing does for you" ("you" added), but simply omitting the word "for" is more terse and gets the intended point across just as well, if not more so. I originally went with the approach of writing "for you", but Junio C Hamano suggested this approach instead. Signed-off-by: Kristoffer Haugsbakk <kristoffer.haugsbakk@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-09doc: make the intent of sentence clearerKristoffer Haugsbakk
By adding the word "just", which might have been accidentally omitted. Adding the word "just" makes it clear that the point is to *not* do an octopus merge simply because you *can* do it. In other words, you should have a reason for doing it beyond simply having two (seemingly) independent commits that you need to merge into another branch, since it's not always the best approach. The previous sentence made it look more like it was trying to say that you shouldn't do an octopus merge *because* you can do an octopus merge. Although this interpretation doesn't make sense and the rest of the paragraph makes the intended meaning clear, this adjustment should make the intent of the sentence more immediately clear to the reader. Signed-off-by: Kristoffer Haugsbakk <kristoffer.haugsbakk@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-09doc: add verb in front of command to runKristoffer Haugsbakk
Instead of using the command 'git clone' as a verb, use "run" as the verb indicating the action of executing the command 'git clone'. Signed-off-by: Kristoffer Haugsbakk <kristoffer.haugsbakk@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-09doc: add articles (grammar)Kristoffer Haugsbakk
Add definite and indefinite articles in three places where they were missing. - Use "the" in front of a directory name - Use "the" in front of "style of cooperation" - Use an indefinite article in front of "CVS background" Signed-off-by: Kristoffer Haugsbakk <kristoffer.haugsbakk@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06http: make redirects more obviousJeff King
We instruct curl to always follow HTTP redirects. This is convenient, but it creates opportunities for malicious servers to create confusing situations. For instance, imagine Alice is a git user with access to a private repository on Bob's server. Mallory runs her own server and wants to access objects from Bob's repository. Mallory may try a few tricks that involve asking Alice to clone from her, build on top, and then push the result: 1. Mallory may simply redirect all fetch requests to Bob's server. Git will transparently follow those redirects and fetch Bob's history, which Alice may believe she got from Mallory. The subsequent push seems like it is just feeding Mallory back her own objects, but is actually leaking Bob's objects. There is nothing in git's output to indicate that Bob's repository was involved at all. The downside (for Mallory) of this attack is that Alice will have received Bob's entire repository, and is likely to notice that when building on top of it. 2. If Mallory happens to know the sha1 of some object X in Bob's repository, she can instead build her own history that references that object. She then runs a dumb http server, and Alice's client will fetch each object individually. When it asks for X, Mallory redirects her to Bob's server. The end result is that Alice obtains objects from Bob, but they may be buried deep in history. Alice is less likely to notice. Both of these attacks are fairly hard to pull off. There's a social component in getting Mallory to convince Alice to work with her. Alice may be prompted for credentials in accessing Bob's repository (but not always, if she is using a credential helper that caches). Attack (1) requires a certain amount of obliviousness on Alice's part while making a new commit. Attack (2) requires that Mallory knows a sha1 in Bob's repository, that Bob's server supports dumb http, and that the object in question is loose on Bob's server. But we can probably make things a bit more obvious without any loss of functionality. This patch does two things to that end. First, when we encounter a whole-repo redirect during the initial ref discovery, we now inform the user on stderr, making attack (1) much more obvious. Second, the decision to follow redirects is now configurable. The truly paranoid can set the new http.followRedirects to false to avoid any redirection entirely. But for a more practical default, we will disallow redirects only after the initial ref discovery. This is enough to thwart attacks similar to (2), while still allowing the common use of redirects at the repository level. Since c93c92f30 (http: update base URLs when we see redirects, 2013-09-28) we re-root all further requests from the redirect destination, which should generally mean that no further redirection is necessary. As an escape hatch, in case there really is a server that needs to redirect individual requests, the user can set http.followRedirects to "true" (and this can be done on a per-server basis via http.*.followRedirects config). Reported-by: Jann Horn <jannh@google.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06docs: warn about possible '=' in clean/smudge filter process valuesLars Schneider
A pathname value in a clean/smudge filter process "key=value" pair can contain the '=' character (introduced in edcc858). Make the user aware of this issue in the docs, add a corresponding test case, and fix the issue in filter process value parser of the example implementation in contrib. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-05commit: make --only --allow-empty work without pathsAndreas Krey
--only is implied when paths are present, and required them unless --amend. But with --allow-empty it should be allowed as well - it is the only way to create an empty commit in the presence of staged changes. Signed-off-by: Andreas Krey <a.krey@gmx.de> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-05Sync with maint-2.10Junio C Hamano
* maint-2.10: preparing for 2.10.3
2016-12-05preparing for 2.10.3Junio C Hamano
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-29Merge branch 'nd/worktree-lock' into maintJunio C Hamano
Typofix. * nd/worktree-lock: git-worktree.txt: fix typo "to"/"two", and add comma
2016-11-29Merge branch 'ps/common-info-doc' into maintJunio C Hamano
Doc fix. * ps/common-info-doc: doc: fix location of 'info/' with $GIT_COMMON_DIR
2016-11-29Merge branch 'sc/fmt-merge-msg-doc-markup-fix' into maintJunio C Hamano
Documentation fix. * sc/fmt-merge-msg-doc-markup-fix: Documentation/fmt-merge-msg: fix markup in example
2016-11-29Merge branch 'jk/rebase-config-insn-fmt-docfix' into maintJunio C Hamano
Documentation fix. * jk/rebase-config-insn-fmt-docfix: doc: fix missing "::" in config list
2016-11-29Git 2.11v2.11.0Junio C Hamano
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-28RelNotes: spelling and phrasing fixupsMarc Branchaud
Signed-off-by: Marc Branchaud <marcnarc@xiplink.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-23Git 2.11-rc3v2.11.0-rc3Junio C Hamano
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-23Merge branch 'jt/trailer-with-cruft'Junio C Hamano
Doc update. * jt/trailer-with-cruft: doc: mention user-configured trailers
2016-11-21doc: mention user-configured trailersJonathan Tan
In commit 1462450 ("trailer: allow non-trailers in trailer block", 2016-10-21), functionality was added (and tested [1]) to allow non-trailer lines in trailer blocks, as long as those blocks contain at least one Git-generated or user-configured trailer, and consists of at least 25% trailers. The documentation was updated to mention this new functionality, but did not mention "user-configured trailer". Further update the documentation to also mention "user-configured trailer". [1] "with non-trailer lines mixed with a configured trailer" in t/t7513-interpret-trailers.sh Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-18upload-pack: optionally allow fetching any sha1David Turner
It seems a little silly to do a reachabilty check in the case where we trust the user to access absolutely everything in the repository. Also, it's racy in a distributed system -- perhaps one server advertises a ref, but another has since had a force-push to that ref, and perhaps the two HTTP requests end up directed to these different servers. Signed-off-by: David Turner <dturner@twosigma.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-17Git 2.11-rc2v2.11.0-rc2Junio C Hamano
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-17Merge branch 'nd/worktree-lock'Junio C Hamano
Typofix. * nd/worktree-lock: git-worktree.txt: fix typo "to"/"two", and add comma
2016-11-16git-gc.txt: expand discussion of races with other processesMatt McCutchen
In general, "git gc" may delete objects that another concurrent process is using but hasn't created a reference to. Git has some mitigations, but they fall short of a complete solution. Document this in the git-gc(1) man page and add a reference from the documentation of the gc.pruneExpire config variable. Based on a write-up by Jeff King: http://marc.info/?l=git&m=147922960131779&w=2 Signed-off-by: Matt McCutchen <matt@mattmccutchen.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-14doc: mention transfer data leaks in more placesMatt McCutchen
The "SECURITY" section of the gitnamespaces(7) man page described two ways for a client to steal data from a server that wasn't intended to be shared. Similar attacks can be performed by a server on a client, so adapt the section to cover both directions and add it to the git-fetch(1), git-pull(1), and git-push(1) man pages. Also add references to this section from the documentation of server configuration options that attempt to control data leakage but may not be fully effective. Signed-off-by: Matt McCutchen <matt@mattmccutchen.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-14git-worktree.txt: fix typo "to"/"two", and add commaBen North
Signed-off-by: Ben North <ben@redfrontdoor.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-11Git 2.11.0-rc1v2.11.0-rc1Junio C Hamano
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-11Merge branch 'ps/common-info-doc'Junio C Hamano
Doc fix. * ps/common-info-doc: doc: fix location of 'info/' with $GIT_COMMON_DIR
2016-11-11doc: fix location of 'info/' with $GIT_COMMON_DIRPatrick Steinhardt
With the introduction of the $GIT_COMMON_DIR variable, the repository layout manual was changed to reflect the location for many files in case the variable is set. While adding the new locations, one typo snuck in regarding the location of the 'info/' folder, which is falsely claimed to reside at "$GIT_COMMON_DIR/index". Fix the typo to point to "$GIT_COMMON_DIR/info/" instead. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-11-01A bit of updates post -rc0Junio C Hamano
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-31Git 2.11-rc0v2.11.0-rc0Junio C Hamano
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-31Merge branch 'sc/fmt-merge-msg-doc-markup-fix'Junio C Hamano
Documentation fix. * sc/fmt-merge-msg-doc-markup-fix: Documentation/fmt-merge-msg: fix markup in example
2016-10-31Merge branch 'jk/rebase-config-insn-fmt-docfix'Junio C Hamano
Documentation fix. * jk/rebase-config-insn-fmt-docfix: doc: fix missing "::" in config list
2016-10-31Merge branch 'aw/numbered-stash'Junio C Hamano
The user always has to say "stash@{$N}" when naming a single element in the default location of the stash, i.e. reflogs in refs/stash. The "git stash" command learned to accept "git stash apply 4" as a short-hand for "git stash apply stash@{4}". * aw/numbered-stash: stash: allow stashes to be referenced by index only
2016-10-31Merge branch 'jt/trailer-with-cruft'Junio C Hamano
Update "interpret-trailers" machinery and teaches it that people in real world write all sorts of crufts in the "trailer" that was originally designed to have the neat-o "Mail-Header: like thing" and nothing else. * jt/trailer-with-cruft: trailer: support values folded to multiple lines trailer: forbid leading whitespace in trailers trailer: allow non-trailers in trailer block trailer: clarify failure modes in parse_trailer trailer: make args have their own struct trailer: streamline trailer item create and add trailer: use list.h for doubly-linked list trailer: improve const correctness
2016-10-31Merge branch 'ls/filter-process'Junio C Hamano
The smudge/clean filter API expect an external process is spawned to filter the contents for each path that has a filter defined. A new type of "process" filter API has been added to allow the first request to run the filter for a path to spawn a single process, and all filtering need is served by this single process for multiple paths, reducing the process creation overhead. * ls/filter-process: contrib/long-running-filter: add long running filter example convert: add filter.<driver>.process option convert: prepare filter.<driver>.process option convert: make apply_filter() adhere to standard Git error handling pkt-line: add functions to read/write flush terminated packet streams pkt-line: add packet_write_gently() pkt-line: add packet_flush_gently() pkt-line: add packet_write_fmt_gently() pkt-line: extract set_packet_header() pkt-line: rename packet_write() to packet_write_fmt() run-command: add clean_on_exit_handler run-command: move check_pipe() from write_or_die to run_command convert: modernize tests convert: quote filter names in error messages
2016-10-30doc: fix missing "::" in config listJeff King
The rebase.instructionFormat option is missing its "::" to tell AsciiDoc that it's a list entry. As a result, the option name gets lumped into the description in one big paragraph. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-28Sync with 2.10.2Junio C Hamano
2016-10-28Git 2.10.2v2.10.2Junio C Hamano
Signed-off-by: Junio C Hamano <gitster@pobox.com>