summaryrefslogtreecommitdiff
path: root/http.c
AgeCommit message (Collapse)Author
2016-01-26http: use credential API to handle proxy authenticationKnut Franke
Currently, the only way to pass proxy credentials to curl is by including them in the proxy URL. Usually, this means they will end up on disk unencrypted, one way or another (by inclusion in ~/.gitconfig, shell profile or history). Since proxy authentication often uses a domain user, credentials can be security sensitive; therefore, a safer way of passing credentials is desirable. If the configured proxy contains a username but not a password, query the credential API for one. Also, make sure we approve/reject proxy credentials properly. For consistency reasons, add parsing of http_proxy/https_proxy/all_proxy environment variables, which would otherwise be evaluated as a fallback by curl. Without this, we would have different semantics for git configuration and environment variables. Helped-by: Junio C Hamano <gitster@pobox.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Elia Pinto <gitter.spiros@gmail.com> Signed-off-by: Knut Franke <k.franke@science-computing.de> Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-26http: allow selection of proxy authentication methodKnut Franke
CURLAUTH_ANY does not work with proxies which answer unauthenticated requests with a 307 redirect to an error page instead of a 407 listing supported authentication methods. Therefore, allow the authentication method to be set using the environment variable GIT_HTTP_PROXY_AUTHMETHOD or configuration variables http.proxyAuthmethod and remote.<name>.proxyAuthmethod (in analogy to http.proxy and remote.<name>.proxy). The following values are supported: * anyauth (default) * basic * digest * negotiate * ntlm Signed-off-by: Knut Franke <k.franke@science-computing.de> Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Helped-by: Junio C Hamano <gitster@pobox.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Elia Pinto <gitter.spiros@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-10Merge branch 'bc/object-id'Junio C Hamano
More transition from "unsigned char[40]" to "struct object_id". This needed a few merge fixups, but is mostly disentangled from other topics. * bc/object-id: remote: convert functions to struct object_id Remove get_object_hash. Convert struct object to object_id Add several uses of get_object_hash. object: introduce get_object_hash macro. ref_newer: convert to use struct object_id push_refs_with_export: convert to struct object_id get_remote_heads: convert to struct object_id parse_fetch: convert to use struct object_id add_sought_entry_mem: convert to struct object_id Convert struct ref to use object_id. sha1_file: introduce has_object_file helper.
2015-12-01Merge branch 'dt/http-range'Jeff King
Portability fix for a topic already in 'master'. * dt/http-range: http: fix some printf format warnings
2015-12-01Merge branch 'maint'Jeff King
* maint: http: treat config options sslCAPath and sslCAInfo as paths Documentation/diff: give --word-diff-regex=. example filter-branch: deal with object name vs. pathname ambiguity in tree-filter check-ignore: correct documentation about output git-p4: clean up after p4 submit failure git-p4: work with a detached head git-p4: add option to system() to return subshell status git-p4: add failing test for submit from detached head remote-http(s): support SOCKS proxies t5813: avoid creating urls that break on cygwin Escape Git's exec path in contrib/rerere-train.sh script allow hooks to ignore their standard input stream rebase-i-exec: Allow space in SHELL_PATH Documentation: make environment variable formatting more consistent
2015-12-01Merge branch 'cb/ssl-config-pathnames' into maintJeff King
Allow tilde-expansion in some http config variables. * cb/ssl-config-pathnames: http: treat config options sslCAPath and sslCAInfo as paths
2015-12-01Merge branch 'pt/http-socks-proxy' into maintJeff King
Add support for talking http/https over socks proxy. * pt/http-socks-proxy: remote-http(s): support SOCKS proxies
2015-11-24http: treat config options sslCAPath and sslCAInfo as pathsCharles Bailey
This enables ~ and ~user expansion for these config options. Signed-off-by: Charles Bailey <cbailey32@bloomberg.net> Signed-off-by: Jeff King <peff@peff.net>
2015-11-20Convert struct ref to use object_id.brian m. carlson
Use struct object_id in three fields in struct ref and convert all the necessary places that use it. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Jeff King <peff@peff.net>
2015-11-20remote-http(s): support SOCKS proxiesPat Thoyts
With this patch we properly support SOCKS proxies, configured e.g. like this: git config http.proxy socks5://192.168.67.1:32767 Without this patch, Git mistakenly tries to use SOCKS proxies as if they were HTTP proxies, resulting in a error message like: fatal: unable to access 'http://.../': Proxy CONNECT aborted This patch was required to work behind a faulty AP and scraped from http://stackoverflow.com/questions/15227130/#15228479 and guarded with an appropriate cURL version check by Johannes Schindelin. Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-12http: fix some printf format warningsRamsay Jones
Commit f8117f55 ("http: use off_t to store partial file size", 02-11-2015) changed the type of some variables from long to off_t. Unfortunately, the off_t type is not portable and can be represented by several different actual types (even multiple types on the same platform). This makes it difficult to print an off_t variable in a platform independent way. As a result, this commit causes gcc to issue some printf format warnings on a couple of different platforms. In order to suppress the warnings, change the format specifier to use the PRIuMAX macro and cast the off_t argument to uintmax_t. (See also the http_opt_request_remainder() function, which uses the same solution). Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Jeff King <peff@peff.net>
2015-11-02http: use off_t to store partial file sizeJeff King
When we try to resume transfer of a partially-downloaded object or pack, we fopen() the existing file for append, then use ftell() to get the current position. We use a "long", which can hold only 2GB on a 32-bit system, even though packfiles may be larger than that. Let's switch to using off_t, which should hold any file size our system is capable of storing. We need to use ftello() to get the off_t. This is in POSIX and hopefully available everywhere; if not, we should be able to wrap it by falling back to ftell(), which would presumably return "-1" on such a large file (and we would simply skip resuming in that case). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-02http.c: use CURLOPT_RANGE for range requestsDavid Turner
A HTTP server is permitted to return a non-range response to a HTTP range request (and Apache httpd in fact does this in some cases). While libcurl knows how to correctly handle this (by skipping bytes before and after the requested range), it only turns on this handling if it is aware that a range request is being made. By manually setting the range header instead of using CURLOPT_RANGE, we were hiding the fact that this was a range request from libcurl. This could cause corruption. Signed-off-by: David Turner <dturner@twopensource.com> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-20Merge branch 'jk/war-on-sprintf'Junio C Hamano
Many allocations that is manually counted (correctly) that are followed by strcpy/sprintf have been replaced with a less error prone constructs such as xstrfmt. Macintosh-specific breakage was noticed and corrected in this reroll. * jk/war-on-sprintf: (70 commits) name-rev: use strip_suffix to avoid magic numbers use strbuf_complete to conditionally append slash fsck: use for_each_loose_file_in_objdir Makefile: drop D_INO_IN_DIRENT build knob fsck: drop inode-sorting code convert strncpy to memcpy notes: document length of fanout path with a constant color: add color_set helper for copying raw colors prefer memcpy to strcpy help: clean up kfmclient munging receive-pack: simplify keep_arg computation avoid sprintf and strcpy with flex arrays use alloc_ref rather than hand-allocating "struct ref" color: add overflow checks for parsing colors drop strcpy in favor of raw sha1_to_hex use sha1_to_hex_r() instead of strcpy daemon: use cld->env_array when re-spawning stat_tracking_info: convert to argv_array http-push: use an argv_array for setup_revisions fetch-pack: use argv_array for index-pack / unpack-objects ...
2015-09-29Sync with v2.5.4Junio C Hamano
2015-09-28Sync with 2.4.10Junio C Hamano
2015-09-28Sync with 2.3.10Junio C Hamano
2015-09-25http: limit redirection depthBlake Burkhart
By default, libcurl will follow circular http redirects forever. Let's put a cap on this so that somebody who can trigger an automated fetch of an arbitrary repository (e.g., for CI) cannot convince git to loop infinitely. The value chosen is 20, which is the same default that Firefox uses. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25http: limit redirection to protocol-whitelistBlake Burkhart
Previously, libcurl would follow redirection to any protocol it was compiled for support with. This is desirable to allow redirection from HTTP to HTTPS. However, it would even successfully allow redirection from HTTP to SFTP, a protocol that git does not otherwise support at all. Furthermore git's new protocol-whitelisting could be bypassed by following a redirect within the remote helper, as it was only enforced at transport selection time. This patch limits redirects within libcurl to HTTP, HTTPS, FTP and FTPS. If there is a protocol-whitelist present, this list is limited to those also allowed by the whitelist. As redirection happens from within libcurl, it is impossible for an HTTP redirect to a protocol implemented within another remote helper. When the curl version git was compiled with is too old to support restrictions on protocol redirection, we warn the user if GIT_ALLOW_PROTOCOL restrictions were requested. This is a little inaccurate, as even without that variable in the environment, we would still restrict SFTP, etc, and we do not warn in that case. But anything else means we would literally warn every time git accesses an http remote. This commit includes a test, but it is not as robust as we would hope. It redirects an http request to ftp, and checks that curl complained about the protocol, which means that we are relying on curl's specific error message to know what happened. Ideally we would redirect to a working ftp server and confirm that we can clone without protocol restrictions, and not with them. But we do not have a portable way of providing an ftp server, nor any other protocol that curl supports (https is the closest, but we would have to deal with certificates). [jk: added test and version warning] Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25use strip_suffix and xstrfmt to replace suffixJeff King
When we want to convert "foo.pack" to "foo.idx", we do it by duplicating the original string and then munging the bytes in place. Let's use strip_suffix and xstrfmt instead, which has several advantages: 1. It's more clear what the intent is. 2. It does not implicitly rely on the fact that strlen(".idx") <= strlen(".pack") to avoid an overflow. 3. We communicate the assumption that the input file ends with ".pack" (and get a run-time check that this is so). 4. We drop calls to strcpy, which makes auditing the code base easier. Likewise, we can do this to convert ".pack" to ".bitmap", avoiding some manual memory computation. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25convert trivial sprintf / strcpy calls to xsnprintfJeff King
We sometimes sprintf into fixed-size buffers when we know that the buffer is large enough to fit the input (either because it's a constant, or because it's numeric input that is bounded in size). Likewise with strcpy of constant strings. However, these sites make it hard to audit sprintf and strcpy calls for buffer overflows, as a reader has to cross-reference the size of the array with the input. Let's use xsnprintf instead, which communicates to a reader that we don't expect this to overflow (and catches the mistake in case we do). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-26Merge branch 'ep/http-configure-ssl-version'Junio C Hamano
A new configuration variable http.sslVersion can be used to specify what specific version of SSL/TLS to use to make a connection. * ep/http-configure-ssl-version: http: add support for specifying the SSL version
2015-08-19Merge branch 'jc/finalize-temp-file'Junio C Hamano
Long overdue micro clean-up. * jc/finalize-temp-file: sha1_file.c: rename move_temp_to_file() to finalize_object_file()
2015-08-17http: add support for specifying the SSL versionElia Pinto
Teach git about a new option, "http.sslVersion", which permits one to specify the SSL version to use when negotiating SSL connections. The setting can be overridden by the GIT_SSL_VERSION environment variable. Signed-off-by: Elia Pinto <gitter.spiros@gmail.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-10sha1_file.c: rename move_temp_to_file() to finalize_object_file()Junio C Hamano
Since 5a688fe4 ("core.sharedrepository = 0mode" should set, not loosen, 2009-03-25), we kept reminding ourselves: NEEDSWORK: this should be renamed to finalize_temp_file() as "moving" is only a part of what it does, when no patch between master to pu changes the call sites of this function. without doing anything about it. Let's do so. The purpose of this function was not to move but to finalize. The detail of the primarily implementation of finalizing was to link the temporary file to its final name and then to unlink, which wasn't even "moving". The alternative implementation did "move" by calling rename(2), which is a fun tangent. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-07-13Merge branch 'et/http-proxyauth'Junio C Hamano
We used to ask libCURL to use the most secure authentication method available when talking to an HTTP proxy only when we were told to talk to one via configuration variables. We now ask libCURL to always use the most secure authentication method, because the user can tell libCURL to use an HTTP proxy via an environment variable without using configuration variables. * et/http-proxyauth: http: always use any proxy auth method available
2015-06-29http: always use any proxy auth method availableEnrique Tobis
We set CURLOPT_PROXYAUTH to use the most secure authentication method available only when the user has set configuration variables to specify a proxy. However, libcurl also supports specifying a proxy through environment variables. In that case libcurl defaults to only using the Basic proxy authentication method, because we do not use CURLOPT_PROXYAUTH. Set CURLOPT_PROXYAUTH to always use the most secure authentication method available, even when there is no git configuration telling us to use a proxy. This allows the user to use environment variables to configure a proxy that requires an authentication method different from Basic. Signed-off-by: Enrique A. Tobis <etobis@twosigma.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-22Merge branch 'ls/http-ssl-cipher-list'Junio C Hamano
Introduce http.<url>.SSLCipherList configuration variable to tweak the list of cipher suite to be used with libcURL when talking with https:// sites. * ls/http-ssl-cipher-list: http: add support for specifying an SSL cipher list
2015-05-08http: add support for specifying an SSL cipher listLars Kellogg-Stedman
Teach git about a new option, "http.sslCipherList", which permits one to specify a list of ciphers to use when negotiating SSL connections. The setting can be overwridden by the GIT_SSL_CIPHER_LIST environment variable. Signed-off-by: Lars Kellogg-Stedman <lars@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-24http: release the memory of a http pack request as wellStefan Beller
The cleanup function is used in 4 places now and it's always safe to free up the memory as well. Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-03-06Merge branch 'ye/http-accept-language'Junio C Hamano
Compilation fix for a recent topic in 'master'. * ye/http-accept-language: gettext.c: move get_preferred_languages() from http.c
2015-02-26gettext.c: move get_preferred_languages() from http.cJeff King
Calling setlocale(LC_MESSAGES, ...) directly from http.c, without including <locale.h>, was causing compilation warnings. Move the helper function to gettext.c that already includes the header and where locale-related issues are handled. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-02-25Merge branch 'tc/missing-http-proxyauth'Junio C Hamano
We did not check the curl library version before using CURLOPT_PROXYAUTH feature that may not exist. * tc/missing-http-proxyauth: http: support curl < 7.10.7
2015-02-25Merge branch 'jk/dumb-http-idx-fetch-fix' into maintJunio C Hamano
A broken pack .idx file in the receiving repository prevented the dumb http transport from fetching a good copy of it from the other side. * jk/dumb-http-idx-fetch-fix: dumb-http: do not pass NULL path to parse_pack_index
2015-02-18Merge branch 'ye/http-accept-language'Junio C Hamano
Using environment variable LANGUAGE and friends on the client side, HTTP-based transports now send Accept-Language when making requests. * ye/http-accept-language: http: add Accept-Language header if possible
2015-02-17Merge branch 'jk/dumb-http-idx-fetch-fix'Junio C Hamano
A broken pack .idx file in the receiving repository prevented the dumb http transport from fetching a good copy of it from the other side. * jk/dumb-http-idx-fetch-fix: dumb-http: do not pass NULL path to parse_pack_index
2015-02-11Merge branch 'jc/unused-symbols'Junio C Hamano
Mark file-local symbols as "static", and drop functions that nobody uses. * jc/unused-symbols: shallow.c: make check_shallow_file_for_update() static remote.c: make clear_cas_option() static urlmatch.c: make match_urls() static revision.c: make save_parents() and free_saved_parents() static line-log.c: make line_log_data_init() static pack-bitmap.c: make pack_bitmap_filename() static prompt.c: remove git_getpass() nobody uses http.c: make finish_active_slot() and handle_curl_result() static
2015-02-03http: support curl < 7.10.7Tom G. Christensen
Commit dd61399 introduced support for http proxies that require authentication but it relies on the CURL_PROXYAUTH option which was added in curl 7.10.7. This makes sure proxy authentication is only enabled if libcurl can support it. Signed-off-by: Tom G. Christensen <tgc@statsbiblioteket.dk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-01-28http: add Accept-Language header if possibleYi EungJun
Add an Accept-Language header which indicates the user's preferred languages defined by $LANGUAGE, $LC_ALL, $LC_MESSAGES and $LANG. Examples: LANGUAGE= -> "" LANGUAGE=ko:en -> "Accept-Language: ko, en;q=0.9, *;q=0.1" LANGUAGE=ko LANG=en_US.UTF-8 -> "Accept-Language: ko, *;q=0.1" LANGUAGE= LANG=en_US.UTF-8 -> "Accept-Language: en-US, *;q=0.1" This gives git servers a chance to display remote error messages in the user's preferred language. Limit the number of languages to 1,000 because q-value must not be smaller than 0.001, and limit the length of Accept-Language header to 4,000 bytes for some HTTP servers which cannot accept such long header. Signed-off-by: Yi EungJun <eungjun.yi@navercorp.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-01-27dumb-http: do not pass NULL path to parse_pack_indexJeff King
Once upon a time, dumb http always fetched .idx files directly into their final location, and then checked their validity with parse_pack_index. This was refactored in commit 750ef42 (http-fetch: Use temporary files for pack-*.idx until verified, 2010-04-19), which uses the following logic: 1. If we have the idx already in place, see if it's valid (using parse_pack_index). If so, use it. 2. Otherwise, fetch the .idx to a tempfile, check that, and if so move it into place. 3. Either way, fetch the pack itself if necessary. However, it got step 1 wrong. We pass a NULL path parameter to parse_pack_index, so an existing .idx file always looks broken. Worse, we do not treat this broken .idx as an opportunity to re-fetch, but instead return an error, ignoring the pack entirely. This can lead to a dumb-http fetch failing to retrieve the necessary objects. This doesn't come up much in practice, because it must be a packfile that we found out about (and whose .idx we stored) during an earlier dumb-http fetch, but whose packfile we _didn't_ fetch. I.e., we did a partial clone of a repository, didn't need some packfiles, and now a followup fetch needs them. Discovery and tests by Charles Bailey <charles@hashpling.org>. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-01-15http.c: make finish_active_slot() and handle_curl_result() staticJunio C Hamano
They used to be used directly by remote-curl.c for the smart-http protocol. But they got wrapped by run_one_slot() in beed336 (http: never use curl_easy_perform, 2014-02-18). Any future users are expected to follow that route. Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-01-08remote-curl: fall back to Basic auth if Negotiate failsbrian m. carlson
Apache servers using mod_auth_kerb can be configured to allow the user to authenticate either using Negotiate (using the Kerberos ticket) or Basic authentication (using the Kerberos password). Often, one will want to use Negotiate authentication if it is available, but fall back to Basic authentication if the ticket is missing or expired. However, libcurl will try very hard to use something other than Basic auth, even over HTTPS. If Basic and something else are offered, libcurl will never attempt to use Basic, even if the other option fails. Teach the HTTP client code to stop trying authentication mechanisms that don't use a password (currently Negotiate) after the first failure, since if they failed the first time, they will never succeed. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-10-14Merge branch 'da/include-compat-util-first-in-c'Junio C Hamano
Code clean-up. * da/include-compat-util-first-in-c: cleanups: ensure that git-compat-util.h is included first
2014-09-15cleanups: ensure that git-compat-util.h is included firstDavid Aguilar
CodingGuidelines states that the first #include in C files should be git-compat-util.h or another header file that includes it, such as cache.h or builtin.h. Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-09-11Merge branch 'br/http-init-fix'Junio C Hamano
Code clean-up. * br/http-init-fix: http: style fixes for curl_multi_init error check http.c: die if curl_*_init fails
2014-08-21http: style fixes for curl_multi_init error checkJeff King
Unless there is a good reason, we should use die() rather than fprintf/exit. We can also shorten the message to match other curl init failures (and match our usual lowercase no-full-stop style). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-20run-command: introduce CHILD_PROCESS_INITRené Scharfe
Most struct child_process variables are cleared using memset first after declaration. Provide a macro, CHILD_PROCESS_INIT, that can be used to initialize them statically instead. That's shorter, doesn't require a function call and is slightly more readable (especially given that we already have STRBUF_INIT, ARGV_ARRAY_INIT etc.). Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-18http.c: die if curl_*_init failsBernhard Reiter
Signed-off-by: Bernhard Reiter <ockham@raz.or.at> Reviewed-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-09Merge branch 'jk/skip-prefix'Junio C Hamano
* jk/skip-prefix: http-push: refactor parsing of remote object names imap-send: use skip_prefix instead of using magic numbers use skip_prefix to avoid repeated calculations git: avoid magic number with skip_prefix fetch-pack: refactor parsing in get_ack fast-import: refactor parsing of spaces stat_opt: check extra strlen call daemon: use skip_prefix to avoid magic numbers fast-import: use skip_prefix for parsing input use skip_prefix to avoid repeating strings use skip_prefix to avoid magic numbers transport-helper: avoid reading past end-of-string fast-import: fix read of uninitialized argv memory apply: use skip_prefix instead of raw addition refactor skip_prefix to return a boolean avoid using skip_prefix as a boolean daemon: mark some strings as const parse_diff_color_slot: drop ofs parameter
2014-06-20use skip_prefix to avoid repeated calculationsJeff King
In some cases, we use starts_with to check for a prefix, and then use an already-calculated prefix length to advance a pointer past the prefix. There are no magic numbers or duplicated strings here, but we can still make the code simpler and more obvious by using skip_prefix. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>