summaryrefslogtreecommitdiff
path: root/upload-pack.c
AgeCommit message (Collapse)Author
2020-12-17Merge branch 'tb/partial-clone-filters-fix'Junio C Hamano
Fix potential server side resource deallocation issues when responding to a partial clone request. * tb/partial-clone-filters-fix: upload-pack.c: don't free allowed_filters util pointers builtin/clone.c: don't ignore transport_fetch_refs() errors
2020-12-14Merge branch 'jk/multi-line-indent-style-fix'Junio C Hamano
Style fix. * jk/multi-line-indent-style-fix: style: indent multiline "if" conditions to align
2020-12-14Merge branch 'jk/check-config-parsing-error-in-upload-pack'Junio C Hamano
Tighten error checking in the codepath that responds to "git fetch". * jk/check-config-parsing-error-in-upload-pack: upload-pack: propagate return value from object filter config callback
2020-12-08Merge branch 'js/trace2-session-id'Junio C Hamano
The transport layer was taught to optionally exchange the session ID assigned by the trace2 subsystem during fetch/push transactions. * js/trace2-session-id: receive-pack: log received client session ID send-pack: advertise session ID in capabilities upload-pack, serve: log received client session ID fetch-pack: advertise session ID in capabilities transport: log received server session ID serve: advertise session ID in v2 capabilities receive-pack: advertise session ID in v0 capabilities upload-pack: advertise session ID in v0 capabilities trace2: add a public function for getting the SID docs: new transfer.advertiseSID option docs: new capability to advertise session IDs
2020-12-03upload-pack.c: don't free allowed_filters util pointersTaylor Blau
To keep track of which object filters are allowed or not, 'git upload-pack' stores the name of each filter in a string_list, and sets it ->util pointer to be either 0 or 1, indicating whether it is banned or allowed. Later on, we attempt to clear that list, but we incorrectly ask for the util pointers to be free()'d, too. This behavior (introduced back in 6dd3456a8c (upload-pack.c: allow banning certain object filter(s), 2020-08-03)) leads to an invalid free, and causes us to crash. In order to trigger this, one needs to fetch from a server that (a) has at least one object filter allowed, and (b) issue a fetch that contains a subset of the allowed filters (i.e., we cannot ask for a banned filter, since this causes us to die() before we hit the bogus string_list_clear()). In that case, whatever banned filters exist will cause a noop free() (since those ->util pointers are set to 0), but the first allowed filter we try to free will crash us. We never noticed this in the tests because we didn't have an example of setting 'uploadPackFilter' configuration variables and then following up with a valid fetch. The first new 'git clone' prevents further regression here. For good measure on top, add a test which checks the same behavior at a tree depth greater than 0. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-12-03style: indent multiline "if" conditions to alignJeff King
Commit 6dc905d974 (config: split repo scope to local and worktree, 2020-02-10) made some "if" statements multiline, but didn't indent the second lines in our usual way. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-12-03upload-pack: propagate return value from object filter config callbackJeff King
If we encounter an error in parse_filter_object_config(), we'll complain to stderr but won't actually propagate the return value up the stack. This is unlike most of our config callbacks, which return the error to git_config() so it can die (this includes the call just below us to parse_hide_refs_config(), which can also produce errors). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-12-03Merge branch 'jk/stop-pack-objects-when-fetch-is-killed'Junio C Hamano
"git fetch" that is killed may leave a pack-objects process behind, still computing to find a good compression, wasting cycles. This has been corrected. * jk/stop-pack-objects-when-fetch-is-killed: upload-pack: kill pack-objects helper on signal or exit
2020-12-01upload-pack: kill pack-objects helper on signal or exitJeff King
We spawn an external pack-objects process to actually send objects to the remote side. If we are killed by a signal during this process, then pack-objects may continue to run. As soon as it starts producing output for the pack, it will see a failure writing to upload-pack and exit itself. But before then, it may do significant work traversing the object graph, compressing deltas, etc, which will all be pointless. So let's make sure to kill as soon as we know that the caller will not read the result. There's no test here, since it's inherently racy, but here's an easy reproduction is on a large-ish repo like linux.git: - make sure you don't have pack bitmaps (since they make the enumerating phase go quickly). For linux.git it takes ~30s or so to walk the whole graph on my machine. - run "git clone --no-local -q . dst"; the "-q" is important because if pack-objects is writing progress to upload-pack (to get multiplexed over the sideband to the client), then it will notice pretty quickly the failure to write to stderr - kill the client-side clone process in another terminal (don't use ^C, as that will send SIGINT to all of the processes) - run "ps au | grep git" or similar to observe upload-pack dying within 5 seconds (it will send a keepalive that will notice the client has gone away) - but you'll still see pack-objects consuming 100% CPU (and 1GB+ of RAM) during the traversal and delta compression phases. It will exit as soon as it starts to write the pack (when it will notice that upload-pack went away). With this patch, pack-objects exits as soon as upload-pack does. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-12upload-pack, serve: log received client session IDJosh Steadmon
When upload-pack (protocol v0/v1) or a protocol v2 server receives a session-id capability from a client, log the received session ID via a trace2 data event. Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-11-12upload-pack: advertise session ID in v0 capabilitiesJosh Steadmon
When transfer.advertiseSID is true, advertise upload-pack's session ID via the new session-id capability. Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-10-31upload-pack: allow stateless client EOF just prior to havesDaniel Duvall
During stateless packfile negotiation where a depth is given, stateless RPC clients (e.g. git-remote-curl) will send multiple upload-pack requests with the first containing only the wants/shallows/deepens/filters and the subsequent containing haves/done. When upload-pack handles such requests, entering get_common_commits without checking whether the client has hung up can result in unexpected EOF during the negotiation loop and a die() with message "fatal: the remote end hung up unexpectedly". Real world effects include: - A client speaking to git-http-backend via a server that doesn't check the exit codes of CGIs (e.g. mod_cgi) doesn't know and doesn't care about the fatal. It continues to process the response body as normal. - A client speaking to a server that does check the exit code and returns an errant HTTP status as a result will fail with the message "error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500." - Admins running servers that surface the failure must workaround it by patching code that handles execution of git-http-backend to ignore exit codes or take other heuristic approaches. - Admins may have to deal with "hung up unexpectedly" log spam related to the failures even in cases where the exit code isn't surfaced as an HTTP server-side error status. To avoid these EOF related fatals, have upload-pack gently peek for an EOF between the sending of shallow/unshallow lines (followed by flush) and the reading of client haves. If the client has hung up at this point, exit normally. Signed-off-by: Daniel Duvall <dan@mutual.io> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-24Merge branch 'rs/more-buffered-io'Junio C Hamano
Use more buffered I/O where we used to call many small write(2)s. * rs/more-buffered-io: upload-pack: use buffered I/O to talk to rev-list midx: use buffered I/O to talk to pack-objects connected: use buffered I/O to talk to rev-list
2020-08-19Merge branch 'rs/upload-pack-sigchain-fix'Junio C Hamano
Code clean-up. * rs/upload-pack-sigchain-fix: upload-pack: remove superfluous sigchain_pop() call
2020-08-17upload-pack: use buffered I/O to talk to rev-listRené Scharfe
Like f0bca72dc77 (send-pack: use buffered I/O to talk to pack-objects, 2016-06-08), significantly reduce the number of system calls and simplify the code for sending object IDs to rev-list by using stdio's buffering. Take care to handle errors immediately to get the correct error code, and to flush the buffer explicitly before closing the stream in order to catch any write errors for these last bytes. Helped-by: Chris Torek <chris.torek@gmail.com> Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-12Merge branch 'tb/upload-pack-filters'Junio C Hamano
The component to respond to "git fetch" request is made more configurable to selectively allow or reject object filtering specification used for partial cloning. * tb/upload-pack-filters: t5616: use test_i18ngrep for upload-pack errors upload-pack.c: introduce 'uploadpackfilter.tree.maxDepth' upload-pack.c: allow banning certain object filter(s) list_objects_filter_options: introduce 'list_object_filter_config_name'
2020-08-11upload-pack: remove superfluous sigchain_pop() callRené Scharfe
2997178ee6 (upload-pack: split check_unreachable() in two, prep for get_reachable_list(), 2016-06-12) moved most code of has_unreachable() into the new function do_reachable_revlist(). The latter takes care to ignore SIGPIPE during its operations, and restores the original signal handler before returning. However, a sigchain_pop(SIGPIPE) call remained in the error handling code of has_unreachable(), which does nothing because the stack is empty after do_reachable_revlist() cleaned up after itself. Remove it. Signed-off-by: René Scharfe <l.s.r@web.de> Reviewed-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-10Merge branch 'jk/strvec'Junio C Hamano
The argv_array API is useful for not just managing argv but any "vector" (NULL-terminated array) of strings, and has seen adoption to a certain degree. It has been renamed to "strvec" to reduce the barrier to adoption. * jk/strvec: strvec: rename struct fields strvec: drop argv_array compatibility layer strvec: update documention to avoid argv_array strvec: fix indentation in renamed calls strvec: convert remaining callers away from argv_array name strvec: convert more callers away from argv_array name strvec: convert builtin/ callers away from argv_array name quote: rename sq_dequote_to_argv_array to mention strvec strvec: rename files from argv-array to strvec argv-array: rename to strvec argv-array: use size_t for count and alloc
2020-08-04upload-pack.c: introduce 'uploadpackfilter.tree.maxDepth'Taylor Blau
In b79cf959b2 (upload-pack.c: allow banning certain object filter(s), 2020-02-26), we introduced functionality to disallow certain object filters from being chosen from within 'git upload-pack'. Traditionally, administrators use this functionality to disallow filters that are known to perform slowly, for e.g., those that do not have bitmap-level filtering. In the past, the '--filter=tree:<n>' was one such filter that does not have bitmap-level filtering support, and so was likely to be banned by administrators. However, in the previous couple of commits, we introduced bitmap-level filtering for the case when 'n' is equal to '0', i.e., as if we had a '--filter=tree:none' choice. While it would be sufficient to simply write $ git config uploadpackfilter.tree.allow true (since it would allow all values of 'n'), we would like to be able to allow this filter for certain values of 'n', i.e., those no greater than some pre-specified maximum. In order to do this, introduce a new configuration key, as follows: $ git config uploadpackfilter.tree.maxDepth <m> where '<m>' specifies the maximum allowed value of 'n' in the filter 'tree:n'. Administrators who wish to allow for only the value '0' can write: $ git config uploadpackfilter.tree.allow true $ git config uploadpackfilter.tree.maxDepth 0 which allows '--filter=tree:0', but no other values. Signed-off-by: Taylor Blau <me@ttaylorr.com> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-08-04upload-pack.c: allow banning certain object filter(s)Taylor Blau
Git clients may ask the server for a partial set of objects, where the set of objects being requested is refined by one or more object filters. Server administrators can configure 'git upload-pack' to allow or ban these filters by setting the 'uploadpack.allowFilter' variable to 'true' or 'false', respectively. However, administrators using bitmaps may wish to allow certain kinds of object filters, but ban others. Specifically, they may wish to allow object filters that can be optimized by the use of bitmaps, while rejecting other object filters which aren't and represent a perceived performance degradation (as well as an increased load factor on the server). Allow configuring 'git upload-pack' to support object filters on a case-by-case basis by introducing two new configuration variables: - 'uploadpackfilter.allow' - 'uploadpackfilter.<kind>.allow' where '<kind>' may be one of 'blobNone', 'blobLimit', 'tree', and so on. Setting the second configuration variable for any valid value of '<kind>' explicitly allows or disallows restricting that kind of object filter. If a client requests the object filter <kind> and the respective configuration value is not set, 'git upload-pack' will default to the value of 'uploadpackfilter.allow', which itself defaults to 'true' to maintain backwards compatibility. Note that this differs from 'uploadpack.allowfilter', which controls whether or not the 'filter' capability is advertised. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-31strvec: rename struct fieldsJeff King
The "argc" and "argv" names made sense when the struct was argv_array, but now they're just confusing. Let's rename them to "nr" (which we use for counts elsewhere) and "v" (which is rather terse, but reads well when combined with typical variable names like "args.v"). Note that we have to update all of the callers immediately. Playing tricks with the preprocessor is hard here, because we wouldn't want to rewrite unrelated tokens. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28strvec: fix indentation in renamed callsJeff King
Code which split an argv_array call across multiple lines, like: argv_array_pushl(&args, "one argument", "another argument", "and more", NULL); was recently mechanically renamed to use strvec, which results in mis-matched indentation like: strvec_pushl(&args, "one argument", "another argument", "and more", NULL); Let's fix these up to align the arguments with the opening paren. I did this manually by sifting through the results of: git jump grep 'strvec_.*,$' and liberally applying my editor's auto-format. Most of the changes are of the form shown above, though I also normalized a few that had originally used a single-tab indentation (rather than our usual style of aligning with the open paren). I also rewrapped a couple of obvious cases (e.g., where previously too-long lines became short enough to fit on one), but I wasn't aggressive about it. In cases broken to three or more lines, the grouping of arguments is sometimes meaningful, and it wasn't worth my time or reviewer time to ponder each case individually. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28strvec: convert remaining callers away from argv_array nameJeff King
We eventually want to drop the argv_array name and just use strvec consistently. There's no particular reason we have to do it all at once, or care about interactions between converted and unconverted bits. Because of our preprocessor compat layer, the names are interchangeable to the compiler (so even a definition and declaration using different names is OK). This patch converts all of the remaining files, as the resulting diff is reasonably sized. The conversion was done purely mechanically with: git ls-files '*.c' '*.h' | xargs perl -i -pe ' s/ARGV_ARRAY/STRVEC/g; s/argv_array/strvec/g; ' We'll deal with any indentation/style fallouts separately. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-28strvec: rename files from argv-array to strvecJeff King
This requires updating #include lines across the code-base, but that's all fairly mechanical, and was done with: git ls-files '*.c' '*.h' | xargs perl -i -pe 's/argv-array.h/strvec.h/' Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-16upload-pack: do not lazy-fetch "have" objectsJonathan Tan
When upload-pack receives a request containing "have" hashes, it (among other things) checks if the served repository has the corresponding objects. However, it does not do so with the OBJECT_INFO_SKIP_FETCH_OBJECT flag, so if serving a partial clone, a lazy fetch will be triggered first. This was discovered at $DAYJOB when a user fetched from a partial clone (into another partial clone - although this would also happen if the repo to be fetched into is not a partial clone). Therefore, whenever "have" hashes are checked for existence, pass the OBJECT_INFO_SKIP_FETCH_OBJECT flag. Also add the OBJECT_INFO_QUICK flag to improve performance, as it is typical that such objects do not exist in the serving repo, and the consequences of a false negative are minor (usually, a slightly larger pack sent). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-07-07Merge branch 'bc/sha-256-part-2'Junio C Hamano
SHA-256 migration work continues. * bc/sha-256-part-2: (44 commits) remote-testgit: adapt for object-format bundle: detect hash algorithm when reading refs t5300: pass --object-format to git index-pack t5704: send object-format capability with SHA-256 t5703: use object-format serve option t5702: offer an object-format capability in the test t/helper: initialize the repository for test-sha1-array remote-curl: avoid truncating refs with ls-remote t1050: pass algorithm to index-pack when outside repo builtin/index-pack: add option to specify hash algorithm remote-curl: detect algorithm for dumb HTTP by size builtin/ls-remote: initialize repository based on fetch t5500: make hash independent serve: advertise object-format capability for protocol v2 connect: parse v2 refs with correct hash algorithm connect: pass full packet reader when parsing v2 refs Documentation/technical: document object-format for protocol v2 t1302: expect repo format version 1 for SHA-256 builtin/show-index: provide options to determine hash algo t5302: modernize test formatting ...
2020-06-25Merge branch 'jt/cdn-offload'Junio C Hamano
The "fetch/clone" protocol has been updated to allow the server to instruct the clients to grab pre-packaged packfile(s) in addition to the packed object data coming over the wire. * jt/cdn-offload: upload-pack: fix a sparse '0 as NULL pointer' warning upload-pack: send part of packfile response as uri fetch-pack: support more than one pack lockfile upload-pack: refactor reading of pack-objects out Documentation: add Packfile URIs design doc Documentation: order protocol v2 sections http-fetch: support fetching packfiles by URL http-fetch: refactor into function http: refactor finish_http_pack_request() http: use --stdin when indexing dumb HTTP pack
2020-06-17upload-pack: fix a sparse '0 as NULL pointer' warningRamsay Jones
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: refactor common code into do_got_oid()Christian Couder
As 'upload-pack.c' is now using 'struct upload_pack_data' thoroughly, let's refactor some common code into a new do_got_oid() function. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: move oldest_have to upload_pack_dataChristian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's move the 'oldest_have' static variable into this struct. It is used by both protocol v0 and protocol v2 code. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: pass upload_pack_data to got_oid()Christian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's pass that struct to got_oid(), so that this function can use all the fields of the struct. This will be used in followup commits to move a static variable into 'upload_pack_data'. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: pass upload_pack_data to ok_to_give_up()Christian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's pass that struct to ok_to_give_up(), so that this function can use all the fields of the struct. This will be used in followup commits to move a static variable into 'upload_pack_data'. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: pass upload_pack_data to send_acks()Christian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's pass that struct to send_acks(), so that this function can use all the fields of the struct. This will be used in followup commits to move a static variable into 'upload_pack_data'. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: pass upload_pack_data to process_haves()Christian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's pass that struct to process_haves(), so that this function can use all the fields of the struct. This will be used in followup commits to move a static variable into 'upload_pack_data'. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: change allow_unadvertised_object_request to an enumChristian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's change allow_unadvertised_object_request, which is now part of 'upload_pack_data', from an 'unsigned int' to an enum. This will make it clear which values this variable can take. While at it let's change this variable name to 'allow_uor' to make it shorter. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: move allow_unadvertised_object_request to upload_pack_dataChristian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's move the 'allow_unadvertised_object_request' static variable into this struct. It is used by code common to protocol v0 and protocol v2. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: move extra_edge_obj to upload_pack_dataChristian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's move the 'extra_edge_obj' static variable into this struct. It is used by code common to protocol v0 and protocol v2. While at it let's properly initialize and clear 'extra_edge_obj' in the appropriate 'upload_pack_data' initialization and clearing functions. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: move shallow_nr to upload_pack_dataChristian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's move the 'shallow_nr' static variable into this struct. It is used by code common to protocol v0 and protocol v2. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: pass upload_pack_data to send_unshallow()Christian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's pass that struct to send_unshallow(), so that this function can use all the fields of the struct. This will be used in followup commits to move static variables into 'upload_pack_data'. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: pass upload_pack_data to deepen_by_rev_list()Christian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's pass that struct to deepen_by_rev_list(), so that this function can use all the fields of the struct. This will be used in followup commits to move static variables into 'upload_pack_data'. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: pass upload_pack_data to deepen()Christian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's pass that struct to deepen(), so that this function can use all the fields of the struct. This will be used in followup commits to move static variables into 'upload_pack_data'. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: pass upload_pack_data to send_shallow_list()Christian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's pass that struct to send_shallow_list(), so that this function can use all the fields of the struct. This will be used in followup commits to move static variables into 'upload_pack_data'. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: send part of packfile response as uriJonathan Tan
Teach upload-pack to send part of its packfile response as URIs. An administrator may configure a repository with one or more "uploadpack.blobpackfileuri" lines, each line containing an OID, a pack hash, and a URI. A client may configure fetch.uriprotocols to be a comma-separated list of protocols that it is willing to use to fetch additional packfiles - this list will be sent to the server. Whenever an object with one of those OIDs would appear in the packfile transmitted by upload-pack, the server may exclude that object, and instead send the URI. The client will then download the packs referred to by those URIs before performing the connectivity check. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-11upload-pack: refactor reading of pack-objects outJonathan Tan
Subsequent patches will change how the output of pack-objects is processed, so extract that processing into its own function. Currently, at most 1 character can be buffered (in the "buffered" local variable). One of those patches will require a larger buffer, so replace that "buffered" local variable with a buffer array. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-04upload-pack: move pack_objects_hook to upload_pack_dataChristian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's move the 'pack_objects_hook' static variable into this struct. It is used by code common to protocol v0 and protocol v2. While at it let's also free() it in upload_pack_data_clear(). Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-04upload-pack: move allow_sideband_all to upload_pack_dataChristian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's move the 'allow_sideband_all' static variable into this struct. It is used only by protocol v2 code. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-04upload-pack: move allow_ref_in_want to upload_pack_dataChristian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's move the 'allow_ref_in_want' static variable into this struct. It is used only by protocol v2 code. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-04upload-pack: move allow_filter to upload_pack_dataChristian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's move the 'allow_filter' static variable into this struct. It is used by both protocol v0 and protocol v2 code. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-04upload-pack: move keepalive to upload_pack_dataChristian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's move the 'keepalive' static variable into this struct. It is used by code common to protocol v0 and protocol v2. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-04upload-pack: pass upload_pack_data to upload_pack_config()Christian Couder
As we cleanup 'upload-pack.c' by using 'struct upload_pack_data' more thoroughly, let's pass that struct to upload_pack_config(), so that this function can use all the fields of the struct. This will be used in followup commits to move static variables that are set in upload_pack_config() into 'upload_pack_data'. Signed-off-by: Christian Couder <chriscool@tuxfamily.org> Acked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>