path: root/t/helper/test-tool.h
AgeCommit message (Collapse)Author
7 hoursMerge branch 'jh/builtin-fsmonitor-part2' (early part) into seenJunio C Hamano
* 'jh/builtin-fsmonitor-part2' (early part): (29 commits) t7527: test status with untracked-cache and fsmonitor--daemon fsmonitor: force update index after large responses fsmonitor--daemon: use a cookie file to sync with file system fsmonitor--daemon: periodically truncate list of modified files t/perf/p7519: add fsmonitor--daemon test cases t/perf/p7519: speed up test on Windows t/helper/test-chmtime: skip directories on Windows t/perf: avoid copying builtin fsmonitor files into test repo t7527: create test for fsmonitor--daemon t/helper/fsmonitor-client: create IPC client to talk to FSMonitor Daemon help: include fsmonitor--daemon feature flag in version info fsmonitor--daemon: implement handle_client callback compat/fsmonitor/fsm-listen-darwin: implement FSEvent listener on MacOS compat/fsmonitor/fsm-listen-darwin: add macos header files for FSEvent compat/fsmonitor/fsm-listen-win32: implement FSMonitor backend on Windows fsmonitor--daemon: create token-based changed path cache fsmonitor--daemon: define token-ids fsmonitor--daemon: add pathname classification fsmonitor--daemon: implement 'start' command fsmonitor--daemon: implement 'run' command ...
3 dayswrapper: add a helper to generate numbers from a CSPRNGbrian m. carlson
There are many situations in which having access to a cryptographically secure pseudorandom number generator (CSPRNG) is helpful. In the future, we'll encounter one of these when dealing with temporary files. To make this possible, let's add a function which reads from a system CSPRNG and returns some bytes. We know that all systems will have such an interface. A CSPRNG is required for a secure TLS or SSH implementation and a Git implementation which provided neither would be of little practical use. In addition, POSIX is set to standardize getentropy(2) in the next version, so in the (potentially distant) future we can rely on that. For systems which lack one of the other interfaces, we provide the ability to use OpenSSL's CSPRNG. OpenSSL is highly portable and functions on practically every known OS, and we know it will have access to some source of cryptographically secure randomness. We also provide support for the arc4random in libbsd for folks who would prefer to use that. Because this is a security sensitive interface, we take some precautions. We either succeed by filling the buffer completely as we requested, or we fail. We don't return partial data because the caller will almost never find that to be a useful behavior. Specify a makefile knob which users can use to specify one or more suitable CSPRNGs, and turn the multiple string options into a set of defines, since we cannot match on strings in the preprocessor. We allow multiple options to make the job of handling this in autoconf easier. The order of options is important here. On systems with arc4random, which is most of the BSDs, we use that, since, except on MirBSD and macOS, it uses ChaCha20, which is extremely fast, and sits entirely in userspace, avoiding a system call. We then prefer getrandom over getentropy, because the former has been available longer on Linux, and then OpenSSL. Finally, if none of those are available, we use /dev/urandom, because most Unix-like operating systems provide that API. We prefer options that don't involve device files when possible because those work in some restricted environments where device files may not be available. Set the configuration variables appropriately for Linux and the BSDs, including macOS, as well as Windows and NonStop. We specifically only consider versions which receive publicly available security support here. For the same reason, we don't specify getrandom(2) on Linux, because CentOS 7 doesn't support it in glibc (although its kernel does) and we don't want to resort to making syscalls. Finally, add a test helper to allow this to be tested by hand and in tests. We don't add any tests, since invoking the CSPRNG is not likely to produce interesting, reproducible results. Signed-off-by: brian m. carlson <> Signed-off-by: Junio C Hamano <>
2021-10-21t/helper/fsmonitor-client: create IPC client to talk to FSMonitor DaemonJeff Hostetler
Create an IPC client to send query and flush commands to the daemon. Signed-off-by: Jeff Hostetler <> Signed-off-by: Junio C Hamano <>
2021-10-08Add "test-tool dump-reftable" command.Han-Wen Nienhuys
This command dumps individual tables or a stack of of tables. Signed-off-by: Han-Wen Nienhuys <> Signed-off-by: Junio C Hamano <>
2021-10-08reftable: utility functionsHan-Wen Nienhuys
This commit provides basic utility classes for the reftable library. Signed-off-by: Han-Wen Nienhuys <> Helped-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2021-08-04Merge branch 'ab/getcwd-test'Junio C Hamano
Portability test update. * ab/getcwd-test: t0001: fix broken not-quite getcwd(3) test in bed67874e2
2021-07-30t0001: fix broken not-quite getcwd(3) test in bed67874e2Ævar Arnfjörð Bjarmason
With a54e938e5b (strbuf: support long paths w/o read rights in strbuf_getcwd() on FreeBSD, 2017-03-26) we had t0001 break on systems like OpenBSD and AIX whose getcwd(3) has standard (but not like glibc et al) behavior. This was partially fixed in bed67874e2 (t0001: skip test with restrictive permissions if getpwd(3) respects them, 2017-08-07). The problem with that fix is that while its analysis of the problem is correct, it doesn't actually call getcwd(3), instead it invokes "pwd -P". There is no guarantee that "pwd -P" is going to call getcwd(3), as opposed to e.g. being a shell built-in. On AIX under both bash and ksh this test breaks because "pwd -P" will happily display the current working directory, but getcwd(3) called by the "git init" we're testing here will fail to get it. I checked whether clobbering the $PWD environment variable would affect it, and it didn't. Presumably these shells keep track of their working directory internally. There's possible follow-up work here in teaching strbuf_getcwd() to get the working directory with whatever method "pwd" uses on these platforms. See [1] for a discussion of that, but let's take the easy way out here and just skip these tests by fixing the GETCWD_IGNORES_PERMS prerequisite to match the limitations of strbuf_getcwd(). 1. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2021-07-28Merge branch 'ew/many-alternate-optim'Junio C Hamano
Optimization for repositories with many alternate object store. * ew/many-alternate-optim: oidtree: a crit-bit tree for odb_loose_cache oidcpy_with_padding: constify `src' arg make object_directory.loose_objects_subdir_seen a bitmap avoid strlen via strbuf_addstr in link_alt_odb_entry speed up alt_odb_usable() with many alternates
2021-07-08oidtree: a crit-bit tree for odb_loose_cacheEric Wong
This saves 8K per `struct object_directory', meaning it saves around 800MB in my case involving 100K alternates (half or more of those alternates are unlikely to hold loose objects). This is implemented in two parts: a generic, allocation-free `cbtree' and the `oidtree' wrapper on top of it. The latter provides allocation using alloc_state as a memory pool to improve locality and reduce free(3) overhead. Unlike oid-array, the crit-bit tree does not require sorting. Performance is bound by the key length, for oidtree that is fixed at sizeof(struct object_id). There's no need to have 256 oidtrees to mitigate the O(n log n) overhead like we did with oid-array. Being a prefix trie, it is natively suited for expanding short object IDs via prefix-limited iteration in `find_short_object_filename'. On my busy workstation, p4205 performance seems to be roughly unchanged (+/-8%). Startup with 100K total alternates with no loose objects seems around 10-20% faster on a hot cache. (800MB in memory savings means more memory for the kernel FS cache). The generic cbtree implementation does impose some extra overhead for oidtree in that it uses memcmp(3) on "struct object_id" so it wastes cycles comparing 12 extra bytes on SHA-1 repositories. I've not yet explored reducing this overhead, but I expect there are many places in our code base where we'd want to investigate this. More information on crit-bit trees: Signed-off-by: Eric Wong <> Signed-off-by: Junio C Hamano <>
2021-06-28promisor-remote: teach lazy-fetch in any repoJonathan Tan
This is one step towards supporting partial clone submodules. Even after this patch, we will still lack partial clone submodules support, primarily because a lot of Git code that accesses submodule objects does so by adding their object stores as alternates, meaning that any lazy fetches that would occur in the submodule would be done based on the config of the superproject, not of the submodule. This also prevents testing of the functionality in this patch by user-facing commands. So for now, test this mechanism using a test helper. Besides that, there is some code that uses the wrapper functions like has_promisor_remote(). Those will need to be checked to see if they could support the non-wrapper functions instead (and thus support any repository, not just the_repository). Signed-off-by: Jonathan Tan <> Signed-off-by: Junio C Hamano <> Reviewed-by: Elijah Newren <> Signed-off-by: Junio C Hamano <>
2021-04-21Merge branch 'ab/userdiff-tests'Junio C Hamano
A bit of code clean-up and a lot of test clean-up around userdiff area. * ab/userdiff-tests: blame tests: simplify userdiff driver test blame tests: don't rely on t/t4018/ directory userdiff: remove support for "broken" tests userdiff tests: list builtin drivers via test-tool userdiff tests: explicitly test "default" pattern userdiff: add and use for_each_userdiff_driver() userdiff style: normalize pascal regex declaration userdiff style: declare patterns with consistent style userdiff style: re-order drivers in alphabetical order
2021-04-13Merge branch 'tb/pack-preferred-tips-to-give-bitmap'Junio C Hamano
A configuration variable has been added to force tips of certain refs to be given a reachability bitmap. * tb/pack-preferred-tips-to-give-bitmap: builtin/pack-objects.c: respect 'pack.preferBitmapTips' t/helper/test-bitmap.c: initial commit pack-bitmap: add 'test_bitmap_commits()' helper
2021-04-08userdiff tests: list builtin drivers via test-toolÆvar Arnfjörð Bjarmason
Change the userdiff test to list the builtin drivers via the test-tool, using the new for_each_userdiff_driver() API function. This gets rid of the need to modify this part of the test every time a new pattern is added, see 2ff6c34612 (userdiff: support Bash, 2020-10-22) and 09dad9256a (userdiff: support Markdown, 2020-05-02) for two recent examples. I only need the "list-builtin-drivers "argument here, but let's add "list-custom-drivers" and "list-drivers" too, just because it's easy. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2021-04-02Merge branch 'jh/simple-ipc'Junio C Hamano
A simple IPC interface gets introduced to build services like fsmonitor on top. * jh/simple-ipc: t0052: add simple-ipc tests and t/helper/test-simple-ipc tool simple-ipc: add Unix domain socket implementation unix-stream-server: create unix domain socket under lock unix-socket: disallow chdir() when creating unix domain sockets unix-socket: add backlog size option to unix_stream_listen() unix-socket: eliminate static unix_stream_socket() helper function simple-ipc: add win32 implementation simple-ipc: design documentation for new IPC mechanism pkt-line: add options argument to read_packetized_to_strbuf() pkt-line: add PACKET_READ_GENTLE_ON_READ_ERROR option pkt-line: do not issue flush packets in write_packetized_*() pkt-line: eliminate the need for static buffer in packet_write_gently()
2021-04-01t/helper/test-bitmap.c: initial commitTaylor Blau
Add a new 'bitmap' test-tool which can be used to list the commits that have received bitmaps. In theory, a determined tester could run 'git rev-list --test-bitmap <commit>' to check if '<commit>' received a bitmap or not, since '--test-bitmap' exits with a non-zero code when it can't find the requested commit. But this is a dubious behavior to rely on, since arguably 'git rev-list' could continue its object walk outside of which commits are covered by bitmaps. This will be used to test the behavior of 'pack.preferBitmapTips', which will be added in the following patch. Signed-off-by: Taylor Blau <> Signed-off-by: Junio C Hamano <>
2021-03-22t0052: add simple-ipc tests and t/helper/test-simple-ipc toolJeff Hostetler
Create with unit tests for the "simple-ipc" mechanism. Create t/helper/test-simple-ipc test tool to exercise the "simple-ipc" functions. When the tool is invoked with "run-daemon", it runs a server to listen for "simple-ipc" connections on a test socket or named pipe and responds to a set of commands to exercise/stress the communication setup. When the tool is invoked with "start-daemon", it spawns a "run-daemon" command in the background and waits for the server to become ready before exiting. (This helps make unit tests in t0052 more predictable and avoids the need for arbitrary sleeps in the test script.) The tool also has a series of client "send" commands to send commands and data to a server instance. Signed-off-by: Jeff Hostetler <> Signed-off-by: Junio C Hamano <>
2021-01-25grep/pcre2: better support invalid UTF-8 haystacksÆvar Arnfjörð Bjarmason
Improve the support for invalid UTF-8 haystacks given a non-ASCII needle when using the PCREv2 backend. This is a more complete fix for a bug I started to fix in 870eea8166 (grep: do not enter PCRE2_UTF mode on fixed matching, 2019-07-26), now that PCREv2 has the PCRE2_MATCH_INVALID_UTF mode we can make use of it. This fixes the sort of case described in 8a5999838e (grep: stess test PCRE v2 on invalid UTF-8 data, 2019-07-26), i.e.: - The subject string is non-ASCII (e.g. "ævar") - We're under a is_utf8_locale(), e.g. "en_US.UTF-8", not "C" - We are using --ignore-case, or we're a non-fixed pattern If those conditions were satisfied and we matched found non-valid UTF-8 data PCREv2 might bark on it, in practice this only happened under the JIT backend (turned on by default on most platforms). Ultimately this fixes a "regression" in b65abcafc7 ("grep: use PCRE v2 for optimized fixed-string search", 2019-07-01), I'm putting that in scare-quotes because before then we wouldn't properly support these complex case-folding, locale etc. cases either, it just broke in different ways. There was a bug related to this the PCRE2_NO_START_OPTIMIZE flag fixed in PCREv2 10.36. It can be worked around by setting the PCRE2_NO_START_OPTIMIZE flag. Let's do that in those cases, and add tests for the bug. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
2020-11-18Merge branch 'en/merge-ort-api-null-impl'Junio C Hamano
Preparation for a new merge strategy. * en/merge-ort-api-null-impl: merge,rebase,revert: select ort or recursive by config or environment fast-rebase: demonstrate merge-ort's API via new test-tool command merge-ort-wrappers: new convience wrappers to mimic the old merge API merge-ort: barebones API of new merge strategy with empty implementation
2020-11-18Merge branch 'ds/maintenance-part-3'Junio C Hamano
Parts of "git maintenance" to ease writing crontab entries (and other scheduling system configuration) for it. * ds/maintenance-part-3: maintenance: add troubleshooting guide to docs maintenance: use 'incremental' strategy by default maintenance: create maintenance.strategy config maintenance: add start/stop subcommands maintenance: add [un]register subcommands for-each-repo: run subcommands on configured repos maintenance: add --schedule option and config maintenance: optionally skip --auto process
2020-10-29fast-rebase: demonstrate merge-ort's API via new test-tool commandElijah Newren
Add a new test-tool command named 'fast-rebase', which is a super-slimmed down and nowhere near as capable version of 'git rebase'. 'test-tool fast-rebase' is not currently planned for usage in the testsuite, but is here for two purposes: 1) Demonstrate the desired API of merge-ort. In particular, fast-rebase takes advantage of the separation of the merging operation from the updating of the index and working tree, to allow it to pick N commits, but only update the index and working tree once at the end. Look for the calls to merge_incore_nonrecursive() and merge_switch_to_result(). 2) Provide a convenient benchmark that isn't polluted by the heavy disk writing and forking of unnecessary processes that comes from sequencer.c and merge-recursive.c. fast-rebase is not meant to replace sequencer.c, just give ideas on how sequencer.c can be changed. Updating sequencer.c with these goals is probably a large amount of work; writing a simple targeted command with no documentation, less-than-useful help messages, numerous limitations in terms of flags it can accept and situations it can handle, and which is flagged off from users is a much easier interim step. Signed-off-by: Elijah Newren <> Signed-off-by: Junio C Hamano <>
2020-09-25maintenance: add start/stop subcommandsDerrick Stolee
Add new subcommands to 'git maintenance' that start or stop background maintenance using 'cron', when available. This integration is as simple as I could make it, barring some implementation complications. The schedule is laid out as follows: 0 1-23 * * * $cmd maintenance run --schedule=hourly 0 0 * * 1-6 $cmd maintenance run --schedule=daily 0 0 * * 0 $cmd maintenance run --schedule=weekly where $cmd is a properly-qualified 'git for-each-repo' execution: $cmd=$path/git --exec-path=$path for-each-repo --config=maintenance.repo where $path points to the location of the Git executable running 'git maintenance start'. This is critical for systems with multiple versions of Git. Specifically, macOS has a system version at '/usr/bin/git' while the version that users can install resides at '/usr/local/bin/git' (symlinked to '/usr/local/libexec/git-core/git'). This will also use your locally-built version if you build and run this in your development environment without installing first. This conditional schedule avoids having cron launch multiple 'git for-each-repo' commands in parallel. Such parallel commands would likely lead to the 'hourly' and 'daily' tasks competing over the object database lock. This could lead to to some tasks never being run! Since the --schedule=<frequency> argument will run all tasks with _at least_ the given frequency, the daily runs will also run the hourly tasks. Similarly, the weekly runs will also run the daily and hourly tasks. The GIT_TEST_CRONTAB environment variable is not intended for users to edit, but instead as a way to mock the 'crontab [-l]' command. This variable is set in to avoid a future test from accidentally running anything with the cron integration from modifying the user's schedule. We use GIT_TEST_CRONTAB='test-tool crontab <file>' in our tests to check how the schedule is modified in 'git maintenance (start|stop)' commands. Signed-off-by: Derrick Stolee <> Signed-off-by: Junio C Hamano <>
2020-08-27receive-pack: add new proc-receive hookJiang Xin
Git calls an internal `execute_commands` function to handle commands sent from client to `git-receive-pack`. Regardless of what references the user pushes, git creates or updates the corresponding references if the user has write-permission. A contributor who has no write-permission, cannot push to the repository directly. So, the contributor has to write commits to an alternate location, and sends pull request by emails or by other ways. We call this workflow as a distributed workflow. It would be more convenient to work in a centralized workflow like what Gerrit provided for some cases. For example, a read-only user who cannot push to a branch directly can run the following `git push` command to push commits to a pseudo reference (has a prefix "refs/for/", not "refs/heads/") to create a code review. git push origin \ HEAD:refs/for/<branch-name>/<session> The `<branch-name>` in the above example can be as simple as "master", or a more complicated branch name like "foo/bar". The `<session>` in the above example command can be the local branch name of the client side, such as "my/topic". We cannot implement a centralized workflow elegantly by using "pre-receive" + "post-receive", because Git will call the internal function "execute_commands" to create references (even the special pseudo reference) between these two hooks. Even though we can delete the temporarily created pseudo reference via the "post-receive" hook, having a temporary reference is not safe for concurrent pushes. So, add a filter and a new handler to support this kind of workflow. The filter will check the prefix of the reference name, and if the command has a special reference name, the filter will turn a specific field (`run_proc_receive`) on for the command. Commands with this filed turned on will be executed by a new handler (a hook named "proc-receive") instead of the internal `execute_commands` function. We can use this "proc-receive" command to create pull requests or send emails for code review. Suggested by Junio, this "proc-receive" hook reads the commands, push-options (optional), and send result using a protocol in pkt-line format. In the following example, the letter "S" stands for "receive-pack" and letter "H" stands for the hook. # Version and features negotiation. S: PKT-LINE(version=1\0push-options atomic...) S: flush-pkt H: PKT-LINE(version=1\0push-options...) H: flush-pkt # Send commands from server to the hook. S: PKT-LINE(<old-oid> <new-oid> <ref>) S: ... ... S: flush-pkt # Send push-options only if the 'push-options' feature is enabled. S: PKT-LINE(push-option) S: ... ... S: flush-pkt # Receive result from the hook. # OK, run this command successfully. H: PKT-LINE(ok <ref>) # NO, I reject it. H: PKT-LINE(ng <ref> <reason>) # Fall through, let 'receive-pack' to execute it. H: PKT-LINE(ok <ref>) H: PKT-LINE(option fall-through) # OK, but has an alternate reference. The alternate reference name # and other status can be given in options H: PKT-LINE(ok <ref>) H: PKT-LINE(option refname <refname>) H: PKT-LINE(option old-oid <old-oid>) H: PKT-LINE(option new-oid <new-oid>) H: PKT-LINE(option forced-update) H: ... ... H: flush-pkt After receiving a command, the hook will execute the command, and may create/update different reference. For example, a command for a pseudo reference "refs/for/master/topic" may create/update different reference such as "refs/pull/123/head". The alternate reference name and other status are given in option lines. The list of commands returned from "proc-receive" will replace the relevant commands that are sent from user to "receive-pack", and "receive-pack" will continue to run the "execute_commands" function and other routines. Finally, the result of the execution of these commands will be reported to end user. The reporting function from "receive-pack" to "send-pack" will be extended in latter commit just like what the "proc-receive" hook reports to "receive-pack". Signed-off-by: Jiang Xin <> Signed-off-by: Junio C Hamano <>
2020-05-01Merge branch 'gs/commit-graph-path-filter'Junio C Hamano
Introduce an extension to the commit-graph to make it efficient to check for the paths that were modified at each commit using Bloom filters. * gs/commit-graph-path-filter: bloom: ignore renames when computing changed paths commit-graph: add GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS test flag t4216: add end to end tests for git log with Bloom filters revision.c: add trace2 stats around Bloom filter usage revision.c: use Bloom filters to speed up path based revision walks commit-graph: add --changed-paths option to write subcommand commit-graph: reuse existing Bloom filters during write commit-graph: write Bloom filters to commit graph file commit-graph: examine commits by generation number commit-graph: examine changed-path objects in pack order commit-graph: compute Bloom filters for changed paths diff: halt tree-diff early after max_changes bloom.c: core Bloom filter implementation for changed paths. bloom.c: introduce core Bloom filter constructs bloom.c: add the murmur3 hash implementation commit-graph: define and use MAX_NUM_CHUNKS
2020-04-22Merge branch 'jk/oid-array-cleanups'Junio C Hamano
Code cleanup. * jk/oid-array-cleanups: oidset: stop referring to sha1-array ref-filter: stop referring to "sha1 array" bisect: stop referring to sha1_array test-tool: rename sha1-array to oid-array oid_array: rename source file from sha1-array oid_array: use size_t for iteration oid_array: use size_t for count and allocation
2020-03-30test-tool: rename sha1-array to oid-arrayJeff King
This matches the actual data structure name, as well as the source file that contains the code we're testing. The test scripts need updating to use the new name, as well. Signed-off-by: Jeff King <> Signed-off-by: Junio C Hamano <>
2020-03-30bloom.c: add the murmur3 hash implementationGarima Singh
In preparation for computing changed paths Bloom filters, implement the Murmur3 hash algorithm as described in [1]. It hashes the given data using the given seed and produces a uniformly distributed hash value. [1] Helped-by: Derrick Stolee <> Helped-by: Szeder Gábor <> Reviewed-by: Jakub Narębski <> Signed-off-by: Garima Singh <> Signed-off-by: Junio C Hamano <>
2020-03-25Merge branch 'hw/advise-ng'Junio C Hamano
Revamping of the advise API to allow more systematic enumeration of advice knobs in the future. * hw/advise-ng: tag: use new advice API to check visibility advice: revamp advise API advice: change "setupStreamFailure" to "setUpstreamFailure" advice: extract vadvise() from advise()
2020-03-05advice: revamp advise APIHeba Waly
Currently it's very easy for the advice library's callers to miss checking the visibility step before printing an advice. Also, it makes more sense for this step to be handled by the advice library. Add a new advise_if_enabled function that checks the visibility of advice messages before printing. Add a new helper advise_enabled to check the visibility of the advice if the caller needs to carry out complicated processing based on that value. A list of advice_settings is added to cache the config variables names and values, it's intended to replace advice_config[] and the global variables once we migrate all the callers to use the new APIs. Signed-off-by: Heba Waly <> Signed-off-by: Junio C Hamano <>
2020-01-15t: directly test parse_pathspec_file()Alexandr Miloslavskiy
Previously, `parse_pathspec_file()` was tested indirectly by invoking git commands with properly crafted inputs. As demonstrated by the previous bugfix, testing complicated black boxes indirectly can lead to tests that silently test the wrong thing. Introduce direct tests for `parse_pathspec_file()`. Signed-off-by: Alexandr Miloslavskiy <> Signed-off-by: Junio C Hamano <>
2019-11-13test-tool: use 'read-graph' helperDerrick Stolee
The 'git commit-graph read' subcommand is used in test scripts to check that the commit-graph contents match the expected data. Mostly, this helps check the header information and the list of chunks. Users do not need this information, so move the functionality to a test helper. Reported-by: Bryan Turner <> Signed-off-by: Derrick Stolee <> Signed-off-by: Junio C Hamano <>
2019-09-17Test the progress displaySZEDER Gábor
'progress.c' has seen a few fixes recently [1], and, unfortunately, some of those fixes required further fixes [2]. It seems it's time to have a few tests focusing on the subtleties of the progress display. Add the 'test-tool progress' subcommand to help testing the progress display, reading instructions from standard input and turning them into calls to the display_progress() and display_throughput() functions with the given parameters. The progress display is, however, critically dependent on timing, because it's only updated once every second or, if the toal is known in advance, every 1%, and there is the throughput rate as well. These make the progress display far too undeterministic for testing as-is. To address this, add a few testing-specific variables and functions to 'progress.c', allowing the the new test helper to: - Disable the triggered-every-second SIGALRM and set the 'progress_update' flag explicitly based in the input instructions. This way the progress line will be updated deterministically when the test wants it to be updated. - Specify the time elapsed since start_progress() to make the throughput rate calculations deterministic. Add the new test script '' to check a few simple cases with and without throughput, and that a shorter progress line properly covers up the previously displayed line in different situations. [1] See commits 545dc345eb (progress: break too long progress bar lines, 2019-04-12) and 9f1fd84e15 (progress: clear previous progress update dynamically, 2019-04-12). [2] 1aed1a5f25 (progress: avoid empty line when breaking the progress line, 2019-05-19) Signed-off-by: SZEDER Gábor <> Signed-off-by: Junio C Hamano <>
2019-07-25Merge branch 'mt/dir-iterator-updates'Junio C Hamano
Adjust the dir-iterator API and apply it to the local clone optimization codepath. * mt/dir-iterator-updates: clone: replace strcmp by fspathcmp clone: use dir-iterator to avoid explicit dir traversal clone: extract function from copy_or_link_directory clone: copy hidden paths at local clone dir-iterator: add flags parameter to dir_iterator_begin dir-iterator: refactor state machine model dir-iterator: use warning_errno when possible dir-iterator: add tests for dir-iterator API clone: better handle symlinked files at .git/objects/ clone: test for our behavior on odd objects/* content
2019-07-11dir-iterator: add tests for dir-iterator APIDaniel Ferreira
Create t/helper/test-dir-iterator.c, which prints relevant information about a directory tree iterated over with dir-iterator. Create t/, which tests that dir-iterator does iterate through a whole directory tree as expected. Signed-off-by: Daniel Ferreira <> [matheus.bernardino: update to use test-tool and some minor aesthetics] Helped-by: Matheus Tavares <> Signed-off-by: Matheus Tavares <> Signed-off-by: Junio C Hamano <>
2019-06-18t/helper: add test-oidmap.cChristian Couder
This new helper is very similar to "test-hashmap.c" and will help test how `struct oidmap` from oidmap.{c,h} can be used. Helped-by: SZEDER Gábor <> Helped-by: Jeff King <> Signed-off-by: Christian Couder <> Signed-off-by: Junio C Hamano <>
2019-05-08Merge branch 'js/misc-doc-fixes'Junio C Hamano
"make check-docs", "git help -a", etc. did not account for cases where a particular build may deliberately omit some subcommands, which has been corrected. * js/misc-doc-fixes: Turn `git serve` into a test helper test-tool: handle the `-C <directory>` option just like `git` check-docs: do not bother checking for legacy scripts' documentation docs: exclude documentation for commands that have been excluded check-docs: allow command-list.txt to contain excluded commands help -a: do not list commands that are excluded from the build Makefile: drop the NO_INSTALL variable remote-testgit: move it into the support directory for t5801
2019-04-19Turn `git serve` into a test helperJohannes Schindelin
The `git serve` built-in was introduced in ed10cb952d31 (serve: introduce git-serve, 2018-03-15) as a backend to serve Git protocol v2, probably originally intended to be spawned by `git upload-pack`. However, in the version that the protocol v2 patches made it into core Git, `git upload-pack` calls the `serve()` function directly instead of spawning `git serve`; The only reason in life for `git serve` to survive as a built-in command is to provide a way to test the protocol v2 functionality. Meaning that it does not even have to be a built-in that is installed with end-user facing Git installations, but it can be a test helper instead. Let's make it so. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2019-02-22trace2: t/helper/test-trace2,,, t0212.shJeff Hostetler
Create unit tests for Trace2. Signed-off-by: Jeff Hostetler <> Signed-off-by: Junio C Hamano <>
2019-02-19tests: teach the test-tool to generate NUL bytes and use itJohannes Schindelin
In cc95bc2025 (t5562: replace /dev/zero with a pipe from generate_zero_bytes, 2019-02-09), we replaced usage of /dev/zero (which is not available on NonStop, apparently) by a Perl script snippet to generate NUL bytes. Sadly, it does not seem to work on NonStop, as t5562 reportedly hangs. Worse, this also hangs in the Ubuntu 16.04 agents of the CI builds on Azure Pipelines: for some reason, the Perl script snippet that is run via `generate_zero_bytes` in t5562's 'CONTENT_LENGTH overflow ssite_t' test case tries to write out an infinite amount of NUL bytes unless a broken pipe is encountered, that snippet never encounters the broken pipe, and keeps going until the build times out. Oddly enough, this does not reproduce on the Windows and macOS agents, nor in a local Ubuntu 18.04. This developer tried for a day to figure out the exact circumstances under which this hang happens, to no avail, the details remain a mystery. In the end, though, what counts is that this here change incidentally fixes that hang (maybe also on NonStop?). Even more positively, it gets rid of yet another unnecessary Perl invocation. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2019-02-07Merge branch 'js/vsts-ci'Junio C Hamano
Prepare to run test suite on Azure Pipeline. * js/vsts-ci: (22 commits) test-date: drop unused parameter to getnanos() ci: parallelize testing on Windows ci: speed up Windows phase tests: optionally skip bin-wrappers/ t0061: workaround issues with --with-dashes and RUNTIME_PREFIX tests: add t/helper/ to the PATH with --with-dashes mingw: try to work around issues with the test cleanup tests: include detailed trace logs with --write-junit-xml upon failure tests: avoid calling Perl just to determine file sizes README: add a build badge (status of the Azure Pipelines build) mingw: be more generous when wrapping up the setitimer() emulation ci: use git-sdk-64-minimal build artifact ci: add a Windows job to the Azure Pipelines definition Add a build definition for Azure DevOps ci/ add support for Azure Pipelines tests: optionally write results as JUnit-style .xml test-date: add a subcommand to measure times in shell scripts ci: use a junction on Windows instead of a symlink ci: inherit --jobs via MAKEFLAGS in run-build-and-tests ci/ encapsulate Travis-specific things ...
2019-02-07Merge branch 'nd/the-index-final'Junio C Hamano
The assumption to work on the single "in-core index" instance has been reduced from the library-ish part of the codebase. * nd/the-index-final: cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch read-cache.c: remove the_* from index_has_changes() merge-recursive.c: remove implicit dependency on the_repository merge-recursive.c: remove implicit dependency on the_index sha1-name.c: remove implicit dependency on the_index read-cache.c: replace update_index_if_able with repo_& read-cache.c: kill read_index() checkout: avoid the_index when possible repository.c: replace hold_locked_index() with repo_hold_locked_index() notes-utils.c: remove the_repository references grep: use grep_opt->repo instead of explict repo argument
2019-01-29Merge branch 'bc/sha-256'Junio C Hamano
Add sha-256 hash and plug it through the code to allow building Git with the "NewHash". * bc/sha-256: hash: add an SHA-256 implementation using OpenSSL sha256: add an SHA-256 implementation using libgcrypt Add a base implementation of SHA-256 support commit-graph: convert to using the_hash_algo t/helper: add a test helper to compute hash speed sha1-file: add a constant for hash block size t: make the sha1 test-tool helper generic t: add basic tests for our SHA-1 implementation cache: make hashcmp and hasheq work with larger hashes hex: introduce functions to print arbitrary hashes sha1-file: provide functions to look up hash algorithms sha1-file: rename algorithm to "sha1"
2019-01-29tests: optionally write results as JUnit-style .xmlJohannes Schindelin
This will come in handy when publishing the results of Git's test suite during an automated Azure DevOps run. Note: we need to make extra sure that invalid UTF-8 encoding is turned into valid UTF-8 (using the Replacement Character, \uFFFD) because t9902's trace contains such invalid byte sequences, and the task in the Azure Pipeline that uploads the test results would refuse to do anything if it was asked to parse an .xml file with invalid UTF-8 in it. Signed-off-by: Johannes Schindelin <> Signed-off-by: Junio C Hamano <>
2019-01-24cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switchNguyễn Thái Ngọc Duy
By default, index compat macros are off from now on, because they could hide the_index dependency. Only those in builtin can use it. Signed-off-by: Nguyễn Thái Ngọc Duy <> Signed-off-by: Junio C Hamano <>
2018-11-14Add a base implementation of SHA-256 supportbrian m. carlson
SHA-1 is weak and we need to transition to a new hash function. For some time, we have referred to this new function as NewHash. Recently, we decided to pick SHA-256 as NewHash. The reasons behind the choice of SHA-256 are outlined in the thread starting at [1] and in the commit history for the hash function transition document. Add a basic implementation of SHA-256 based off libtomcrypt, which is in the public domain. Optimize it and restructure it to meet our coding standards. Pull in the update and final functions from the SHA-1 block implementation, as we know these function correctly with all compilers. This implementation is slower than SHA-1, but more performant implementations will be introduced in future commits. Wire up SHA-256 in the list of hash algorithms, and add a test that the algorithm works correctly. Note that with this patch, it is still not possible to switch to using SHA-256 in Git. Additional patches are needed to prepare the code to handle a larger hash algorithm and further test fixes are needed. [1] Signed-off-by: brian m. carlson <> Signed-off-by: Junio C Hamano <>
2018-11-14t/helper: add a test helper to compute hash speedbrian m. carlson
Add a utility (which is less for the testsuite and more for developers) that can compute hash speeds for whatever hash algorithms are implemented. This allows developers to test their personal systems to determine the performance characteristics of various algorithms. Signed-off-by: brian m. carlson <> Signed-off-by: Junio C Hamano <>
2018-11-14t: make the sha1 test-tool helper genericbrian m. carlson
Since we're going to have multiple hash algorithms to test, it makes sense to share as much of the test code as possible. Convert the sha1 helper for the test-tool to be generic and move it out into its own module. This will allow us to share most of this code with our NewHash implementation. Signed-off-by: brian m. carlson <> Signed-off-by: Junio C Hamano <>
2018-11-13Merge branch 'ao/submodule-wo-gitmodules-checked-out'Junio C Hamano
The submodule support has been updated to read from the blob at HEAD:.gitmodules when the .gitmodules file is missing from the working tree. * ao/submodule-wo-gitmodules-checked-out: t/helper: add test-submodule-nested-repo-config submodule: support reading .gitmodules when it's not in the working tree submodule: add a helper to check if it is safe to write to .gitmodules t7506: clean up .gitmodules properly before setting up new scenario submodule: use the 'submodule--helper config' command submodule--helper: add a new 'config' subcommand t7411: be nicer to future tests and really clean things up t7411: merge tests 5 and 6 submodule: factor out a config_set_in_gitmodules_file_gently function submodule: add a print_config_from_gitmodules() helper
2018-10-31t/helper: add test-submodule-nested-repo-configAntonio Ospite
Add a test tool to exercise config_from_gitmodules(), in particular for the case of nested submodules. Add also a test to document that reading the submoudles config of nested submodules does not work yet when the .gitmodules file is not in the working tree but it still in the index. This is because the git API does not always make it possible access the object store of an arbitrary repository (see get_oid() usage in config_from_gitmodules()). When this git limitation gets fixed the aforementioned use case will be supported too. Signed-off-by: Antonio Ospite <> Signed-off-by: Junio C Hamano <>
2018-10-18headers: normalize the spelling of some header guardsRamsay Jones
Signed-off-by: Ramsay Jones <> Signed-off-by: Junio C Hamano <>
2018-10-10Merge branch 'nd/test-tool'Junio C Hamano
Test helper binaries clean-up. * nd/test-tool: Makefile: add a hint about TEST_BUILTINS_OBJS t/helper: merge test-dump-fsmonitor into test-tool t/helper: merge test-parse-options into test-tool t/helper: merge test-pkt-line into test-tool t/helper: merge test-dump-untracked-cache into test-tool t/helper: keep test-tool command list sorted