path: root/t/helper/test-tool.c
diff options
authorÆvar Arnfjörð Bjarmason <>2021-01-24 17:28:13 (GMT)
committerJunio C Hamano <>2021-01-25 00:09:17 (GMT)
commit95ca1f987edd23389e3079d0a7fe6d0f89927b68 (patch)
tree848a3df16861c43f43e8161eeae352e597337ce6 /t/helper/test-tool.c
parenta4fea08b6ebc2782891abdf6fd4bb9feeb21ff4d (diff)
grep/pcre2: better support invalid UTF-8 haystacks
Improve the support for invalid UTF-8 haystacks given a non-ASCII needle when using the PCREv2 backend. This is a more complete fix for a bug I started to fix in 870eea8166 (grep: do not enter PCRE2_UTF mode on fixed matching, 2019-07-26), now that PCREv2 has the PCRE2_MATCH_INVALID_UTF mode we can make use of it. This fixes the sort of case described in 8a5999838e (grep: stess test PCRE v2 on invalid UTF-8 data, 2019-07-26), i.e.: - The subject string is non-ASCII (e.g. "ævar") - We're under a is_utf8_locale(), e.g. "en_US.UTF-8", not "C" - We are using --ignore-case, or we're a non-fixed pattern If those conditions were satisfied and we matched found non-valid UTF-8 data PCREv2 might bark on it, in practice this only happened under the JIT backend (turned on by default on most platforms). Ultimately this fixes a "regression" in b65abcafc7 ("grep: use PCRE v2 for optimized fixed-string search", 2019-07-01), I'm putting that in scare-quotes because before then we wouldn't properly support these complex case-folding, locale etc. cases either, it just broke in different ways. There was a bug related to this the PCRE2_NO_START_OPTIMIZE flag fixed in PCREv2 10.36. It can be worked around by setting the PCRE2_NO_START_OPTIMIZE flag. Let's do that in those cases, and add tests for the bug. Signed-off-by: Ævar Arnfjörð Bjarmason <> Signed-off-by: Junio C Hamano <>
Diffstat (limited to 't/helper/test-tool.c')
1 files changed, 1 insertions, 0 deletions
diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
index 9d6d14d..f97cd9f 100644
--- a/t/helper/test-tool.c
+++ b/t/helper/test-tool.c
@@ -46,6 +46,7 @@ static struct test_cmd cmds[] = {
{ "parse-options", cmd__parse_options },
{ "parse-pathspec-file", cmd__parse_pathspec_file },
{ "path-utils", cmd__path_utils },
+ { "pcre2-config", cmd__pcre2_config },
{ "pkt-line", cmd__pkt_line },
{ "prio-queue", cmd__prio_queue },
{ "proc-receive", cmd__proc_receive},