pcre2 1.1.5 → 2.0.0

raw patch · 23 files changed

+1421/−485 lines, 23 filesPVP ok

version bump matches the API change (PVP)

API changes (from Hackage documentation)

Files

• ChangeLog.md +42/−11
• README.md +23/−16
• bench/Bench.hs +4/−4
• pcre2.cabal +4/−5
• src/c/config.h +97/−32
• src/c/pcre2.h +2/−4
• src/c/pcre2/src/pcre2_auto_possess.c +9/−4
• src/c/pcre2/src/pcre2_compile.c +35/−17
• src/c/pcre2/src/pcre2_jit_compile.c +18/−22
• src/c/pcre2/src/pcre2_jit_simd_inc.h +786/−42
• src/c/pcre2/src/pcre2_match.c +3/−1
• src/c/pcre2/src/pcre2posix.c +16/−2
• src/c/pcre2/src/sljit/sljitConfigInternal.h +2/−0
• src/c/pcre2/src/sljit/sljitExecAllocator.c +71/−53
• src/c/pcre2/src/sljit/sljitNativeS390X.c +22/−21
• src/c/pcre2/src/sljit/sljitUtils.c +7/−3
• src/c/pcre2/src/sljit/sljitWXExecAllocator.c +6/−2
• src/c/pcre2_chartables.c +0/−2
• src/hs/Text/Regex/Pcre2.hs +42/−38
• src/hs/Text/Regex/Pcre2/Internal.hs +94/−124
• src/hs/Text/Regex/Pcre2/TH.hs +69/−42
• src/hs/Text/Regex/Pcre2/Unsafe.hs +32/−0
• test/Spec.hs +37/−40

@@ -1,9 +1,40 @@ # Changelog and Acknowledgements +## 2.0.0+This release introduces significant breaking changes in order to make the API+smaller, more consistent, and safer.+* Implemented [#18](https://github.com/sjshuck/hs-pcre2/issues/18):+    * Removed `matchAll`, `matchAllOpt`, `capturesAll`, and `capturesAllOpt`.+    * Upgraded `match`, `matchOpt`, `captures`, and `capturesOpt` to offer their+      functionality, respectively.+    * Renamed `capturesA` and `capturesAOpt` to `captures` and `capturesOpt`,+      replacing the latter two functions altogether.  `captures`/-`Opt` were+      intended to be extreme+      [convenience functions](https://hackage.haskell.org/package/pcre2-1.1.5/docs/Text-Regex-Pcre2.html#v:captures)+      that required no special datatypes beyond the `Prelude`.  However, this+      was of doubtful benefit, since that's false anyway—they required+      `Text`, not to mention `{-# LANGUAGE OverloadedStrings #-}`.  Their names+      are simple and valuable, and no other `Alternative`-producing function has+      the naming convention "-`A`", so repurposing their names was in order.+* Moved the callout interface to a new module, `Text.Regex.Pcre2.Unsafe`.  This+  includes the options `UnsafeCompileRecGuard`, `UnsafeCallout`,+  `UnsafeSubCallout`, and `AutoCallout`, and the types `CalloutInfo`,+  `CalloutIndex`, `CalloutResult`, `SubCalloutInfo`, and `SubCalloutResult`.+* Also moved option `BadEscapeIsLiteral` there.+* Removed the ineffectual options `DupNames` and `Utf`.++Other improvements with no API impact:+* Updated PCRE2 to 10.37.+* Replaced copied C files with symlinks, diminishing codebase by 1.5K lines and+  simplifying future PCRE2 updates.+* Reduced size of Template Haskell splices to make error messages less+  obnoxious.+* Moderate refactoring of internals and documentation.+ ## 1.1.5-* Fixed [#17](https://github.com/sjshuck/pcre2/17), where functions returning-  `Alternative` containers were not restricted to single results despite their-  documentation.+* Fixed [#17](https://github.com/sjshuck/hs-pcre2/issues/17), where functions+  returning `Alternative` containers were not restricted to single results+  despite their documentation. * Minor improvements to docs and examples.  ## 1.1.4@@ -21,12 +52,12 @@  ## 1.1.2 * Refactored using the `streaming` library.  Fixed-  [#11](https://github.com/sjshuck/pcre2/11), where large global matches were-  very slow.+  [#11](https://github.com/sjshuck/hs-pcre2/issues/11), where large global+  matches were very slow.  ## 1.1.1-* Fixed [#12](https://github.com/sjshuck/pcre2/4), where some functions returned-  too many match results.+* Fixed [#12](https://github.com/sjshuck/hs-pcre2/issues/12), where some+  functions returned too many match results.  ## 1.1.0 * Added global matching.@@ -35,8 +66,8 @@ * Changed `capturesOptA` to `capturesAOpt` for naming consistency.  ## 1.0.2-* Fixed [#4](https://github.com/sjshuck/pcre2/4), where multiple named captures-  were not type-indexed correctly.+* Fixed [#4](https://github.com/sjshuck/hs-pcre2/4), where multiple named+  captures were not type-indexed correctly. * Established automated builds using Github Workflows.  Thanks amesgen!  ## 1.0.1.1@@ -44,8 +75,8 @@   Hackage.  ## 1.0.1-* Fixed [#1](https://github.com/sjshuck/pcre2/1), where building on Windows-  would succeed but not run.  Thanks Andrew!+* Fixed [#1](https://github.com/sjshuck/hs-pcre2/issues/1), where building on+  Windows would succeed but not run.  Thanks Andrew! * Try to adjust dependency version bounds to get it building on Hackage.  Thanks   snoyberg! 

@@ -7,13 +7,16 @@  ## Teasers ```haskell-embeddedLicensePlate :: Text -> Maybe Text-embeddedLicensePlate = match "[A-Z]{3}[0-9]{3,4}"+licensePlate :: Text -> Maybe Text+licensePlate = match "[A-Z]{3}[0-9]{3,4}"++licensePlates :: Text -> [Text]+licensePlates = match "[A-Z]{3}[0-9]{3,4}" ``` ```haskell case "The quick brown fox" of-    [regex|brown\s+(?<animal>\w+)|] -> Text.putStrLn animal-    _                               -> error "nothing brown"+    [regex|\bbrown\s+(?<animal>[A-z]+)\b|] -> Text.putStrLn animal+    _                                      -> error "nothing brown" ``` ```haskell let kv'd = lined . packed . [_regex|(?x)  # Extended PCRE2 syntax@@ -39,13 +42,14 @@ ```  ## Features-* Quiet functions with simple types&mdash;for the most part it's  -`Text` _(pattern)_ `-> Text` _(subject)_ `-> result`.-* Use partial application to create performant, compile-once-match-many code.-* Low cognitive overhead&mdash;there's just one custom datatype for both compile-  and match options, the `Option` monoid.-* `Text` everywhere for interop with both C and the broader Haskell ecosystem.-* Match failures expressed via `Alternative` or pattern match failures.+* No opaque "`Regex`" object.  Instead, quiet functions with simple+  types&mdash;for the most part it's `Text` _(pattern)_ `-> Text` _(subject)_+  `-> result`.  Use partial application to create performant,+  compile-once-match-many code.+* No [custom typeclasses](https://hackage.haskell.org/package/regex-base/docs/Text-Regex-Base-RegexLike.html#t:RegexContext).+* A single datatype for both compile and match options, the `Option` monoid.+* `Text` everywhere.+* Match success expressed via `Alternative`. * Opt-in Template Haskell facilities for compile-time verification of patterns,   indexing captures, and memoizing inline regexes. * Opt-in `lens` support.@@ -55,8 +59,11 @@   Both are first-class. * Vast presentation of PCRE2 functionality.  We can even register Haskell   callbacks to run during matching!+* Zero-copying of substrings where beneficial.  Benchmarks show a 10&times;+  speedup over `pcre-light`, and 20&times; over `regex-pcre`, for longer+  captures. * Few dependencies.-* Bundled, statically-linked UTF-16 build of up-to-date PCRE2 (version 10.36),+* Bundled, statically-linked UTF-16 build of up-to-date PCRE2 (version 10.37),   with a complete, exposed Haskell binding.  ## Wishlist@@ -64,11 +71,11 @@   slower than other libraries for some operations, although things are   improving.  (We are already faster than   [regex-base](https://hackage.haskell.org/package/regex-base)/[regex-pcre](https://hackage.haskell.org/package/regex-pcre)-  when working with `Text`.)  If it's really regex processing that's causing a-  bottleneck,+  when working with `Text`, even without zero-copying.)  If it's really regex+  processing that's causing a bottleneck,   [pcre-light](https://hackage.haskell.org/package/pcre-light)/[-heavy](https://hackage.haskell.org/package/pcre-heavy)/[lens-regex-pcre](https://hackage.haskell.org/package/lens-regex-pcre)   are recommended instead of this library for the very best performance.-* Make use of DFA and JIT compilation.+* Make use of DFA matching and JIT compilation. * Improve PCRE2 C compile time. * Add splitting support. @@ -77,4 +84,4 @@ PCRE2 is distributed under the [3-clause BSD](https://www.pcre.org/licence.txt) license.  ## Main Author-&copy;2020 Shlomo Shuck+&copy;2020&ndash;2021 Shlomo Shuck

@@ -21,7 +21,7 @@ main = defaultMain [     bgroup "creating single-use regexes" [         bench "pcre2" $ flip nf strings $ \(patt, subj) ->-            let Just (_ :| [bar]) = capturesA (Text.pack patt) (Text.pack subj)+            let Just (_ :| [bar]) = captures (Text.pack patt) (Text.pack subj)             in Text.unpack bar,          bench "regex-pcre-builtin" $ flip nf strings $ \(patt, subj) ->@@ -34,9 +34,9 @@             let r = Text.Regex.PCRE.Light.Char8.compile patt []                 Just [_, bar] = Text.Regex.PCRE.Light.Char8.match r subj []             in bar],-    +     bgroup "strings" [-        let r = capturesA $ Text.pack stringPattern+        let r = captures $ Text.pack stringPattern         in bench "pcre2" $ flip nf stringSubject $ \subj ->             let Just (_ :| [bar]) = r $ Text.pack subj in Text.unpack bar, @@ -117,6 +117,6 @@     where     textPattern = Text.pack patt     textSubject = Text.pack subj-    pcre2R = capturesA textPattern+    pcre2R = captures textPattern     regexBaseR = mkRegexBaseR patt     pcreLightR = Text.Regex.PCRE.Light.Char8.compile patt []

@@ -3,11 +3,9 @@ -- This file has been generated from package.yaml by hpack version 0.34.4. -- -- see: https://github.com/sol/hpack------ hash: cb8596609200fe862c181845b59caabfdf1c2a19dc0f410828ecba4f7c2c4fe5  name:           pcre2-version:        1.1.5+version:        2.0.0 synopsis:       Regular expressions via the PCRE2 C library (included) description:    Please see the README on GitHub at <https://github.com/sjshuck/hs-pcre2> category:       Text@@ -15,7 +13,7 @@ bug-reports:    https://github.com/sjshuck/hs-pcre2/issues author:         Shlomo Shuck and contributors maintainer:     stevenjshuck@gmail.com-copyright:      2020 Shlomo Shuck+copyright:      2020-2021 Shlomo Shuck license:        Apache-2.0 license-file:   LICENSE build-type:     Simple@@ -102,6 +100,7 @@   exposed-modules:       Text.Regex.Pcre2       Text.Regex.Pcre2.Foreign+      Text.Regex.Pcre2.Unsafe   other-modules:       Text.Regex.Pcre2.Foreign.TH       Text.Regex.Pcre2.Internal@@ -109,7 +108,7 @@       Paths_pcre2   hs-source-dirs:       src/hs-  ghc-options: -optc=-DPCRE2_CODE_UNIT_WIDTH=16 -optc=-DPCRE2_STATIC=1 -optc=-Wno-discarded-qualifiers -optc=-Wno-incompatible-pointer-types+  ghc-options: -optc=-DPCRE2_CODE_UNIT_WIDTH=16 -optc=-DPCRE2_STATIC=1 -optc=-DHAVE_INTTYPES_H=1 -optc=-DHAVE_LIMITS_H=1 -optc=-DHAVE_MEMMOVE_H=1 -optc=-DHAVE_STDINT_H=1 -optc=-DHAVE_STDLIB_H=1 -optc=-DHAVE_STRERROR_H=1 -optc=-DHAVE_STRING_H=1 -optc=-DSUPPORT_JIT=1 -optc=-DSUPPORT_PCRE2_16=1 -optc=-DSUPPORT_UNICODE=1 -optc=-Wno-discarded-qualifiers -optc=-Wno-incompatible-pointer-types   cc-options: -DHAVE_CONFIG_H -DPCRE2_CODE_UNIT_WIDTH=16   include-dirs:       src/c

@@ -1,4 +1,5 @@-// Copied and edited per the PCRE2 source's NON-AUTOTOOLS-BUILD instructions.+/* src/config.h.  Generated from config.h.in by configure.  */+/* src/config.h.in.  Generated from configure.ac by autoheader.  */  /* PCRE2 is written in Standard C, but there are a few non-standard things it can cope with, allowing it to run on SunOS4 and other "close to standard"@@ -73,19 +74,19 @@ /* #undef HAVE_EDIT_READLINE_READLINE_H */  /* Define to 1 if you have the <inttypes.h> header file. */-#define HAVE_INTTYPES_H 1+/* #undef HAVE_INTTYPES_H */  /* Define to 1 if you have the <limits.h> header file. */-#define HAVE_LIMITS_H 1+/* #undef HAVE_LIMITS_H */  /* Define to 1 if you have the `memfd_create' function. */ /* #undef HAVE_MEMFD_CREATE */  /* Define to 1 if you have the `memmove' function. */-#define HAVE_MEMMOVE 1+/* #undef HAVE_MEMMOVE */ -/* Define to 1 if you have the <memory.h> header file. */-/* #undef HAVE_MEMORY_H */+/* Define to 1 if you have the <minix/config.h> header file. */+/* #undef HAVE_MINIX_CONFIG_H */  /* Define to 1 if you have the `mkostemp' function. */ /* #undef HAVE_MKOSTEMP */@@ -106,19 +107,22 @@ /* #undef HAVE_SECURE_GETENV */  /* Define to 1 if you have the <stdint.h> header file. */-#define HAVE_STDINT_H 1+/* #undef HAVE_STDINT_H */ +/* Define to 1 if you have the <stdio.h> header file. */+/* #undef HAVE_STDIO_H */+ /* Define to 1 if you have the <stdlib.h> header file. */-#define HAVE_STDLIB_H 1+/* #undef HAVE_STDLIB_H */  /* Define to 1 if you have the `strerror' function. */-#define HAVE_STRERROR 1+/* #undef HAVE_STRERROR */  /* Define to 1 if you have the <strings.h> header file. */ /* #undef HAVE_STRINGS_H */  /* Define to 1 if you have the <string.h> header file. */-#define HAVE_STRING_H 1+/* #undef HAVE_STRING_H */  /* Define to 1 if you have the <sys/stat.h> header file. */ /* #undef HAVE_SYS_STAT_H */@@ -135,6 +139,9 @@ /* Define to 1 if the compiler supports simple visibility declarations. */ /* #undef HAVE_VISIBILITY */ +/* Define to 1 if you have the <wchar.h> header file. */+/* #undef HAVE_WCHAR_H */+ /* Define to 1 if you have the <windows.h> header file. */ /* #undef HAVE_WINDOWS_H */ @@ -223,7 +230,7 @@ #define PACKAGE_NAME "PCRE2"  /* Define to the full name and version of this package. */-#define PACKAGE_STRING "PCRE2 10.36"+#define PACKAGE_STRING "PCRE2 10.37"  /* Define to the one symbol short name of this package. */ #define PACKAGE_TARNAME "pcre2"@@ -232,7 +239,7 @@ #define PACKAGE_URL ""  /* Define to the version of this package. */-#define PACKAGE_VERSION "10.36"+#define PACKAGE_VERSION "10.37"  /* The value of PARENS_NEST_LIMIT specifies the maximum depth of nested    parentheses (of any kind) in a pattern. This limits the amount of system@@ -285,11 +292,13 @@    unless SUPPORT_JIT is also defined. */ /* #undef SLJIT_PROT_EXECUTABLE_ALLOCATOR */ -/* Define to 1 if you have the ANSI C header files. */-#define STDC_HEADERS 1+/* Define to 1 if all of the C90 standard headers exist (not just the ones+   required in a freestanding environment). This macro is provided for+   backward compatibility; new code need not use it. */+/* #undef STDC_HEADERS */  /* Define to any value to enable support for Just-In-Time compiling. */-#define SUPPORT_JIT 1+/* #undef SUPPORT_JIT */  /* Define to any value to allow pcre2grep to be linked with libbz2, so that it    is able to handle .bz2 files. */@@ -318,7 +327,7 @@ /* #undef SUPPORT_PCRE2GREP_JIT */  /* Define to any value to enable the 16 bit PCRE2 library. */-#define SUPPORT_PCRE2_16 1+/* #undef SUPPORT_PCRE2_16 */  /* Define to any value to enable the 32 bit PCRE2 library. */ /* #undef SUPPORT_PCRE2_32 */@@ -330,7 +339,7 @@    will work even in an EBCDIC environment, but it is incompatible with the    EBCDIC macro. That is, PCRE2 can support *either* EBCDIC code *or*    ASCII/Unicode, but not both at once. */-#define SUPPORT_UNICODE 1+/* #undef SUPPORT_UNICODE */  /* Define to any value for valgrind support to find invalid memory reads. */ /* #undef SUPPORT_VALGRIND */@@ -339,35 +348,91 @@ #ifndef _ALL_SOURCE # define _ALL_SOURCE 1 #endif+/* Enable general extensions on macOS.  */+#ifndef _DARWIN_C_SOURCE+# define _DARWIN_C_SOURCE 1+#endif+/* Enable general extensions on Solaris.  */+#ifndef __EXTENSIONS__+# define __EXTENSIONS__ 1+#endif /* Enable GNU extensions on systems that have them.  */ #ifndef _GNU_SOURCE # define _GNU_SOURCE 1 #endif-/* Enable threading extensions on Solaris.  */+/* Enable X/Open compliant socket functions that do not require linking+   with -lxnet on HP-UX 11.11.  */+#ifndef _HPUX_ALT_XOPEN_SOCKET_API+# define _HPUX_ALT_XOPEN_SOCKET_API 1+#endif+/* Identify the host operating system as Minix.+   This macro does not affect the system headers' behavior.+   A future release of Autoconf may stop defining this macro.  */+#ifndef _MINIX+/* # undef _MINIX */+#endif+/* Enable general extensions on NetBSD.+   Enable NetBSD compatibility extensions on Minix.  */+#ifndef _NETBSD_SOURCE+# define _NETBSD_SOURCE 1+#endif+/* Enable OpenBSD compatibility extensions on NetBSD.+   Oddly enough, this does nothing on OpenBSD.  */+#ifndef _OPENBSD_SOURCE+# define _OPENBSD_SOURCE 1+#endif+/* Define to 1 if needed for POSIX-compatible behavior.  */+#ifndef _POSIX_SOURCE+/* # undef _POSIX_SOURCE */+#endif+/* Define to 2 if needed for POSIX-compatible behavior.  */+#ifndef _POSIX_1_SOURCE+/* # undef _POSIX_1_SOURCE */+#endif+/* Enable POSIX-compatible threading on Solaris.  */ #ifndef _POSIX_PTHREAD_SEMANTICS # define _POSIX_PTHREAD_SEMANTICS 1 #endif+/* Enable extensions specified by ISO/IEC TS 18661-5:2014.  */+#ifndef __STDC_WANT_IEC_60559_ATTRIBS_EXT__+# define __STDC_WANT_IEC_60559_ATTRIBS_EXT__ 1+#endif+/* Enable extensions specified by ISO/IEC TS 18661-1:2014.  */+#ifndef __STDC_WANT_IEC_60559_BFP_EXT__+# define __STDC_WANT_IEC_60559_BFP_EXT__ 1+#endif+/* Enable extensions specified by ISO/IEC TS 18661-2:2015.  */+#ifndef __STDC_WANT_IEC_60559_DFP_EXT__+# define __STDC_WANT_IEC_60559_DFP_EXT__ 1+#endif+/* Enable extensions specified by ISO/IEC TS 18661-4:2015.  */+#ifndef __STDC_WANT_IEC_60559_FUNCS_EXT__+# define __STDC_WANT_IEC_60559_FUNCS_EXT__ 1+#endif+/* Enable extensions specified by ISO/IEC TS 18661-3:2015.  */+#ifndef __STDC_WANT_IEC_60559_TYPES_EXT__+# define __STDC_WANT_IEC_60559_TYPES_EXT__ 1+#endif+/* Enable extensions specified by ISO/IEC TR 24731-2:2010.  */+#ifndef __STDC_WANT_LIB_EXT2__+# define __STDC_WANT_LIB_EXT2__ 1+#endif+/* Enable extensions specified by ISO/IEC 24747:2009.  */+#ifndef __STDC_WANT_MATH_SPEC_FUNCS__+# define __STDC_WANT_MATH_SPEC_FUNCS__ 1+#endif /* Enable extensions on HP NonStop.  */ #ifndef _TANDEM_SOURCE # define _TANDEM_SOURCE 1 #endif-/* Enable general extensions on Solaris.  */-#ifndef __EXTENSIONS__-# define __EXTENSIONS__ 1+/* Enable X/Open extensions.  Define to 500 only if necessary+   to make mbstate_t available.  */+#ifndef _XOPEN_SOURCE+/* # undef _XOPEN_SOURCE */ #endif  /* Version number of package */-#define VERSION "10.36"--/* Define to 1 if on MINIX. */-/* #undef _MINIX */--/* Define to 2 if the system does not provide POSIX.1 features except with-   this defined. */-/* #undef _POSIX_1_SOURCE */--/* Define to 1 if you need to in order for `stat' and other things to work. */-/* #undef _POSIX_SOURCE */+#define VERSION "10.37"  /* Define to empty if `const' does not conform to ANSI C. */ /* #undef const */

@@ -1,5 +1,3 @@-// Copied per the PCRE2 source's NON-AUTOTOOLS-BUILD instructions.- /************************************************* *       Perl-Compatible Regular Expressions      * *************************************************/@@ -44,9 +42,9 @@ /* The current PCRE version information. */  #define PCRE2_MAJOR           10-#define PCRE2_MINOR           36+#define PCRE2_MINOR           37 #define PCRE2_PRERELEASE      -#define PCRE2_DATE            2020-12-04+#define PCRE2_DATE            2021-05-26  /* When an application links to a PCRE DLL in Windows, the symbols that are imported have to be identified as such. When building PCRE2, the appropriate

@@ -7,7 +7,7 @@                         Written by Philip Hazel      Original API code Copyright (c) 1997-2012 University of Cambridge-          New API code Copyright (c) 2016-2020 University of Cambridge+          New API code Copyright (c) 2016-2021 University of Cambridge  ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without@@ -490,6 +490,7 @@   list[2] = (uint32_t)(end - code);   return end;   }+ return NULL;    /* Opcode not accepted */ } @@ -1186,12 +1187,16 @@     c = *repeat_opcode;     if (c >= OP_CRSTAR && c <= OP_CRMINRANGE)       {-      /* end must not be NULL. */-      end = get_chr_property_list(code, utf, ucp, cb->fcc, list);+      /* The return from get_chr_property_list() will never be NULL when+      *code (aka c) is one of the three class opcodes. However, gcc with+      -fanalyzer notes that a NULL return is possible, and grumbles. Hence we+      put in a check. */ +      end = get_chr_property_list(code, utf, ucp, cb->fcc, list);       list[1] = (c & 1) == 0; -      if (compare_opcodes(end, utf, ucp, cb, list, end, &rec_limit))+      if (end != NULL &&+          compare_opcodes(end, utf, ucp, cb, list, end, &rec_limit))         {         switch (c)           {

@@ -1398,32 +1398,47 @@ read_repeat_counts(PCRE2_SPTR *ptrptr, PCRE2_SPTR ptrend, uint32_t *minp,   uint32_t *maxp, int *errorcodeptr) {-PCRE2_SPTR p = *ptrptr;+PCRE2_SPTR p; BOOL yield = FALSE;+BOOL had_comma = FALSE; int32_t min = 0; int32_t max = REPEAT_UNLIMITED; /* This value is larger than MAX_REPEAT_COUNT */ -/* NB read_number() initializes the error code to zero. The only error is for a-number that is too big. */+/* Check the syntax */ +*errorcodeptr = 0;+for (p = *ptrptr;; p++)+  {+  uint32_t c;+  if (p >= ptrend) return FALSE;+  c = *p;+  if (IS_DIGIT(c)) continue;+  if (c == CHAR_RIGHT_CURLY_BRACKET) break;+  if (c == CHAR_COMMA)+    {+    if (had_comma) return FALSE;+    had_comma = TRUE;+    }+  else return FALSE;+  }++/* The only error from read_number() is for a number that is too big. */++p = *ptrptr; if (!read_number(&p, ptrend, -1, MAX_REPEAT_COUNT, ERR5, &min, errorcodeptr))   goto EXIT; -if (p >= ptrend) goto EXIT;- if (*p == CHAR_RIGHT_CURLY_BRACKET)   {   p++;   max = min;   }- else   {-  if (*p++ != CHAR_COMMA || p >= ptrend) goto EXIT;-  if (*p != CHAR_RIGHT_CURLY_BRACKET)+  if (*(++p) != CHAR_RIGHT_CURLY_BRACKET)     {     if (!read_number(&p, ptrend, -1, MAX_REPEAT_COUNT, ERR5, &max,-        errorcodeptr) || p >= ptrend ||  *p != CHAR_RIGHT_CURLY_BRACKET)+        errorcodeptr))       goto EXIT;     if (max < min)       {@@ -1438,11 +1453,10 @@ if (minp != NULL) *minp = (uint32_t)min; if (maxp != NULL) *maxp = (uint32_t)max; -/* Update the pattern pointer on success, or after an error, but not when-the result is "not a repeat quantifier". */+/* Update the pattern pointer */  EXIT:-if (yield || *errorcodeptr != 0) *ptrptr = p;+*ptrptr = p; return yield; } @@ -1776,19 +1790,23 @@       {       oldptr = ptr;       ptr--;   /* Back to the digit */-      if (!read_number(&ptr, ptrend, -1, INT_MAX/10 - 1, ERR61, &s,-          errorcodeptr))-        break; -      /* \1 to \9 are always back references. \8x and \9x are too; \1x to \7x+      /* As we know we are at a digit, the only possible error from+      read_number() is a number that is too large to be a group number. In this+      case we fall through handle this as not a group reference. If we have+      read a small enough number, check for a back reference.++      \1 to \9 are always back references. \8x and \9x are too; \1x to \7x       are octal escapes if there are not that many previous captures. */ -      if (s < 10 || oldptr[-1] >= CHAR_8 || s <= (int)cb->bracount)+      if (read_number(&ptr, ptrend, -1, INT_MAX/10 - 1, 0, &s, errorcodeptr) &&+          (s < 10 || oldptr[-1] >= CHAR_8 || s <= (int)cb->bracount))         {         if (s > (int)MAX_GROUP_NUMBER) *errorcodeptr = ERR61;           else escape = -s;     /* Indicates a back reference */         break;         }+       ptr = oldptr;      /* Put the pointer back and fall through */       } 

@@ -1226,7 +1226,7 @@ return TRUE; } -#define EARLY_FAIL_ENHANCE_MAX (1 + 1)+#define EARLY_FAIL_ENHANCE_MAX (1 + 3)  /* start:@@ -1238,6 +1238,7 @@ */ static int detect_early_fail(compiler_common *common, PCRE2_SPTR cc, int *private_data_start, sljit_s32 depth, int start) {+PCRE2_SPTR begin = cc; PCRE2_SPTR next_alt; PCRE2_SPTR end; PCRE2_SPTR accelerated_start;@@ -1475,31 +1476,19 @@       case OP_CBRA:       end = cc + GET(cc, 1); -      if (*end == OP_KET && PRIVATE_DATA(end) == 0)-        {-        if (*cc == OP_CBRA)-          {-          if (common->optimized_cbracket[GET2(cc, 1 + LINK_SIZE)] == 0)-            break;-          cc += IMM2_SIZE;-          }--        cc += 1 + LINK_SIZE;-        continue;-        }-       fast_forward_allowed = FALSE;       if (depth >= 4)         break;        end = bracketend(cc) - (1 + LINK_SIZE);-      if (*end != OP_KET || PRIVATE_DATA(end) != 0)-        break;--      if (*cc == OP_CBRA && common->optimized_cbracket[GET2(cc, 1 + LINK_SIZE)] == 0)+      if (*end != OP_KET || (*cc == OP_CBRA && common->optimized_cbracket[GET2(cc, 1 + LINK_SIZE)] == 0))         break;        count = detect_early_fail(common, cc, private_data_start, depth + 1, count);++      if (PRIVATE_DATA(cc) != 0)+        common->private_data_ptrs[begin - common->start] = 1;+       if (count < EARLY_FAIL_ENHANCE_MAX)         {         cc = end + (1 + LINK_SIZE);@@ -1555,6 +1544,8 @@           return EARLY_FAIL_ENHANCE_MAX;         } +      /* Cannot be part of a repeat. */+      common->private_data_ptrs[begin - common->start] = 1;       count++;        if (count < EARLY_FAIL_ENHANCE_MAX)@@ -1620,11 +1611,12 @@ sljit_s32 min, max, i;  /* Detect fixed iterations first. */-if (end[-(1 + LINK_SIZE)] != OP_KET)+if (end[-(1 + LINK_SIZE)] != OP_KET || PRIVATE_DATA(begin) != 0)   return FALSE; -/* Already detected repeat. */-if (common->private_data_ptrs[end - common->start - LINK_SIZE] != 0)+/* /(?:AB){4,6}/ is currently converted to /(?:AB){3}(?AB){1,3}/+ * Skip the check of the second part. */+if (PRIVATE_DATA(end - LINK_SIZE) == 0)   return TRUE;  next = end;@@ -1763,6 +1755,7 @@   if (private_data_ptr > SLJIT_MAX_LOCAL_SIZE)     break; +  /* When the bracket is prefixed by a zero iteration, skip the repeat check (at this point). */   if (repeat_check && (*cc == OP_ONCE || *cc == OP_BRA || *cc == OP_CBRA || *cc == OP_COND))     {     if (detect_repeat(common, cc))@@ -1813,6 +1806,7 @@      case OP_COND:     /* Might be a hidden SCOND. */+    common->private_data_ptrs[cc - common->start] = 0;     alternative = cc + GET(cc, 1);     if (*alternative == OP_KETRMAX || *alternative == OP_KETRMIN)       {@@ -13661,9 +13655,11 @@ memset(common->private_data_ptrs, 0, total_length * sizeof(sljit_s32));  private_data_size = common->cbra_ptr + (re->top_bracket + 1) * sizeof(sljit_sw);-set_private_data_ptrs(common, &private_data_size, ccend);+ if ((re->overall_options & PCRE2_ANCHORED) == 0 && (re->overall_options & PCRE2_NO_START_OPTIMIZE) == 0 && !common->has_skip_in_assert_back)   detect_early_fail(common, common->start, &private_data_size, 0, 0);++set_private_data_ptrs(common, &private_data_size, ccend);  SLJIT_ASSERT(common->early_fail_start_ptr <= common->early_fail_end_ptr); 

@@ -39,8 +39,30 @@ ----------------------------------------------------------------------------- */ -#if (defined SLJIT_CONFIG_X86 && SLJIT_CONFIG_X86) && !(defined SUPPORT_VALGRIND)+#if !(defined SUPPORT_VALGRIND) +#if ((defined SLJIT_CONFIG_X86 && SLJIT_CONFIG_X86) \+     || (defined SLJIT_CONFIG_S390X && SLJIT_CONFIG_S390X))++typedef enum {+  vector_compare_match1,+  vector_compare_match1i,+  vector_compare_match2,+} vector_compare_type;++static SLJIT_INLINE sljit_u32 max_fast_forward_char_pair_offset(void)+{+#if PCRE2_CODE_UNIT_WIDTH == 8+return 15;+#elif PCRE2_CODE_UNIT_WIDTH == 16+return 7;+#elif PCRE2_CODE_UNIT_WIDTH == 32+return 3;+#else+#error "Unsupported unit width"+#endif+}+ #if defined SUPPORT_UNICODE && PCRE2_CODE_UNIT_WIDTH != 32 static struct sljit_jump *jump_if_utf_char_start(struct sljit_compiler *compiler, sljit_s32 reg) {@@ -56,6 +78,10 @@ } #endif +#endif /* SLJIT_CONFIG_X86 || SLJIT_CONFIG_S390X */++#if (defined SLJIT_CONFIG_X86 && SLJIT_CONFIG_X86)+ static sljit_s32 character_to_int32(PCRE2_UCHAR chr) { sljit_u32 value = chr;@@ -97,13 +123,7 @@ sljit_emit_op_custom(compiler, instruction, 5); } -typedef enum {-    sse2_compare_match1,-    sse2_compare_match1i,-    sse2_compare_match2,-} sse2_compare_type;--static void fast_forward_char_pair_sse2_compare(struct sljit_compiler *compiler, sse2_compare_type compare_type,+static void fast_forward_char_pair_sse2_compare(struct sljit_compiler *compiler, vector_compare_type compare_type,   int step, sljit_s32 dst_ind, sljit_s32 cmp1_ind, sljit_s32 cmp2_ind, sljit_s32 tmp_ind) { sljit_u8 instruction[4];@@ -112,11 +132,11 @@  SLJIT_ASSERT(step >= 0 && step <= 3); -if (compare_type != sse2_compare_match2)+if (compare_type != vector_compare_match2)   {   if (step == 0)     {-    if (compare_type == sse2_compare_match1i)+    if (compare_type == vector_compare_match1i)       {       /* POR xmm1, xmm2/m128 */       /* instruction[0] = 0x66; */@@ -185,14 +205,14 @@ static void fast_forward_char_simd(compiler_common *common, PCRE2_UCHAR char1, PCRE2_UCHAR char2, sljit_s32 offset) { DEFINE_COMPILER;+sljit_u8 instruction[8]; struct sljit_label *start; #if defined SUPPORT_UNICODE && PCRE2_CODE_UNIT_WIDTH != 32 struct sljit_label *restart; #endif struct sljit_jump *quit; struct sljit_jump *partial_quit[2];-sse2_compare_type compare_type = sse2_compare_match1;-sljit_u8 instruction[8];+vector_compare_type compare_type = vector_compare_match1; sljit_s32 tmp1_reg_ind = sljit_get_register_index(TMP1); sljit_s32 str_ptr_reg_ind = sljit_get_register_index(STR_PTR); sljit_s32 data_ind = 0;@@ -207,12 +227,12 @@ if (char1 != char2)   {   bit = char1 ^ char2;-  compare_type = sse2_compare_match1i;+  compare_type = vector_compare_match1i;    if (!is_powerof2(bit))     {     bit = 0;-    compare_type = sse2_compare_match2;+    compare_type = vector_compare_match2;     }   } @@ -349,11 +369,11 @@ static jump_list *fast_requested_char_simd(compiler_common *common, PCRE2_UCHAR char1, PCRE2_UCHAR char2) { DEFINE_COMPILER;+sljit_u8 instruction[8]; struct sljit_label *start; struct sljit_jump *quit; jump_list *not_found = NULL;-sse2_compare_type compare_type = sse2_compare_match1;-sljit_u8 instruction[8];+vector_compare_type compare_type = vector_compare_match1; sljit_s32 tmp1_reg_ind = sljit_get_register_index(TMP1); sljit_s32 str_ptr_reg_ind = sljit_get_register_index(STR_PTR); sljit_s32 data_ind = 0;@@ -366,12 +386,12 @@ if (char1 != char2)   {   bit = char1 ^ char2;-  compare_type = sse2_compare_match1i;+  compare_type = vector_compare_match1i;    if (!is_powerof2(bit))     {     bit = 0;-    compare_type = sse2_compare_match2;+    compare_type = vector_compare_match2;     }   } @@ -476,27 +496,15 @@  #ifndef _WIN64 -static SLJIT_INLINE sljit_u32 max_fast_forward_char_pair_offset(void)-{-#if PCRE2_CODE_UNIT_WIDTH == 8-return 15;-#elif PCRE2_CODE_UNIT_WIDTH == 16-return 7;-#elif PCRE2_CODE_UNIT_WIDTH == 32-return 3;-#else-#error "Unsupported unit width"-#endif-}- #define JIT_HAS_FAST_FORWARD_CHAR_PAIR_SIMD (sljit_has_cpu_feature(SLJIT_HAS_SSE2))  static void fast_forward_char_pair_simd(compiler_common *common, sljit_s32 offs1,   PCRE2_UCHAR char1a, PCRE2_UCHAR char1b, sljit_s32 offs2, PCRE2_UCHAR char2a, PCRE2_UCHAR char2b) { DEFINE_COMPILER;-sse2_compare_type compare1_type = sse2_compare_match1;-sse2_compare_type compare2_type = sse2_compare_match1;+sljit_u8 instruction[8];+vector_compare_type compare1_type = vector_compare_match1;+vector_compare_type compare2_type = vector_compare_match1; sljit_u32 bit1 = 0; sljit_u32 bit2 = 0; sljit_u32 diff = IN_UCHARS(offs1 - offs2);@@ -516,7 +524,6 @@ struct sljit_label *restart; #endif struct sljit_jump *jump[2];-sljit_u8 instruction[8]; int i;  SLJIT_ASSERT(common->mode == PCRE2_JIT_COMPLETE && offs1 > offs2);@@ -549,13 +556,13 @@   bit1 = char1a ^ char1b;   if (is_powerof2(bit1))     {-    compare1_type = sse2_compare_match1i;+    compare1_type = vector_compare_match1i;     OP1(SLJIT_MOV, TMP1, 0, SLJIT_IMM, character_to_int32(char1a | bit1));     OP1(SLJIT_MOV, TMP2, 0, SLJIT_IMM, character_to_int32(bit1));     }   else     {-    compare1_type = sse2_compare_match2;+    compare1_type = vector_compare_match2;     bit1 = 0;     OP1(SLJIT_MOV, TMP1, 0, SLJIT_IMM, character_to_int32(char1a));     OP1(SLJIT_MOV, TMP2, 0, SLJIT_IMM, character_to_int32(char1b));@@ -578,13 +585,13 @@   bit2 = char2a ^ char2b;   if (is_powerof2(bit2))     {-    compare2_type = sse2_compare_match1i;+    compare2_type = vector_compare_match1i;     OP1(SLJIT_MOV, TMP1, 0, SLJIT_IMM, character_to_int32(char2a | bit2));     OP1(SLJIT_MOV, TMP2, 0, SLJIT_IMM, character_to_int32(bit2));     }   else     {-    compare2_type = sse2_compare_match2;+    compare2_type = vector_compare_match2;     bit2 = 0;     OP1(SLJIT_MOV, TMP1, 0, SLJIT_IMM, character_to_int32(char2a));     OP1(SLJIT_MOV, TMP2, 0, SLJIT_IMM, character_to_int32(char2b));@@ -731,9 +738,6 @@  add_jump(compiler, &common->failed_match, CMP(SLJIT_GREATER_EQUAL, STR_PTR, 0, STR_END, 0)); -if (common->match_end_ptr != 0)-  OP1(SLJIT_MOV, STR_END, 0, SLJIT_MEM1(SLJIT_SP), common->match_end_ptr);- #if defined SUPPORT_UNICODE && PCRE2_CODE_UNIT_WIDTH != 32 if (common->utf)   {@@ -760,7 +764,7 @@  #undef SSE2_COMPARE_TYPE_INDEX -#endif /* SLJIT_CONFIG_X86 && !SUPPORT_VALGRIND */+#endif /* SLJIT_CONFIG_X86 */  #if (defined SLJIT_CONFIG_ARM_64 && SLJIT_CONFIG_ARM_64 && (defined __ARM_NEON || defined __ARM_NEON__)) @@ -1121,3 +1125,743 @@ }  #endif /* SLJIT_CONFIG_ARM_64 && SLJIT_CONFIG_ARM_64 */++#if (defined SLJIT_CONFIG_S390X && SLJIT_CONFIG_S390X)++#if PCRE2_CODE_UNIT_WIDTH == 8+#define VECTOR_ELEMENT_SIZE 0+#elif PCRE2_CODE_UNIT_WIDTH == 16+#define VECTOR_ELEMENT_SIZE 1+#elif PCRE2_CODE_UNIT_WIDTH == 32+#define VECTOR_ELEMENT_SIZE 2+#else+#error "Unsupported unit width"+#endif++static void load_from_mem_vector(struct sljit_compiler *compiler, BOOL vlbb, sljit_s32 dst_vreg,+  sljit_s32 base_reg, sljit_s32 index_reg)+{+sljit_u16 instruction[3];++instruction[0] = (sljit_u16)(0xe700 | (dst_vreg << 4) | index_reg);+instruction[1] = (sljit_u16)(base_reg << 12);+instruction[2] = (sljit_u16)((0x8 << 8) | (vlbb ? 0x07 : 0x06));++sljit_emit_op_custom(compiler, instruction, 6);+}++#if PCRE2_CODE_UNIT_WIDTH == 32++static void replicate_imm_vector(struct sljit_compiler *compiler, int step, sljit_s32 dst_vreg,+  PCRE2_UCHAR chr, sljit_s32 tmp_general_reg)+{+sljit_u16 instruction[3];++SLJIT_ASSERT(step >= 0 && step <= 1);++if (chr < 0x7fff)+  {+  if (step == 1)+    return;++  /* VREPI */+  instruction[0] = (sljit_u16)(0xe700 | (dst_vreg << 4));+  instruction[1] = (sljit_u16)chr;+  instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0x8 << 8) | 0x45);+  sljit_emit_op_custom(compiler, instruction, 6);+  return;+  }++if (step == 0)+  {+  OP1(SLJIT_MOV, tmp_general_reg, 0, SLJIT_IMM, chr);++  /* VLVG */+  instruction[0] = (sljit_u16)(0xe700 | (dst_vreg << 4) | sljit_get_register_index(tmp_general_reg));+  instruction[1] = 0;+  instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0x8 << 8) | 0x22);+  sljit_emit_op_custom(compiler, instruction, 6);+  return;+  }++/* VREP */+instruction[0] = (sljit_u16)(0xe700 | (dst_vreg << 4) | dst_vreg);+instruction[1] = 0;+instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0xc << 8) | 0x4d);+sljit_emit_op_custom(compiler, instruction, 6);+}++#endif++static void fast_forward_char_pair_sse2_compare(struct sljit_compiler *compiler, vector_compare_type compare_type,+  int step, sljit_s32 dst_ind, sljit_s32 cmp1_ind, sljit_s32 cmp2_ind, sljit_s32 tmp_ind)+{+sljit_u16 instruction[3];++SLJIT_ASSERT(step >= 0 && step <= 2);++if (step == 1)+  {+  /* VCEQ */+  instruction[0] = (sljit_u16)(0xe700 | (dst_ind << 4) | dst_ind);+  instruction[1] = (sljit_u16)(cmp1_ind << 12);+  instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0xe << 8) | 0xf8);+  sljit_emit_op_custom(compiler, instruction, 6);+  return;+  }++if (compare_type != vector_compare_match2)+  {+  if (step == 0 && compare_type == vector_compare_match1i)+    {+    /* VO */+    instruction[0] = (sljit_u16)(0xe700 | (dst_ind << 4) | dst_ind);+    instruction[1] = (sljit_u16)(cmp2_ind << 12);+    instruction[2] = (sljit_u16)((0xe << 8) | 0x6a);+    sljit_emit_op_custom(compiler, instruction, 6);+    }+  return;+  }++switch (step)+  {+  case 0:+  /* VCEQ */+  instruction[0] = (sljit_u16)(0xe700 | (tmp_ind << 4) | dst_ind);+  instruction[1] = (sljit_u16)(cmp2_ind << 12);+  instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0xe << 8) | 0xf8);+  sljit_emit_op_custom(compiler, instruction, 6);+  return;++  case 2:+  /* VO */+  instruction[0] = (sljit_u16)(0xe700 | (dst_ind << 4) | dst_ind);+  instruction[1] = (sljit_u16)(tmp_ind << 12);+  instruction[2] = (sljit_u16)((0xe << 8) | 0x6a);+  sljit_emit_op_custom(compiler, instruction, 6);+  return;+  }+}++#define JIT_HAS_FAST_FORWARD_CHAR_SIMD 1++static void fast_forward_char_simd(compiler_common *common, PCRE2_UCHAR char1, PCRE2_UCHAR char2, sljit_s32 offset)+{+DEFINE_COMPILER;+sljit_u16 instruction[3];+struct sljit_label *start;+#if defined SUPPORT_UNICODE && PCRE2_CODE_UNIT_WIDTH != 32+struct sljit_label *restart;+#endif+struct sljit_jump *quit;+struct sljit_jump *partial_quit[2];+vector_compare_type compare_type = vector_compare_match1;+sljit_s32 tmp1_reg_ind = sljit_get_register_index(TMP1);+sljit_s32 str_ptr_reg_ind = sljit_get_register_index(STR_PTR);+sljit_s32 data_ind = 0;+sljit_s32 tmp_ind = 1;+sljit_s32 cmp1_ind = 2;+sljit_s32 cmp2_ind = 3;+sljit_s32 zero_ind = 4;+sljit_u32 bit = 0;+int i;++SLJIT_UNUSED_ARG(offset);++if (char1 != char2)+  {+  bit = char1 ^ char2;+  compare_type = vector_compare_match1i;++  if (!is_powerof2(bit))+    {+    bit = 0;+    compare_type = vector_compare_match2;+    }+  }++partial_quit[0] = CMP(SLJIT_GREATER_EQUAL, STR_PTR, 0, STR_END, 0);+if (common->mode == PCRE2_JIT_COMPLETE)+  add_jump(compiler, &common->failed_match, partial_quit[0]);++/* First part (unaligned start) */++OP2(SLJIT_ADD, TMP2, 0, STR_PTR, 0, SLJIT_IMM, 16);++#if PCRE2_CODE_UNIT_WIDTH != 32++/* VREPI */+instruction[0] = (sljit_u16)(0xe700 | (cmp1_ind << 4));+instruction[1] = (sljit_u16)(char1 | bit);+instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0x8 << 8) | 0x45);+sljit_emit_op_custom(compiler, instruction, 6);++if (char1 != char2)+  {+  /* VREPI */+  instruction[0] = (sljit_u16)(0xe700 | (cmp2_ind << 4));+  instruction[1] = (sljit_u16)(bit != 0 ? bit : char2);+  /* instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0x8 << 8) | 0x45); */+  sljit_emit_op_custom(compiler, instruction, 6);+  }++#else /* PCRE2_CODE_UNIT_WIDTH == 32 */++for (int i = 0; i < 2; i++)+  {+  replicate_imm_vector(compiler, i, cmp1_ind, char1 | bit, TMP1);++  if (char1 != char2)+    replicate_imm_vector(compiler, i, cmp2_ind, bit != 0 ? bit : char2, TMP1);+  }++#endif /* PCRE2_CODE_UNIT_WIDTH != 32 */++if (compare_type == vector_compare_match2)+  {+  /* VREPI */+  instruction[0] = (sljit_u16)(0xe700 | (zero_ind << 4));+  instruction[1] = 0;+  instruction[2] = (sljit_u16)((0x8 << 8) | 0x45);+  sljit_emit_op_custom(compiler, instruction, 6);+  }++#if defined SUPPORT_UNICODE && PCRE2_CODE_UNIT_WIDTH != 32+restart = LABEL();+#endif++load_from_mem_vector(compiler, TRUE, data_ind, str_ptr_reg_ind, 0);+OP2(SLJIT_AND, TMP2, 0, TMP2, 0, SLJIT_IMM, ~15);++if (compare_type != vector_compare_match2)+  {+  if (compare_type == vector_compare_match1i)+    fast_forward_char_pair_sse2_compare(compiler, compare_type, 0, data_ind, cmp1_ind, cmp2_ind, tmp_ind);++  /* VFEE */+  instruction[0] = (sljit_u16)(0xe700 | (data_ind << 4) | data_ind);+  instruction[1] = (sljit_u16)((cmp1_ind << 12) | (1 << 4));+  instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0xe << 8) | 0x80);+  sljit_emit_op_custom(compiler, instruction, 6);+  }+else+  {+  for (i = 0; i < 3; i++)+    fast_forward_char_pair_sse2_compare(compiler, compare_type, i, data_ind, cmp1_ind, cmp2_ind, tmp_ind);++  /* VFENE */+  instruction[0] = (sljit_u16)(0xe700 | (data_ind << 4) | data_ind);+  instruction[1] = (sljit_u16)((zero_ind << 12) | (1 << 4));+  instruction[2] = (sljit_u16)((0xe << 8) | 0x81);+  sljit_emit_op_custom(compiler, instruction, 6);+  }++/* TODO: use sljit_set_current_flags */++/* VLGVB */+instruction[0] = (sljit_u16)(0xe700 | (tmp1_reg_ind << 4) | data_ind);+instruction[1] = 7;+instruction[2] = (sljit_u16)((0x4 << 8) | 0x21);+sljit_emit_op_custom(compiler, instruction, 6);++OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, TMP1, 0);+quit = CMP(SLJIT_LESS, STR_PTR, 0, TMP2, 0);++OP2(SLJIT_SUB, STR_PTR, 0, TMP2, 0, SLJIT_IMM, 16);++/* Second part (aligned) */+start = LABEL();++OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, 16);++partial_quit[1] = CMP(SLJIT_GREATER_EQUAL, STR_PTR, 0, STR_END, 0);+if (common->mode == PCRE2_JIT_COMPLETE)+  add_jump(compiler, &common->failed_match, partial_quit[1]);++load_from_mem_vector(compiler, TRUE, data_ind, str_ptr_reg_ind, 0);++if (compare_type != vector_compare_match2)+  {+  if (compare_type == vector_compare_match1i)+    fast_forward_char_pair_sse2_compare(compiler, compare_type, 0, data_ind, cmp1_ind, cmp2_ind, tmp_ind);++  /* VFEE */+  instruction[0] = (sljit_u16)(0xe700 | (data_ind << 4) | data_ind);+  instruction[1] = (sljit_u16)((cmp1_ind << 12) | (1 << 4));+  instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0xe << 8) | 0x80);+  sljit_emit_op_custom(compiler, instruction, 6);+  }+else+  {+  for (i = 0; i < 3; i++)+    fast_forward_char_pair_sse2_compare(compiler, compare_type, i, data_ind, cmp1_ind, cmp2_ind, tmp_ind);++  /* VFENE */+  instruction[0] = (sljit_u16)(0xe700 | (data_ind << 4) | data_ind);+  instruction[1] = (sljit_u16)((zero_ind << 12) | (1 << 4));+  instruction[2] = (sljit_u16)((0xe << 8) | 0x81);+  sljit_emit_op_custom(compiler, instruction, 6);+  }++/* TODO: use sljit_set_current_flags */++/* VLGVB */+instruction[0] = (sljit_u16)(0xe700 | (tmp1_reg_ind << 4) | data_ind);+instruction[1] = 7;+instruction[2] = (sljit_u16)((0x4 << 8) | 0x21);+sljit_emit_op_custom(compiler, instruction, 6);++CMPTO(SLJIT_GREATER_EQUAL, TMP1, 0, SLJIT_IMM, 16, start);++OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, TMP1, 0);++JUMPHERE(quit);++if (common->mode != PCRE2_JIT_COMPLETE)+  {+  JUMPHERE(partial_quit[0]);+  JUMPHERE(partial_quit[1]);+  OP2(SLJIT_SUB | SLJIT_SET_GREATER, SLJIT_UNUSED, 0, STR_PTR, 0, STR_END, 0);+  CMOV(SLJIT_GREATER, STR_PTR, STR_END, 0);+  }+else+  add_jump(compiler, &common->failed_match, CMP(SLJIT_GREATER_EQUAL, STR_PTR, 0, STR_END, 0));++#if defined SUPPORT_UNICODE && PCRE2_CODE_UNIT_WIDTH != 32+if (common->utf && offset > 0)+  {+  SLJIT_ASSERT(common->mode == PCRE2_JIT_COMPLETE);++  OP1(MOV_UCHAR, TMP1, 0, SLJIT_MEM1(STR_PTR), IN_UCHARS(-offset));++  quit = jump_if_utf_char_start(compiler, TMP1);++  OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1));+  add_jump(compiler, &common->failed_match, CMP(SLJIT_GREATER_EQUAL, STR_PTR, 0, STR_END, 0));++  OP2(SLJIT_ADD, TMP2, 0, STR_PTR, 0, SLJIT_IMM, 16);+  JUMPTO(SLJIT_JUMP, restart);++  JUMPHERE(quit);+  }+#endif+}++#define JIT_HAS_FAST_REQUESTED_CHAR_SIMD 1++static jump_list *fast_requested_char_simd(compiler_common *common, PCRE2_UCHAR char1, PCRE2_UCHAR char2)+{+DEFINE_COMPILER;+sljit_u16 instruction[3];+struct sljit_label *start;+struct sljit_jump *quit;+jump_list *not_found = NULL;+vector_compare_type compare_type = vector_compare_match1;+sljit_s32 tmp1_reg_ind = sljit_get_register_index(TMP1);+sljit_s32 tmp3_reg_ind = sljit_get_register_index(TMP3);+sljit_s32 data_ind = 0;+sljit_s32 tmp_ind = 1;+sljit_s32 cmp1_ind = 2;+sljit_s32 cmp2_ind = 3;+sljit_s32 zero_ind = 4;+sljit_u32 bit = 0;+int i;++if (char1 != char2)+  {+  bit = char1 ^ char2;+  compare_type = vector_compare_match1i;++  if (!is_powerof2(bit))+    {+    bit = 0;+    compare_type = vector_compare_match2;+    }+  }++add_jump(compiler, &not_found, CMP(SLJIT_GREATER_EQUAL, TMP1, 0, STR_END, 0));++/* First part (unaligned start) */++OP2(SLJIT_ADD, TMP2, 0, TMP1, 0, SLJIT_IMM, 16);++#if PCRE2_CODE_UNIT_WIDTH != 32++/* VREPI */+instruction[0] = (sljit_u16)(0xe700 | (cmp1_ind << 4));+instruction[1] = (sljit_u16)(char1 | bit);+instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0x8 << 8) | 0x45);+sljit_emit_op_custom(compiler, instruction, 6);++if (char1 != char2)+  {+  /* VREPI */+  instruction[0] = (sljit_u16)(0xe700 | (cmp2_ind << 4));+  instruction[1] = (sljit_u16)(bit != 0 ? bit : char2);+  /* instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0x8 << 8) | 0x45); */+  sljit_emit_op_custom(compiler, instruction, 6);+  }++#else /* PCRE2_CODE_UNIT_WIDTH == 32 */++for (int i = 0; i < 2; i++)+  {+  replicate_imm_vector(compiler, i, cmp1_ind, char1 | bit, TMP3);++  if (char1 != char2)+    replicate_imm_vector(compiler, i, cmp2_ind, bit != 0 ? bit : char2, TMP3);+  }++#endif /* PCRE2_CODE_UNIT_WIDTH != 32 */++if (compare_type == vector_compare_match2)+  {+  /* VREPI */+  instruction[0] = (sljit_u16)(0xe700 | (zero_ind << 4));+  instruction[1] = 0;+  instruction[2] = (sljit_u16)((0x8 << 8) | 0x45);+  sljit_emit_op_custom(compiler, instruction, 6);+  }++load_from_mem_vector(compiler, TRUE, data_ind, tmp1_reg_ind, 0);+OP2(SLJIT_AND, TMP2, 0, TMP2, 0, SLJIT_IMM, ~15);++if (compare_type != vector_compare_match2)+  {+  if (compare_type == vector_compare_match1i)+    fast_forward_char_pair_sse2_compare(compiler, compare_type, 0, data_ind, cmp1_ind, cmp2_ind, tmp_ind);++  /* VFEE */+  instruction[0] = (sljit_u16)(0xe700 | (data_ind << 4) | data_ind);+  instruction[1] = (sljit_u16)((cmp1_ind << 12) | (1 << 4));+  instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0xe << 8) | 0x80);+  sljit_emit_op_custom(compiler, instruction, 6);+  }+else+  {+  for (i = 0; i < 3; i++)+    fast_forward_char_pair_sse2_compare(compiler, compare_type, i, data_ind, cmp1_ind, cmp2_ind, tmp_ind);++  /* VFENE */+  instruction[0] = (sljit_u16)(0xe700 | (data_ind << 4) | data_ind);+  instruction[1] = (sljit_u16)((zero_ind << 12) | (1 << 4));+  instruction[2] = (sljit_u16)((0xe << 8) | 0x81);+  sljit_emit_op_custom(compiler, instruction, 6);+  }++/* TODO: use sljit_set_current_flags */++/* VLGVB */+instruction[0] = (sljit_u16)(0xe700 | (tmp3_reg_ind << 4) | data_ind);+instruction[1] = 7;+instruction[2] = (sljit_u16)((0x4 << 8) | 0x21);+sljit_emit_op_custom(compiler, instruction, 6);++OP2(SLJIT_ADD, TMP1, 0, TMP1, 0, TMP3, 0);+quit = CMP(SLJIT_LESS, TMP1, 0, TMP2, 0);++OP2(SLJIT_SUB, TMP1, 0, TMP2, 0, SLJIT_IMM, 16);++/* Second part (aligned) */+start = LABEL();++OP2(SLJIT_ADD, TMP1, 0, TMP1, 0, SLJIT_IMM, 16);++add_jump(compiler, &not_found, CMP(SLJIT_GREATER_EQUAL, TMP1, 0, STR_END, 0));++load_from_mem_vector(compiler, TRUE, data_ind, tmp1_reg_ind, 0);++if (compare_type != vector_compare_match2)+  {+  if (compare_type == vector_compare_match1i)+    fast_forward_char_pair_sse2_compare(compiler, compare_type, 0, data_ind, cmp1_ind, cmp2_ind, tmp_ind);++  /* VFEE */+  instruction[0] = (sljit_u16)(0xe700 | (data_ind << 4) | data_ind);+  instruction[1] = (sljit_u16)((cmp1_ind << 12) | (1 << 4));+  instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0xe << 8) | 0x80);+  sljit_emit_op_custom(compiler, instruction, 6);+  }+else+  {+  for (i = 0; i < 3; i++)+    fast_forward_char_pair_sse2_compare(compiler, compare_type, i, data_ind, cmp1_ind, cmp2_ind, tmp_ind);++  /* VFENE */+  instruction[0] = (sljit_u16)(0xe700 | (data_ind << 4) | data_ind);+  instruction[1] = (sljit_u16)((zero_ind << 12) | (1 << 4));+  instruction[2] = (sljit_u16)((0xe << 8) | 0x81);+  sljit_emit_op_custom(compiler, instruction, 6);+  }++/* TODO: use sljit_set_current_flags */++/* VLGVB */+instruction[0] = (sljit_u16)(0xe700 | (tmp3_reg_ind << 4) | data_ind);+instruction[1] = 7;+instruction[2] = (sljit_u16)((0x4 << 8) | 0x21);+sljit_emit_op_custom(compiler, instruction, 6);++CMPTO(SLJIT_GREATER_EQUAL, TMP3, 0, SLJIT_IMM, 16, start);++OP2(SLJIT_ADD, TMP1, 0, TMP1, 0, TMP3, 0);++JUMPHERE(quit);+add_jump(compiler, &not_found, CMP(SLJIT_GREATER_EQUAL, TMP1, 0, STR_END, 0));++return not_found;+}++#define JIT_HAS_FAST_FORWARD_CHAR_PAIR_SIMD 1++static void fast_forward_char_pair_simd(compiler_common *common, sljit_s32 offs1,+  PCRE2_UCHAR char1a, PCRE2_UCHAR char1b, sljit_s32 offs2, PCRE2_UCHAR char2a, PCRE2_UCHAR char2b)+{+DEFINE_COMPILER;+sljit_u16 instruction[3];+struct sljit_label *start;+#if defined SUPPORT_UNICODE && PCRE2_CODE_UNIT_WIDTH != 32+struct sljit_label *restart;+#endif+struct sljit_jump *quit;+struct sljit_jump *jump[2];+vector_compare_type compare1_type = vector_compare_match1;+vector_compare_type compare2_type = vector_compare_match1;+sljit_u32 bit1 = 0;+sljit_u32 bit2 = 0;+sljit_s32 diff = IN_UCHARS(offs2 - offs1);+sljit_s32 tmp1_reg_ind = sljit_get_register_index(TMP1);+sljit_s32 tmp2_reg_ind = sljit_get_register_index(TMP2);+sljit_s32 str_ptr_reg_ind = sljit_get_register_index(STR_PTR);+sljit_s32 data1_ind = 0;+sljit_s32 data2_ind = 1;+sljit_s32 tmp1_ind = 2;+sljit_s32 tmp2_ind = 3;+sljit_s32 cmp1a_ind = 4;+sljit_s32 cmp1b_ind = 5;+sljit_s32 cmp2a_ind = 6;+sljit_s32 cmp2b_ind = 7;+sljit_s32 zero_ind = 8;+int i;++SLJIT_ASSERT(common->mode == PCRE2_JIT_COMPLETE && offs1 > offs2);+SLJIT_ASSERT(-diff <= (sljit_s32)IN_UCHARS(max_fast_forward_char_pair_offset()));+SLJIT_ASSERT(tmp1_reg_ind != 0 && tmp2_reg_ind != 0);++if (char1a != char1b)+  {+  bit1 = char1a ^ char1b;+  compare1_type = vector_compare_match1i;++  if (!is_powerof2(bit1))+    {+    bit1 = 0;+    compare1_type = vector_compare_match2;+    }+  }++if (char2a != char2b)+  {+  bit2 = char2a ^ char2b;+  compare2_type = vector_compare_match1i;++  if (!is_powerof2(bit2))+    {+    bit2 = 0;+    compare2_type = vector_compare_match2;+    }+  }++/* Initialize. */+if (common->match_end_ptr != 0)+  {+  OP1(SLJIT_MOV, TMP1, 0, SLJIT_MEM1(SLJIT_SP), common->match_end_ptr);+  OP1(SLJIT_MOV, TMP3, 0, STR_END, 0);+  OP2(SLJIT_ADD, TMP1, 0, TMP1, 0, SLJIT_IMM, IN_UCHARS(offs1 + 1));++  OP2(SLJIT_SUB | SLJIT_SET_LESS, SLJIT_UNUSED, 0, TMP1, 0, STR_END, 0);+  CMOV(SLJIT_LESS, STR_END, TMP1, 0);+  }++OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(offs1));+add_jump(compiler, &common->failed_match, CMP(SLJIT_GREATER_EQUAL, STR_PTR, 0, STR_END, 0));+OP2(SLJIT_AND, TMP2, 0, STR_PTR, 0, SLJIT_IMM, ~15);++#if PCRE2_CODE_UNIT_WIDTH != 32++OP2(SLJIT_SUB, TMP1, 0, STR_PTR, 0, SLJIT_IMM, -diff);++/* VREPI */+instruction[0] = (sljit_u16)(0xe700 | (cmp1a_ind << 4));+instruction[1] = (sljit_u16)(char1a | bit1);+instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0x8 << 8) | 0x45);+sljit_emit_op_custom(compiler, instruction, 6);++if (char1a != char1b)+  {+  /* VREPI */+  instruction[0] = (sljit_u16)(0xe700 | (cmp1b_ind << 4));+  instruction[1] = (sljit_u16)(bit1 != 0 ? bit1 : char1b);+  /* instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0x8 << 8) | 0x45); */+  sljit_emit_op_custom(compiler, instruction, 6);+  }++/* VREPI */+instruction[0] = (sljit_u16)(0xe700 | (cmp2a_ind << 4));+instruction[1] = (sljit_u16)(char2a | bit2);+/* instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0x8 << 8) | 0x45); */+sljit_emit_op_custom(compiler, instruction, 6);++if (char2a != char2b)+  {+  /* VREPI */+  instruction[0] = (sljit_u16)(0xe700 | (cmp2b_ind << 4));+  instruction[1] = (sljit_u16)(bit2 != 0 ? bit2 : char2b);+  /* instruction[2] = (sljit_u16)((VECTOR_ELEMENT_SIZE << 12) | (0x8 << 8) | 0x45); */+  sljit_emit_op_custom(compiler, instruction, 6);+  }++#else /* PCRE2_CODE_UNIT_WIDTH == 32 */++for (int i = 0; i < 2; i++)+  {+  replicate_imm_vector(compiler, i, cmp1a_ind, char1a | bit1, TMP1);++  if (char1a != char1b)+    replicate_imm_vector(compiler, i, cmp1b_ind, bit1 != 0 ? bit1 : char1b, TMP1);++  replicate_imm_vector(compiler, i, cmp2a_ind, char2a | bit2, TMP1);++  if (char2a != char2b)+    replicate_imm_vector(compiler, i, cmp2b_ind, bit2 != 0 ? bit2 : char2b, TMP1);+  }++OP2(SLJIT_SUB, TMP1, 0, STR_PTR, 0, SLJIT_IMM, -diff);++#endif /* PCRE2_CODE_UNIT_WIDTH != 32 */++/* VREPI */+instruction[0] = (sljit_u16)(0xe700 | (zero_ind << 4));+instruction[1] = 0;+instruction[2] = (sljit_u16)((0x8 << 8) | 0x45);+sljit_emit_op_custom(compiler, instruction, 6);++#if defined SUPPORT_UNICODE && PCRE2_CODE_UNIT_WIDTH != 32+restart = LABEL();+#endif++jump[0] = CMP(SLJIT_LESS, TMP1, 0, TMP2, 0);+load_from_mem_vector(compiler, TRUE, data2_ind, tmp1_reg_ind, 0);+jump[1] = JUMP(SLJIT_JUMP);+JUMPHERE(jump[0]);+load_from_mem_vector(compiler, FALSE, data2_ind, tmp1_reg_ind, 0);+JUMPHERE(jump[1]);++load_from_mem_vector(compiler, TRUE, data1_ind, str_ptr_reg_ind, 0);+OP2(SLJIT_ADD, TMP2, 0, TMP2, 0, SLJIT_IMM, 16);++for (i = 0; i < 3; i++)+  {+  fast_forward_char_pair_sse2_compare(compiler, compare1_type, i, data1_ind, cmp1a_ind, cmp1b_ind, tmp1_ind);+  fast_forward_char_pair_sse2_compare(compiler, compare2_type, i, data2_ind, cmp2a_ind, cmp2b_ind, tmp2_ind);+  }++/* VN */+instruction[0] = (sljit_u16)(0xe700 | (data1_ind << 4) | data1_ind);+instruction[1] = (sljit_u16)(data2_ind << 12);+instruction[2] = (sljit_u16)((0xe << 8) | 0x68);+sljit_emit_op_custom(compiler, instruction, 6);++/* VFENE */+instruction[0] = (sljit_u16)(0xe700 | (data1_ind << 4) | data1_ind);+instruction[1] = (sljit_u16)((zero_ind << 12) | (1 << 4));+instruction[2] = (sljit_u16)((0xe << 8) | 0x81);+sljit_emit_op_custom(compiler, instruction, 6);++/* TODO: use sljit_set_current_flags */++/* VLGVB */+instruction[0] = (sljit_u16)(0xe700 | (tmp1_reg_ind << 4) | data1_ind);+instruction[1] = 7;+instruction[2] = (sljit_u16)((0x4 << 8) | 0x21);+sljit_emit_op_custom(compiler, instruction, 6);++OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, TMP1, 0);+quit = CMP(SLJIT_LESS, STR_PTR, 0, TMP2, 0);++OP2(SLJIT_SUB, STR_PTR, 0, TMP2, 0, SLJIT_IMM, 16);+OP1(SLJIT_MOV, TMP1, 0, SLJIT_IMM, diff);++/* Main loop. */+start = LABEL();++OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, 16);+add_jump(compiler, &common->failed_match, CMP(SLJIT_GREATER_EQUAL, STR_PTR, 0, STR_END, 0));++load_from_mem_vector(compiler, FALSE, data1_ind, str_ptr_reg_ind, 0);+load_from_mem_vector(compiler, FALSE, data2_ind, str_ptr_reg_ind, tmp1_reg_ind);++for (i = 0; i < 3; i++)+  {+  fast_forward_char_pair_sse2_compare(compiler, compare1_type, i, data1_ind, cmp1a_ind, cmp1b_ind, tmp1_ind);+  fast_forward_char_pair_sse2_compare(compiler, compare2_type, i, data2_ind, cmp2a_ind, cmp2b_ind, tmp2_ind);+  }++/* VN */+instruction[0] = (sljit_u16)(0xe700 | (data1_ind << 4) | data1_ind);+instruction[1] = (sljit_u16)(data2_ind << 12);+instruction[2] = (sljit_u16)((0xe << 8) | 0x68);+sljit_emit_op_custom(compiler, instruction, 6);++/* VFENE */+instruction[0] = (sljit_u16)(0xe700 | (data1_ind << 4) | data1_ind);+instruction[1] = (sljit_u16)((zero_ind << 12) | (1 << 4));+instruction[2] = (sljit_u16)((0xe << 8) | 0x81);+sljit_emit_op_custom(compiler, instruction, 6);++/* TODO: use sljit_set_current_flags */++/* VLGVB */+instruction[0] = (sljit_u16)(0xe700 | (tmp2_reg_ind << 4) | data1_ind);+instruction[1] = 7;+instruction[2] = (sljit_u16)((0x4 << 8) | 0x21);+sljit_emit_op_custom(compiler, instruction, 6);++CMPTO(SLJIT_GREATER_EQUAL, TMP2, 0, SLJIT_IMM, 16, start);++OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, TMP2, 0);++JUMPHERE(quit);++add_jump(compiler, &common->failed_match, CMP(SLJIT_GREATER_EQUAL, STR_PTR, 0, STR_END, 0));++#if defined SUPPORT_UNICODE && PCRE2_CODE_UNIT_WIDTH != 32+if (common->utf)+  {+  SLJIT_ASSERT(common->mode == PCRE2_JIT_COMPLETE);++  OP1(MOV_UCHAR, TMP1, 0, SLJIT_MEM1(STR_PTR), IN_UCHARS(-offs1));++  quit = jump_if_utf_char_start(compiler, TMP1);++  OP2(SLJIT_ADD, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(1));+  add_jump(compiler, &common->failed_match, CMP(SLJIT_GREATER_EQUAL, STR_PTR, 0, STR_END, 0));++  /* TMP1 contains diff. */+  OP2(SLJIT_AND, TMP2, 0, STR_PTR, 0, SLJIT_IMM, ~15);+  OP2(SLJIT_SUB, TMP1, 0, STR_PTR, 0, SLJIT_IMM, -diff);+  JUMPTO(SLJIT_JUMP, restart);++  JUMPHERE(quit);+  }+#endif++OP2(SLJIT_SUB, STR_PTR, 0, STR_PTR, 0, SLJIT_IMM, IN_UCHARS(offs1));++if (common->match_end_ptr != 0)+  OP1(SLJIT_MOV, STR_END, 0, TMP3, 0);+}++#endif /* SLJIT_CONFIG_S390X */++#endif /* !SUPPORT_VALGRIND */

@@ -818,10 +818,12 @@        /* N is now the frame of the recursion; the previous frame is at the       OP_RECURSE position. Go back there, copying the current subject position-      and mark, and move on past the OP_RECURSE. */+      and mark, and the start_match position (\K might have changed it), and+      then move on past the OP_RECURSE. */        P->eptr = Feptr;       P->mark = Fmark;+      P->start_match = Fstart_match;       F = P;       Fecode += 1 + LINK_SIZE;       continue;

@@ -7,7 +7,7 @@                         Written by Philip Hazel      Original API code Copyright (c) 1997-2012 University of Cambridge-          New API code Copyright (c) 2016-2019 University of Cambridge+          New API code Copyright (c) 2016-2021 University of Cambridge  ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without@@ -175,6 +175,20 @@   +#if 0  /* REMOVE THIS CODE */++The code below was created for 10.33 (see ChangeLog 10.33 #4) when the+POSIX functions were given pcre2_... names instead of the traditional POSIX+names. However, it has proved to be more troublesome than useful. There have+been at least two cases where a program links with two others, one of which+uses the POSIX library and the other uses the PCRE2 POSIX functions, thus+causing two instances of the POSIX runctions to exist, leading to trouble. For+10.37 this code is commented out. In due course it can be removed if there are+no issues. The only small worry is the comment below about languages that do+not include pcre2posix.h. If there are any such cases, they will have to use+the PCRE2 names.++ /************************************************* *      Wrappers with traditional POSIX names     * *************************************************/@@ -218,7 +232,7 @@ { return pcre2_regexec(preg, string, nmatch, pmatch, eflags); }-+#endif   /*************************************************

@@ -158,6 +158,8 @@ #define SLJIT_CONFIG_MIPS_64 1 #elif defined(__sparc__) || defined(__sparc) #define SLJIT_CONFIG_SPARC_32 1+#elif defined(__s390x__)+#define SLJIT_CONFIG_S390X 1 #else /* Unsupported architecture */ #define SLJIT_CONFIG_UNSUPPORTED 1

@@ -79,6 +79,7 @@ */  #ifdef _WIN32+#define SLJIT_UPDATE_WX_FLAGS(from, to, enable_exec)  static SLJIT_INLINE void* alloc_chunk(sljit_uw size) {@@ -91,96 +92,108 @@ 	VirtualFree(chunk, 0, MEM_RELEASE); } -#else--#ifdef __APPLE__-#ifdef MAP_ANON-/* Configures TARGET_OS_OSX when appropriate */-#include <TargetConditionals.h>--#if TARGET_OS_OSX && defined(MAP_JIT)-#include <sys/utsname.h>-#endif /* TARGET_OS_OSX && MAP_JIT */--#ifdef MAP_JIT+#else /* POSIX */ +#if defined(__APPLE__) && defined(MAP_JIT) /*    On macOS systems, returns MAP_JIT if it is defined _and_ we're running on a-   version where it's OK to have more than one JIT block.+   version where it's OK to have more than one JIT block or where MAP_JIT is+   required.    On non-macOS systems, returns MAP_JIT if it is defined. */+#include <TargetConditionals.h>+#if TARGET_OS_OSX+#if defined SLJIT_CONFIG_X86 && SLJIT_CONFIG_X86+#ifdef MAP_ANON+#include <sys/utsname.h>+#include <stdlib.h>++#define SLJIT_MAP_JIT	(get_map_jit_flag())+ static SLJIT_INLINE int get_map_jit_flag() {-#if TARGET_OS_OSX-	sljit_sw page_size = get_page_alignment() + 1;+	sljit_sw page_size; 	void *ptr;+	struct utsname name; 	static int map_jit_flag = -1; -	/*-	  The following code is thread safe because multiple initialization-	  sets map_jit_flag to the same value and the code has no side-effects.-	  Changing the kernel version witout system restart is (very) unlikely.-	*/-	if (map_jit_flag == -1) {-		struct utsname name;-+	if (map_jit_flag < 0) { 		map_jit_flag = 0; 		uname(&name); -		/* Kernel version for 10.14.0 (Mojave) */+		/* Kernel version for 10.14.0 (Mojave) or later */ 		if (atoi(name.release) >= 18) {+			page_size = get_page_alignment() + 1; 			/* Only use MAP_JIT if a hardened runtime is used */--			ptr = mmap(NULL, page_size, PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANON, -1, 0);+			ptr = mmap(NULL, page_size, PROT_WRITE | PROT_EXEC,+					MAP_PRIVATE | MAP_ANON, -1, 0); -			if (ptr == MAP_FAILED) {-				map_jit_flag = MAP_JIT;-			} else {+			if (ptr != MAP_FAILED) 				munmap(ptr, page_size);-			}+			else+				map_jit_flag = MAP_JIT; 		} 	}- 	return map_jit_flag;-#else /* !TARGET_OS_OSX */-	return MAP_JIT;-#endif /* TARGET_OS_OSX */ }--#endif /* MAP_JIT */ #endif /* MAP_ANON */-#endif /* __APPLE__ */+#else /* !SLJIT_CONFIG_X86 */+#if !(defined SLJIT_CONFIG_ARM && SLJIT_CONFIG_ARM)+#error Unsupported architecture+#endif /* SLJIT_CONFIG_ARM */+#include <pthread.h> +#define SLJIT_MAP_JIT	(MAP_JIT)+#define SLJIT_UPDATE_WX_FLAGS(from, to, enable_exec) \+                        apple_update_wx_flags(enable_exec)++static SLJIT_INLINE void apple_update_wx_flags(sljit_s32 enable_exec)+{+	pthread_jit_write_protect_np(enable_exec);+}+#endif /* SLJIT_CONFIG_X86 */+#else /* !TARGET_OS_OSX */+#define SLJIT_MAP_JIT	(MAP_JIT)+#endif /* TARGET_OS_OSX */+#endif /* __APPLE__ && MAP_JIT */+#ifndef SLJIT_UPDATE_WX_FLAGS+#define SLJIT_UPDATE_WX_FLAGS(from, to, enable_exec)+#endif /* !SLJIT_UPDATE_WX_FLAGS */+#ifndef SLJIT_MAP_JIT+#define SLJIT_MAP_JIT	(0)+#endif /* !SLJIT_MAP_JIT */+ static SLJIT_INLINE void* alloc_chunk(sljit_uw size) { 	void *retval;-	const int prot = PROT_READ | PROT_WRITE | PROT_EXEC;--#ifdef MAP_ANON--	int flags = MAP_PRIVATE | MAP_ANON;+	int prot = PROT_READ | PROT_WRITE | PROT_EXEC;+	int flags = MAP_PRIVATE;+	int fd = -1; -#ifdef MAP_JIT-	flags |= get_map_jit_flag();+#ifdef PROT_MAX+	prot |= PROT_MAX(prot); #endif -	retval = mmap(NULL, size, prot, flags, -1, 0);+#ifdef MAP_ANON+	flags |= MAP_ANON | SLJIT_MAP_JIT; #else /* !MAP_ANON */ 	if (SLJIT_UNLIKELY((dev_zero < 0) && open_dev_zero())) 		return NULL; -	retval = mmap(NULL, size, prot, MAP_PRIVATE, dev_zero, 0);+	fd = dev_zero; #endif /* MAP_ANON */ +	retval = mmap(NULL, size, prot, flags, fd, 0); 	if (retval == MAP_FAILED)-		retval = NULL;-	else {-		if (mprotect(retval, size, prot) < 0) {-			munmap(retval, size);-			retval = NULL;-		}+		return NULL;++	if (mprotect(retval, size, PROT_READ | PROT_WRITE | PROT_EXEC) < 0) {+		munmap(retval, size);+		return NULL; 	} +	SLJIT_UPDATE_WX_FLAGS(retval, (uint8_t *)retval + size, 0);+ 	return retval; } @@ -189,7 +202,7 @@ 	munmap(chunk, size); } -#endif+#endif /* windows */  /* --------------------------------------------------------------------- */ /*  Common functions                                                     */@@ -261,6 +274,7 @@ 	while (free_block) { 		if (free_block->size >= size) { 			chunk_size = free_block->size;+			SLJIT_UPDATE_WX_FLAGS(NULL, NULL, 0); 			if (chunk_size > size + 64) { 				/* We just cut a block from the end of the free block. */ 				chunk_size -= size;@@ -326,6 +340,7 @@ 	allocated_size -= header->size;  	/* Connecting free blocks together if possible. */+	SLJIT_UPDATE_WX_FLAGS(NULL, NULL, 0);  	/* If header->prev_size == 0, free_block will equal to header. 	   In this case, free_block->header.size will be > 0. */@@ -358,6 +373,7 @@ 		} 	} +	SLJIT_UPDATE_WX_FLAGS(NULL, NULL, 1); 	SLJIT_ALLOCATOR_UNLOCK(); } @@ -367,6 +383,7 @@ 	struct free_block* next_free_block;  	SLJIT_ALLOCATOR_LOCK();+	SLJIT_UPDATE_WX_FLAGS(NULL, NULL, 0);  	free_block = free_blocks; 	while (free_block) {@@ -381,5 +398,6 @@ 	}  	SLJIT_ASSERT((total_size && free_blocks) || (!total_size && !free_blocks));+	SLJIT_UPDATE_WX_FLAGS(NULL, NULL, 1); 	SLJIT_ALLOCATOR_UNLOCK(); }

@@ -42,7 +42,7 @@ typedef sljit_uw sljit_ins;  /* Instruction tags (most significant halfword). */-const sljit_ins sljit_ins_const = (sljit_ins)1 << 48;+static const sljit_ins sljit_ins_const = (sljit_ins)1 << 48;  static const sljit_u8 reg_map[SLJIT_NUMBER_OF_REGISTERS + 4] = { 	14, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 0, 1@@ -66,22 +66,22 @@  * will be retired ASAP (TODO: carenas)  */ -const sljit_gpr r0 = 0;	/* reg_map[SLJIT_NUMBER_OF_REGISTERS + 2]: 0 in address calculations; reserved */-const sljit_gpr r1 = 1;	/* reg_map[SLJIT_NUMBER_OF_REGISTERS + 3]: reserved */-const sljit_gpr r2 = 2;	/* reg_map[1]: 1st argument */-const sljit_gpr r3 = 3;	/* reg_map[2]: 2nd argument */-const sljit_gpr r4 = 4;	/* reg_map[3]: 3rd argument */-const sljit_gpr r5 = 5;	/* reg_map[4]: 4th argument */-const sljit_gpr r6 = 6;	/* reg_map[5]: 5th argument; 1st saved register */-const sljit_gpr r7 = 7;	/* reg_map[6] */-const sljit_gpr r8 = 8;	/* reg_map[7] */-const sljit_gpr r9 = 9;	/* reg_map[8] */-const sljit_gpr r10 = 10;	/* reg_map[9] */-const sljit_gpr r11 = 11;	/* reg_map[10] */-const sljit_gpr r12 = 12;	/* reg_map[11]: GOT */-const sljit_gpr r13 = 13;	/* reg_map[12]: Literal Pool pointer */-const sljit_gpr r14 = 14;	/* reg_map[0]: return address and flag register */-const sljit_gpr r15 = 15;	/* reg_map[SLJIT_NUMBER_OF_REGISTERS + 1]: stack pointer */+static const sljit_gpr r0 = 0;		/* reg_map[SLJIT_NUMBER_OF_REGISTERS + 2]: 0 in address calculations; reserved */+static const sljit_gpr r1 = 1;		/* reg_map[SLJIT_NUMBER_OF_REGISTERS + 3]: reserved */+static const sljit_gpr r2 = 2;		/* reg_map[1]: 1st argument */+static const sljit_gpr r3 = 3;		/* reg_map[2]: 2nd argument */+static const sljit_gpr r4 = 4;		/* reg_map[3]: 3rd argument */+static const sljit_gpr r5 = 5;		/* reg_map[4]: 4th argument */+static const sljit_gpr r6 = 6;		/* reg_map[5]: 5th argument; 1st saved register */+static const sljit_gpr r7 = 7;		/* reg_map[6] */+static const sljit_gpr r8 = 8;		/* reg_map[7] */+static const sljit_gpr r9 = 9;		/* reg_map[8] */+static const sljit_gpr r10 = 10;	/* reg_map[9] */+static const sljit_gpr r11 = 11;	/* reg_map[10] */+static const sljit_gpr r12 = 12;	/* reg_map[11]: GOT */+static const sljit_gpr r13 = 13;	/* reg_map[12]: Literal Pool pointer */+static const sljit_gpr r14 = 14;	/* reg_map[0]: return address and flag register */+static const sljit_gpr r15 = 15;	/* reg_map[SLJIT_NUMBER_OF_REGISTERS + 1]: stack pointer */  /* WARNING: r12 and r13 shouldn't be used as per ABI recommendation */ /* TODO(carenas): r12 might conflict in PIC code, reserve? */@@ -100,8 +100,8 @@ /* Link registers. The normal link register is r14, but since    we use that for flags we need to use r0 instead to do fast    calls so that flags are preserved. */-const sljit_gpr link_r = 14;     /* r14 */-const sljit_gpr fast_link_r = 0; /* r0 */+static const sljit_gpr link_r = 14;     /* r14 */+static const sljit_gpr fast_link_r = 0; /* r0 */  /* Flag register layout: @@ -110,7 +110,7 @@    |      ZERO     | 0 | 0 |  C C  |///////|    +---------------+---+---+-------+-------+ */-const sljit_gpr flag_r = 14; /* r14 */+static const sljit_gpr flag_r = 14; /* r14 */  struct sljit_s390x_const { 	struct sljit_const const_; /* must be first */@@ -1465,7 +1465,8 @@ 	op = GET_OPCODE(op) | (op & SLJIT_I32_OP); 	switch (op) { 	case SLJIT_BREAKPOINT:-		/* TODO(mundaym): insert real breakpoint? */+		/* The following invalid instruction is emitted by gdb. */+		return push_inst(compiler, 0x0001 /* 2-byte trap */); 	case SLJIT_NOP: 		return push_inst(compiler, 0x0700 /* 2-byte nop */); 	case SLJIT_LMUL_UW:

@@ -48,7 +48,7 @@ static SLJIT_INLINE void allocator_grab_lock(void) { 	HANDLE lock;-	if (SLJIT_UNLIKELY(!allocator_lock)) {+	if (SLJIT_UNLIKELY(!InterlockedCompareExchangePointer(&allocator_lock, NULL, NULL))) { 		lock = CreateMutex(NULL, FALSE, NULL); 		if (InterlockedCompareExchangePointer(&allocator_lock, lock, NULL)) 			CloseHandle(lock);@@ -146,9 +146,13 @@ #include <unistd.h>  static SLJIT_INLINE sljit_sw get_page_alignment(void) {-	static sljit_sw sljit_page_align;-	if (!sljit_page_align) {+	static sljit_sw sljit_page_align = -1;+	if (sljit_page_align < 0) {+#ifdef _SC_PAGESIZE 		sljit_page_align = sysconf(_SC_PAGESIZE);+#else+		sljit_page_align = getpagesize();+#endif 		/* Should never happen. */ 		if (sljit_page_align < 0) 			sljit_page_align = 4096;

@@ -121,14 +121,18 @@ 	static pthread_mutex_t se_lock = PTHREAD_MUTEX_INITIALIZER; #endif 	static int se_protected = !SLJIT_PROT_WX;+	int prot = PROT_READ | PROT_WRITE | SLJIT_PROT_WX; 	sljit_uw* ptr;  	if (SLJIT_UNLIKELY(se_protected < 0)) 		return NULL; +#ifdef PROT_MAX+	prot |= PROT_MAX(PROT_READ | PROT_WRITE | PROT_EXEC);+#endif+ 	size += sizeof(sljit_uw);-	ptr = (sljit_uw*)mmap(NULL, size, PROT_READ | PROT_WRITE | SLJIT_PROT_WX,-				MAP_PRIVATE | MAP_ANON, -1, 0);+	ptr = (sljit_uw*)mmap(NULL, size, prot, MAP_PRIVATE | MAP_ANON, -1, 0);  	if (ptr == MAP_FAILED) 		return NULL;

@@ -1,5 +1,3 @@-// Copied per the PCRE2 source's NON-AUTOTOOLS-BUILD instructions.- /************************************************* *      Perl-Compatible Regular Expressions       * *************************************************/

@@ -1,7 +1,7 @@ module Text.Regex.Pcre2 (     -- * Matching and substitution     {-|-    /__Introduction__/+    === __Introduction__      Atop the low-level binding to the C API, we present a high-level interface     to add regular expressions to Haskell programs.@@ -45,7 +45,7 @@     cognitive overhead, and (for lenses) library dependencies, so it\'s really a     matter of finding the best trade-offs for her case. -    /__Definitions__/+    === __Definitions__      [Pattern]:  The string defining a regular expression.  Refer to     syntax [here](https://pcre.org/current/doc/html/pcre2pattern.html).@@ -75,7 +75,7 @@     @(?\<foo\>...)@.  Whether they have names or not, captures are always     numbered as described above. -    /__Performance__/+    === __Performance__      Each API function is designed such that, when a regex is obtained, the     underlying C data generated from the pattern and any options is reused for@@ -99,23 +99,50 @@     Note: Template Haskell regexes are immune from this problem and may be     freely inlined; see below. -    /__Handling errors__/+    Also of note is the optimization that, for each capture that\'s more than+    half the length of the subject, a zero-copy `Text` is produced in constant+    time and space.  This can yield a large performance boost in many cases,+    for example when splitting lines into key-value pairs as in+    the [teaser](https://github.com/sjshuck/hs-pcre2#teasers).  A downside,+    however, is that retaining these slices in memory will carry the overhead+    of the dead portions of the subject (still guaranteed to be less than the+    slices in total size). -    In a few places we use the `Alternative` typeclass to optionally return-    match results, expressing success via `pure` and failure via `empty`.-    Typically the user will choose the instance `Maybe`, but other useful ones-    exist, notably @[STM](https://hackage.haskell.org/package/stm/docs/Control-Monad-STM.html#t:STM)@,+    === __Handling results and errors__++    In contrast to [other](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/match)+    [APIs](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/matchAll)&mdash;including+    previous versions of [this library](https://github.com/sjshuck/hs-pcre2/issues/17)&mdash;where+    there are separate functions to request single versus global matching, we+    accomplish this /(since 2.0.0)/ in a unified fashion using the `Alternative`+    typeclass.  Typically the user will choose from two instances, `Maybe` and+    `[]`:++    > b2 :: (Alternative f) => Text -> f Text+    > b2 = match "b.."+    >+    > -- Zero or one match+    > findB2 :: Text -> Maybe Text+    > findB2 = b2+    >+    > -- Zero or more matches+    > findAllB2s :: Text -> [Text]+    > findAllB2s = b2++    Other instances exist for niche uses,+    notably @[STM](https://hackage.haskell.org/package/stm/docs/Control-Monad-STM.html#t:STM)@,     that of [optparse-applicative](https://hackage.haskell.org/package/optparse-applicative/docs/Options-Applicative.html#t:Parser),     and those of parser combinator libraries such     as [megaparsec](https://hackage.haskell.org/package/megaparsec/docs/Text-Megaparsec.html#t:ParsecT).      By contrast, user errors are thrown purely.  If a user error is to be     caught, it must be at the site where the match or substitution results are-    evaluated&#x2014;in other words, wherever the regex is applied to a subject.-    Even pattern compile errors are deferred to match sites, due to the way this-    library employs `System.IO.Unsafe.unsafePerformIO` to implement laziness.+    evaluated.  As a particular consequence, pattern compile errors are deferred+    to match sites.      >>> broken = match "*"+    >>> :t broken+    broken :: Alternative f => Text -> f Text     >>> broken "foo"     *** Exception: pcre2_compile: quantifier does not follow a repeatable item                         *@@ -124,30 +151,23 @@     `Control.Exception.evaluate` comes in handy to force results into the `IO`     monad in order to catch errors reliably: -    >>> evaluate (broken "foo") `catch` \(_ :: SomePcre2Exception) -> return Nothing+    >>> :set -XTypeApplications+    >>> handle @SomePcre2Exception (\_ -> return Nothing) $ evaluate $ broken "foo"     Nothing      Or simply select `IO` as the `Alternative` instance: -    >>> :t broken-    broken :: Alternative f => Text -> f Text-    >>> broken "foo" `catch` \(_ :: SomePcre2Exception) -> return "broken"+    >>> handle @SomePcre2Exception (\_ -> return "broken") $ broken "foo"     "broken"     -}      -- ** Basic matching functions     match,     matchOpt,-    matchAll,-    matchAllOpt,     matches,     matchesOpt,     captures,     capturesOpt,-    capturesA,-    capturesAOpt,-    capturesAll,-    capturesAllOpt,      -- ** PCRE2-native substitution     sub,@@ -260,14 +280,11 @@         AltCircumflex,         AltVerbNames,         Anchored,-        AutoCallout,-        BadEscapeIsLiteral,         Bsr,         Caseless,         DepthLimit,         DollarEndOnly,         DotAll,-        DupNames,         EndAnchored,         EscapedCrIsLf,         Extended,@@ -302,23 +319,10 @@         SubUnknownUnset,         SubUnsetEmpty,         Ucp,-        Ungreedy,-        UnsafeCallout,-        UnsafeCompileRecGuard,-        UnsafeSubCallout,-        Utf),+        Ungreedy),      Bsr(..),     Newline(..),--    -- ** Callout interface--    CalloutInfo(..),-    CalloutIndex(..),-    CalloutResult(..),-    -- *** Substitution callouts-    SubCalloutInfo(..),-    SubCalloutResult(..),      -- * User errors     SomePcre2Exception(),

@@ -19,9 +19,11 @@ import           Control.Exception          hiding (TypeError) import           Control.Monad.State.Strict import           Data.Either                (partitionEithers)+import           Data.Foldable              (toList) import           Data.Function              ((&)) import           Data.Functor               ((<&>)) import           Data.Functor.Const         (Const(..))+import           Data.Functor.Identity      (Identity(..)) import           Data.IORef import           Data.IntMap.Strict         (IntMap) import qualified Data.IntMap.Strict         as IM@@ -47,6 +49,8 @@  -- * General utilities +type FfiWrapper f = f -> IO (FunPtr f)+ -- | There is no @nullForeignPtr@ to pass to `withForeignPtr`, so we have to -- fake it with a `Maybe`. withForeignOrNullPtr :: Maybe (ForeignPtr a) -> (Ptr a -> IO b) -> IO b@@ -93,23 +97,23 @@  -- ** Fast @Text@ slicing -data SliceRange = SliceRange+data Slice = Slice     {-# UNPACK #-} !Text.I16     {-# UNPACK #-} !Text.I16  -- | Zero-copy slice a 'Text'.  An unset capture is represented by a -- `pcre2_UNSET` range and is interpreted in this library as `Text.empty`.-thinSlice :: Text -> SliceRange -> Text-thinSlice text (SliceRange off offEnd)+thinSlice :: Text -> Slice -> Text+thinSlice text (Slice off offEnd)     | off == fromIntegral pcre2_UNSET = Text.empty     | otherwise                       = text         & Text.takeWord16 offEnd         & Text.dropWord16 off  -- | Slice a 'Text', copying if it\'s less than half of the original.-slice :: Text -> SliceRange -> Text-slice text sliceRange =-    let substring = thinSlice text sliceRange+smartSlice :: Text -> Slice -> Text+smartSlice text slice =+    let substring = thinSlice text slice     in if Text.length substring > Text.length text `div` 2         then substring         else Text.copy substring@@ -147,13 +151,8 @@ toAlternativeOf :: (Alternative f) => Getting (Alt f a) s a -> s -> f a toAlternativeOf l = let alt = Alt . pure in getAlt . view (l . to alt) --- | See https://github.com/sjshuck/hs-pcre2/issues/17.--- This should go away with https://github.com/sjshuck/hs-pcre2/issues/18.-toAlternativeOf1 :: (Alternative f) => Getting (Alt Maybe a) s a -> s -> f a-toAlternativeOf1 l = maybe empty pure . preview l--_headNE :: Lens' (NonEmpty a) a-_headNE f (x :| xs) = f x <&> (:| xs)+_Identity :: Lens' (Identity a) a+_Identity f (Identity x) = Identity <$> f x  -- ** Streaming support @@ -236,6 +235,9 @@ -- an alternative to specifying them as an `Option`.  See `Caseless` for -- example. --+-- Most options are exported in "Text.Regex.Pcre2".  The callout interface is+-- found in "Text.Regex.Pcre2.Unsafe".+-- -- Documentation is scant here.  For more complete, accurate information, -- including discussions of corner cases arising from specific combinations of -- options and pattern items, please see the [C API@@ -255,8 +257,6 @@     | AltVerbNames -- ^ Enable backslash escapes in verb names.  E.g.,     -- @(*MARK:L\\(O\\)L)@.     | Anchored -- ^ Equivalent to beginning pattern with @^@.-    | AutoCallout -- ^ Run callout for every pattern item.  Only relevant if a-    -- callout is set.     | BadEscapeIsLiteral -- ^ Do not throw an error for unrecognized or     -- malformed escapes.  /\"This is a dangerous option.\"/     | Bsr Bsr -- ^ Override what @\\R@ matches (default given by `defaultBsr`).@@ -268,7 +268,6 @@     -- subject.     | DotAll -- ^ A dot also matches a (single-character) newline.  Equivalent     -- to @(?s)@.-    | DupNames -- ^ Allow non-unique capture names.     | EndAnchored -- ^ More or less like ending pattern with @$@.     | EscapedCrIsLf -- ^ Interpret @\\r@ as @\\n@.     | Extended -- ^ In the pattern, ignore whitespace, and enable comments@@ -338,31 +337,26 @@     -- Incompatible with `NeverUcp`.     | Ungreedy -- ^ Invert the effect of @?@.  Without it, quantifiers are     -- non-greedy; with it, they are greedy.  Equivalent to @(?U)@.-    | UnsafeCallout (CalloutInfo -> IO CalloutResult) -- ^ Run the given callout-    -- at every callout point-    -- (see [the docs](https://pcre.org/current/doc/html/pcre2callout.html) for-    -- more info).  Multiples of this option before the rightmost are ignored.-    ---    -- /NOTE: The callout is run via `unsafePerformIO` within pure code!/+     | UnsafeCompileRecGuard (Int -> IO Bool) -- ^ Run the given guard on every     -- new descent into a level of parentheses, passing the current depth as     -- argument.  Returning @False@ aborts pattern compilation with an     -- exception.  Multiples of this option before the rightmost are ignored.     ---    -- /NOTE: Currently (PCRE2 version 10\.35) patterns seem to be parsed in/+    -- /NOTE: Currently (PCRE2 version 10\.37) patterns seem to be compiled in/     -- /two passes, both times triggering the recursion guard.  Also, it is/     -- /triggered at the beginning of the pattern, passing 0.  None of this is/     -- /documented; expect the unexpected in the presence of side effects!/-    ---    -- /NOTE: The guard is run via `unsafePerformIO` within pure code!/+    | UnsafeCallout (CalloutInfo -> IO CalloutResult) -- ^ Run the given callout+    -- at every callout point+    -- (see [the docs](https://pcre.org/current/doc/html/pcre2callout.html) for+    -- more info).  Multiples of this option before the rightmost are ignored.+    | AutoCallout -- ^ Run callout for every pattern item.  Only relevant if a+    -- callout is set.     | UnsafeSubCallout (SubCalloutInfo -> IO SubCalloutResult) -- ^ Run the     -- given callout on every substitution.  This is at most once unless     -- `SubGlobal` is set.  Multiples of this option before the rightmost are     -- ignored.-    ---    -- /NOTE: The callout is run via `unsafePerformIO` within pure code!/-    | Utf -- ^ Treat both the pattern and subject as UTF rather than fixed-width-    -- 16-bit code units.  instance Semigroup Option where     (<>) = TwoOptions@@ -500,7 +494,6 @@     Caseless          -> [CompileOption pcre2_CASELESS]     DollarEndOnly     -> [CompileOption pcre2_DOLLAR_ENDONLY]     DotAll            -> [CompileOption pcre2_DOTALL]-    DupNames          -> [CompileOption pcre2_DUPNAMES]     EndAnchored       -> [CompileOption pcre2_ENDANCHORED]     Extended          -> [CompileOption pcre2_EXTENDED]     ExtendedMore      -> [CompileOption pcre2_EXTENDED_MORE]@@ -516,7 +509,6 @@     NoStartOptimize   -> [CompileOption pcre2_NO_START_OPTIMIZE]     Ucp               -> [CompileOption pcre2_UCP]     Ungreedy          -> [CompileOption pcre2_UNGREEDY]-    Utf               -> [CompileOption pcre2_UTF]      -- ExtraCompileOption     AltBsux            -> [CompileExtraOption pcre2_EXTRA_ALT_BSUX]@@ -611,8 +603,8 @@  -- | Use a fake @Prism'@ to extract a category of options. extractOptsOf :: Getting (Alt Maybe a) AppliedOption a -> ExtractOpts [a]-extractOptsOf traversal = state $ partitionEithers . map discrim where-    discrim opt = maybe (Right opt) Left $ preview traversal opt+extractOptsOf prism = state $ partitionEithers . map discrim where+    discrim opt = maybe (Right opt) Left $ preview prism opt  -- | Prepare to compile a `Code`. extractCompileEnv :: ExtractOpts CompileEnv@@ -962,15 +954,11 @@     calloutStateException :: Maybe SomeException,     calloutStateSubsLog   :: IntMap SubCalloutResult} --- | FFI wrapper.-foreign import ccall "wrapper" mkRecursionGuard-    :: (CUInt -> Ptr a -> IO CInt)-    -> IO (FunPtr (CUInt -> Ptr a -> IO CInt))+foreign import ccall "wrapper" mkRecursionGuard :: FfiWrapper+    (CUInt -> Ptr a -> IO CInt) --- | FFI wrapper.-foreign import ccall "wrapper" mkCallout-    :: (Ptr block -> Ptr a -> IO CInt)-    -> IO (FunPtr (Ptr block -> Ptr a -> IO CInt))+foreign import ccall "wrapper" mkCallout :: FfiWrapper+    (Ptr block -> Ptr a -> IO CInt)  -- | Within a callout, marshal the original subject and @pcre2_callout_block@ -- data to Haskell and present to the user function.  Ensure no pointers are@@ -1000,7 +988,7 @@             [start, end] <- forM [0, 1] $ peekElemOff ovecPtr . (n * 2 +)             return $ if start == pcre2_UNSET                 then Nothing-                else Just $ slice calloutSubject $ SliceRange+                else Just $ smartSlice calloutSubject $ Slice                     (fromIntegral start)                     (fromIntegral end) @@ -1039,7 +1027,7 @@             [start, end] <- forM [0, 1] $ peekElemOff ovecPtr . (n * 2 +)             return $ if start == pcre2_UNSET                 then Nothing-                else Just $ slice subCalloutSubject $ SliceRange+                else Just $ smartSlice subCalloutSubject $ Slice                     (fromIntegral start)                     (fromIntegral end) @@ -1060,25 +1048,24 @@ -- * Packaging @Matcher@s and @Subber@s as public API functions  -- | The most general form of a matching function, which can also be used as a--- @Setter'@ to perform substitutions at the Haskell level.  Operates globally.------ Internal only!  Users should not (have to) know about `Matcher`.-_capturesInternal :: Matcher -> FromMatch -> Traversal' Text (NonEmpty Text)+-- @Setter'@ to perform substitutions at the Haskell level.+_capturesInternal :: (Traversable t) =>+    Matcher -> FromMatch t -> Traversal' Text (t Text) _capturesInternal matcher fromMatch f subject =-    traverse f captureLists <&> \captureLists' ->+    traverse f captureTs <&> \captureTs' ->         -- Swag foldl-as-foldr to create only as many segments as we need to         -- stitch back together and no more.         let triples = concat $ zipWith3 zip3-                (map NE.toList sliceRangeLists)-                (map NE.toList captureLists)-                (map NE.toList captureLists')+                (map toList sliceTs)+                (map toList captureTs)+                (map toList captureTs')         in Text.concat $ foldr mkSegments termSegments triples 0      where-    sliceRangeLists = unsafeLazyStreamToList $ mapMS fromMatch $ matcher subject-    captureLists = map (NE.map $ slice subject) sliceRangeLists+    sliceTs = unsafeLazyStreamToList $ mapMS fromMatch $ matcher subject+    captureTs = map (smartSlice subject <$>) sliceTs -    mkSegments (SliceRange off offEnd, c, c') r prevOffEnd+    mkSegments (Slice off offEnd, c, c') r prevOffEnd         | off == fromIntegral pcre2_UNSET || c == c' =             -- This substring is unset or unchanged.  Keep going without making             -- cuts.@@ -1086,84 +1073,74 @@         | otherwise =             -- Emit the subject up until here, and the new substring, and keep             -- going, remembering where we are now.-            thinSlice subject (SliceRange prevOffEnd off) : c' : r offEnd+            thinSlice subject (Slice prevOffEnd off) : c' : r offEnd     termSegments off =         let offEnd = fromIntegral $ Text.length subject         -- If the terminal segment is empty, omit it altogether.  That way,-        -- `Text.concat` can just return the subject without copying anything in+        -- Text.concat can just return the subject without copying anything in         -- cases where no substring is changed.-        in [thinSlice subject (SliceRange off offEnd) | off /= offEnd]+        in [thinSlice subject (Slice off offEnd) | off /= offEnd] --- | A function that takes a C match result and extracts a list of captures.  We--- need to pass this effectful callback to `_capturesInternal` because of the--- latter\'s imperative loop that reuses the same @pcre2_match_data@ block.-type FromMatch = Ptr Pcre2_match_data -> IO (NonEmpty SliceRange)+-- | A function that takes a C match result and extracts captures into a+-- container.  We need to pass this effectful callback to `_capturesInternal`+-- because of the latter\'s imperative loop that reuses the same+-- @pcre2_match_data@ block.+--+-- The container type is polymorphic and in practice carries a `Traversable`+-- constraint.  Currently the following containers are used:+--+-- * `NonEmpty` when we want multiple capture groups;+--+-- * `Identity` when we only want the 0th;+--+-- * `Proxy` when we are only checking if a match succeeded.+type FromMatch t = Ptr Pcre2_match_data -> IO (t Slice)  -- | Read all specifically indexed captures\' offsets from match results.-getWhitelistedSliceRanges :: NonEmpty Int -> FromMatch-getWhitelistedSliceRanges whitelist matchDataPtr = do+getWhitelistedSlices :: (Traversable t) => t Int -> FromMatch t+getWhitelistedSlices whitelist matchDataPtr = do     ovecPtr <- pcre2_get_ovector_pointer matchDataPtr     let peekOvec :: Int -> IO Text.I16         peekOvec = fmap fromIntegral . peekElemOff ovecPtr -    forM whitelist $ \i -> liftM2 SliceRange+    forM whitelist $ \i -> liftM2 Slice         (peekOvec $ i * 2)         (peekOvec $ i * 2 + 1)  -- | Read just the 0th capture\'s offsets from match results.-get0thSliceRanges :: FromMatch-get0thSliceRanges = getWhitelistedSliceRanges $ 0 :| []+get0thSlice :: FromMatch Identity+get0thSlice = getWhitelistedSlices $ Identity 0  -- | Read all captures\' offsets from match results.-getAllSliceRanges :: FromMatch-getAllSliceRanges matchDataPtr = do+getAllSlices :: FromMatch NonEmpty+getAllSlices matchDataPtr = do     count <- fromIntegral <$> pcre2_get_ovector_count matchDataPtr     let whitelist = 0 :| [1 .. count - 1] -    getWhitelistedSliceRanges whitelist matchDataPtr---- | Placeholder for half-building a `Traversal'` to be passed to `has`.-errorFromMatch :: FromMatch-errorFromMatch _ = return $ error "BUG! Tried to use match results"---- | Match a pattern to a subject once and return a list of captures, or @[]@ if--- no match.-captures :: Text -> Text -> [Text]-captures = capturesOpt mempty+    getWhitelistedSlices whitelist matchDataPtr --- | @capturesOpt mempty = captures@-capturesOpt :: Option -> Text -> Text -> [Text]-capturesOpt option patt = maybe [] NE.toList . capturesAOpt option patt+-- | Placeholder for building a `Traversal'` to be passed to `has`.+nilFromMatch :: FromMatch Proxy+nilFromMatch _ = return Proxy --- | Match a pattern to a subject once and return a non-empty list of captures+-- | Match a pattern to a subject and return some non-empty list(s) of captures -- in an `Alternative`, or `empty` if no match.  The non-empty list constructor -- `:|` serves as a cue to differentiate the 0th capture from the others: ----- > let parseDate = capturesA "(\\d{4})-(\\d{2})-(\\d{2})"+-- > let parseDate = captures "(\\d{4})-(\\d{2})-(\\d{2})" -- > in case parseDate "submitted 2020-10-20" of -- >     Just (date :| [y, m, d]) -> ... -- >     Nothing                  -> putStrLn "didn't match"-capturesA :: (Alternative f) => Text -> Text -> f (NonEmpty Text)-capturesA = capturesAOpt mempty---- | @capturesAOpt mempty = capturesA@ ----- @since 1.1.0-capturesAOpt :: (Alternative f) => Option -> Text -> Text -> f (NonEmpty Text)-capturesAOpt option patt = toAlternativeOf1 $ _capturesOpt option patt---- | Match a pattern to a subject and lazily produce a list of all--- non-overlapping portions, with all capture groups, that matched.------ @since 1.1.0-capturesAll :: Text -> Text -> [NonEmpty Text]-capturesAll = capturesAllOpt mempty+-- @since 2.0.0+captures :: (Alternative f) => Text -> Text -> f (NonEmpty Text)+captures = capturesOpt mempty --- | @capturesAllOpt mempty = capturesAll@+-- | @capturesOpt mempty = captures@ ----- @since 1.1.0-capturesAllOpt :: Option -> Text -> Text -> [NonEmpty Text]-capturesAllOpt option patt = toListOf $ _capturesOpt option patt+-- @since 2.0.0+capturesOpt :: (Alternative f) => Option -> Text -> Text -> f (NonEmpty Text)+capturesOpt option patt = toAlternativeOf $ _capturesOpt option patt  -- | Does the pattern match the subject at least once? matches :: Text -> Text -> Bool@@ -1171,30 +1148,19 @@  -- | @matchesOpt mempty = matches@ matchesOpt :: Option -> Text -> Text -> Bool-matchesOpt option patt = has $ _capturesInternal matcher errorFromMatch where+matchesOpt option patt = has $ _capturesInternal matcher nilFromMatch where     matcher = unsafePerformIO $ assembleMatcher option patt --- | Match a pattern to a subject once and return the portion that matched in an+-- | Match a pattern to a subject and return the portion(s) that matched in an -- `Alternative`, or `empty` if no match.+--+-- @since 2.0.0 match :: (Alternative f) => Text -> Text -> f Text match = matchOpt mempty  -- | @matchOpt mempty = match@ matchOpt :: (Alternative f) => Option -> Text -> Text -> f Text-matchOpt option patt = toAlternativeOf1 $ _matchOpt option patt---- | Match a pattern to a subject and lazily return a list of all--- non-overlapping portions that matched.------ @since 1.1.0-matchAll :: Text -> Text -> [Text]-matchAll = matchAllOpt mempty---- | @matchAllOpt mempty = matchAll@------ @since 1.1.0-matchAllOpt :: Option -> Text -> Text -> [Text]-matchAllOpt option patt = toListOf $ _matchOpt option patt+matchOpt option patt = toAlternativeOf $ _matchOpt option patt  -- | Perform at most one substitution.  See -- [the docs](https://pcre.org/current/doc/html/pcre2api.html#SEC36) for the@@ -1225,7 +1191,7 @@     subber = unsafePerformIO $ assembleSubber replacement option patt  -- | Given a pattern, produce a traversal (0 or more targets) that focuses from--- a subject to each non-empty list of captures that pattern matches globally.+-- a subject to each non-empty list of captures that pattern matches. -- -- Substitution works in the following way:  If a capture is set such that the -- new `Text` is not equal to the old one, a substitution occurs, otherwise it@@ -1266,11 +1232,11 @@  -- | @_capturesOpt mempty = _captures@ _capturesOpt :: Option -> Text -> Traversal' Text (NonEmpty Text)-_capturesOpt option patt = _capturesInternal matcher getAllSliceRanges where+_capturesOpt option patt = _capturesInternal matcher getAllSlices where     matcher = unsafePerformIO $ assembleMatcher option patt  -- | Given a pattern, produce a traversal (0 or more targets) that focuses from--- a subject to the portions of it that match.+-- a subject to the non-overlapping portions of it that match. -- -- @_match = `_captures` patt . ix 0@ _match :: Text -> Traversal' Text Text@@ -1278,8 +1244,8 @@  -- | @_matchOpt mempty = _match@ _matchOpt :: Option -> Text -> Traversal' Text Text-_matchOpt option patt = _cs . _headNE where-    _cs = _capturesInternal matcher get0thSliceRanges+_matchOpt option patt = _cs . _Identity where+    _cs = _capturesInternal matcher get0thSlice     matcher = unsafePerformIO $ assembleMatcher option patt  -- * Support for Template Haskell compile-time regex analysis@@ -1352,7 +1318,11 @@             -- else             num -    CaptNum (name :: Symbol) '(_, '(name, num) ': _) = num+    -- FIXME See Text.Regex.Pcre2.TH, where we end the lookup table with a+    -- placeholder entry due to a GHC bug.  We must ensure a successful name+    -- lookup does not occur on this last entry; hence the extra promoted cons+    -- operators in this pattern.+    CaptNum (name :: Symbol) '(_, '(name, num) ': _ ': _) = num     CaptNum (name :: Symbol) '(hi, _ ': kvs) = CaptNum name '(hi, kvs)     CaptNum (name :: Symbol) _ = TypeError         (TypeLits.Text "No capture named " :<>: ShowType name)@@ -1374,7 +1344,7 @@ -- > capture @3 -- > capture @"bar" ----- Specifying a nonexistant number or name will result in a type error.+-- Specifying a nonexistent number or name will result in a type error. capture :: forall i info num. (CaptNum i info ~ num, KnownNat num) =>     Captures info -> Text capture = view $ _capture @i

@@ -1,14 +1,18 @@+{-# LANGUAGE DataKinds #-} {-# LANGUAGE LambdaCase #-}-{-# LANGUAGE QuasiQuotes #-}+{-# LANGUAGE RankNTypes #-} {-# LANGUAGE TemplateHaskell #-}  module Text.Regex.Pcre2.TH where +import           Control.Applicative        (Alternative(..))+import           Data.Functor               ((<&>)) import           Data.IORef-import           Data.List.NonEmpty         (NonEmpty)+import           Data.List.NonEmpty         (NonEmpty(..)) import qualified Data.List.NonEmpty         as NE import           Data.Map.Lazy              (Map) import qualified Data.Map.Lazy              as Map+import           Data.Proxy                 (Proxy(..)) import           Data.Text                  (Text) import qualified Data.Text                  as Text import           Language.Haskell.TH@@ -23,7 +27,7 @@ globalMatcherCache = unsafePerformIO $ newIORef Map.empty {-# NOINLINE globalMatcherCache #-} --- | Given a `Text`, create or retrieve a `Matcher` from the global cache.+-- | Given a pattern, create or retrieve a `Matcher` from the global cache. memoMatcher :: Text -> Matcher memoMatcher patt = unsafePerformIO $ do     cache <- readIORef globalMatcherCache@@ -35,10 +39,6 @@                 (Map.insert patt matcher cache, ())             return matcher --- | Generate code to produce \(and memoize\) a `Matcher` from a pattern.-matcherQ :: String -> ExpQ-matcherQ s = [e| memoMatcher $ Text.pack $(stringE s) |]- -- | Predict parenthesized captures \(maybe named\) of a pattern at splice time. predictCaptureNamesQ :: String -> Q [Maybe Text] predictCaptureNamesQ = runIO . predictCaptureNames mempty . Text.pack@@ -55,21 +55,57 @@     [] -> return Nothing      -- One or more parenthesized captures.  Present-    --     [Just "foo", Just "bar", Nothing]+    --     [Just "foo", Nothing, Nothing, Just "bar"]     -- as-    --     '(3, '[ '("foo", 1), '("bar", 2)]).+    --     '(4, '[ '("foo", 1), '("bar", 4), '("", 0)]).     captureNames -> Just <$> promotedTupleT 2 `appT` hi `appT` kvs where-        -- 3+        -- 4         hi = litT $ numTyLit $ fromIntegral $ length captureNames-        -- '[ '("foo", 1), '("bar", 2)]-        kvs = foldr f promotedNilT $ toKVs captureNames where+        -- '[ '("foo", 1), '("bar", 4), '("", 0)]+        kvs = foldr f end (toKVs captureNames) where             -- '("foo", 1) ': ...-            f (number, name) = appT $ appT promotedConsT $       -- ':-                promotedTupleT 2                                 -- '(,)+            f (number, name) r = promotedConsT `appT` kv `appT` r where+                kv = promotedTupleT 2                            -- '(,)                     `appT` litT (strTyLit $ Text.unpack name)    -- "foo"                     `appT` litT (numTyLit $ fromIntegral number) -- 1+        -- FIXME GHC kind-checks empty '[] as [*] instead of [k], which breaks+        -- quasi-quoted regexes with parenthesized captures but no names.+        -- Therefore, we avoid ever splicing an empty lookup table by ending it+        -- with a placeholder entry (which cannot arise from a pattern since ""+        -- is an invalid capture group name).+        end = [t| '[ '("", 0)] |] --- | /__As an expression__/+-- | Helper for `regex` with no parenthesized captures.+matchTH :: (Alternative f) => Text -> Text -> f Text+matchTH patt = toAlternativeOf $+    _capturesInternal (memoMatcher patt) get0thSlice . _Identity++-- | Helper for `regex` with parenthesized captures.+capturesTH :: (Alternative f) => Text -> Proxy info -> Text -> f (Captures info)+capturesTH patt _ = toAlternativeOf $+    _capturesInternal (memoMatcher patt) getAllSlices . to Captures++-- | Helper for `regex` as a guard pattern.+matchesTH :: Text -> Text -> Bool+matchesTH patt = has $ _capturesInternal (memoMatcher patt) nilFromMatch++-- | Helper for `regex` as a pattern that binds local variables.+capturesNumberedTH :: Text -> NonEmpty Int -> Text -> [Text]+capturesNumberedTH patt numbers = concatMap NE.toList . toListOf _cs where+    _cs = _capturesInternal (memoMatcher patt) fromMatch+    fromMatch = getWhitelistedSlices numbers++-- | Helper for `_regex` with no parenthesized captures.+_matchTH :: Text -> Traversal' Text Text+_matchTH patt = _capturesInternal (memoMatcher patt) get0thSlice . _Identity++-- | Helper for `_regex` with parenthesized captures.+_capturesTH :: Text -> Proxy info -> Traversal' Text (Captures info)+_capturesTH patt _ = _cs . wrapped where+    _cs = _capturesInternal (memoMatcher patt) getAllSlices+    wrapped f cs = f (Captures cs) <&> \(Captures cs') -> cs'++-- | === As an expression -- -- > regex :: (Alternative f) => String -> Text -> f (Captures info) --@@ -78,8 +114,8 @@ -- > regex :: (Alternative f) => String -> Text -> f Text -- -- if there are none.  In other words, if there is more than the 0th capture,--- this behaves like `capturesA` (except returning an opaque `Captures` instead--- of a list), otherwise it behaves like `match`.+-- this behaves like `captures` (except returning an opaque `Captures` instead+-- of a `NonEmpty` list), otherwise it behaves like `match`. -- -- To retrieve an individual capture from a `Captures`, use `capture`. --@@ -92,7 +128,7 @@ -- > forM_ ([regex|\s+$|] line :: Maybe Text) $ \spaces -> error $ -- >     "line has trailing spaces (" ++ show (Text.length spaces) ++ " characters)" ----- /__As a pattern__/+-- === As a pattern -- -- This matches when the regex first matches, whereupon any named captures are -- bound to variables of the same names.@@ -108,14 +144,11 @@ -- If there are no named captures, this simply acts as a guard. regex :: QuasiQuoter regex = QuasiQuoter {-    quoteExp = \s -> do-        let fromCsQ = capturesInfoQ s >>= maybe [e| _headNE |] toWrapQ-            toWrapQ info = [e|-                let wrap cs = Captures cs :: Captures $(return info)-                in to wrap |]--        [e| toAlternativeOf1 $-            _capturesInternal $(matcherQ s) getAllSliceRanges . $(fromCsQ) |],+    quoteExp = \s -> capturesInfoQ s >>= \case+        Nothing   -> [e| matchTH (Text.pack $(stringE s)) |]+        Just info -> [e| capturesTH+            (Text.pack $(stringE s))+            (Proxy :: Proxy $(return info)) |],      quotePat = \s -> do         captureNames <- predictCaptureNamesQ s@@ -124,18 +157,16 @@             -- No named captures.  Test whether the string matches without             -- creating any new Text values.             Nothing -> viewP-                [e| has $ _capturesInternal $(matcherQ s) errorFromMatch |]+                [e| matchesTH (Text.pack $(stringE s)) |]                 [p| True |]              -- One or more named captures.  Attempt to bind only those to local             -- variables of the same names.             Just numberedNames -> viewP e p where                 (numbers, names) = NE.unzip numberedNames-                e = [e|-                    let _cs = _capturesInternal-                            $(matcherQ s)-                            (getWhitelistedSliceRanges $(liftData numbers))-                    in view $ _cs . to NE.toList |]+                e = [e| capturesNumberedTH+                    (Text.pack $(stringE s))+                    $(liftData numbers) |]                 p = foldr f wildP names where                     f name r = conP '(:) [varP $ mkName $ Text.unpack name, r], @@ -143,7 +174,7 @@      quoteDec = const $ fail "regex: cannot produce declarations"} --- | A global, optical variant of `regex`.  Can only be used as an expression.+-- | An optical variant of `regex`.  Can only be used as an expression. -- -- > _regex :: String -> Traversal' Text (Captures info) -- > _regex :: String -> Traversal' Text Text@@ -158,17 +189,13 @@ -- > main = putStrLn $ "There are 14 competing standards" & embeddedNumber %~ (+ 1) -- > -- > -- There are 15 competing standards--- _regex :: QuasiQuoter _regex = QuasiQuoter {-    quoteExp = \s -> do-        let fromCsQ = capturesInfoQ s >>= maybe [e| _headNE |] wrappedQ-            wrappedQ info = [e|-                let wrapped :: Lens' (NonEmpty Text) (Captures $(return info))-                    wrapped f cs = f (Captures cs) <&> \(Captures cs') -> cs'-                in wrapped |]--        [e| _capturesInternal $(matcherQ s) getAllSliceRanges . $(fromCsQ) |],+    quoteExp  = \s -> capturesInfoQ s >>= \case+        Nothing   -> [e| _matchTH (Text.pack $(stringE s)) |]+        Just info -> [e| _capturesTH+            (Text.pack $(stringE s))+            (Proxy :: Proxy $(return info)) |],      quotePat = const $ fail "_regex: cannot produce a pattern", 

@@ -0,0 +1,32 @@+-- | These items are unsafe for one reason or another, and are sequestered here+-- to require the user to do an extra import to get them.+--+-- Chief among+-- them is the callout interface: these options and associated datatypes may be+-- used to register effectful callbacks, sometimes referred to as /callouts/ in+-- the PCRE2 API, for regex compilation, matching, and substitution.  We include+-- them here for completeness and use them to implement unit tests for this+-- library; for ordinary use, however, seek other means to accomplish whatever+-- is needed (such as accreting effects with optics), since they carry all the+-- problems of `unsafePerformIO`.  See+-- the [C API docs](https://pcre.org/current/doc/html/pcre2callout.html)+-- for more information.+module Text.Regex.Pcre2.Unsafe (+    -- ** Options+    Option(+        AutoCallout,+        BadEscapeIsLiteral,+        UnsafeCallout,+        UnsafeCompileRecGuard,+        UnsafeSubCallout),++    -- ** Types+    CalloutInfo(..),+    CalloutIndex(..),+    CalloutResult(..),+    SubCalloutInfo(..),+    SubCalloutResult(..))+where++import System.IO.Unsafe          (unsafePerformIO)+import Text.Regex.Pcre2.Internal

@@ -1,22 +1,23 @@ {-# LANGUAGE DataKinds #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE QuasiQuotes #-}-{-# LANGUAGE ScopedTypeVariables #-} {-# LANGUAGE TypeApplications #-} {-# LANGUAGE ViewPatterns #-}  module Main where -import           Control.Applicative    (Alternative)-import           Control.Exception      (catch, evaluate, handle)-import           Control.Monad.RWS.Lazy (ask, evalRWS, forM_, tell, void)+import           Control.Applicative     (Alternative)+import           Control.Exception       (catch, evaluate, handle)+import           Control.Monad.RWS.Lazy  (ask, evalRWS, forM_, tell, void) import           Data.IORef-import           Data.List.NonEmpty     (NonEmpty(..))-import           Data.Text              (Text)-import qualified Data.Text              as Text+import           Data.List.NonEmpty      (NonEmpty(..))+import           Data.Proxy              (Proxy(..))+import           Data.Text               (Text)+import qualified Data.Text               as Text import           Lens.Micro.Platform import           Test.Hspec import           Text.Regex.Pcre2+import           Text.Regex.Pcre2.Unsafe  main :: IO () main = hspec $ do@@ -39,11 +40,20 @@         it "works using matches" $ do             matches "(?i)foo" "FOO" `shouldBe` True -        it "works using capturesA" $ do-            case capturesA "(\\d{4})-(\\d{2})-(\\d{2})" submitted of+        it "works using captures" $ do+            case captures "(\\d{4})-(\\d{2})-(\\d{2})" submitted of                 Just ne -> ne `shouldBe` "2020-10-20" :| ["2020", "10", "20"]                 Nothing -> expectationFailure "didn't match" +        it "is lazy" $ do+            counter <- newIORef (0 :: Int)+            let callout = UnsafeCallout $ \_ -> do+                    modifyIORef counter (+ 1)+                    return CalloutProceed+            take 3 (matchOpt callout "(?C1)a" "apples and bananas")+                `shouldBe` ["a", "a", "a"]+            readIORef counter `shouldReturn` 3+     describe "lens-powered matching" $ do         let _nee :: Traversal' Text Text             _nee = _match "(?i)\\bnee\\b"@@ -53,14 +63,14 @@             let result = evalRWS                     (promptNee "We are the knights who say...NEE!")                     "NOO"-                    undefined+                    Proxy             result `shouldBe` ("We are the knights who say...NOO!", ["NEE"])          it "signals match failure by not targeting anything" $ do             let result = evalRWS                     (promptNee "Shhhhh")-                    undefined-                    undefined+                    (error "should be unreachable")+                    Proxy             result `shouldBe` ("Shhhhh", [])          it "does not substitute when setting equal Text" $ do@@ -113,13 +123,13 @@                 `shouldThrow` anyPcre2Exception          it "are catchable using Control.Exception.evaluate" $ do-            let example = evaluate (broken "foo") `catch`-                    \(_ :: SomePcre2Exception) -> return Nothing+            let example = handle @SomePcre2Exception (\_ -> return Nothing) $+                    evaluate $ broken "foo"             example `shouldReturn` Nothing          it "are catchable using instance Alternative IO" $ do-            let example = broken "foo" `catch`-                    \(_ :: SomePcre2Exception) -> return "broken"+            let example = handle @SomePcre2Exception (\_ -> return "broken") $+                    broken "foo"             example `shouldReturn` "broken"      describe "native substitution" $ do@@ -180,30 +190,21 @@      describe "an unset capture" $ do         it "is treated as empty" $ do-            captures "(a)?" "" `shouldBe` ["", ""]+            captures "(a)?" "" `shouldBe` Just ("" :| [""])          it "is unchanged via Traversal'" $ do-            ("" & [_regex|(a)?|] . _capture @1 .~ "foo") `shouldBe` ""+            set ([_regex|(a)?|] . _capture @1) "foo" "" `shouldBe` ""          it "permits other captures to be changed via Traversal'" $ do-            ("" & [_regex|(a)?|] . _capture @0 .~ "foo") `shouldBe` "foo"+            set ([_regex|(a)?|] . _capture @0) "foo" "" `shouldBe` "foo"      describe "Traversal'" $ do         it "supports global substitutions" $ do-            ("apples and bananas" & _match "a" .~ "o")+            set (_match "a") "o" "apples and bananas"                 `shouldBe` "opples ond bononos" -        it "is lazy" $ do-            counter <- newIORef (0 :: Int)-            let callout = UnsafeCallout $ \_ -> do-                    modifyIORef counter (+ 1)-                    return CalloutProceed-            take 3 ("apples and bananas" ^.. _matchOpt callout "(?C1)a")-                `shouldBe` ["a", "a", "a"]-            readIORef counter `shouldReturn` 3-         it "converges in the presence of empty matches" $ do-            length (toListOf [_regex||] "12345") `shouldBe` 6+            length (match @[] "" "12345") `shouldBe` 6      describe "bug fixes" bugFixes @@ -214,15 +215,11 @@         f "a" `shouldReturn` ""         f "b" `shouldReturn` "b" -    issue 12 $ do-        captures ".a." "foo bar baz" `shouldBe` ["bar"]-        case "foo bar baz" of-            [regex|.a(?<x>.)|] -> return ()-            _                  ->-                expectationFailure "quasi-quoted pattern didn't match"--    issue 17 $ do-        match "\\d+" "123 456" `shouldBe` ["123"]+    issue 18 $ do+        let result :: (Alternative f) => f Text+            result = match "\\d+" "123 456"+        result `shouldBe` Just "123"+        result `shouldBe` ["123", "456"]      where     issue :: Int -> Expectation -> Spec@@ -252,7 +249,7 @@ broken :: (Alternative f) => Text -> f Text broken = match "*" --- microlens doesn't have this yet as of 12/5/2020+-- microlens doesn't have this yet as of 06/26/2021 _Show :: (Read a, Show a) => Traversal' String a _Show f s = case reads s of     [(x, "")] -> show <$> f x