lens-regex-pcre 0.3.1.0 → 1.0.0.0

raw patch · 10 files changed

+1041/−523 lines, 10 filesdep +gaugedep −pcre-lightPVP ok

version bump matches the API change (PVP)

Dependencies added: gauge

Dependencies removed: pcre-light

API changes (from Hackage documentation)

Files

• ChangeLog.md +17/−7
• README.md +84/−117
• bench/Bench.hs +43/−0
• lens-regex-pcre.cabal +28/−7
• src/Control/Lens/Regex.hs +0/−257
• src/Control/Lens/Regex/ByteString.hs +319/−0
• src/Control/Lens/Regex/Text.hs +251/−0
• test/ByteString.hs +143/−0
• test/Spec.hs +4/−135
• test/Text.hs +152/−0

@@ -1,20 +1,30 @@ # Changelog for lens-regex-pcre +# 1.0.0.0+- Add `regexing` and `makeRegexTraversalQQ`+- Replace `regex` traversal maker with `regex` QuasiQuoter+- Split Control.Lens.Regex into Control.Lens.Regex.Text and Control.Lens.Regex.ByteString+- Move regexBS to `Control.Lens.Regex.ByteString.regex`+- Change whole implementation to use ByteString Builders for a massive speedup+- Monomorphise `Match text` -> `Match`+- Add groups to index of `match` and match to index of `groups` & `group`+- Add `group = groups . ix n` for accessing a single group.+ # 0.3.1.0 -Match -> Match text-Added regexBS to run regex on ByteStrings directly+- Match -> Match text+- Added regexBS to run regex on ByteStrings directly  # 0.3.0.0 -Unify `iregex` into `regex` as a single indexed traversal+- Unify `iregex` into `regex` as a single indexed traversal  # 0.2.0.0 -Unify `grouped`, `groups`, and `igroups` into just `groups` with optional traversal+- Unify `grouped`, `groups`, and `igroups` into just `groups` with optional traversal  # 0.1.1.0 -Adds `grouped` and `matchAndGroups`+- Adds `grouped` and `matchAndGroups`  # 0.1.0.1 -Doc fixes+- Doc fixes  # 0.1.0.0 -Initial Release+- Initial Release

@@ -2,179 +2,146 @@  [Hackage and Docs](http://hackage.haskell.org/package/lens-regex-pcre) -* NOTE: I don't promise that this is __fast__ yet, nor do I have any benchmarks;-* NOTE: currently only supports `Text` but should be generalizable to more string-likes; open an issue if you need it+Based on `pcre-heavy`; so it should support any regexes or options which it supports. -Based on `pcre-heavy`; so it should support any regexes which it supports.-I'll likely add a way to pass settings in soon; make an issue if you need this :)+Performance is [equal, sometimes **better**](#performance) than that of `pcre-heavy` alone. +Which module should you use?++If you need unicode support, use `Control.Lens.Regex.Text`, if not then `Control.Lens.Regex.ByteString` is faster.+ Working with Regexes in Haskell kinda sucks; it's tough to figure out which libs-to use, and even after you pick one it's tough to figure out how to use it.+to use, and even after you pick one it's tough to figure out how to use it; `lens-regex-pcre` hopes to replace most other solutions by being fast, easy to set up, more adaptable with a more consistent interface. +It helps that there are already HUNDREDS of combinators which interop with lenses :smile:.+ As it turns out; regexes are a very lens-like tool; Traversals allow you to select and alter zero or more matches; traversals can even carry indexes so you know which match or group you're working on. -Note that all traversals in this library are not techically lawful, as the semantics of regular expressions don't allow for it;-They break the 'multi-set' idempotence law of traversals (same one broken by `filtered` from `lens`); in reality this isn't usually a problem (I've literally never encountered an issue with it); but consider yourself warned. Test your code.--Here are a few examples:+# Examples  ```haskell txt :: Text txt = "raindrops on roses and whiskers on kittens"  -- Search-λ> has (regex [rx|whisk|]) txt+>>> has [regex|whisk|] . match txt True  -- Get matches-λ> txt ^.. regex [rx|\br\w+|] . match+>>> txt ^.. [regex|\br\w+|] . match ["raindrops","roses"]  -- Edit matches-λ> txt & regex [rx|\br\w+|] . match %~ T.intersperse '-' . T.toUpper+>>> txt & [regex|\br\w+|] . match %~ T.intersperse '-' . T.toUpper "R-A-I-N-D-R-O-P-S on R-O-S-E-S and whiskers on kittens"  -- Get Groups-λ> txt ^.. regex [rx|(\w+) on (\w+)|] . groups+>>> txt ^.. [regex|(\w+) on (\w+)|] . groups [["raindrops","roses"],["whiskers","kittens"]]  -- Edit Groups-λ> txt & regex [rx|(\w+) on (\w+)|] . groups %~ reverse+>>> txt & [regex|(\w+) on (\w+)|] . groups %~ reverse "roses on raindrops and kittens on whiskers"  -- Get the third match-λ> txt ^? regex [rx|\w+|] . index 2 . match+>>> txt ^? [regex|\w+|] . index 2 . match Just "roses"  -- Match integers, 'Read' them into ints, then sort them in-place -- dumping them back into the source text afterwards.-λ> "Monday: 29, Tuesday: 99, Wednesday: 3" -   & partsOf (regex [rx|\d+|] . match . unpacked . _Show @Int) %~ sort+>>> "Monday: 29, Tuesday: 99, Wednesday: 3" +   & partsOf ([regex|\d+|] . match . unpacked . _Show @Int) %~ sort "Monday: 3, Tuesday: 29, Wednesday: 99"  ```  Basically anything you want to do is possible somehow. -Expected behaviour (and examples) can be found in the test suite:--```haskell-describe "regex" $ do-    describe "match" $ do-        describe "getting" $ do-            it "should find one match" $ do-                "abc" ^.. regex [rx|b|] . match-                `shouldBe` ["b"]--            it "should find many matches" $ do-                "a b c" ^.. regex [rx|\w|] . match-                `shouldBe` ["a", "b", "c"]--            it "should fold" $ do-                "a b c" ^. regex [rx|\w|] . match-                `shouldBe` "abc"--            it "should match with a group" $ do-                "a b c" ^.. regex [rx|(\w)|] . match-                `shouldBe` ["a", "b", "c"]--            it "should match with many groups" $ do-                "a b c" ^.. regex [rx|(\w) (\w)|] . match-                `shouldBe` ["a b"]--            it "should be greedy when overlapping" $ do-                "abc" ^.. regex [rx|\w+|] . match-                `shouldBe`["abc"]--            it "should respect lazy modifiers" $ do-                "abc" ^.. regex [rx|\w+?|] . match-                `shouldBe`["a", "b", "c"]+# Performance -            it "should allow folding with index" $ do-                ("one two three" ^.. (regex [rx|\w+|] <. match) . withIndex)-                `shouldBe` [(0, "one"), (1, "two"), (2, "three")]+See the [benchmarks](./bench/Bench.hs). -            it "should allow getting with index" $ do-                ("one two three" ^.. regex [rx|\w+|] . index 1 . match)-                `shouldBe` ["two"]+## Summary -        describe "setting" $ do-            it "should allow setting" $ do-                ("one two three" & regex [rx|two|] . match .~ "new")-                `shouldBe` "one new three"+Caveat: I'm by no means a benchmarking expert; if you have tips on how to do this better I'm all ears! -            it "should allow setting many" $ do-                ("one <two> three" & regex [rx|\w+|] . match .~ "new")-                `shouldBe` "new <new> new"+* **Search** `lens-regex-pcre` is *marginally* slower than `pcre-heavy`, but well within acceptable margins (within 0.6%)+* **Replace** `lens-regex-pcre` beats `pcre-heavy` by ~10%+* **Modify** `pcre-heavy` doesn't support this operation at all, so I guess `lens-regex-pcre` wins here :) -            it "should allow mutating" $ do-                ("one two three" & regex [rx|two|] . match %~ (<> "!!"). T.toUpper)-                `shouldBe` "one TWO!! three"+How can it possibly be **faster** if it's based on `pcre-heavy`? `lens-regex-pcre` only uses `pcre-heavy` for **finding** the matches, not substitution/replacement. After that it splits the text into chunks and traverses over them with whichever operation you've chosen. The nature of this implementation makes it a lot easier to understand than imperative implementations of the same thing. This means it's pretty easy to make edits, and is also the reason we can support arbitrary traversals/actions. It was easy enough, so I went ahead and made the whole thing use ByteString Builders, which sped it up a lot. I suspect that `pcre-heavy` can benefit from the same optimization if anyone feels like back-porting it; it could be (almost) as nicely using simple `traverse` without any lenses. The whole thing is only about 25 LOC. -            it "should allow mutating many" $ do-                ("one two three" & regex [rx|two|] . match %~ T.toUpper)-                `shouldBe` "one TWO three"+I'm neither a benchmarks nor stats person, so please open an issue if anything here seems fishy. -            it "should allow setting with index" $ do-                ("one two three" & regex [rx|\w+|] <. match .@~ T.pack . show)-                `shouldBe` "0 1 2"+Without `pcre-light` and `pcre-heavy` this library wouldn't be possible, so huge thanks to all contributors! -            it "should allow mutating with index" $ do-                ("one two three" & regex [rx|\w+|] <. match %@~ \i s -> (T.pack $ show i) <> ": " <> s)-                `shouldBe` "0: one 1: two 2: three"+Here are the benchmarks on my 2013 Macbook (2.6 Ghz i5) -describe "groups" $ do-    describe "getting" $ do-        it "should get groups" $ do-            "a b c" ^.. regex [rx|(\w)|] . groups-            `shouldBe` [["a"], ["b"], ["c"]]+```haskell+benchmarking static pattern search/pcre-heavy ... took 20.78 s, total 56 iterations+benchmarked static pattern search/pcre-heavy+time                 375.3 ms   (372.0 ms .. 378.5 ms)+                     1.000 R²   (0.999 R² .. 1.000 R²)+mean                 378.1 ms   (376.4 ms .. 380.8 ms)+std dev              3.747 ms   (922.3 μs .. 5.609 ms) -        it "should get multiple groups" $ do-            "raindrops on roses and whiskers on kittens" ^.. regex [rx|(\w+) on (\w+)|] . groups-            `shouldBe` [["raindrops","roses"],["whiskers","kittens"]]+benchmarking static pattern search/lens-regex-pcre ... took 20.79 s, total 56 iterations+benchmarked static pattern search/lens-regex-pcre+time                 379.5 ms   (376.2 ms .. 382.4 ms)+                     1.000 R²   (1.000 R² .. 1.000 R²)+mean                 377.3 ms   (376.5 ms .. 378.4 ms)+std dev              1.667 ms   (1.075 ms .. 2.461 ms) -        it "should allow getting a specific index" $ do-            ("one two three four" ^.. regex [rx|(\w+) (\w+)|] . groups . ix 1)-            `shouldBe` ["two", "four"]+benchmarking complex pattern search/pcre-heavy ... took 95.95 s, total 56 iterations+benchmarked complex pattern search/pcre-heavy+time                 1.741 s    (1.737 s .. 1.746 s)+                     1.000 R²   (1.000 R² .. 1.000 R²)+mean                 1.746 s    (1.744 s .. 1.749 s)+std dev              4.499 ms   (3.186 ms .. 6.080 ms) -    describe "setting" $ do-        it "should allow setting groups as a list" $ do-            ("one two three" & regex [rx|(\w+) (\w+)|] . groups .~ ["1", "2"])-            `shouldBe` "1 2 three"+benchmarking complex pattern search/lens-regex-pcre ... took 97.26 s, total 56 iterations+benchmarked complex pattern search/lens-regex-pcre+time                 1.809 s    (1.736 s .. 1.908 s)+                     0.996 R²   (0.991 R² .. 1.000 R²)+mean                 1.757 s    (1.742 s .. 1.810 s)+std dev              42.83 ms   (11.51 ms .. 70.69 ms) -        it "should allow editing when result list is the same length" $ do-            ("raindrops on roses and whiskers on kittens" & regex [rx|(\w+) on (\w+)|] . groups %~ reverse)-            `shouldBe` "roses on raindrops and kittens on whiskers"+benchmarking simple replacement/pcre-heavy ... took 23.32 s, total 56 iterations+benchmarked simple replacement/pcre-heavy+time                 423.8 ms   (422.4 ms .. 425.3 ms)+                     1.000 R²   (1.000 R² .. 1.000 R²)+mean                 424.0 ms   (422.9 ms .. 426.2 ms)+std dev              2.684 ms   (1.239 ms .. 4.270 ms) -    describe "traversed" $ do-        it "should allow setting all group matches" $ do-            ("one two three" & regex [rx|(\w+) (\w+)|] . groups . traversed .~ "new")-            `shouldBe` "new new three"+benchmarking simple replacement/lens-regex-pcre ... took 20.84 s, total 56 iterations+benchmarked simple replacement/lens-regex-pcre+time                 382.8 ms   (374.3 ms .. 391.5 ms)+                     0.999 R²   (0.999 R² .. 1.000 R²)+mean                 378.2 ms   (376.3 ms .. 381.0 ms)+std dev              3.794 ms   (2.577 ms .. 5.418 ms) -        it "should allow mutating" $ do-            ("one two three four" & regex [rx|one (two) (three)|] . groups . traversed %~ (<> "!!") . T.toUpper)-            `shouldBe` "one TWO!! THREE!! four"+benchmarking complex replacement/pcre-heavy ... took 24.77 s, total 56 iterations+benchmarked complex replacement/pcre-heavy+time                 448.1 ms   (444.7 ms .. 450.0 ms)+                     1.000 R²   (1.000 R² .. 1.000 R²)+mean                 450.8 ms   (449.5 ms .. 453.9 ms)+std dev              3.129 ms   (947.0 μs .. 4.841 ms) -        it "should allow folding with index" $ do-            ("one two three four" ^.. regex [rx|(\w+) (\w+)|] . groups . traversed . withIndex)-            `shouldBe` [(0, "one"), (1, "two"), (0, "three"), (1, "four")]+benchmarking complex replacement/lens-regex-pcre ... took 21.99 s, total 56 iterations+benchmarked complex replacement/lens-regex-pcre+time                 399.9 ms   (398.4 ms .. 402.2 ms)+                     1.000 R²   (1.000 R² .. 1.000 R²)+mean                 399.6 ms   (399.0 ms .. 400.4 ms)+std dev              1.135 ms   (826.2 μs .. 1.604 ms) -        it "should allow setting with index" $ do-            ("one two three four" & regex [rx|(\w+) (\w+)|] . groups . traversed .@~ T.pack . show)-            `shouldBe` "0 1 0 1"+Benchmark lens-regex-pcre-bench: FINISH+``` -        it "should allow mutating with index" $ do-            ("one two three four" & regex [rx|(\w+) (\w+)|] . groups . traversed %@~ \i s -> (T.pack $ show i) <> ": " <> s)-            `shouldBe` "0: one 1: two 0: three 1: four"+# Behaviour -        it "should compose indices with matches" $ do-            ("one two three four" ^.. (regex [rx|(\w+) (\w+)|] <.> groups . traversed) . withIndex)-            `shouldBe` [((0, 0), "one"), ((0, 1), "two"), ((1, 0), "three"), ((1, 1), "four")]+Precise Expected behaviour (and examples) can be found in the test suites: -describe "matchAndGroups" $ do-    it "should get match and groups" $ do-        "raindrops on roses and whiskers on kittens" ^.. regex [rx|(\w+) on (\w+)|] . matchAndGroups-        `shouldBe` [("raindrops on roses",["raindrops","roses"]),("whiskers on kittens",["whiskers","kittens"])]-```+* [ByteString tests](./test/ByteString.hs)+* [Text tests](./test/Text.hs)

@@ -0,0 +1,43 @@+{-# LANGUAGE QuasiQuotes #-}+{-# LANGUAGE OverloadedStrings #-}++import Gauge.Benchmark+import Gauge.Main+import Data.ByteString as BS+import qualified Text.Regex.PCRE.Heavy as PCRE+import Control.Lens+import Control.Lens.Regex.ByteString++main :: IO ()+main = do+    srcFile <- BS.readFile "./bench/data/small-bible.txt"+    defaultMain+      [ bgroup "static pattern search"+        [ bench "pcre-heavy" $ nf (heavySearch [PCRE.re|Moses|]) srcFile+        , bench "lens-regex-pcre" $ nf (lensSearch [PCRE.re|Moses|]) srcFile+        ]+      , bgroup "complex pattern search"+        [ bench "pcre-heavy" $ nf (heavySearch [PCRE.re|l\w+e|]) srcFile+        , bench "lens-regex-pcre" $ nf (lensSearch [PCRE.re|l\w+e|]) srcFile+        ]+      , bgroup "simple replacement"+        [ bench "pcre-heavy" $ nf (heavyReplace [PCRE.re|Moses|] "Jarvis") srcFile+        , bench "lens-regex-pcre" $ nf (lensReplace [PCRE.re|Moses|] "Jarvis") srcFile+        ]+      , bgroup "complex replacement"+        [ bench "pcre-heavy" $ nf (heavyReplace [PCRE.re|M\w*s\w*s|] "Jarvis") srcFile+        , bench "lens-regex-pcre" $ nf (lensReplace [PCRE.re|M\w*s\w*s|] "Jarvis") srcFile+        ]+      ]++heavySearch :: Regex -> BS.ByteString -> [BS.ByteString]+heavySearch pat src = fst <$> PCRE.scan pat src++lensSearch :: Regex -> BS.ByteString -> [BS.ByteString]+lensSearch pat src = src ^.. regexing pat . match++heavyReplace :: Regex -> BS.ByteString -> BS.ByteString -> BS.ByteString+heavyReplace pat replacement src = PCRE.gsub pat replacement src++lensReplace :: Regex -> BS.ByteString -> BS.ByteString -> BS.ByteString+lensReplace pat replacement src = src & regexing pat . match .~ replacement

@@ -1,13 +1,13 @@ cabal-version: 1.12 --- This file has been generated from package.yaml by hpack version 0.31.1.+-- This file has been generated from package.yaml by hpack version 0.31.2. -- -- see: https://github.com/sol/hpack ----- hash: c1baf100f3d2d1c413b88695a96d578b0a8d18aace0929759a5428dcfa30b1f1+-- hash: 19bd65f2de279f3777dc4f946f7d0efb2805098d91aa727f525e3a4375b09351  name:           lens-regex-pcre-version:        0.3.1.0+version:        1.0.0.0 synopsis:       A lensy interface to regular expressions description:    Please see the README on GitHub at <https://github.com/ChrisPenner/lens-regex-pcre#readme> category:       Regex@@ -29,7 +29,8 @@  library   exposed-modules:-      Control.Lens.Regex+      Control.Lens.Regex.ByteString+      Control.Lens.Regex.Text   other-modules:       Paths_lens_regex_pcre   hs-source-dirs:@@ -37,29 +38,49 @@   ghc-options: -Wall   build-depends:       base >=4.7 && <5+    , bytestring     , lens     , pcre-heavy-    , pcre-light     , template-haskell     , text-    , bytestring   default-language: Haskell2010  test-suite lens-regex-pcre-test   type: exitcode-stdio-1.0   main-is: Spec.hs   other-modules:+      ByteString+      Text       Paths_lens_regex_pcre   hs-source-dirs:       test   ghc-options: -Wall -threaded -rtsopts -with-rtsopts=-N   build-depends:       base >=4.7 && <5+    , bytestring     , hspec     , lens     , lens-regex-pcre     , pcre-heavy-    , pcre-light+    , template-haskell+    , text+  default-language: Haskell2010++benchmark lens-regex-pcre-bench+  type: exitcode-stdio-1.0+  main-is: Bench.hs+  other-modules:+      Paths_lens_regex_pcre+  hs-source-dirs:+      bench+  ghc-options: -Wall -threaded -rtsopts -with-rtsopts=-N+  build-depends:+      base >=4.7 && <5+    , bytestring+    , gauge+    , lens+    , lens-regex-pcre+    , pcre-heavy     , template-haskell     , text   default-language: Haskell2010

@@ -1,257 +0,0 @@-{-|-Module      : Control.Lens.Regex-Description : PCRE regex combinators for interop with lens-Copyright   : (c) Chris Penner, 2019-License     : BSD3--}--{-# LANGUAGE FlexibleContexts #-}-{-# LANGUAGE OverloadedStrings #-}-{-# LANGUAGE PartialTypeSignatures #-}-{-# LANGUAGE QuasiQuotes #-}-{-# LANGUAGE RankNTypes #-}-{-# LANGUAGE TemplateHaskell #-}--module Control.Lens.Regex-    (-    -- * Combinators-      regex-    , regexBS-    , match-    , groups-    , matchAndGroups--    -- * Compiling regex-    , rx-    , mkRegexQQ-    , compile-    , compileM--    -- * Types-    , Match-    , Regex-    ) where--import qualified Data.Text as T hiding (index)-import qualified Data.Text.Encoding as T-import qualified Data.Text.Encoding.Error as T-import qualified Data.ByteString as BS-import Text.Regex.PCRE.Heavy-import Text.Regex.PCRE.Light (compile)-import Control.Lens hiding (re, matching)-import Data.Data (Data)-import Data.Data.Lens (biplate)-import Language.Haskell.TH.Quote---- $setup--- >>> :set -XQuasiQuotes--- >>> :set -XOverloadedStrings--- >>> :set -XTypeApplications--- >>> import Data.Text.Lens (unpacked)--- >>> import Data.Text (Text)--- >>> import Data.List (sort)---- | Match represents a whole regex match; you can drill into it using 'match' or 'groups' or 'matchAndGroups'------ @text@ is either "Text" or "ByteString" depending on whether you use 'regex' or 'regexBS'------ Consider this to be internal; don't depend on its representation.-type Match text = [Either text text]-type MatchRange = (Int, Int)-type GroupRanges = [(Int, Int)]---- | Access all groups of a match at once.------ Note that you can edit the groups through this traversal,--- Changing the length of the list has behaviour similar to 'partsOf'.------ Get all matched groups:------ >>> "raindrops on roses and whiskers on kittens" ^.. regex [rx|(\w+) on (\w+)|] . groups--- [["raindrops","roses"],["whiskers","kittens"]]------ You can access a specific group by combining with `ix`------ >>> "raindrops on roses and whiskers on kittens" ^.. regex [rx|(\w+) on (\w+)|] . groups .  ix 1--- ["roses","kittens"]------ @groups@ is a traversal; you can mutate matches through it.------ >>> "raindrops on roses and whiskers on kittens" & regex [rx|(\w+) on (\w+)|] . groups .  ix 1 %~ T.toUpper--- "raindrops on ROSES and whiskers on KITTENS"------ Editing the list rearranges groups------ >>> "raindrops on roses and whiskers on kittens" & regex [rx|(\w+) on (\w+)|] . groups %~ Prelude.reverse--- "roses on raindrops and kittens on whiskers"------ You can traverse the list to flatten out all groups------ >>> "raindrops on roses and whiskers on kittens" ^.. regex [rx|(\w+) on (\w+)|] . groups . traversed--- ["raindrops","roses","whiskers","kittens"]-groups :: Traversal' (Match text) [text]-groups = partsOf (traversed . _Right)---- | Traverse each match------  Get a match if one exists:------ >>> "find a needle in a haystack" ^? regex [rx|n..dle|] . match--- Just "needle"------  Collect all matches------ >>> "one _two_ three _four_" ^.. regex [rx|_\w+_|] . match--- ["_two_","_four_"]------ You can edit the traversal to perform a regex replace/substitution------ >>> "one _two_ three _four_" & regex [rx|_\w+_|] . match %~ T.toUpper--- "one _TWO_ three _FOUR_"-match :: Monoid text => Traversal' (Match text) text-match f grps = (:[]) . Right <$> f (grps ^. traversed . chosen)---- | The base combinator for doing regex searches.--- It's a traversal which selects 'Match'es; you can compose it with 'match' or 'groups'--- to get the relevant parts of your match.------ >>> txt = "raindrops on roses and whiskers on kittens" :: Text------ Search------ >>> has (regex [rx|whisk|]) txt--- True------ Get matches------ >>> txt ^.. regex [rx|\br\w+|] . match--- ["raindrops","roses"]------ Edit matches------ >>> txt & regex [rx|\br\w+|] . match %~ T.intersperse '-' . T.toUpper--- "R-A-I-N-D-R-O-P-S on R-O-S-E-S and whiskers on kittens"------ Get Groups------ >>> txt ^.. regex [rx|(\w+) on (\w+)|] . groups--- [["raindrops","roses"],["whiskers","kittens"]]------ Edit Groups------ >>> txt & regex [rx|(\w+) on (\w+)|] . groups %~ Prelude.reverse--- "roses on raindrops and kittens on whiskers"------ Get the third match------ >>> txt ^? regex [rx|\w+|] . index 2 . match--- Just "roses"------ Match integers, 'Read' them into ints, then sort them in-place--- dumping them back into the source text afterwards.------ >>> "Monday: 29, Tuesday: 99, Wednesday: 3" & partsOf (regex [rx|\d+|] . match . unpacked . _Show @Int) %~ sort--- "Monday: 3, Tuesday: 29, Wednesday: 99"------ To alter behaviour of the regex you may wish to pass 'PCREOption's when compiling it.--- The default behaviour may seem strange in certain cases; e.g. it operates in 'single-line'--- mode. You can 'compile' the 'Regex' separately and add any options you like, then pass the resulting--- 'Regex' into 'regex';--- Alternatively can make your own version of the QuasiQuoter with any options you want embedded--- by using 'mkRegexQQ'.-regex :: Regex -> IndexedTraversal' Int T.Text (Match T.Text)-regex pattern = utf8 . regexBS pattern . matchBsText-  where-    utf8 :: Iso' T.Text BS.ByteString-    utf8 = iso T.encodeUtf8 (T.decodeUtf8With T.lenientDecode)-    matchBsText :: Iso' [Either BS.ByteString BS.ByteString] (Match T.Text)-    matchBsText = iso (traversed . chosen %~ T.decodeUtf8With T.lenientDecode) (traversed . chosen %~ T.encodeUtf8)---- | A version of 'regex' which operates directly on 'BS.ByteString's.--- This is more efficient than using 'regex' as it avoids converting back and forth--- between 'BS.ByteString' and 'T.Text'.-regexBS :: Regex -> IndexedTraversal' Int BS.ByteString (Match BS.ByteString)-regexBS pattern = indexing (regexT pattern)---- | Base regex traversal. Used only to define 'regex' traversals-regexT :: Regex -> Traversal' BS.ByteString [Either BS.ByteString BS.ByteString]-regexT pattern f txt = collapseMatch <$> apply (fmap splitAgain <$> splitter txt matches)-  where-    matches :: [(MatchRange, GroupRanges)]-    matches = scanRanges pattern txt-    collapseMatch :: [Either BS.ByteString [Either BS.ByteString BS.ByteString]] -> BS.ByteString-    collapseMatch xs = xs ^. folded . beside id (traversed . chosen)-    -- apply :: [Either Text [Either Text Text]] -> _ [Either Text [Either Text Text]]-    apply xs = xs & traversed . _Right %%~ f----- | Get the full match text from a match-matchText :: Monoid text => Match text -> text-matchText m = m ^. traversed . chosen---- | Collect both the match text AND all the matching groups------ >>> "raindrops on roses and whiskers on kittens" ^.. regex [rx|(\w+) on (\w+)|] . matchAndGroups--- [("raindrops on roses",["raindrops","roses"]),("whiskers on kittens",["whiskers","kittens"])]-matchAndGroups :: Monoid text => Getter (Match text) (text, [text])-matchAndGroups = to $ \m -> (matchText m, m ^. groups)---- | 'QuasiQuoter' for compiling regexes.--- This is just 're' re-exported under a different name so as not to conflict with @re@ from--- 'Control.Lens'-rx :: QuasiQuoter-rx = re---- | This allows you to "stash" the match text into an index for use later in the traversal.--- This is a slight abuse of indices; but it can sometimes be handy. This allows you to--- have the full match in scope when editing groups using indexed combinators.------ If you're viewing or folding you should probably just use 'matchAndGroups'.------ >>> "raindrops on roses and whiskers on kittens" ^.. regex [rx|(\w+) on (\w+)|] . (withGroups <. match) . withIndex--- [(["raindrops","roses"],"raindrops on roses"),(["whiskers","kittens"],"whiskers on kittens")]-withMatch :: Monoid text => IndexedTraversal' text (Match text) (Match text)-withMatch p mtch = indexed p (matchText mtch) mtch---- | This allows you to "stash" the match text into an index for use later in the traversal.--- This is a slight abuse of indices; but it can sometimes be handy. This allows you to--- have the full match in scope when editing groups using indexed combinators.------ If you're viewing or folding you should probably just use 'matchAndGroups'.------ >>> "raindrops on roses and whiskers on kittens" ^.. regex [rx|(\w+) on (\w+)|] . (withMatch <. groups) . withIndex--- [("raindrops on roses",["raindrops","roses"]),("whiskers on kittens",["whiskers","kittens"])]-withGroups :: IndexedTraversal' [text] (Match text) (Match text)-withGroups p mtch = indexed p (mtch ^. groups) mtch---- split up text into matches paired with groups; Left is unmatched text-splitter :: BS.ByteString -> [(MatchRange, GroupRanges)] -> [Either BS.ByteString (BS.ByteString, GroupRanges)]-splitter t [] = wrapIfNotEmpty t-splitter t (((start, end), grps) : rest) =-    splitOnce t ((start, end), grps)-    <> splitter (BS.drop end t) (subtractFromAll end rest)--splitOnce :: BS.ByteString -> (MatchRange, GroupRanges) -> [Either BS.ByteString (BS.ByteString, GroupRanges)]-splitOnce t ((start, end), grps) = do-    let (before, mid) = BS.splitAt start t-    let focused = BS.take (end - start) mid-    wrapIfNotEmpty before <> [Right (focused, subtractFromAll start grps)]--splitAgain :: (BS.ByteString, GroupRanges) -> [Either BS.ByteString BS.ByteString]-splitAgain (t, []) | BS.null t = []-                   | otherwise = [Left t]-splitAgain (t, (start, end) : rest) = do-    let (before, mid) = BS.splitAt start t-    let focused = BS.take (end - start) mid-    wrapIfNotEmpty before-        <> [Right focused]-        <> splitAgain ((BS.drop end t), (subtractFromAll end rest))----- helpers-subtractFromAll :: (Data b) => Int -> b -> b-subtractFromAll n = biplate -~ n--wrapIfNotEmpty :: BS.ByteString -> [Either BS.ByteString a]-wrapIfNotEmpty txt-    | BS.null txt = []-    | otherwise = [Left txt]

@@ -0,0 +1,319 @@+{-|+Module      : Control.Lens.Regex.ByteString+Description : ByteString PCRE Regex library with a lensy interface.+Copyright   : (c) Chris Penner, 2019+License     : BSD3+-}++{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE PartialTypeSignatures #-}+{-# LANGUAGE QuasiQuotes #-}+{-# LANGUAGE RankNTypes #-}+{-# LANGUAGE TemplateHaskell #-}+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE UndecidableInstances #-}+{-# LANGUAGE TypeOperators #-}++module Control.Lens.Regex.ByteString+    (+    -- * Basics+      regex+    , match+    , groups+    , group+    , matchAndGroups++    -- * Compiling regexes to Traversals+    , regexing+    , mkRegexTraversalQQ++    -- * Types+    , Match+    , PCRE.Regex+    ) where++import qualified Data.ByteString as BS+import qualified Data.ByteString.Lazy as BL+import qualified Data.ByteString.Builder as BS+import qualified Text.Regex.PCRE.Heavy as PCRE+import Control.Lens hiding (re)+import Data.Bifunctor+import qualified Language.Haskell.TH.Quote as TH+import qualified Language.Haskell.TH.Syntax as TH+import qualified Language.Haskell.TH as TH+import GHC.TypeLits++-- $setup+-- >>> :set -XQuasiQuotes+-- >>> :set -XOverloadedStrings+-- >>> :set -XTypeApplications+-- >>> import qualified Data.ByteString.Char8 as Char8+-- >>> import Data.Char+-- >>> import Data.List hiding (group)+-- >>> import Data.ByteString.Lens++type MatchRange = (Int, Int)+type GroupRanges = [(Int, Int)]++-- | Match represents an opaque regex match.+-- You can drill into it using 'match', 'groups', 'group' or 'matchAndGroups'+newtype Match = Match [Either BS.Builder BS.Builder]++instance TypeError+  ('Text "You're trying to 'show' a raw 'Match' object."+   ':$$: 'Text "You likely missed adding a 'match' or 'groups' or 'group' call after your 'regex' call :)")+  => Show Match where+  show _ = "This is a raw Match object, did you miss a 'match' or 'groups' or 'group' call after your 'regex'?"++chunks :: Iso' Match [Either BS.Builder BS.Builder]+chunks = coerced++unBuilder :: BS.Builder -> BS.ByteString+unBuilder = BL.toStrict . BS.toLazyByteString++building :: Iso' BS.Builder BS.ByteString+building = iso unBuilder BS.byteString++-- | Access all groups of a match as a list. Stashes the full match text as the index in case+-- you need it.+--+-- Note that you can edit the groups through this traversal,+-- Changing the length of the list has behaviour similar to 'partsOf'.+--+-- Get all matched groups:+--+-- >>> "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . groups+-- [["raindrops","roses"],["whiskers","kittens"]]+--+-- You can access a specific group combining with 'ix', or just use 'group' instead+--+-- >>> "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . groups .  ix 1+-- ["roses","kittens"]+--+-- Editing groups:+--+-- >>> "raindrops on roses and whiskers on kittens" & [regex|(\w+) on (\w+)|] . groups .  ix 1 %~ Char8.map toUpper+-- "raindrops on ROSES and whiskers on KITTENS"+--+-- Editing the list rearranges groups+--+-- >>> "raindrops on roses and whiskers on kittens" & [regex|(\w+) on (\w+)|] . groups %~ reverse+-- "roses on raindrops and kittens on whiskers"+--+-- You can traverse the list to flatten out all groups+--+-- >>> "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . groups . traversed+-- ["raindrops","roses","whiskers","kittens"]+--+-- Use indexed helpers to access the full match when operating on a group.+--+-- This replaces each group with the full match text wrapped in parens:+--+-- >>> "one-two" & [regex|(\w+)-(\w+)|] . groups <. traversed %@~ \mtch grp -> grp <> ":(" <> mtch <> ")"+-- "one:(one-two)-two:(one-two)"+groups :: IndexedTraversal' BS.ByteString Match [BS.ByteString]+groups = conjoined groupsT (reindexed (view match) selfIndex <. groupsT)+    where+      groupsT :: Traversal' Match [BS.ByteString]+      groupsT = chunks . partsOf (traversed . _Right . building)++-- | Access a specific group of a match. Numbering starts at 0.+--+-- Stashes the full match text as the index in case you need it.+--+-- See 'groups' for more info on grouping+--+-- >>> "key:value, a:b" ^.. [regex|(\w+):(\w+)|] . group 0+-- ["key","a"]+--+-- >>> "key:value, a:b" ^.. [regex|(\w+):(\w+)|] . group 1+-- ["value","b"]+--+-- >>> "key:value, a:b" & [regex|(\w+):(\w+)|] . group 1 %~ Char8.map toUpper+-- "key:VALUE, a:B"+--+-- Replace the first capture group with the full match:+--+-- >>> "a, b" & [regex|(\w+), (\w+)|] . group 0 .@~ \i -> "(" <> i <> ")"+-- "(a, b), b"+group :: Int -> IndexedTraversal' BS.ByteString Match BS.ByteString+group n = groups <. ix n++-- | Traverse each match+--+-- Stashes any matched groups into the index in case you need them.+--+--  Get a match if one exists:+--+-- >>> "find a needle in a haystack" ^? [regex|n..dle|] . match+-- Just "needle"+--+--  Collect all matches+--+-- >>> "one _two_ three _four_" ^.. [regex|_\w+_|] . match+-- ["_two_","_four_"]+--+-- You can edit the traversal to perform a regex replace/substitution+--+-- >>> "one _two_ three _four_" & [regex|_\w+_|] . match %~ Char8.map toUpper+-- "one _TWO_ three _FOUR_"+--+-- Here we use the group matches stored in the index to form key-value pairs, replacing the entire match.+--+-- >>> "abc-def, ghi-jkl" & [regex|(\w+)-(\w+)|] . match %@~ \[k, v] _ -> "{" <> k <> ":" <> v <> "}"+-- "{abc:def}, {ghi:jkl}"+match :: IndexedTraversal' [BS.ByteString] Match BS.ByteString+match = conjoined matchBS (reindexed (view groups) selfIndex <. matchBS)+  where+    matchBS :: Traversal' Match BS.ByteString+    matchBS = chunks . matchT . building+    matchT :: Traversal' [Either BS.Builder BS.Builder] BS.Builder+    matchT f grps =+        (:[]) . Right <$> f (grps ^. folded . chosen)++-- | Build a traversal from the provided 'PCRE.Regex', this is handy if you're QuasiQuoter+-- averse, or if you already have a 'PCRE.Regex' object floating around.+--+-- Also see 'mkRegexTraversalQQ'+regexing :: PCRE.Regex -> IndexedTraversal' Int BS.ByteString Match+regexing pattern = conjoined (regexT pattern) (indexing (regexT pattern)) . from chunks++-- | Base regex traversal helper+regexT :: PCRE.Regex -> Traversal' BS.ByteString [Either BS.Builder BS.Builder]+regexT pattern f txt = unBuilder . collapseMatch <$> apply (splitAll txt matches)+  where+    matches :: [(MatchRange, GroupRanges)]+    matches = PCRE.scanRanges pattern txt+    collapseMatch :: [Either BS.Builder [Either BS.Builder BS.Builder]] -> BS.Builder+    collapseMatch xs = xs ^. folded . beside id (traversed . chosen)+    apply xs = xs & traversed . _Right %%~ f++-- | Collect both the match text AND all the matching groups+--+-- >>> "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . matchAndGroups+-- [("raindrops on roses",["raindrops","roses"]),("whiskers on kittens",["whiskers","kittens"])]+matchAndGroups :: Getter Match (BS.ByteString, [BS.ByteString])+matchAndGroups = to $ \m -> (m ^. match, m ^. groups)++-- | Builds a traversal over text using a Regex pattern+--+-- It's a 'QuasiQuoter' which creates a Traversal out of the given regex string.+-- It's equivalent to calling 'regexing' on a 'Regex' created using the+-- 're' QuasiQuoter.+--+-- The "real" type is:+--+-- > regex :: Regex -> IndexedTraversal' Int BS.ByteString Match+--+-- It's a traversal which selects 'Match'es; compose it with 'match' or 'groups'+-- to get the relevant parts of your match.+--+-- >>> txt = "raindrops on roses and whiskers on kittens"+--+-- Search+--+-- >>> has ([regex|whisk|]) txt+-- True+--+-- Get matches+--+-- >>> txt ^.. [regex|\br\w+|] . match+-- ["raindrops","roses"]+--+-- Edit matches+--+-- >>> txt & [regex|\br\w+|] . match %~ Char8.intersperse '-' . Char8.map toUpper+-- "R-A-I-N-D-R-O-P-S on R-O-S-E-S and whiskers on kittens"+--+-- Get Groups+--+-- >>> txt ^.. [regex|(\w+) on (\w+)|] . groups+-- [["raindrops","roses"],["whiskers","kittens"]]+--+-- Edit Groups+--+-- >>> txt & [regex|(\w+) on (\w+)|] . groups %~ reverse+-- "roses on raindrops and kittens on whiskers"+--+-- Get the third match+--+-- >>> txt ^? [regex|\w+|] . index 2 . match+--Just "roses"+--+-- Edit matches+--+-- >>> txt & [regex|\br\w+|] . match %~ Char8.intersperse '-' . Char8.map toUpper+-- "R-A-I-N-D-R-O-P-S on R-O-S-E-S and whiskers on kittens"+--+-- Get Groups+--+-- >>> txt ^.. [regex|(\w+) on (\w+)|] . groups+-- [["raindrops","roses"],["whiskers","kittens"]]+--+-- Edit Groups+--+-- >>> txt & [regex|(\w+) on (\w+)|] . groups %~ reverse+-- "roses on raindrops and kittens on whiskers"+--+-- Get the third match+--+-- >>> txt ^? [regex|\w+|] . index 2 . match+-- Just "roses"+--+-- Match integers, 'Read' them into ints, then sort them in-place+-- dumping them back into the source text afterwards.+--+-- >>> "Monday: 29, Tuesday: 99, Wednesday: 3" & partsOf ([regex|\d+|] . match . from packedChars . _Show @Int) %~ sort+-- "Monday: 3, Tuesday: 29, Wednesday: 99"+--+-- To alter behaviour of the regex you may wish to pass 'PCREOption's when compiling it.+-- The default behaviour may seem strange in certain cases; e.g. it operates in 'single-line'+-- mode. You can 'compile' the 'Regex' separately and add any options you like, then pass the resulting+-- 'Regex' into 'regex';+-- Alternatively can make your own version of the QuasiQuoter with any options you want embedded+-- by using 'mkRegexQQ'.+regex :: TH.QuasiQuoter+regex = PCRE.re{TH.quoteExp=quoter}+  where+    quoter str = do+        rgx <- TH.quoteExp PCRE.re str+        regexExpr <- TH.varE 'regexing+        return $ TH.AppE regexExpr rgx++-- | Build a QuasiQuoter just like 'regex' but with the provided 'PCRE.PCREOption' overrides.+mkRegexTraversalQQ :: [PCRE.PCREOption] -> TH.QuasiQuoter+mkRegexTraversalQQ opts = (PCRE.mkRegexQQ opts){TH.quoteExp=quoter}+  where+    quoter str = do+        rgx <- TH.quoteExp (PCRE.mkRegexQQ opts) str+        regexExpr <- TH.varE 'regexing+        return $ TH.AppE regexExpr rgx++---------------------------------------------------------------------------------------------++splitAll :: BS.ByteString -> [(MatchRange, GroupRanges)] -> [Either BS.Builder [Either BS.Builder BS.Builder]]+splitAll txt matches = fmap (second (\(txt', (start,_), grps) -> groupSplit txt' start grps)) splitUp+  where+    splitUp = splits txt 0 matches++groupSplit :: BS.ByteString -> Int -> GroupRanges -> [Either BS.Builder BS.Builder]+groupSplit txt _ [] = [Left $ BS.byteString txt]+groupSplit txt offset ((grpStart, grpEnd) : rest) | offset == grpStart =+    let (prefix, suffix) = BS.splitAt (grpEnd - offset) txt+     in Right (BS.byteString prefix) : groupSplit suffix grpEnd rest+groupSplit txt offset ((grpStart, grpEnd) : rest) =+    let (prefix, suffix) = BS.splitAt (grpStart - offset) txt+     in Left (BS.byteString prefix) : groupSplit suffix grpStart ((grpStart, grpEnd) : rest)++splits :: BS.ByteString -> Int -> [(MatchRange, GroupRanges)] -> [Either BS.Builder (BS.ByteString, MatchRange, GroupRanges)]+-- No more matches left+splits txt _ [] = [Left $ BS.byteString txt]+-- We're positioned at a match+splits txt offset (((start, end), grps) : rest) | offset == start =+    let (prefix, suffix) = BS.splitAt (end - offset) txt+     in (Right (prefix, (start, end), grps)) : splits suffix end rest+-- jump to the next match+splits txt offset matches@(((start, _), _) : _) =+    let (prefix, suffix) = BS.splitAt (start - offset) txt+     in (Left $ BS.byteString prefix) : splits suffix start matches

@@ -0,0 +1,251 @@+{-|+Module      : Control.Lens.Regex.Text+Description : Text PCRE Regex library with a lensy interface.+Copyright   : (c) Chris Penner, 2019+License     : BSD3+-}++{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE PartialTypeSignatures #-}+{-# LANGUAGE QuasiQuotes #-}+{-# LANGUAGE RankNTypes #-}+{-# LANGUAGE TemplateHaskell #-}++module Control.Lens.Regex.Text+    (+    -- * Basics+      regex+    , match+    , groups+    , group+    , matchAndGroups++    -- * Compiling regexes to Traversals+    , regexing+    , mkRegexTraversalQQ++    -- * Types+    , RBS.Match+    , PCRE.Regex+    ) where++import qualified Data.Text as T+import qualified Data.Text.Encoding as T+import qualified Data.Text.Encoding.Error as T+import qualified Data.ByteString as BS+import qualified Text.Regex.PCRE.Heavy as PCRE+import Control.Lens hiding (re, matching)+import qualified Language.Haskell.TH as TH+import qualified Language.Haskell.TH.Quote as TH++import qualified Control.Lens.Regex.ByteString as RBS++-- $setup+-- >>> :set -XQuasiQuotes+-- >>> :set -XOverloadedStrings+-- >>> :set -XTypeApplications+-- >>> import Data.Text.Lens (unpacked)+-- >>> import qualified Data.Text as T+-- >>> import Data.List (sort)++utf8 :: Iso' T.Text BS.ByteString+utf8 = iso T.encodeUtf8 (T.decodeUtf8With T.lenientDecode)++-- | Builds a traversal over text using a Regex pattern+--+-- It's a 'QuasiQuoter' which creates a Traversal out of the given regex string.+-- It's equivalent to calling 'regexing' on a 'Regex' created using the+-- 're' QuasiQuoter.+--+-- The "real" type is:+--+-- > regex :: Regex -> IndexedTraversal' Int T.Text Match+--+-- It's a traversal which selects 'Match'es; compose it with 'match' or 'groups'+-- to get the relevant parts of your match.+--+-- >>> txt = "raindrops on roses and whiskers on kittens"+--+-- Search+--+-- >>> has ([regex|whisk|]) txt+-- True+--+-- Get matches+--+-- >>> txt ^.. [regex|\br\w+|] . match+-- ["raindrops","roses"]+--+-- Edit matches+--+-- >>> txt & [regex|\br\w+|] . match %~ T.intersperse '-' . T.toUpper+-- "R-A-I-N-D-R-O-P-S on R-O-S-E-S and whiskers on kittens"+--+-- Get Groups+--+-- >>> txt ^.. [regex|(\w+) on (\w+)|] . groups+-- [["raindrops","roses"],["whiskers","kittens"]]+--+-- Edit Groups+--+-- >>> txt & [regex|(\w+) on (\w+)|] . groups %~ reverse+-- "roses on raindrops and kittens on whiskers"+--+-- Get the third match+--+-- >>> txt ^? [regex|\w+|] . index 2 . match+--Just "roses"+--+-- Edit matches+--+-- >>> txt & [regex|\br\w+|] . match %~ T.intersperse '-' . T.toUpper+-- "R-A-I-N-D-R-O-P-S on R-O-S-E-S and whiskers on kittens"+--+-- Get Groups+--+-- >>> txt ^.. [regex|(\w+) on (\w+)|] . groups+-- [["raindrops","roses"],["whiskers","kittens"]]+--+-- Edit Groups+--+-- >>> txt & [regex|(\w+) on (\w+)|] . groups %~ reverse+-- "roses on raindrops and kittens on whiskers"+--+-- Get the third match+--+-- >>> txt ^? [regex|\w+|] . index 2 . match+-- Just "roses"+--+-- Match integers, 'Read' them into ints, then sort them in-place+-- dumping them back into the source text afterwards.+--+-- >>> "Monday: 29, Tuesday: 99, Wednesday: 3" & partsOf ([regex|\d+|] . match . unpacked . _Show @Int) %~ sort+-- "Monday: 3, Tuesday: 29, Wednesday: 99"+--+-- To alter behaviour of the regex you may wish to pass 'PCREOption's when compiling it.+-- The default behaviour may seem strange in certain cases; e.g. it operates in 'single-line'+-- mode. You can 'compile' the 'Regex' separately and add any options you like, then pass the resulting+-- 'Regex' into 'regex';+-- Alternatively can make your own version of the QuasiQuoter with any options you want embedded+-- by using 'mkRegexQQ'.+-- regex :: Regex -> IndexedTraversal' Int T.Text RBS.Match+regex :: TH.QuasiQuoter+regex = PCRE.re{TH.quoteExp=quoter}+  where+    quoter str = do+        rgx <- TH.quoteExp PCRE.re str+        regexExpr <- TH.varE 'regexing+        return $ TH.AppE regexExpr rgx++-- | Build a QuasiQuoter just like 'regex' but with the provided 'PCRE.PCREOption' overrides.+mkRegexTraversalQQ :: [PCRE.PCREOption] -> TH.QuasiQuoter+mkRegexTraversalQQ opts = (PCRE.mkRegexQQ opts){TH.quoteExp=quoter}+  where+    quoter str = do+        rgx <- TH.quoteExp (PCRE.mkRegexQQ opts) str+        regexExpr <- TH.varE 'regexing+        return $ TH.AppE regexExpr rgx++-- | Build a traversal from the provided 'PCRE.Regex', this is handy if you're QuasiQuoter+-- averse, or if you already have a 'PCRE.Regex' object floating around.+--+-- Also see 'mkRegexTraversalQQ'+regexing :: PCRE.Regex -> IndexedTraversal' Int T.Text RBS.Match+regexing pat = utf8 . RBS.regexing pat++-- | Access all groups of a match as a list. Also keeps full match text as the index in case+-- you need it.+--+-- Note that you can edit the groups through this traversal,+-- Changing the length of the list has behaviour similar to 'partsOf'.+--+-- Get all matched groups:+--+-- >>> "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . groups+-- [["raindrops","roses"],["whiskers","kittens"]]+--+-- You can access a specific group combining with 'ix', or just use 'group' instead+--+-- >>> "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . groups .  ix 1+-- ["roses","kittens"]+--+-- Editing groups:+--+-- >>> "raindrops on roses and whiskers on kittens" & [regex|(\w+) on (\w+)|] . groups .  ix 1 %~ T.toUpper+-- "raindrops on ROSES and whiskers on KITTENS"+--+-- Editing the list rearranges groups+--+-- >>> "raindrops on roses and whiskers on kittens" & [regex|(\w+) on (\w+)|] . groups %~ Prelude.reverse+-- "roses on raindrops and kittens on whiskers"+--+-- You can traverse the list to flatten out all groups+--+-- >>> "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . groups . traversed+-- ["raindrops","roses","whiskers","kittens"]+--+-- This replaces each group with the full match text wrapped in parens:+--+-- >>> "one-two" & [regex|(\w+)-(\w+)|] . groups <. traversed %@~ \mtch grp -> grp <> ":(" <> mtch <> ")"+-- "one:(one-two)-two:(one-two)"+groups :: IndexedTraversal' T.Text RBS.Match [T.Text]+groups = reindexed (view $ from utf8) RBS.groups <. mapping (from utf8)++-- | Access a specific group of a match. Numbering starts at 0.+--+-- Stashes the full match text as the index in case you need it.+--+-- See 'groups' for more info on grouping+--+-- >>> "key:value, a:b" ^.. [regex|(\w+):(\w+)|] . group 0+-- ["key","a"]+--+-- >>> "key:value, a:b" ^.. [regex|(\w+):(\w+)|] . group 1+-- ["value","b"]+--+-- >>> "key:value, a:b" & [regex|(\w+):(\w+)|] . group 1 %~ T.toUpper+-- "key:VALUE, a:B"+--+-- >>> "key:value, a:b" & [regex|(\w+):(\w+)|] . group 1 %~ T.toUpper+-- "key:VALUE, a:B"+--+-- Replace the first capture group with the full match:+--+-- >>> "a, b" & [regex|(\w+), (\w+)|] . group 0 .@~ \i -> "(" <> i <> ")"+-- "(a, b), b"+group :: Int -> IndexedTraversal' T.Text RBS.Match T.Text+group n = groups <. ix n++-- | Traverse each match+--+-- Stashes any matched groups into the index in case you need them.+--+--  Get a match if one exists:+--+-- >>> "find a needle in a haystack" ^? [regex|n..dle|] . match+-- Just "needle"+--+--  Collect all matches+--+-- >>> "one _two_ three _four_" ^.. [regex|_\w+_|] . match+-- ["_two_","_four_"]+--+-- You can edit the traversal to perform a regex replace/substitution+--+-- >>> "one _two_ three _four_" & [regex|_\w+_|] . match %~ T.toUpper+-- "one _TWO_ three _FOUR_"+--+-- Here we use the group matches stored in the index to form key-value pairs, replacing the entire match.+--+-- >>> "abc-def, ghi-jkl" & [regex|(\w+)-(\w+)|] . match %@~ \[k, v] _ -> "{" <> k <> ":" <> v <> "}"+-- "{abc:def}, {ghi:jkl}"+match :: IndexedTraversal' [T.Text] RBS.Match T.Text+match = reindexed (fmap (view $ from utf8)) RBS.match <. from utf8++-- | Collect both the match text AND all the matching groups+--+-- >>> "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . matchAndGroups+-- [("raindrops on roses",["raindrops","roses"]),("whiskers on kittens",["whiskers","kittens"])]+matchAndGroups :: Getter RBS.Match (T.Text, [T.Text])+matchAndGroups = to $ \m -> (m ^. match, m ^. groups)

@@ -0,0 +1,143 @@+{-# LANGUAGE QuasiQuotes #-}+{-# LANGUAGE OverloadedStrings #-}+module ByteString where++import Control.Lens+import Control.Lens.Regex.ByteString+import qualified Data.ByteString.Char8 as C8 hiding (index)+import Data.Char+import Test.Hspec++spec :: Spec+spec = do+    describe "regex" $ do+        describe "match" $ do+            describe "getting" $ do+                it "should find one match" $ do+                    "abc" ^.. [regex|b|] . match+                    `shouldBe` ["b"]++                it "should find many matches" $ do+                    "a b c" ^.. [regex|\w|] . match+                    `shouldBe` ["a", "b", "c"]++                it "should fold" $ do+                    "a b c" ^. [regex|\w|] . match+                    `shouldBe` "abc"++                it "should match with a group" $ do+                    "a b c" ^.. [regex|(\w)|] . match+                    `shouldBe` ["a", "b", "c"]++                it "should match with many groups" $ do+                    "a b c" ^.. [regex|(\w) (\w)|] . match+                    `shouldBe` ["a b"]++                it "should be greedy when overlapping" $ do+                    "abc" ^.. [regex|\w+|] . match+                    `shouldBe`["abc"]++                it "should respect lazy modifiers" $ do+                    "abc" ^.. [regex|\w+?|] . match+                    `shouldBe`["a", "b", "c"]++            describe "setting" $ do+                it "should allow setting" $ do+                    ("one two three" & [regex|two|] . match .~ "new")+                    `shouldBe` "one new three"++                it "should allow setting many" $ do+                    ("one <two> three" & [regex|\w+|] . match .~ "new")+                    `shouldBe` "new <new> new"++                it "should allow mutating" $ do+                    ("one two three" & [regex|two|] . match %~ (<> "!!"). C8.map toUpper)+                    `shouldBe` "one TWO!! three"++                it "should allow mutating many" $ do+                    ("one two three" & [regex|two|] . match %~ C8.map toUpper)+                    `shouldBe` "one TWO three"++        describe "indexed" $ do+            it "should allow folding with index" $ do+                ("one two three" ^.. ([regex|\w+|] <. match) . withIndex)+                `shouldBe` [(0, "one"), (1, "two"), (2, "three")]++            it "should allow getting with index" $ do+                ("one two three" ^.. [regex|\w+|] . index 1 . match)+                `shouldBe` ["two"]++            it "should allow setting with index" $ do+                ("one two three" & [regex|\w+|] <. match .@~ C8.pack . show)+                `shouldBe` "0 1 2"++            it "should allow mutating with index" $ do+                ("one two three" & [regex|\w+|] <. match %@~ \i s -> (C8.pack $ show i) <> ": " <> s)+                `shouldBe` "0: one 1: two 2: three"++    describe "groups" $ do+        describe "getting" $ do+            it "should get groups" $ do+                "a b c" ^.. [regex|(\w)|] . groups+                `shouldBe` [["a"], ["b"], ["c"]]++            it "should get multiple groups" $ do+                "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . groups+                `shouldBe` [["raindrops","roses"],["whiskers","kittens"]]++            it "should allow getting a specific index" $ do+                ("one two three four" ^.. [regex|(\w+) (\w+)|] . groups . ix 1)+                `shouldBe` ["two", "four"]++            xit "should handle weird group alternation" $ do+                "1:2 a=b" ^.. [regex|(\d):(\d)|(\w)=(\w)|] . groups+                `shouldBe` [[ "not entirely sure what I expect this to be yet" ]]++        describe "setting" $ do+            it "should allow setting groups as a list" $ do+                ("one two three" & [regex|(\w+) (\w+)|] . groups .~ ["1", "2"])+                `shouldBe` "1 2 three"++            it "should allow editing when result list is the same length" $ do+                ("raindrops on roses and whiskers on kittens" & [regex|(\w+) on (\w+)|] . groups %~ reverse)+                `shouldBe` "roses on raindrops and kittens on whiskers"++    describe "group" $ do+        it "should get a single group" $ do+                "a:b c:d" ^.. [regex|(\w):(\w)|] . group 1+                `shouldBe` ["b", "d"]++        it "should set a single group" $ do+                "a:b c:d" & [regex|(\w):(\w)|] . group 1 %~ C8.map toUpper+                `shouldBe` "a:B c:D"++        describe "traversed" $ do+            it "should allow setting all group matches" $ do+                ("one two three" & [regex|(\w+) (\w+)|] . groups . traversed .~ "new")+                `shouldBe` "new new three"++            it "should allow mutating" $ do+                ("one two three four" & [regex|one (two) (three)|] . groups . traversed %~ (<> "!!") . C8.map toUpper)+                `shouldBe` "one TWO!! THREE!! four"++            it "should allow folding with index" $ do+                ("one two three four" ^.. [regex|(\w+) (\w+)|] . groups . traversed . withIndex)+                `shouldBe` [(0, "one"), (1, "two"), (0, "three"), (1, "four")]++            it "should allow setting with index" $ do+                ("one two three four" & [regex|(\w+) (\w+)|] . groups . traversed .@~ C8.pack . show)+                `shouldBe` "0 1 0 1"++            it "should allow mutating with index" $ do+                ("one two three four" & [regex|(\w+) (\w+)|] . groups . traversed %@~ \i s -> (C8.pack $ show i) <> ": " <> s)+                `shouldBe` "0: one 1: two 0: three 1: four"++            it "should compose indices with matches" $ do+                ("one two three four" ^.. ([regex|(\w+) (\w+)|] <.> groups . traversed) . withIndex)+                `shouldBe` [((0, 0), "one"), ((0, 1), "two"), ((1, 0), "three"), ((1, 1), "four")]++    describe "matchAndGroups" $ do+        it "should get match and groups" $ do+            "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . matchAndGroups+            `shouldBe` [("raindrops on roses",["raindrops","roses"]),("whiskers on kittens",["whiskers","kittens"])]+

@@ -1,141 +1,10 @@ {-# LANGUAGE QuasiQuotes #-} {-# LANGUAGE OverloadedStrings #-}-import Control.Lens-import Control.Lens.Regex-import qualified Data.Text as T import Test.Hspec+import Text+import ByteString  main :: IO () main = hspec $ do-    describe "regex" $ do-        describe "match" $ do-            describe "getting" $ do-                it "should find one match" $ do-                    "abc" ^.. regex [rx|b|] . match-                    `shouldBe` ["b"]--                it "should find many matches" $ do-                    "a b c" ^.. regex [rx|\w|] . match-                    `shouldBe` ["a", "b", "c"]--                it "should fold" $ do-                    "a b c" ^. regex [rx|\w|] . match-                    `shouldBe` "abc"--                it "should match with a group" $ do-                    "a b c" ^.. regex [rx|(\w)|] . match-                    `shouldBe` ["a", "b", "c"]--                it "should match with many groups" $ do-                    "a b c" ^.. regex [rx|(\w) (\w)|] . match-                    `shouldBe` ["a b"]--                it "should be greedy when overlapping" $ do-                    "abc" ^.. regex [rx|\w+|] . match-                    `shouldBe`["abc"]--                it "should respect lazy modifiers" $ do-                    "abc" ^.. regex [rx|\w+?|] . match-                    `shouldBe`["a", "b", "c"]--                it "should handle unicode in source text properly" $ do-                    "🍕 test 🍔" ^. regex [rx|test|] . match-                        `shouldBe` "test"-                    ("🍕 test 🍔" & regex [rx|🍔|] . match .~ "👻🙈")-                        `shouldBe` "🍕 test 👻🙈"--                it "should handle unicode in patterns properly" $ do-                    "*🍕 test 🍔*" ^. regex [rx|🍕 \w+ 🍔|] . match-                    `shouldBe` "🍕 test 🍔"--            describe "setting" $ do-                it "should allow setting" $ do-                    ("one two three" & regex [rx|two|] . match .~ "new")-                    `shouldBe` "one new three"--                it "should allow setting many" $ do-                    ("one <two> three" & regex [rx|\w+|] . match .~ "new")-                    `shouldBe` "new <new> new"--                it "should allow mutating" $ do-                    ("one two three" & regex [rx|two|] . match %~ (<> "!!"). T.toUpper)-                    `shouldBe` "one TWO!! three"--                it "should allow mutating many" $ do-                    ("one two three" & regex [rx|two|] . match %~ T.toUpper)-                    `shouldBe` "one TWO three"--        describe "indexed" $ do-            it "should allow folding with index" $ do-                ("one two three" ^.. (regex [rx|\w+|] <. match) . withIndex)-                `shouldBe` [(0, "one"), (1, "two"), (2, "three")]--            it "should allow getting with index" $ do-                ("one two three" ^.. regex [rx|\w+|] . index 1 . match)-                `shouldBe` ["two"]--            it "should allow setting with index" $ do-                ("one two three" & regex [rx|\w+|] <. match .@~ T.pack . show)-                `shouldBe` "0 1 2"--            it "should allow mutating with index" $ do-                ("one two three" & regex [rx|\w+|] <. match %@~ \i s -> (T.pack $ show i) <> ": " <> s)-                `shouldBe` "0: one 1: two 2: three"--    describe "groups" $ do-        describe "getting" $ do-            it "should get groups" $ do-                "a b c" ^.. regex [rx|(\w)|] . groups-                `shouldBe` [["a"], ["b"], ["c"]]--            it "should get multiple groups" $ do-                "raindrops on roses and whiskers on kittens" ^.. regex [rx|(\w+) on (\w+)|] . groups-                `shouldBe` [["raindrops","roses"],["whiskers","kittens"]]--            it "should allow getting a specific index" $ do-                ("one two three four" ^.. regex [rx|(\w+) (\w+)|] . groups . ix 1)-                `shouldBe` ["two", "four"]--            xit "should handle weird group alternation" $ do-                "1:2 a=b" ^.. regex [rx|(\d):(\d)|(\w)=(\w)|] . groups-                `shouldBe` [[ "not entirely sure what I expect this to be yet" ]]--        describe "setting" $ do-            it "should allow setting groups as a list" $ do-                ("one two three" & regex [rx|(\w+) (\w+)|] . groups .~ ["1", "2"])-                `shouldBe` "1 2 three"--            it "should allow editing when result list is the same length" $ do-                ("raindrops on roses and whiskers on kittens" & regex [rx|(\w+) on (\w+)|] . groups %~ reverse)-                `shouldBe` "roses on raindrops and kittens on whiskers"--        describe "traversed" $ do-            it "should allow setting all group matches" $ do-                ("one two three" & regex [rx|(\w+) (\w+)|] . groups . traversed .~ "new")-                `shouldBe` "new new three"--            it "should allow mutating" $ do-                ("one two three four" & regex [rx|one (two) (three)|] . groups . traversed %~ (<> "!!") . T.toUpper)-                `shouldBe` "one TWO!! THREE!! four"--            it "should allow folding with index" $ do-                ("one two three four" ^.. regex [rx|(\w+) (\w+)|] . groups . traversed . withIndex)-                `shouldBe` [(0, "one"), (1, "two"), (0, "three"), (1, "four")]--            it "should allow setting with index" $ do-                ("one two three four" & regex [rx|(\w+) (\w+)|] . groups . traversed .@~ T.pack . show)-                `shouldBe` "0 1 0 1"--            it "should allow mutating with index" $ do-                ("one two three four" & regex [rx|(\w+) (\w+)|] . groups . traversed %@~ \i s -> (T.pack $ show i) <> ": " <> s)-                `shouldBe` "0: one 1: two 0: three 1: four"--            it "should compose indices with matches" $ do-                ("one two three four" ^.. (regex [rx|(\w+) (\w+)|] <.> groups . traversed) . withIndex)-                `shouldBe` [((0, 0), "one"), ((0, 1), "two"), ((1, 0), "three"), ((1, 1), "four")]--    describe "matchAndGroups" $ do-        it "should get match and groups" $ do-            "raindrops on roses and whiskers on kittens" ^.. regex [rx|(\w+) on (\w+)|] . matchAndGroups-            `shouldBe` [("raindrops on roses",["raindrops","roses"]),("whiskers on kittens",["whiskers","kittens"])]-+    describe "text" Text.spec+    describe "bytestring" ByteString.spec

@@ -0,0 +1,152 @@+{-# LANGUAGE QuasiQuotes #-}+{-# LANGUAGE OverloadedStrings #-}+module Text where++import Control.Lens+import Control.Lens.Regex.Text+import qualified Data.Text as T+import Test.Hspec++spec :: Spec+spec = do+    describe "regex" $ do+        describe "match" $ do+            describe "getting" $ do+                it "should find one match" $ do+                    "abc" ^.. [regex|b|] . match+                    `shouldBe` ["b"]++                it "should find many matches" $ do+                    "a b c" ^.. [regex|\w|] . match+                    `shouldBe` ["a", "b", "c"]++                it "should fold" $ do+                    "a b c" ^. [regex|\w|] . match+                    `shouldBe` "abc"++                it "should match with a group" $ do+                    "a b c" ^.. [regex|(\w)|] . match+                    `shouldBe` ["a", "b", "c"]++                it "should match with many groups" $ do+                    "a b c" ^.. [regex|(\w) (\w)|] . match+                    `shouldBe` ["a b"]++                it "should be greedy when overlapping" $ do+                    "abc" ^.. [regex|\w+|] . match+                    `shouldBe`["abc"]++                it "should respect lazy modifiers" $ do+                    "abc" ^.. [regex|\w+?|] . match+                    `shouldBe`["a", "b", "c"]++                it "should handle unicode in source text properly" $ do+                    "🍕 test 🍔" ^. [regex|test|] . match+                        `shouldBe` "test"+                    ("🍕 test 🍔" & [regex|🍔|] . match .~ "👻🙈")+                        `shouldBe` "🍕 test 👻🙈"++                it "should handle unicode in patterns properly" $ do+                    "*🍕 test 🍔*" ^. [regex|🍕 \w+ 🍔|] . match+                    `shouldBe` "🍕 test 🍔"++            describe "setting" $ do+                it "should allow setting" $ do+                    ("one two three" & [regex|two|] . match .~ "new")+                    `shouldBe` "one new three"++                it "should allow setting many" $ do+                    ("one <two> three" & [regex|\w+|] . match .~ "new")+                    `shouldBe` "new <new> new"++                it "should allow mutating" $ do+                    ("one two three" & [regex|two|] . match %~ (<> "!!"). T.toUpper)+                    `shouldBe` "one TWO!! three"++                it "should allow mutating many" $ do+                    ("one two three" & [regex|two|] . match %~ T.toUpper)+                    `shouldBe` "one TWO three"++        describe "indexed" $ do+            it "should allow folding with index" $ do+                ("one two three" ^.. ([regex|\w+|] <. match) . withIndex)+                `shouldBe` [(0, "one"), (1, "two"), (2, "three")]++            it "should allow getting with index" $ do+                ("one two three" ^.. [regex|\w+|] . index 1 . match)+                `shouldBe` ["two"]++            it "should allow setting with index" $ do+                ("one two three" & [regex|\w+|] <. match .@~ T.pack . show)+                `shouldBe` "0 1 2"++            it "should allow mutating with index" $ do+                ("one two three" & [regex|\w+|] <. match %@~ \i s -> (T.pack $ show i) <> ": " <> s)+                `shouldBe` "0: one 1: two 2: three"++    describe "groups" $ do+        describe "getting" $ do+            it "should get groups" $ do+                "a b c" ^.. [regex|(\w)|] . groups+                `shouldBe` [["a"], ["b"], ["c"]]++            it "should get multiple groups" $ do+                "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . groups+                `shouldBe` [["raindrops","roses"],["whiskers","kittens"]]++            it "should allow getting a specific index" $ do+                ("one two three four" ^.. [regex|(\w+) (\w+)|] . groups . ix 1)+                `shouldBe` ["two", "four"]++            xit "should handle weird group alternation" $ do+                "1:2 a=b" ^.. [regex|(\d):(\d)|(\w)=(\w)|] . groups+                `shouldBe` [[ "not entirely sure what I expect this to be yet" ]]++        describe "setting" $ do+            it "should allow setting groups as a list" $ do+                ("one two three" & [regex|(\w+) (\w+)|] . groups .~ ["1", "2"])+                `shouldBe` "1 2 three"++            it "should allow editing when result list is the same length" $ do+                ("raindrops on roses and whiskers on kittens" & [regex|(\w+) on (\w+)|] . groups %~ reverse)+                `shouldBe` "roses on raindrops and kittens on whiskers"++    describe "group" $ do+        it "should get a single group" $ do+                "a:b c:d" ^.. [regex|(\w):(\w)|] . group 1+                `shouldBe` ["b", "d"]++        it "should set a single group" $ do+                "a:b c:d" & [regex|(\w):(\w)|] . group 1 %~ T.toUpper+                `shouldBe` "a:B c:D"++        describe "traversed" $ do+            it "should allow setting all group matches" $ do+                ("one two three" & [regex|(\w+) (\w+)|] . groups . traversed .~ "new")+                `shouldBe` "new new three"++            it "should allow mutating" $ do+                ("one two three four" & [regex|one (two) (three)|] . groups . traversed %~ (<> "!!") . T.toUpper)+                `shouldBe` "one TWO!! THREE!! four"++            it "should allow folding with index" $ do+                ("one two three four" ^.. [regex|(\w+) (\w+)|] . groups . traversed . withIndex)+                `shouldBe` [(0, "one"), (1, "two"), (0, "three"), (1, "four")]++            it "should allow setting with index" $ do+                ("one two three four" & [regex|(\w+) (\w+)|] . groups . traversed .@~ T.pack . show)+                `shouldBe` "0 1 0 1"++            it "should allow mutating with index" $ do+                ("one two three four" & [regex|(\w+) (\w+)|] . groups . traversed %@~ \i s -> (T.pack $ show i) <> ": " <> s)+                `shouldBe` "0: one 1: two 0: three 1: four"++            it "should compose indices with matches" $ do+                ("one two three four" ^.. ([regex|(\w+) (\w+)|] <.> groups . traversed) . withIndex)+                `shouldBe` [((0, 0), "one"), ((0, 1), "two"), ((1, 0), "three"), ((1, 1), "four")]++    describe "matchAndGroups" $ do+        it "should get match and groups" $ do+            "raindrops on roses and whiskers on kittens" ^.. [regex|(\w+) on (\w+)|] . matchAndGroups+            `shouldBe` [("raindrops on roses",["raindrops","roses"]),("whiskers on kittens",["whiskers","kittens"])]+