unicode-collation 0.1.3 → 0.1.3.1
raw patch · 6 files changed
+29/−25 lines, 6 filesdep ~base
Dependency ranges changed: base
Files
- CHANGELOG.md +6/−0
- README.md +10/−10
- src/Text/Collate/Collation.hs +1/−2
- src/Text/Collate/Normalize.hs +9/−10
- stack.yaml +1/−1
- unicode-collation.cabal +2/−2
CHANGELOG.md view
@@ -2,6 +2,12 @@ `unicode-collation` uses [PVP Versioning](https://pvp.haskell.org). +## 0.1.3.1++ * Allow base 4.16 (so the library can compile with ghc 9.2).++ * Micro-optimization in normalize; update benchmarks.+ ## 0.1.3 * Add `collateWithUnpacker` (#4). This allows the library to be
README.md view
@@ -33,9 +33,9 @@ ``` sort a list of 10000 random Texts (en):- 6.0 ms ± 580 μs, 22 MB allocated, 911 KB copied+ 5.9 ms ± 487 μs, 22 MB allocated, 899 KB copied sort same list with text-icu (en):- 2.1 ms ± 122 μs, 7.1 MB allocated, 149 KB copied+ 2.1 ms ± 87 μs, 7.1 MB allocated, 148 KB copied ``` Performance is worse on a sample drawn from a smaller character@@ -44,9 +44,9 @@ ``` sort a list of 10000 Texts (composed latin) (en):- 15 ms ± 1.1 ms, 40 MB allocated, 921 KB copied+ 12 ms ± 1.1 ms, 34 MB allocated, 910 KB copied sort same list with text-icu (en):- 2.3 ms ± 212 μs, 6.9 MB allocated, 140 KB copied+ 2.3 ms ± 56 μs, 7.0 MB allocated, 146 KB copied ``` Much of the impact here comes from normalization (decomposition).@@ -55,16 +55,16 @@ ``` sort same list but pre-normalized (en-u-kk-false):- 5.7 ms ± 508 μs, 19 MB allocated, 887 KB copied+ 5.4 ms ± 168 μs, 19 MB allocated, 909 KB copied ``` On plain ASCII, we get a factor of 3 again: ``` sort a list of 10000 ASCII Texts (en):- 4.3 ms ± 66 μs, 16 MB allocated, 892 KB copied+ 4.6 ms ± 405 μs, 17 MB allocated, 880 KB copied sort same list with text-icu (en):- 1.4 ms ± 107 μs, 6.2 MB allocated, 140 KB copied+ 1.6 ms ± 114 μs, 6.2 MB allocated, 130 KB copied ``` Note that this library does incremental normalization,@@ -76,9 +76,9 @@ ``` sort a list of 10000 random Texts that agree in first 32 chars:- 118 ms ± 8.2 ms, 430 MB allocated, 713 KB copied+ 116 ms ± 8.6 ms, 430 MB allocated, 710 KB copied sort same list with text-icu (en):- 3.0 ms ± 226 μs, 8.8 MB allocated, 222 KB copied+ 3.2 ms ± 251 μs, 8.8 MB allocated, 222 KB copied ``` However, in the special case where the texts are identical,@@ -87,7 +87,7 @@ ``` sort a list of 10000 identical Texts (en):- 911 μs ± 34 μs, 468 KB allocated, 10 KB copied+ 877 μs ± 54 μs, 462 KB allocated, 9.7 KB copied ``` ## Localized collations
src/Text/Collate/Collation.hs view
@@ -266,8 +266,7 @@ | otherwise -> (0xFBC0 + (cp `shiftR` 15), (cp .&. 0x7FFFF) .|. 0x8000) --- | Parse a 'Collation' from a 'ByteString' in the format of--- @allkeys.txt@.+-- | Parse a 'Collation' from a Text in the format of @allkeys.txt@. parseCollation :: Text -> Collation parseCollation = foldl' processLine mempty . T.lines where
src/Text/Collate/Normalize.hs view
@@ -30,18 +30,17 @@ go [] = [] go (c:cs) = if canonicalCombiningClass c == 0- then- c : case reorderMarks cs of- ([], rest) -> go rest- (marks, rest) -> foldr (:) (go rest) marks- else- case reorderMarks (c:cs) of- ([], rest) -> go rest- (marks, rest) -> foldr (:) (go rest) marks+ then c : reorderMarks cs+ else reorderMarks (c:cs) reorderMarks zs = case break (\z -> canonicalCombiningClass z == 0) zs of- ([], ys) -> ([], ys)- (xs, ys) -> (sortOn canonicalCombiningClass xs, ys)+ ([], ys) -> go ys+ ([x], ys) -> x : go ys+ ([x1,x2], ys)+ | canonicalCombiningClass x1 <= canonicalCombiningClass x2+ -> x1 : x2 : go ys+ | otherwise -> x2 : x1 : go ys+ (xs, ys) -> sortOn canonicalCombiningClass xs ++ go ys recursivelyDecompose :: [Int] -> [Int] recursivelyDecompose = foldr go mempty
stack.yaml view
@@ -1,4 +1,4 @@-resolver: lts-17.5+resolver: lts-18.10 flags: unicode-collation: executable: true
unicode-collation.cabal view
@@ -1,6 +1,6 @@ cabal-version: 2.2 name: unicode-collation-version: 0.1.3+version: 0.1.3.1 synopsis: Haskell implementation of the Unicode Collation Algorithm description: This library provides a pure Haskell implementation of the Unicode Collation Algorithm described at@@ -48,7 +48,7 @@ Default: False common common-options- build-depends: base >= 4.9 && < 4.16+ build-depends: base >= 4.9 && < 4.17 ghc-options: -Wall -Wcompat