base16 1.0 → 2.0

raw patch · 27 files changed

+1944/−464 lines, 27 filesdep ~basedep ~textPVP ok

version bump matches the API change (PVP)

Dependency ranges changed: base, text

API changes (from Hackage documentation)

Files

• CHANGELOG.md +29/−0
• LICENSE +1/−1
• MIGRATION-1.0.md +1/−1
• MIGRATION-2.0.md +106/−0
• README.md +63/−16
• base16.cabal +45/−21
• benchmarks/Base16Bench.hs +40/−102
• benchmarks/simd-native.c +163/−0
• cbits/base16_simd.c +274/−0
• cbits/base16_simd.h +39/−0
• src/Data/Base16/Types.hs +47/−6
• src/Data/Base16/Types/Internal.hs +0/−56
• src/Data/ByteString/Base16.hs +20/−31
• src/Data/ByteString/Base16/Internal/Head.hs +29/−20
• src/Data/ByteString/Base16/Internal/SIMD.hs +80/−0
• src/Data/ByteString/Base16/Internal/W16/Loop.hs +29/−8
• src/Data/ByteString/Base16/Internal/W16/ShortLoop.hs +39/−14
• src/Data/ByteString/Lazy/Base16.hs +36/−35
• src/Data/ByteString/Short/Base16.hs +19/−27
• src/Data/Text/Encoding/Base16.hs +38/−30
• src/Data/Text/Encoding/Base16/Error.hs +1/−1
• src/Data/Text/Lazy/Encoding/Base16.hs +37/−35
• src/Data/Text/Short/Encoding/Base16.hs +30/−33
• test/Internal.hs +8/−15
• test/Main.hs +198/−12
• test/Simd.hs +215/−0
• test/simd-native.c +357/−0

@@ -1,5 +1,34 @@ # Revision history for base16 +## 2.0++This major release removes two public functions. See+[MIGRATION-2.0.md](MIGRATION-2.0.md) for migration examples.++- Replace `isBase16` and `isValidBase16` with `parseBase16` in all six+  ByteString and Text modules. Parsing checks even length and the mixed-case+  hexadecimal alphabet, then preserves the original encoded representation.+  For a Boolean check a la `isBase16`, just use `isRight` . For `parseBase16`, odd-length alphabet-only inputs+  previously accepted by `isValidBase16` are now rejected.+- Add an optional manual `simd` flag, which is off by default, using the bundled `cbits`+  which give runtime-detected x86-64 AVX2/SSSE3 encoding and decoding.+  Unsupported targets and inputs under the simd threshold use the scalar fallbacks;+  lenient decoding remains scalar.+- Accelerate ShortByteString/ShortText encoding and checked/typed decoding with+  direct byte-array SIMD bindings, without pinning or copying through strict+  ByteString. Short parsers now validate through the short decoder.+- Preserve scalar decoding errors and input-relative offsets with SIMD.+- Fix lazy typed decoding when hex pairs cross chunk boundaries. Lazy checked+  decoding now materializes the complete input for consistent total-length+  checks and global error offsets. Lazy parsing retains the original chunks.+- Fix [#32](https://github.com/emilypi/Base16/issues/32): lenient Text decoding+  no longer throws on malformed UTF-8. Valid UTF-8 is preserved; invalid bytes+  fall back to their Latin-1 characters. Apply the same policy to lazy Text+  and ShortText, avoiding invalid UTF-8 inside the short-text result.+- Require GHC 9.6 or later, support GHC 9.6 through 9.14, refresh CI, and retire+  the old i386 32 bit support.+- Guard encoded output size calculations against integer overflow.+ ## 1.0  - Revamp api (see: [MIGRATION-1.0.md](MIGRATION-1.0.md) in the source repo)

@@ -1,4 +1,4 @@-Copyright (c) 2020, Emily Pillmore+Copyright (c) 2020-2026, Emily Pillmore  All rights reserved. 

@@ -174,4 +174,4 @@ "decodeBase16W*With" -> "decodeBase16*WithUntyped" ``` -And that is all. In order to make use of the new loops, one must only `assertBase16` and proceed with using `decodeBase16` as usual in order to decode. You'll note that an untyped `encodeBase16` is not supplied, and this is due to the fact that it's trivial to extract a `Base16` encoded value once you have it. However, I want to encourage people to use the new API, so I have only supplied a decode with error checking in the untyped case, because sometimes we deal with other people's data and cannot establish provenance. In the encode case, I would rather keep that provenance a part of the API, and the user may opt to strip that data upon sending to others or around their systems. It's not my problem at that point!+And that is all. In order to make use of the new loops, one must only use one of the blessed encode functions to generate a wrapped `Base16` value, or call `assertBase16` and proceed with using `decodeBase16` as usual in order to decode. You'll note that an untyped `encodeBase16` is not supplied, and this is due to the fact that it's trivial to extract a `Base16` encoded value once you have it. However, I want to encourage people to use the new API, so I have only supplied a decode with error checking in the untyped case, because sometimes we deal with other people's data and cannot establish provenance. In the encode case, I would rather keep that provenance a part of the API, and the user may opt to strip that data upon sending to others or around their systems. It's not my problem at that point!

@@ -0,0 +1,106 @@+# Migration guide for 2.0++Version 2.0 replaces `isBase16` and `isValidBase16` with `parseBase16` in all+six ByteString and Text modules. The existing strict ByteString parser keeps+its signature; the other modules now provide the corresponding parser:++```haskell+parseBase16 :: a -> Either Text (Base16 a)+```++Here `Text` in `Either` is strict `Data.Text.Text`, including in the lazy and+short modules. The result retains the original encoded value, including its+case and lazy chunk boundaries.++## Replacing Boolean checks++```haskell+import Data.Either (isRight)+import qualified Data.ByteString.Base16 as B16++-- Previously: B16.isBase16 input+isRight (B16.parseBase16 input)+```++The parser accepts empty input and even-length ASCII hexadecimal strings in+lowercase, uppercase, or mixed case. It rejects odd lengths before checking+characters. An odd-length string such as `"abc"` was accepted by+`isValidBase16`, which checked only the alphabet; `parseBase16` rejects it with+`Left "invalid bytestring size"`. For even-length inputs, invalid characters+produce `Left "invalid character at offset: N"`, using a zero-based byte offset+from the start of the complete input (UTF-8 bytes for Text inputs).++Both predicates are removed in this major release, without a deprecation+period. If an application specifically needs an alphabet-only check, define+that separate policy in the application.++## Validate once, then decode++```haskell+case B16.parseBase16 input of+  Left err      -> handleError err+  Right encoded -> useBytes (B16.decodeBase16 encoded)+```++Use `decodeBase16Untyped` when you want validation and decoded bytes together.+`parseBase16` currently validates via the checked decoder and discards its+output, so parsing followed by typed decoding processes the input twice.+`assertBase16` remains an unchecked assertion for values with known provenance.++Text parsers validate the encoded hex, not the character encoding of the+decoded payload. For example, `"ff"` parses successfully even though byte+`0xff` is not valid UTF-8. Use the ByteString decoding functions or+`decodeBase16With` with a checked text conversion when payloads may be binary.++Lazy typed decoding now handles hex pairs split across chunks. Lazy checked+decoding validates the complete input, giving total-length checks and global+error offsets; it materializes the input as a strict ByteString. Parsing+preserves the original chunks despite this validation step.++## Lenient text decoding++`decodeBase16Lenient` in all three Text modules now handles invalid UTF-8+without throwing or constructing invalid ShortText. After skipping non-hex+characters and dropping a final unpaired hex digit, it decodes valid UTF-8+sequences normally and maps each invalid byte to its Latin-1 character.++This fixes [#32](https://github.com/emilypi/Base16/issues/32): decoding+`"0014df1047454e53"` again yields `"\NUL\DC4ß\DLEGENS"`. Both `"df"` (a lone+Latin-1 byte) and `"c39f"` (valid UTF-8) produce `"ß"`. This is a recovery+policy, not automatic encoding detection: valid UTF-8 takes precedence even+inside otherwise malformed input. Lazy decoding handles sequences across+chunk boundaries and remains streaming.++Typed and checked decoders keep their existing UTF-8 behavior. For explicit+encoding control, decode to bytes and choose a conversion, or use+`decodeBase16With` with a checked conversion.++## Optional SIMD acceleration++The manual `simd` flag is disabled by default. To enable it in `cabal.project`:++```cabal+package base16+  flags: +simd+```++For this repository, use `cabal test --flags=simd`. The bundled C kernels use+ARM64 NEON or runtime-detected x86-64 AVX2/SSSE3. Unsupported targets use the+Haskell scalar implementation. No external SIMD library or global CPU-specific+compiler flags are required. Strict ByteString and ShortByteString encoding,+checked decoding, and typed decoding use SIMD where applicable, as do the+lazy/Text APIs built on them. ShortByteString and ShortText use direct+byte-array bindings without pinning or copying through strict ByteString;+their parsers validate through the short decoder. Inputs below the SIMD+thresholds and lenient decoding retain scalar paths.++## Compiler and module support++Like base64 2.0, this release requires GHC 9.6 or later (`base >= 4.18`),+`bytestring >= 0.11`, and `text >= 2.0`. CI covers GHC 9.6 through 9.14 with+SIMD enabled and disabled; the old i386 CI job has been retired.++The `Base16` definition and instances now live together in `Data.Base16.Types`.+Its constructor remains hidden. The remaining implementation modules are+exposed for advanced use, with their `Internal` names marking implementation+details; `Data.ByteString.Base16.Internal.SIMD` is available only with `simd`.

@@ -3,26 +3,73 @@ ![Build Status](https://github.com/emilypi/base16/workflows/ci/badge.svg) [![Hackage](https://img.shields.io/hackage/v/base16.svg)](https://hackage.haskell.org/package/base16) -RFC 4648-compliant Base16 encoding and decoding.+RFC 4648-compliant Base16 encoding and decoding for strict, lazy, and short+ByteString and Text values. Encoding produces lowercase hex; decoding accepts+lowercase, uppercase, and mixed case. -For the companion optics and pattern synonyms, see [base16-lens](https://hackage.haskell.org/package/base16-lens).+The API tracks encoded values with the `Base16` wrapper. Use `parseBase16` to+validate external input and retain its original representation: -### Summary+```haskell+{-# LANGUAGE OverloadedStrings #-}+import Data.Base16.Types (extractBase16)+import qualified Data.ByteString.Base16 as B16 -The following types have supported codecs:+extractBase16 (B16.encodeBase16' "Sun") -- "53756e"+B16.parseBase16 "53756E"               -- Right "53756E"+B16.decodeBase16Untyped "53756E"       -- Right "Sun"+B16.parseBase16 "abc"                  -- Left "invalid bytestring size"+``` -- `Data.ByteString`-- `Data.ByteString.Lazy`-- `Data.ByteString.Short`-- `Data.Text`-- `Data.Text.Lazy`-- `Data.Text.Short`+For a Boolean check, use `isRight . parseBase16` from `Data.Either`. Version 2.0+removes `isBase16` and `isValidBase16`; see [MIGRATION-2.0.md](MIGRATION-2.0.md)+for replacement examples, lazy decoding changes, and compiler requirements.+For companion optics and pattern synonyms, see+[base16-lens](https://hackage.haskell.org/package/base16-lens). -Additionally this library has+Lenient Text decoding ignores non-hex characters and a final unpaired hex+digit. It preserves valid UTF-8 sequences and interprets each invalid UTF-8+byte as Latin-1, so malformed input does not raise Unicode exceptions. This+also applies to lazy Text and ShortText; see the migration guide for details. -- Much better performance than `base16-bytestring` for encode and decode, with a more conventional api.-- Support for mixed-case hex decoding (defaults to lower-case encoding by convention)-- Optics for handling more complex structures with Base16 representations via the `base16-lens` package-- Checks for both validity and correctness of Base16 encodings.+## SIMD -There are no dependencies aside from those bundled with GHC.+The optional, manual `simd` flag enables bundled ARM64 NEON and runtime-detected+x86-64 AVX2/SSSE3 kernels for strict ByteString and ShortByteString encoding,+checked decoding, and typed decoding, including the lazy and Text APIs built+on those operations. ShortByteString and ShortText use direct byte-array+bindings without pinning or copying through strict ByteString. Their parsers+validate through the short decoder. Unsupported targets and inputs below the+SIMD thresholds fall back to Haskell scalar loops. Lenient decoding remains+scalar.++The flag is off by default. Enable it in your `cabal.project`:++```cabal+package base16+  flags: +simd+```++No external native library is needed. SIMD uses the same alphabet checks and+error offsets as the scalar implementation.++## Development++GHC 9.6 or later is required. Test both build configurations:++```sh+cabal test --flags=-simd+cabal test --flags=simd+cabal bench --flags=-simd --benchmark-options='--iters 1'+cabal bench --flags=simd --benchmark-options='--iters 1'+```++The benchmark suite covers small inputs, vector/call boundaries, and buffers+up to one million bytes. For timings, omit `--iters 1` and use separate build+directories (`--builddir=dist-scalar` and `--builddir=dist-simd`). Compare on the+same machine and compiler; gains depend on input size and CPU.++Native differential tests check exact buffer bounds, alignment, invalid bytes,+and scalar fallback. CI runs them with address/undefined-behavior sanitizers+and guard pages, and builds the Haskell tests, benchmarks, and Haddock from the+source distribution with both flag settings.

@@ -1,6 +1,6 @@ cabal-version:   3.0 name:            base16-version:         1.0+version:         2.0 synopsis:        Fast RFC 4648-compliant Base16 encoding description:   RFC 4648-compliant Base16 encodings and decodings.@@ -11,22 +11,33 @@ license:         BSD-3-Clause license-file:    LICENSE author:          Emily Pillmore-maintainer:      emilypi@cohomolo.gy-copyright:       (c) 2020-2023 Emily Pillmore+maintainer:      Emily Pillmore <emilypi@cohomolo.gy>+copyright:       (c) 2020-2026 Emily Pillmore category:        Data build-type:      Simple extra-doc-files:   CHANGELOG.md   README.md   MIGRATION-1.0.md+  MIGRATION-2.0.md  tested-with:-  GHC ==8.10.7-   || ==9.0.2-   || ==9.2.5-   || ==9.4.5-   || ==9.6.2+  GHC ==9.6.7+   || ==9.8.4+   || ==9.10.3+   || ==9.12.4+   || ==9.14.1 +extra-source-files:+  cbits/base16_simd.h+  test/simd-native.c+  benchmarks/simd-native.c++flag simd+  description:     Use bundled SIMD kernels where supported, with a scalar fallback+  default:         False+  manual:          True+ source-repository head   type:     git   location: https://github.com/emilypi/base16.git@@ -42,19 +53,24 @@     Data.Text.Lazy.Encoding.Base16     Data.Text.Short.Encoding.Base16 -  other-modules:-    Data.Base16.Types.Internal     Data.ByteString.Base16.Internal.Head     Data.ByteString.Base16.Internal.Utils     Data.ByteString.Base16.Internal.W16.Loop     Data.ByteString.Base16.Internal.W16.ShortLoop +  if flag(simd)+    exposed-modules: Data.ByteString.Base16.Internal.SIMD+    cpp-options:    -DBASE16_SIMD+    c-sources:      cbits/base16_simd.c+    include-dirs:   cbits+    cc-options:     -O2 -std=c99+   build-depends:-      base        >=4.14 && <4.19-    , bytestring  ^>=0.11-    , deepseq     ^>=1.4.4.0-    , primitive   >=0.6  && <0.9-    , text        ^>= 2.0+      base        >=4.18 && <4.23+    , bytestring  >=0.11 && <0.13+    , deepseq     >=1.4.4.0 && <1.6+    , primitive   >=0.6  && <0.10+    , text        >=2.0 && <2.3     , text-short  ^>=0.1    hs-source-dirs:   src@@ -63,34 +79,42 @@  test-suite base16-tests   other-modules:    Internal+                    Simd   default-language: Haskell2010   type:             exitcode-stdio-1.0   hs-source-dirs:   test   main-is:          Main.hs   build-depends:-      base               >=4.14 && <4.19+      base               >=4.18 && <4.23     , base16     , base16-bytestring  >=1.0-    , bytestring         ^>=0.11+    , bytestring         >=0.11     , QuickCheck     , random-bytestring     , tasty     , tasty-hunit     , tasty-quickcheck-    , text               ^>=2.0+    , text               >=2.0     , text-short +  ghc-options:      -Wall -threaded -with-rtsopts=-N+ benchmark bench   default-language: Haskell2010   type:             exitcode-stdio-1.0   hs-source-dirs:   benchmarks   main-is:          Base16Bench.hs   build-depends:-      base               >=4.14 && <4.19+      base               >=4.18 && <4.23     , base16     , base16-bytestring  >=1.0-    , bytestring         ^>=0.11+    , bytestring         >=0.11     , criterion     , deepseq     , random-bytestring-    , text               ^>=2.0+    , text               >=2.0++  if flag(simd)+    cpp-options:    -DBASE16_SIMD++  ghc-options:      -Wall -rtsopts

@@ -1,113 +1,51 @@+{-# LANGUAGE CPP #-} {-# LANGUAGE PackageImports #-}-module Main-( main-)where-+-- Copyright (c) 2020-2026 Emily Pillmore. BSD-3-Clause; see LICENSE.+module Main (main) where -import Criterion import Criterion.Main--import Data.Base16.Types-import Data.ByteString-import Data.ByteString.Short-import "base16" Data.ByteString.Base16 as B16-import "base16" Data.ByteString.Short.Base16 as BS16-import "base16-bytestring" Data.ByteString.Base16 as Bos+import Data.Base16.Types (assertBase16, extractBase16)+import qualified Data.ByteString.Short as SBS+import qualified "base16" Data.ByteString.Base16 as B16+import qualified "base16" Data.ByteString.Short.Base16 as S16+import qualified "base16-bytestring" Data.ByteString.Base16 as Reference import Data.ByteString.Random (random) - main :: IO ()-main =-  defaultMain-    [ env bs $ \ ~((bs25,bs100,bs1k,bs10k,bs100k,bs1mm),(bs25L,bs100L,bs1kL,bs10kL,bs100kL,bs1mmL)) ->-      bgroup "encode"-      [ bgroup "25"-        [ bench "base16-short" $ whnf BS16.encodeBase16' bs25L-        , bench "base16-bytestring" $ whnf Bos.encode bs25-        , bench "base16" $ whnf B16.encodeBase16' bs25-        ]-      , bgroup "100"-        [ bench "base16-short" $ whnf BS16.encodeBase16' bs100L-        , bench "base16-bytestring" $ whnf Bos.encode bs100-        , bench "base16" $ whnf B16.encodeBase16' bs100-        ]-      , bgroup "1k"-        [ bench "base16-short" $ whnf BS16.encodeBase16' bs1kL-        , bench "base16-bytestring" $ whnf Bos.encode bs1k-        , bench "base16" $ whnf B16.encodeBase16' bs1k-        ]-      , bgroup "10k"-        [ bench "base16-short" $ whnf BS16.encodeBase16' bs10kL-        , bench "base16-bytestring" $ whnf Bos.encode bs10k-        , bench "base16" $ whnf B16.encodeBase16' bs10k-        ]-      , bgroup "100k"-        [ bench "base16-short" $ whnf BS16.encodeBase16' bs100kL-        , bench "base16-bytestring" $ whnf Bos.encode bs100k-        , bench "base16" $ whnf B16.encodeBase16' bs100k-        ]-      , bgroup "1mm"-        [ bench "base16-short" $ whnf BS16.encodeBase16' bs1mmL-        , bench "base16-bytestring" $ whnf Bos.encode bs1mm-        , bench "base16" $ whnf B16.encodeBase16' bs1mm-        ]+main = defaultMain [benchSize n | n <- sizes]+  where+    sizes = [0, 1, 15, 16, 31, 32, 63, 64, 65, 127, 128, 129,+             1000, 10000, 100000, 1000000]++benchSize :: Int -> Benchmark+benchSize size = env inputs $ \ ~(raw, encoded, shortRaw, shortEncoded) ->+  bgroup (show size)+    [ bgroup "encode"+      [ bench variant $ nf (extractBase16 . B16.encodeBase16') raw+      , bench "base16-short" $ nf (extractBase16 . S16.encodeBase16') shortRaw+      , bench "base16-bytestring" $ nf Reference.encode raw       ]-    , env bs' $ \ ~((bs25,bs100,bs1k,bs10k,bs100k,bs1mm),(bs25L,bs100L,bs1kL,bs10kL,bs100kL,bs1mmL)) ->-      bgroup "decode"-      [ bgroup "25"-        [ bench "base16-short" $ whnf BS16.decodeBase16 bs25L-        , bench "base16-bytestring" $ whnf Bos.decode $ extractBase16 bs25-        , bench "base16" $ whnf B16.decodeBase16 bs25-        , bench "base16-untyped" $ whnf B16.decodeBase16Untyped $ extractBase16 bs25-        ]-      , bgroup "100"-        [ bench "base16-short" $ whnf BS16.decodeBase16 bs100L-        , bench "base16-bytestring" $ whnf Bos.decode $ extractBase16 bs100-        , bench "base16" $ whnf B16.decodeBase16 bs100-        , bench "base16-untyped" $ whnf B16.decodeBase16Untyped $ extractBase16 bs100-        ]-      , bgroup "1k"-        [ bench "base16-short" $ whnf BS16.decodeBase16 bs1kL-        , bench "base16-bytestring" $ whnf Bos.decode $ extractBase16 bs1k-        , bench "base16" $ whnf B16.decodeBase16 bs1k-        , bench "base16-untyped" $ whnf B16.decodeBase16Untyped $ extractBase16 bs1k-        ]-      , bgroup "10k"-        [ bench "base16-short" $ whnf BS16.decodeBase16 bs10kL-        , bench "base16-bytestring" $ whnf Bos.decode $ extractBase16 bs10k-        , bench "base16" $ whnf B16.decodeBase16 bs10k-        , bench "base16-untyped" $ whnf B16.decodeBase16Untyped $ extractBase16 bs10k-        ]-      , bgroup "100k"-        [ bench "base16-short" $ whnf BS16.decodeBase16 bs100kL-        , bench "base16-bytestring" $ whnf Bos.decode $ extractBase16 bs100k-        , bench "base16" $ whnf B16.decodeBase16 bs100k-        , bench "base16-untyped" $ whnf B16.decodeBase16Untyped $ extractBase16 bs100k-        ]-      , bgroup "1mm"-        [ bench "base16-short" $ whnf BS16.decodeBase16 bs1mmL-        , bench "base16-bytestring" $ whnf Bos.decode $ extractBase16 bs1mm-        , bench "base16" $ whnf B16.decodeBase16 bs1mm-        , bench "base16-untyped" $ whnf B16.decodeBase16Untyped $ extractBase16 bs1mm-        ]+    , bgroup "decode"+      [ bench (variant ++ "-typed") $ nf B16.decodeBase16 (assertBase16 encoded)+      , bench (variant ++ "-checked") $ nf B16.decodeBase16Untyped encoded+      , bench "base16-short-typed" $ nf S16.decodeBase16 (assertBase16 shortEncoded)+      , bench "base16-short-checked" $ nf S16.decodeBase16Untyped shortEncoded+      , bench "base16-bytestring" $ nf Reference.decode encoded       ]+    , bgroup "parse"+      [ bench variant $ nf B16.parseBase16 encoded+      , bench "base16-short" $ nf S16.parseBase16 shortEncoded+      ]     ]   where-    bs = do-      a <- random 25-      b <- random 100-      c <- random 1000-      d <- random 10000-      e <- random 100000-      f <- random 1000000-      return ((a,b,c,d,e,f),(toShort a,toShort b,toShort c,toShort d,toShort e,toShort f))+    inputs = do+      raw <- random (fromIntegral size)+      let encoded = Reference.encode raw+      pure (raw, encoded, SBS.toShort raw, SBS.toShort encoded) -    bs' = do-      let k = fmap toShort-      a <- B16.encodeBase16' <$> random 25-      b <- B16.encodeBase16' <$> random 100-      c <- B16.encodeBase16' <$> random 1000-      d <- B16.encodeBase16' <$> random 10000-      e <- B16.encodeBase16' <$> random 100000-      f <- B16.encodeBase16' <$> random 1000000-      return ((a,b,c,d,e,f),(k a,k b,k c,k d,k e,k f))+variant :: String+#ifdef BASE16_SIMD+variant = "base16-simd"+#else+variant = "base16-scalar"+#endif

@@ -0,0 +1,163 @@+/* Copyright (c) 2026, Emily Pillmore. BSD-3-Clause; see LICENSE. */++/* Native diagnostic benchmark; this does not measure Haskell/FFI overhead. */+#define _POSIX_C_SOURCE 200809L+#include <errno.h>+#include <math.h>+#include <stdio.h>+#include <stdlib.h>+#include <string.h>+#include <time.h>++#include "base16_simd.h"++static volatile size_t sink;+static const uint8_t alphabet[] = "0123456789abcdef";++static double now(void)+{+    struct timespec time;+    if (clock_gettime(CLOCK_MONOTONIC, &time) != 0) {+        perror("clock_gettime");+        exit(EXIT_FAILURE);+    }+    return (double)time.tv_sec + (double)time.tv_nsec / 1000000000.0;+}++static void encode(const uint8_t *src, size_t len, uint8_t *dst, int simd)+{+    size_t consumed = simd ? hs_base16_simd_encode(src, len, dst) : 0;+    for (size_t i = consumed; i < len; i++) {+        dst[i * 2] = alphabet[src[i] >> 4];+        dst[i * 2 + 1] = alphabet[src[i] & 15];+    }+}++static uint8_t nibble(uint8_t value)+{+    return value <= '9' ? (uint8_t)(value - '0') : (uint8_t)(value - 'a' + 10);+}++/* The benchmark only supplies independently generated valid lowercase input. */+static void decode(const uint8_t *src, size_t len, uint8_t *dst, int simd)+{+    size_t consumed = simd ? hs_base16_simd_decode(src, len, dst) : 0;+    for (size_t i = consumed; i < len; i += 2) {+        dst[i / 2] = (uint8_t)(nibble(src[i]) * 16 + nibble(src[i + 1]));+    }+}++typedef void (*operation)(const uint8_t *, size_t, uint8_t *, int);++static double measure(operation op, const uint8_t *src, size_t len,+                      uint8_t *dst, int simd, double seconds)+{+    const double start = now();+    double elapsed;+    size_t iterations = 0;++    do {+        for (size_t batch = 0; batch < 16; batch++) {+            op(src, len, dst, simd);+            sink += dst[0];+            iterations++;+        }+        elapsed = now() - start;+    } while (elapsed < seconds);+    return (double)iterations * (double)len / elapsed / 1048576.0;+}++static void usage(const char *name)+{+    fprintf(stderr, "usage: %s [--seconds 0.001..10] [--rounds 1..20] "+                    "[--pattern random|repeating]\n", name);+}++int main(int argc, char **argv)+{+    double seconds = 0.05;+    unsigned long rounds = 3;+    int random_pattern = 1;+    const size_t sizes[] = {64, 128, 1024, 16384, 1048576};+    const int backend = hs_base16_simd_backend();++    for (int i = 1; i < argc; i += 2) {+        char *end;+        if (i + 1 >= argc) {+            usage(argv[0]);+            return EXIT_FAILURE;+        }+        errno = 0;+        if (strcmp(argv[i], "--seconds") == 0) {+            seconds = strtod(argv[i + 1], &end);+            if (errno != 0 || end == argv[i + 1] || *end != '\0' ||+                !isfinite(seconds) || seconds < 0.001 || seconds > 10) {+                usage(argv[0]);+                return EXIT_FAILURE;+            }+        } else if (strcmp(argv[i], "--rounds") == 0) {+            rounds = strtoul(argv[i + 1], &end, 10);+            if (errno != 0 || end == argv[i + 1] || *end != '\0' ||+                rounds < 1 || rounds > 20) {+                usage(argv[0]);+                return EXIT_FAILURE;+            }+        } else if (strcmp(argv[i], "--pattern") == 0 &&+                   (strcmp(argv[i + 1], "random") == 0 ||+                    strcmp(argv[i + 1], "repeating") == 0)) {+            random_pattern = strcmp(argv[i + 1], "random") == 0;+        } else {+            usage(argv[0]);+            return EXIT_FAILURE;+        }+    }+    printf("SIMD backend: %d (%s); input throughput MiB/s; pattern=%s\n", backend,+           backend == 3 ? "AVX2" : backend == 2 ? "NEON" :+           backend == 1 ? "SSSE3" : "scalar fallback",+           random_pattern ? "random" : "repeating");+    puts("bytes operation scalar simd");+    for (size_t sample = 0; sample < sizeof(sizes) / sizeof(sizes[0]); sample++) {+        const size_t len = sizes[sample];+        uint8_t *src = malloc(len);+        uint8_t *encoded = malloc(len * 2);+        uint8_t *output = malloc(len * 2);+        uint32_t state = UINT32_C(0x6d253fb9);++        if (src == NULL || encoded == NULL || output == NULL) {+            fputs("allocation failed\n", stderr);+            free(src);+            free(encoded);+            free(output);+            return EXIT_FAILURE;+        }+        for (size_t i = 0; i < len; i++) {+            state ^= state << 13;+            state ^= state >> 17;+            state ^= state << 5;+            src[i] = random_pattern ? (uint8_t)state : 0xfb;+        }+        encode(src, len, encoded, 0);+        encode(src, len, output, 1);+        if (memcmp(encoded, output, len * 2) != 0) {+            fputs("SIMD encoding differs from scalar reference\n", stderr);+            return EXIT_FAILURE;+        }+        decode(encoded, len * 2, output, 1);+        if (memcmp(src, output, len) != 0) {+            fputs("SIMD decoding differs from scalar reference\n", stderr);+            return EXIT_FAILURE;+        }+        for (unsigned long round = 0; round < rounds; round++) {+            const double scalar_encode = measure(encode, src, len, output, 0, seconds);+            const double simd_encode = measure(encode, src, len, output, 1, seconds);+            const double scalar_decode = measure(decode, encoded, len * 2, output, 0, seconds);+            const double simd_decode = measure(decode, encoded, len * 2, output, 1, seconds);+            printf("%zu encode %.1f %.1f\n", len, scalar_encode, simd_encode);+            printf("%zu decode %.1f %.1f\n", len * 2, scalar_decode, simd_decode);+        }+        free(output);+        free(encoded);+        free(src);+    }+    return EXIT_SUCCESS;+}

@@ -0,0 +1,274 @@+/* Copyright (c) 2026, Emily Pillmore. BSD-3-Clause; see LICENSE. */++#include "base16_simd.h"++/*+ * Original bounded prefix kernels. Target attributes keep x86 instructions+ * out of baseline dispatch, with no global ISA flags or mutable state.+ * HS_BASE16_SIMD_FORCE_SCALAR exercises the unsupported-target fallback.+ * HS_BASE16_SIMD_FORCE_SSSE3 caps x86 dispatch for older-processor testing.+ */+#if !defined(HS_BASE16_SIMD_FORCE_SCALAR)+#if defined(__aarch64__) && defined(__ARM_NEON) && \+    (defined(__GNUC__) || defined(__clang__))+#define HS_BASE16_NEON 1+#include <arm_neon.h>+#elif defined(__x86_64__) && (defined(__GNUC__) || defined(__clang__))+#define HS_BASE16_SSSE3 1+#include <immintrin.h>+#define HS_BASE16_TARGET_SSSE3 __attribute__((target("ssse3")))+#if !defined(HS_BASE16_SIMD_FORCE_SSSE3)+#define HS_BASE16_AVX2 1+#define HS_BASE16_TARGET_AVX2 __attribute__((target("avx2")))+#endif+#endif+#endif++#if defined(HS_BASE16_NEON) || defined(HS_BASE16_SSSE3)+static const uint8_t hs_base16_alphabet[] = "0123456789abcdef";+#endif++#if defined(HS_BASE16_NEON)++static size_t hs_base16_neon_encode(const uint8_t *src, size_t len, uint8_t *dst)+{+    size_t consumed = 0;+    const uint8x16_t alphabet = vld1q_u8(hs_base16_alphabet);+    const uint8x16_t mask = vdupq_n_u8(15);++    while (len - consumed >= 16) {+        const uint8x16_t input = vld1q_u8(src + consumed);+        uint8x16x2_t output;++        output.val[0] = vqtbl1q_u8(alphabet, vshrq_n_u8(input, 4));+        output.val[1] = vqtbl1q_u8(alphabet, vandq_u8(input, mask));+        vst2q_u8(dst, output);+        consumed += 16;+        dst += 32;+    }+    return consumed;+}++static inline uint8x16_t hs_base16_neon_nibbles(uint8x16_t input,+                                               uint8x16_t *valid)+{+    const uint8x16_t digit = vsubq_u8(input, vdupq_n_u8('0'));+    const uint8x16_t letter = vsubq_u8(vorrq_u8(input, vdupq_n_u8(0x20)),+                                      vdupq_n_u8('a'));+    const uint8x16_t is_digit = vcleq_u8(digit, vdupq_n_u8(9));+    const uint8x16_t is_letter = vcleq_u8(letter, vdupq_n_u8(5));++    *valid = vorrq_u8(is_digit, is_letter);+    return vbslq_u8(is_letter, vaddq_u8(letter, vdupq_n_u8(10)), digit);+}++static size_t hs_base16_neon_decode(const uint8_t *src, size_t len, uint8_t *dst)+{+    size_t consumed = 0;++    while (len - consumed >= 32) {+        const uint8x16x2_t input = vld2q_u8(src + consumed);+        uint8x16_t valid_hi, valid_lo;+        const uint8x16_t hi = hs_base16_neon_nibbles(input.val[0], &valid_hi);+        const uint8x16_t lo = hs_base16_neon_nibbles(input.val[1], &valid_lo);++        /* Validate the whole block before committing even its first byte. */+        if (vminvq_u8(vandq_u8(valid_hi, valid_lo)) == 0) {+            break;+        }+        vst1q_u8(dst, vorrq_u8(vshlq_n_u8(hi, 4), lo));+        consumed += 32;+        dst += 16;+    }+    return consumed;+}++#elif defined(HS_BASE16_SSSE3)++HS_BASE16_TARGET_SSSE3+static size_t hs_base16_ssse3_encode(const uint8_t *src, size_t len, uint8_t *dst)+{+    size_t consumed = 0;+    const __m128i alphabet = _mm_loadu_si128((const __m128i *)hs_base16_alphabet);+    const __m128i mask = _mm_set1_epi8(15);++    while (len - consumed >= 16) {+        const __m128i input = _mm_loadu_si128((const __m128i *)(src + consumed));+        const __m128i hi = _mm_shuffle_epi8(alphabet,+            _mm_and_si128(_mm_srli_epi16(input, 4), mask));+        const __m128i lo = _mm_shuffle_epi8(alphabet, _mm_and_si128(input, mask));++        _mm_storeu_si128((__m128i *)dst, _mm_unpacklo_epi8(hi, lo));+        _mm_storeu_si128((__m128i *)(dst + 16), _mm_unpackhi_epi8(hi, lo));+        consumed += 16;+        dst += 32;+    }+    return consumed;+}++HS_BASE16_TARGET_SSSE3+static inline __m128i hs_base16_ssse3_nibbles(__m128i input, __m128i *valid)+{+    const __m128i lower = _mm_or_si128(input, _mm_set1_epi8(0x20));+    const __m128i is_digit = _mm_and_si128(+        _mm_cmpgt_epi8(input, _mm_set1_epi8('0' - 1)),+        _mm_cmpgt_epi8(_mm_set1_epi8('9' + 1), input));+    const __m128i is_letter = _mm_and_si128(+        _mm_cmpgt_epi8(lower, _mm_set1_epi8('a' - 1)),+        _mm_cmpgt_epi8(_mm_set1_epi8('f' + 1), lower));++    /* Bytes >= 128 fail the signed lower-bound comparisons in both ranges. */+    *valid = _mm_or_si128(is_digit, is_letter);+    return _mm_add_epi8(_mm_and_si128(input, _mm_set1_epi8(15)),+                        _mm_and_si128(is_letter, _mm_set1_epi8(9)));+}++HS_BASE16_TARGET_SSSE3+static size_t hs_base16_ssse3_decode(const uint8_t *src, size_t len, uint8_t *dst)+{+    size_t consumed = 0;++    while (len - consumed >= 16) {+        const __m128i input = _mm_loadu_si128((const __m128i *)(src + consumed));+        __m128i valid;+        const __m128i value = hs_base16_ssse3_nibbles(input, &valid);+        __m128i pairs;+        __m128i output;++        if (_mm_movemask_epi8(valid) != 0xffff) {+            break;+        }+        /* Adjacent nibbles form unsigned 0..255 words, without saturation. */+        pairs = _mm_maddubs_epi16(value, _mm_set1_epi16(0x0110));+        output = _mm_packus_epi16(pairs, _mm_setzero_si128());+        _mm_storel_epi64((__m128i *)dst, output);+        consumed += 16;+        dst += 8;+    }+    return consumed;+}++#if defined(HS_BASE16_AVX2)++HS_BASE16_TARGET_AVX2+static size_t hs_base16_avx2_encode(const uint8_t *src, size_t len, uint8_t *dst)+{+    size_t consumed = 0;+    const __m256i alphabet = _mm256_broadcastsi128_si256(+        _mm_loadu_si128((const __m128i *)hs_base16_alphabet));+    const __m256i mask = _mm256_set1_epi8(15);++    while (len - consumed >= 32) {+        const __m256i input = _mm256_loadu_si256((const __m256i *)(src + consumed));+        const __m256i hi = _mm256_shuffle_epi8(alphabet,+            _mm256_and_si256(_mm256_srli_epi16(input, 4), mask));+        const __m256i lo = _mm256_shuffle_epi8(alphabet, _mm256_and_si256(input, mask));+        const __m256i first = _mm256_unpacklo_epi8(hi, lo);+        const __m256i second = _mm256_unpackhi_epi8(hi, lo);++        /* Unpacks operate within 128-bit lanes; restore original byte order. */+        _mm256_storeu_si256((__m256i *)dst,+            _mm256_permute2x128_si256(first, second, 0x20));+        _mm256_storeu_si256((__m256i *)(dst + 32),+            _mm256_permute2x128_si256(first, second, 0x31));+        consumed += 32;+        dst += 64;+    }+    return consumed;+}++HS_BASE16_TARGET_AVX2+static size_t hs_base16_avx2_decode(const uint8_t *src, size_t len, uint8_t *dst)+{+    size_t consumed = 0;++    while (len - consumed >= 32) {+        const __m256i input = _mm256_loadu_si256((const __m256i *)(src + consumed));+        const __m256i lower = _mm256_or_si256(input, _mm256_set1_epi8(0x20));+        const __m256i is_digit = _mm256_and_si256(+            _mm256_cmpgt_epi8(input, _mm256_set1_epi8('0' - 1)),+            _mm256_cmpgt_epi8(_mm256_set1_epi8('9' + 1), input));+        const __m256i is_letter = _mm256_and_si256(+            _mm256_cmpgt_epi8(lower, _mm256_set1_epi8('a' - 1)),+            _mm256_cmpgt_epi8(_mm256_set1_epi8('f' + 1), lower));+        __m256i value;+        __m256i pairs;+        __m128i output;++        if (_mm256_movemask_epi8(_mm256_or_si256(is_digit, is_letter)) != -1) {+            break;+        }+        value = _mm256_add_epi8(_mm256_and_si256(input, _mm256_set1_epi8(15)),+            _mm256_and_si256(is_letter, _mm256_set1_epi8(9)));+        pairs = _mm256_maddubs_epi16(value, _mm256_set1_epi16(0x0110));+        /* Pack both lanes into exactly 16 output bytes, without lane gaps. */+        output = _mm_packus_epi16(_mm256_castsi256_si128(pairs),+                                  _mm256_extracti128_si256(pairs, 1));+        _mm_storeu_si128((__m128i *)dst, output);+        consumed += 32;+        dst += 16;+    }+    return consumed;+}++#endif /* HS_BASE16_AVX2 */+#endif++int hs_base16_simd_backend(void)+{+#if defined(HS_BASE16_NEON)+    return 2;+#elif defined(HS_BASE16_SSSE3)+#if defined(HS_BASE16_AVX2)+    /* The compiler's query also checks OS support for AVX register state. */+    if (__builtin_cpu_supports("avx2")) {+        return 3;+    }+#endif+    return __builtin_cpu_supports("ssse3") ? 1 : 0;+#else+    return 0;+#endif+}++size_t hs_base16_simd_encode(const uint8_t *src, size_t len, uint8_t *dst)+{+#if defined(HS_BASE16_NEON)+    return hs_base16_neon_encode(src, len, dst);+#elif defined(HS_BASE16_SSSE3)+    const int backend = hs_base16_simd_backend();+#if defined(HS_BASE16_AVX2)+    if (backend == 3) {+        return hs_base16_avx2_encode(src, len, dst);+    }+#endif+    if (backend == 1) {+        return hs_base16_ssse3_encode(src, len, dst);+    }+#endif+    (void)src;+    (void)len;+    (void)dst;+    return 0;+}++size_t hs_base16_simd_decode(const uint8_t *src, size_t len, uint8_t *dst)+{+#if defined(HS_BASE16_NEON)+    return hs_base16_neon_decode(src, len, dst);+#elif defined(HS_BASE16_SSSE3)+    const int backend = hs_base16_simd_backend();+#if defined(HS_BASE16_AVX2)+    if (backend == 3) {+        return hs_base16_avx2_decode(src, len, dst);+    }+#endif+    if (backend == 1) {+        return hs_base16_ssse3_decode(src, len, dst);+    }+#endif+    (void)src;+    (void)len;+    (void)dst;+    return 0;+}

@@ -0,0 +1,39 @@+/* Copyright (c) 2026, Emily Pillmore. BSD-3-Clause; see LICENSE. */++#ifndef HS_BASE16_SIMD_H+#define HS_BASE16_SIMD_H++#include <stddef.h>+#include <stdint.h>++#ifdef __cplusplus+extern "C" {+#endif++/*+ * Process complete vector blocks at the start of a buffer. The caller handles+ * the remaining input and error reporting. Source and destination must not+ * overlap; source has len readable bytes and destination has room for the+ * produced prefix. Loads never extend past len, and stores cover only the+ * returned prefix. Buffers must remain alive throughout the call.+ *+ * Encode emits lowercase ASCII and writes exactly 2 * result bytes.+ * Decode accepts ASCII 0-9, a-f and A-F; it returns an even count and writes+ * exactly result / 2 bytes. An entire vector block containing an invalid byte+ * is left untouched, including its destination, so the caller can report the+ * first invalid byte. The final incomplete block is also left to the caller.+ *+ * Unsupported targets return zero without reading or writing any buffers.+ * The caller must reject output-size overflow before allocating buffers.+ */+size_t hs_base16_simd_encode(const uint8_t *src, size_t len, uint8_t *dst);+size_t hs_base16_simd_decode(const uint8_t *src, size_t len, uint8_t *dst);++/* 0: scalar fallback; 1: x86-64 SSSE3; 2: ARM64 NEON; 3: x86-64 AVX2. */+int hs_base16_simd_backend(void);++#ifdef __cplusplus+}+#endif++#endif /* HS_BASE16_SIMD_H */

@@ -1,7 +1,7 @@ {-# language ExplicitNamespaces #-} -- |--- Module       : Data.ByteString.Base16.Types--- Copyright    : (c) 2019-2022 Emily Pillmore+-- Module       : Data.Base16.Types+-- Copyright    : (c) 2019-2026 Emily Pillmore -- License      : BSD-style -- -- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>,@@ -9,9 +9,8 @@ -- Stability    : stable -- Portability  : non-portable ----- This module contains the 'Base16' type definition, 'Alphabet'--- datatype, alphabet constraints, and various quality of life--- combinators for working with 'Base16'-wrapped data.+-- This module contains the opaque 'Base16' type and combinators for+-- working with Base16-encoded values. -- module Data.Base16.Types ( type Base16@@ -20,9 +19,51 @@ ) where  -import Data.Base16.Types.Internal (Base16(..))+import Control.DeepSeq (NFData(..))+import Data.Functor.Classes (Eq1(..), Ord1(..)) +-- | Wrap a value that is or is intended to be Base16-encoded.+--+-- Use the @parseBase16@ function in the corresponding ByteString or Text+-- module to validate external input. Encoding functions also produce this+-- wrapper. 'assertBase16' and the 'Functor', 'Applicative', and 'Monad'+-- instances do not validate their results.+--+newtype Base16 a = Base16 a++instance Eq a => Eq (Base16 a) where+  Base16 a == Base16 b = a == b++instance Eq1 Base16 where+  liftEq f (Base16 a) (Base16 b) = f a b++instance Ord a => Ord (Base16 a) where+  compare (Base16 a) (Base16 b) = compare a b++instance Ord1 Base16 where+  liftCompare f (Base16 a) (Base16 b) = f a b++instance Functor Base16 where+  fmap f (Base16 a) = Base16 (f a)++instance Applicative Base16 where+  pure = Base16+  Base16 f <*> Base16 a = Base16 (f a)++instance Monad Base16 where+  return = pure+  Base16 a >>= k = k a++instance Show a => Show (Base16 a) where+  show (Base16 a) = show a++instance NFData a => NFData (Base16 a) where+  rnf (Base16 a) = rnf a+ -- | Assert the provenance of a value.+--+-- For untrusted input, use the @parseBase16@ function in the corresponding+-- ByteString or Text module to validate it first. -- -- /Warning/: This is a blind assertion that a particular -- value is base16 encoded in some alphabet. If you are not

@@ -1,56 +0,0 @@--- |--- Module       : Data.ByteString.Base16.Types.Internal--- Copyright    : (c) 2019-2022 Emily Pillmore--- License      : BSD-style------ Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>,---                sofia-m-a <https://github.com/sofia-m-a>--- Stability    : stable--- Portability  : non-portable------ This module contains the 'Base16' newtype.----module Data.Base16.Types.Internal-( Base16(..)-) where---import Control.DeepSeq--import Data.Functor.Classes---- | Wraps a value, asserting that it is or is intended to be--- in a particular kind of Base16 encoding use 'extractBase16'--- to extract the value, and 'assertBase16' to tag a value--- as base16-encoded----newtype Base16 a = Base16 a--instance Eq a => Eq (Base16 a) where-  Base16 a == Base16 b = a == b--instance Eq1 Base16 where-  liftEq f (Base16 a) (Base16 b) = f a b--instance Ord a => Ord (Base16 a) where-  compare (Base16 a) (Base16 b) = compare a b--instance Ord1 Base16 where-  liftCompare f (Base16 a) (Base16 b) = f a b--instance Functor Base16 where-  fmap f (Base16 a) = Base16 (f a)--instance Applicative Base16 where-  pure = Base16-  Base16 f <*> Base16 a = Base16 (f a)--instance Monad Base16 where-  return = pure-  Base16 a >>= k = k a--instance Show a => Show (Base16 a) where-  show (Base16 a) = show a--instance NFData a => NFData (Base16 a) where-  rnf (Base16 a) = rnf a

@@ -1,7 +1,6 @@-{-# LANGUAGE OverloadedStrings #-} -- | -- Module       : Data.ByteString.Base16--- Copyright    : (c) 2020-2023 Emily Pillmore+-- Copyright    : (c) 2020-2026 Emily Pillmore -- License      : BSD-style -- -- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>@@ -20,17 +19,13 @@ , decodeBase16' , decodeBase16Untyped , decodeBase16Lenient-, isBase16-, isValidBase16+, parseBase16 ) where  -import Prelude hiding (all, elem)- import Data.Base16.Types-import Data.ByteString (ByteString, all, elem)+import Data.ByteString (ByteString) import Data.ByteString.Base16.Internal.Head-import Data.Either import Data.Text (Text) import qualified Data.Text.Encoding as T @@ -125,34 +120,28 @@ decodeBase16Lenient = decodeBase16Lenient_ {-# INLINE decodeBase16Lenient #-} --- | Tell whether an untyped 'ByteString' value is base16 encoded.------ === __Examples__:+-- | Validate an untyped Base16 value and retain its encoded representation. ----- >>> isBase16 "666f6"--- False+-- Accepts even-length ASCII hexadecimal input, including upper-case and+-- mixed-case digits. Empty input is valid. The original input is preserved,+-- and errors match 'decodeBase16Untyped'. ----- >>> isBase16 "666f"--- True+-- === __Examples__: ---isBase16 :: ByteString -> Bool-isBase16 bs = isValidBase16 bs && isRight (decodeBase16Untyped bs)-{-# INLINE isBase16 #-}---- | Tell whether an untyped 'ByteString' value is a valid Base16 format.+-- >>> parseBase16 "666f6"+-- Left "invalid bytestring size" ----- This will not tell you whether or not this is a correct Base16 representation,--- only that it conforms to the correct alphabet. To check whether it is a true--- Base16 encoded 'ByteString' value, use 'isBase16'.+-- >>> parseBase16 "666F"+-- Right "666F" ----- === __Examples__:+-- >>> fmap decodeBase16 (parseBase16 "666F")+-- Right "fo" ----- >>> isValidBase16 "666f+/6"--- False+-- >>> parseBase16 "6x"+-- Left "invalid character at offset: 1" ----- >>> isValidBase16 "666f6"--- True+-- @since 1.0 ---isValidBase16 :: ByteString -> Bool-isValidBase16 = all (`elem` "0123456789abcdefABCDEF")-{-# INLINE isValidBase16 #-}+parseBase16 :: ByteString -> Either Text (Base16 ByteString)+parseBase16 bs = assertBase16 bs <$ decodeBase16Untyped bs+{-# INLINE parseBase16 #-}

@@ -13,7 +13,7 @@ ) where  -import Data.Base16.Types.Internal (Base16(..))+import Data.Base16.Types (Base16, extractBase16) import qualified Data.ByteString as BS (empty) import Data.ByteString.Internal import qualified Data.ByteString.Short as SBS (empty)@@ -36,19 +36,19 @@ -- | Head of the base16 encoding loop - marshal data, assemble loops -- encodeBase16_ :: ByteString -> ByteString-encodeBase16_ (PS sfp soff slen) =+encodeBase16_ (BS sfp slen) =     unsafeCreate dlen $ \dptr ->       withForeignPtr sfp $ \sptr -> do         innerLoop           dptr-          (plusPtr sptr soff)-          (plusPtr sptr $ slen + soff)+          sptr+          (plusPtr sptr slen)   where-    !dlen = 2 * slen+    !dlen = encodedLength slen {-# INLINE encodeBase16_ #-}  decodeBase16_ :: ByteString -> Either Text ByteString-decodeBase16_ (PS sfp soff slen)+decodeBase16_ (BS sfp slen)   | slen == 0 = Right BS.empty   | r /= 0 = Left "invalid bytestring size"   | otherwise = unsafeDupablePerformIO $ do@@ -58,8 +58,8 @@         decodeLoop           dfp           dptr-          (plusPtr sptr soff)-          (plusPtr sptr $ slen + soff)+          sptr+          (plusPtr sptr slen)           q   where     !q = slen `quot` 2@@ -67,19 +67,20 @@ {-# INLINE decodeBase16_ #-}  decodeBase16Typed_ :: Base16 ByteString -> ByteString-decodeBase16Typed_ (Base16 (PS sfp soff slen)) =+decodeBase16Typed_ encoded =   unsafeCreate q $ \dptr ->     withForeignPtr sfp $ \sptr ->       decodeLoopTyped         dptr-        (plusPtr sptr soff)-        (plusPtr sptr $ slen + soff)+        sptr+        (plusPtr sptr slen)   where+    BS sfp slen = extractBase16 encoded     !q = slen `quot` 2 {-# INLINE decodeBase16Typed_ #-}  decodeBase16Lenient_ :: ByteString -> ByteString-decodeBase16Lenient_ (PS sfp soff slen) =+decodeBase16Lenient_ (BS sfp slen) =   unsafeDupablePerformIO $ do     dfp <- mallocPlainForeignPtrBytes q     withForeignPtr dfp $ \dptr ->@@ -87,8 +88,8 @@         lenientLoop           dfp           dptr-          (plusPtr sptr soff)-          (plusPtr sptr $ slen + soff)+          sptr+          (plusPtr sptr slen)           0   where     !q = slen `quot` 2@@ -100,11 +101,11 @@ encodeBase16Short_ :: ShortByteString -> ShortByteString encodeBase16Short_ (SBS !ba#) = runShortST $ do     dst <- newByteArray l'-    Short.innerLoop l dst (MutableByteArray (unsafeCoerce# ba#))+    Short.innerLoop l dst (ByteArray ba#)     unsafeFreezeByteArray dst   where     !l = I# (sizeofByteArray# ba#)-    !l' = l * 2+    !l' = encodedLength l {-# INLINE encodeBase16Short_ #-}  decodeBase16Short_ :: ShortByteString -> Either Text ShortByteString@@ -113,7 +114,7 @@     | r /= 0 = Left "invalid bytestring size"     | otherwise = runDecodeST $ do       dst <- newByteArray q-      Short.decodeLoop l dst (MutableByteArray (unsafeCoerce# ba#))+      Short.decodeLoop l dst (ByteArray ba#)   where     !l = I# (sizeofByteArray# ba#)     !q = l `quot` 2@@ -121,10 +122,11 @@ {-# INLINE decodeBase16Short_ #-}  decodeBase16ShortTyped_ :: Base16 ShortByteString -> ShortByteString-decodeBase16ShortTyped_ (Base16 (SBS !ba#)) = runDecodeST' $ do+decodeBase16ShortTyped_ encoded = runDecodeST' $ do     dst <- newByteArray q-    Short.decodeLoopTyped l dst (MutableByteArray (unsafeCoerce# ba#))+    Short.decodeLoopTyped l dst (ByteArray ba#)   where+    !(SBS ba#) = extractBase16 encoded     !l = I# (sizeofByteArray# ba#)     !q = l `quot` 2 {-# INLINE decodeBase16ShortTyped_ #-}@@ -135,10 +137,17 @@     | l == 0 = SBS.empty     | otherwise = runShortST $ do       dst <- newByteArray q-      q' <- Short.lenientLoop l dst (MutableByteArray (unsafeCoerce# ba#))+      q' <- Short.lenientLoop l dst (ByteArray ba#)       !_ <- resizeMutableByteArray dst q'       unsafeFreezeByteArray dst   where     !l = I# (sizeofByteArray# ba#)     !q = l `quot` 2 {-# INLINE decodeBase16ShortLenient_ #-}++-- Check before allocating or passing the output buffer to native code.+encodedLength :: Int -> Int+encodedLength n+    | n > maxBound `quot` 2 = error "base16: encoded output too large"+    | otherwise = 2 * n+{-# INLINE encodedLength #-}

@@ -0,0 +1,80 @@+{-# LANGUAGE ForeignFunctionInterface #-}+{-# LANGUAGE MagicHash #-}+{-# LANGUAGE UnliftedFFITypes #-}+-- |+-- Module       : Data.ByteString.Base16.Internal.SIMD+-- Copyright    : (c) 2026 Emily Pillmore+-- License      : BSD-style+--+-- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>+-- Stability    : Experimental+-- Portability  : portable+--+-- Bounded vector prefixes for foreign pointers and byte arrays. Pointer+-- callers retain both buffers with @withForeignPtr@. Byte-array callers pass+-- the arrays directly to unsafe FFI calls, during which GHC cannot run GC;+-- neither pinning nor copying is needed. The kernels retain no pointers and+-- make no Haskell callbacks. Invalid decode blocks are left untouched so+-- scalar tails preserve exact error offsets.+module Data.ByteString.Base16.Internal.SIMD+( encodePrefix+, decodePrefix+, encodeByteArrayPrefix+, decodeByteArrayPrefix+) where++import Control.Monad.ST (ST)+import Control.Monad.ST.Unsafe (unsafeIOToST)+import Data.Primitive.ByteArray (ByteArray(..), MutableByteArray(..))+import Data.Word (Word8)+import Foreign.C.Types (CSize(..))+import Foreign.Ptr (Ptr)+import GHC.Exts (ByteArray#, MutableByteArray#)++foreign import ccall unsafe "hs_base16_simd_encode"+  c_encodePrefix :: Ptr Word8 -> CSize -> Ptr Word8 -> IO CSize++foreign import ccall unsafe "hs_base16_simd_decode"+  c_decodePrefix :: Ptr Word8 -> CSize -> Ptr Word8 -> IO CSize++foreign import ccall unsafe "hs_base16_simd_encode"+  c_encodeByteArrayPrefix :: ByteArray# -> CSize -> MutableByteArray# s -> IO CSize++foreign import ccall unsafe "hs_base16_simd_decode"+  c_decodeByteArrayPrefix :: ByteArray# -> CSize -> MutableByteArray# s -> IO CSize++-- | Encode complete blocks; return consumed input bytes and write twice as+-- many bytes. The caller must allocate a sufficiently large output buffer.+encodePrefix :: Ptr Word8 -> Int -> Ptr Word8 -> IO Int+encodePrefix src len dst+  | len < 64 = pure 0+  | otherwise = fromIntegral <$> c_encodePrefix src (fromIntegral len) dst+{-# INLINE encodePrefix #-}++-- | Decode complete valid blocks, returning an even input byte count and+-- writing half as many bytes. Small inputs avoid dispatch/foreign-call cost.+decodePrefix :: Ptr Word8 -> Int -> Ptr Word8 -> IO Int+decodePrefix src len dst+  | len < 128 = pure 0+  | otherwise = fromIntegral <$> c_decodePrefix src (fromIntegral len) dst+{-# INLINE decodePrefix #-}++-- | Encode a prefix of an immutable byte array into a distinct mutable+-- destination with room for twice the input length. Return consumed bytes.+-- The foreign call only reads the source and writes the destination, so it+-- can be embedded in ST without letting mutable state escape.+encodeByteArrayPrefix :: ByteArray -> Int -> MutableByteArray s -> ST s Int+encodeByteArrayPrefix (ByteArray src) len (MutableByteArray dst)+  | len < 64 = pure 0+  | otherwise = unsafeIOToST $+      fromIntegral <$> c_encodeByteArrayPrefix src (fromIntegral len) dst+{-# INLINE encodeByteArrayPrefix #-}++-- | Decode complete valid blocks into a distinct mutable destination with+-- room for half the input length. Return an even count of consumed bytes.+decodeByteArrayPrefix :: ByteArray -> Int -> MutableByteArray s -> ST s Int+decodeByteArrayPrefix (ByteArray src) len (MutableByteArray dst)+  | len < 128 = pure 0+  | otherwise = unsafeIOToST $+      fromIntegral <$> c_decodeByteArrayPrefix src (fromIntegral len) dst+{-# INLINE decodeByteArrayPrefix #-}

@@ -6,7 +6,7 @@ {-# LANGUAGE TypeApplications #-} -- | -- Module       : Data.ByteString.Base16.Internal.W16.Loop--- Copyright 	: (c) 2020 Emily Pillmore+-- Copyright 	: (c) 2020-2026 Emily Pillmore -- License	: BSD-style -- -- Maintainer	: Emily Pillmore <emilypi@cohomolo.gy>@@ -26,6 +26,9 @@ import Data.Bits import Data.ByteString.Internal import Data.ByteString.Base16.Internal.Utils+#ifdef BASE16_SIMD+import qualified Data.ByteString.Base16.Internal.SIMD as SIMD+#endif import Data.Text (Text) import qualified Data.Text as T @@ -43,7 +46,13 @@     -> Ptr Word8     -> Ptr Word8     -> IO ()-innerLoop !dptr !sptr !end = go dptr sptr+innerLoop !dptr !sptr !end = do+#ifdef BASE16_SIMD+    consumed <- SIMD.encodePrefix sptr (end `minusPtr` sptr) dptr+    go (plusPtr dptr (2 * consumed)) (plusPtr sptr consumed)+#else+    go dptr sptr+#endif   where     !hex = "0123456789abcdef"# @@ -66,7 +75,13 @@   -> Ptr Word8   -> Int   -> IO (Either Text ByteString)-decodeLoop !dfp !dptr !sptr !end !l = go dptr sptr+decodeLoop !dfp !dptr !sptr !end !l = do+#ifdef BASE16_SIMD+    consumed <- SIMD.decodePrefix sptr (end `minusPtr` sptr) dptr+    go (plusPtr dptr (consumed `quot` 2)) (plusPtr sptr consumed)+#else+    go dptr sptr+#endif   where     err !src = return . Left . T.pack       $ "invalid character at offset: "@@ -77,7 +92,7 @@     !hi = "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x10\x20\x30\x40\x50\x60\x70\x80\x90\xff\xff\xff\xff\xff\xff\xff\xa0\xb0\xc0\xd0\xe0\xf0\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xa0\xb0\xc0\xd0\xe0\xf0\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"#      go !dst !src-      | src == end = return $ Right (PS dfp 0 l)+      | src == end = return $ Right (BS dfp l)       | otherwise = do         !x <- peek @Word8 src         !y <- peek @Word8 (plusPtr src 1)@@ -99,7 +114,13 @@   -> Ptr Word8   -> Ptr Word8   -> IO ()-decodeLoopTyped !dptr !sptr !end = go dptr sptr+decodeLoopTyped !dptr !sptr !end = do+#ifdef BASE16_SIMD+    consumed <- SIMD.decodePrefix sptr (end `minusPtr` sptr) dptr+    go (plusPtr dptr (consumed `quot` 2)) (plusPtr sptr consumed)+#else+    go dptr sptr+#endif   where     !lo = "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\xff\xff\xff\xff\xff\xff\xff\x0a\x0b\x0c\x0d\x0e\x0f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x0a\x0b\x0c\x0d\x0e\x0f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"# @@ -136,7 +157,7 @@     !hi = "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x10\x20\x30\x40\x50\x60\x70\x80\x90\xff\xff\xff\xff\xff\xff\xff\xa0\xb0\xc0\xd0\xe0\xf0\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xa0\xb0\xc0\xd0\xe0\xf0\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"#      goHi !dst !src !n-      | src == end = return (PS dfp 0 n)+      | src == end = return (BS dfp n)       | otherwise = do         !x <- peek @Word8 src @@ -147,7 +168,7 @@         else goLo dst (plusPtr src 1) a n      goLo !dst !src !a !n-      | src == end = return (PS dfp 0 n)+      | src == end = return (BS dfp n)       | otherwise = do         !y <- peek @Word8 src @@ -158,4 +179,4 @@         else do           poke dst (a .|. b)           goHi (plusPtr dst 1) (plusPtr src 1) (n + 1)-{-# LANGUAGE lenientLoop #-}+{-# INLINE lenientLoop #-}

@@ -1,4 +1,5 @@ {-# LANGUAGE BangPatterns #-}+{-# LANGUAGE CPP #-} {-# LANGUAGE MagicHash #-} {-# LANGUAGE MultiWayIf #-} {-# LANGUAGE OverloadedStrings #-}@@ -18,20 +19,30 @@ import Data.Text (Text) import qualified Data.Text as T +#ifdef BASE16_SIMD+import qualified Data.ByteString.Base16.Internal.SIMD as SIMD+#endif  innerLoop     :: Int     -> MutableByteArray s-    -> MutableByteArray s+    -> ByteArray     -> ST s ()-innerLoop !slen !dst !src = go (0 :: Int) (0 :: Int)+innerLoop !slen !dst !src =+#ifdef BASE16_SIMD+  do+    consumed <- SIMD.encodeByteArrayPrefix src slen dst+    go (2 * consumed) consumed+#else+  go 0 0+#endif   where     !hex = "0123456789abcdef"#      go !doff !soff       | soff == slen = return ()       | otherwise = do-        x <- readByteArray src soff+        let !x = indexByteArray src soff         writeByteArray dst doff (aix (unsafeShiftR x 4) hex)         writeByteArray dst (doff + 1) (aix (x .&. 0x0f) hex)         go (doff + 2) (soff + 1)@@ -40,9 +51,16 @@ decodeLoop     :: Int     -> MutableByteArray s-    -> MutableByteArray s+    -> ByteArray     -> ST s (Either Text ByteArray)-decodeLoop !slen !dst !src = go (0 :: Int) (0 :: Int)+decodeLoop !slen !dst !src =+#ifdef BASE16_SIMD+  do+    consumed <- SIMD.decodeByteArrayPrefix src slen dst+    go (consumed `quot` 2) consumed+#else+  go 0 0+#endif   where     err i = return . Left . T.pack       $ "invalid character at offset: "@@ -55,8 +73,8 @@     go !doff !soff       | soff == slen = Right <$> unsafeFreezeByteArray dst       | otherwise = do-        x <- readByteArray src soff-        y <- readByteArray src (soff + 1)+        let !x = indexByteArray src soff+            !y = indexByteArray src (soff + 1)          let !a = aix x hi             !b = aix y lo@@ -72,9 +90,16 @@ decodeLoopTyped     :: Int     -> MutableByteArray s-    -> MutableByteArray s+    -> ByteArray     -> ST s ByteArray-decodeLoopTyped !slen !dst !src = go (0 :: Int) (0 :: Int)+decodeLoopTyped !slen !dst !src =+#ifdef BASE16_SIMD+  do+    consumed <- SIMD.decodeByteArrayPrefix src slen dst+    go (consumed `quot` 2) consumed+#else+  go 0 0+#endif   where     !lo = "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\xff\xff\xff\xff\xff\xff\xff\x0a\x0b\x0c\x0d\x0e\x0f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x0a\x0b\x0c\x0d\x0e\x0f\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff"# @@ -83,8 +108,8 @@     go !doff !soff       | soff == slen = unsafeFreezeByteArray dst       | otherwise = do-        x <- readByteArray src soff-        y <- readByteArray src (soff + 1)+        let !x = indexByteArray src soff+            !y = indexByteArray src (soff + 1)          let !a = aix x hi             !b = aix y lo@@ -97,7 +122,7 @@ lenientLoop     :: Int     -> MutableByteArray s-    -> MutableByteArray s+    -> ByteArray     -> ST s Int lenientLoop !slen !dst !src = goHi (0 :: Int) (0 :: Int)   where@@ -108,7 +133,7 @@     goHi !doff !soff       | soff == slen = return doff       | otherwise = do-        x <- readByteArray src soff+        let !x = indexByteArray src soff          let !a = aix x hi @@ -119,7 +144,7 @@     goLo !doff !soff !a       | soff == slen = return doff       | otherwise = do-        y <- readByteArray src soff+        let !y = indexByteArray src soff          let !b = aix y lo 

@@ -1,7 +1,7 @@ {-# LANGUAGE OverloadedStrings #-} -- | -- Module       : Data.ByteString.Lazy.Base16--- Copyright    : (c) 2020-2023 Emily Pillmore+-- Copyright    : (c) 2020-2026 Emily Pillmore -- License      : BSD-style -- -- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>@@ -20,21 +20,20 @@ , decodeBase16' , decodeBase16Untyped , decodeBase16Lenient-, isBase16-, isValidBase16+, parseBase16 ) where   -import Prelude hiding (all, elem)+import Prelude hiding (elem)  import Data.Base16.Types import qualified Data.ByteString as B-import Data.ByteString.Lazy (all, elem, fromChunks, toChunks)+import Data.ByteString.Lazy (elem, fromChunks, fromStrict, toChunks, toStrict) import Data.ByteString.Lazy.Internal (ByteString(..)) import qualified Data.ByteString.Base16.Internal.Head as B16+import qualified Data.ByteString.Base16 as B16 (decodeBase16Untyped, parseBase16) import Data.ByteString.Base16.Internal.Utils (reChunk)-import Data.Either import Data.Text.Lazy (Text) import qualified Data.Text as T import qualified Data.Text.Lazy.Encoding as TL@@ -75,6 +74,9 @@  -- | Decode a Base16-encoded lazy 'ByteString' value. --+-- Hexadecimal byte pairs may cross chunk boundaries. Decoding remains lazy+-- in the input chunks.+-- -- See: <https://tools.ietf.org/html/rfc4648#section-8 RFC-4648 section 8> -- -- === __Examples__:@@ -83,11 +85,11 @@ -- "Sun" -- decodeBase16 :: Base16 ByteString -> ByteString-decodeBase16 bs = case extractBase16 bs of-    Empty -> Empty-    Chunk b bs' -> Chunk-      (B16.decodeBase16Typed_ (assertBase16 b))-      (decodeBase16 $ assertBase16 bs')+decodeBase16 = fromChunks+  . fmap (B16.decodeBase16Typed_ . assertBase16)+  . reChunk+  . toChunks+  . extractBase16 {-# INLINE decodeBase16 #-}  -- | Decode a Base16-encoded 'Text' value.@@ -105,6 +107,9 @@  -- | Decode an untyped Base16-encoded lazy 'ByteString' value. --+-- Checks the whole input, joining chunks before decoding so that lengths+-- and error offsets refer to the complete encoded value.+-- -- See: <https://tools.ietf.org/html/rfc4648#section-8 RFC-4648 section 8> -- -- === __Examples__:@@ -116,8 +121,7 @@ -- Left "invalid character at offset: 1" -- decodeBase16Untyped :: ByteString -> Either T.Text ByteString-decodeBase16Untyped Empty = Right Empty-decodeBase16Untyped (Chunk b bs) = Chunk <$> B16.decodeBase16_ b <*> decodeBase16Untyped bs+decodeBase16Untyped = fmap fromStrict . B16.decodeBase16Untyped . toStrict {-# INLINE decodeBase16Untyped #-}  @@ -142,34 +146,31 @@   . toChunks {-# INLINE decodeBase16Lenient #-} --- | Tell whether an untyped lazy 'ByteString' value is base16 encoded.+-- | Validate an untyped Base16 value and retain its encoded representation. ----- === __Examples__:+-- Accepts even-length ASCII hexadecimal input, including upper-case and+-- mixed-case digits. Empty input is valid. The original input is preserved,+-- and errors match 'decodeBase16Untyped'. ----- >>> isBase16 "666f6"--- False+-- The whole input is validated before returning 'Right', regardless of+-- chunk boundaries. The returned wrapper preserves those boundaries. ----- >>> isBase16 "666f"--- True+-- === __Examples__: ---isBase16 :: ByteString -> Bool-isBase16 bs = isValidBase16 bs && isRight (decodeBase16Untyped bs)-{-# INLINE isBase16 #-}---- | Tell whether an untyped lazy 'ByteString' value is a valid Base16 format.+-- >>> parseBase16 "666f6"+-- Left "invalid bytestring size" ----- This will not tell you whether or not this is a correct Base16 representation,--- only that it conforms to the correct alphabet. To check whether it is a true--- Base16 encoded 'ByteString' value, use 'isBase16'.+-- >>> parseBase16 "666F"+-- Right "666F" ----- === __Examples__:+-- >>> fmap decodeBase16 (parseBase16 "666F")+-- Right "fo" ----- >>> isValidBase16 "666f+/6"--- False+-- >>> parseBase16 "6x"+-- Left "invalid character at offset: 1" ----- >>> isValidBase16 "666f6"--- True+-- @since 2.0 ---isValidBase16 :: ByteString -> Bool-isValidBase16 = all (`elem` "0123456789abcdefABCDEF")-{-# INLINE isValidBase16 #-}+parseBase16 :: ByteString -> Either T.Text (Base16 ByteString)+parseBase16 bs = assertBase16 bs <$ B16.parseBase16 (toStrict bs)+{-# INLINE parseBase16 #-}

@@ -1,6 +1,6 @@ -- | -- Module       : Data.ByteString.Short.Base16--- Copyright    : (c) 2020-2023 Emily Pillmore+-- Copyright    : (c) 2020-2026 Emily Pillmore -- License      : BSD-style -- -- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>@@ -19,15 +19,13 @@ , decodeBase16' , decodeBase16Untyped , decodeBase16Lenient-, isBase16-, isValidBase16+, parseBase16 ) where   import Data.Base16.Types import Data.ByteString.Short import Data.ByteString.Base16.Internal.Head-import qualified Data.ByteString.Base16 as B16 import Data.Text (Text) import Data.Text.Short import Data.Text.Short.Unsafe@@ -126,34 +124,28 @@ decodeBase16Lenient = decodeBase16ShortLenient_ {-# INLINE decodeBase16Lenient #-} --- | Tell whether an untyped 'ShortByteString' value is base16 encoded.------ === __Examples__:+-- | Validate an untyped Base16 value and retain its encoded representation. ----- >>> isBase16 "666f6"--- False+-- Accepts even-length ASCII hexadecimal input, including upper-case and+-- mixed-case digits. Empty input is valid. The original input is preserved,+-- and errors match 'decodeBase16Untyped'. ----- >>> isBase16 "666f"--- True+-- === __Examples__: ---isBase16 :: ShortByteString -> Bool-isBase16 = B16.isBase16 . fromShort-{-# INLINE isBase16 #-}---- | Tell whether an untyped 'ShortByteString' value is a valid Base16 format.+-- >>> parseBase16 "666f6"+-- Left "invalid bytestring size" ----- This will not tell you whether or not this is a correct Base16 representation,--- only that it conforms to the correct alphabet. To check whether it is a true--- Base16 encoded 'ShortByteString' value, use 'isBase16'.+-- >>> parseBase16 "666F"+-- Right "666F" ----- === __Examples__:+-- >>> fmap decodeBase16 (parseBase16 "666F")+-- Right "fo" ----- >>> isValidBase16 "666f+/6"--- False+-- >>> parseBase16 "6x"+-- Left "invalid character at offset: 1" ----- >>> isValidBase16 "666f6"--- True+-- @since 2.0 ---isValidBase16 :: ShortByteString -> Bool-isValidBase16 = B16.isValidBase16 . fromShort-{-# INLINE isValidBase16 #-}+parseBase16 :: ShortByteString -> Either Text (Base16 ShortByteString)+parseBase16 bs = assertBase16 bs <$ decodeBase16Untyped bs+{-# INLINE parseBase16 #-}

@@ -1,6 +1,6 @@ -- | -- Module       : Data.Text.Encoding.Base16--- Copyright    : (c) 2020-2023 Emily Pillmore+-- Copyright    : (c) 2020-2026 Emily Pillmore -- License      : BSD-style -- -- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>@@ -18,8 +18,7 @@ , decodeBase16Untyped , decodeBase16With , decodeBase16Lenient-, isBase16-, isValidBase16+, parseBase16 ) where  @@ -27,6 +26,7 @@ import Data.Bifunctor (first) import Data.ByteString (ByteString) import qualified Data.ByteString.Base16 as B16+import Data.Char (chr) import Data.Text (Text) import Data.Text.Encoding.Base16.Error (Base16Error(..)) import qualified Data.Text.Encoding as T@@ -37,7 +37,7 @@ -- >>> :set -XOverloadedStrings -- --- | Encode a 'Text' value in Base16 with padding.+-- | Encode a 'Text' value in Base16. -- -- See: <https://tools.ietf.org/html/rfc4648#section-8 RFC-4648 section 8> --@@ -105,10 +105,13 @@     Right a -> first ConversionError (f a) {-# INLINE decodeBase16With #-} --- | Decode an untyped Base16-encoded 'Text' value leniently, using a--- strategy that never fails, catching unicode exceptions raised in the--- process of converting to text values.+-- | Decode an untyped Base16-encoded 'Text' value leniently. --+-- Non-hexadecimal characters and a final unpaired hex digit are ignored.+-- Valid UTF-8 sequences in the decoded bytes are preserved. Each invalid+-- UTF-8 byte is interpreted as its Latin-1 character (U+0000 through U+00FF),+-- so malformed input does not raise Unicode exceptions.+-- -- N.B.: this is not RFC 4648-compliant. -- -- === __Examples__:@@ -119,38 +122,43 @@ -- >>> decodeBase16Lenient "6x6x" -- "f" --+-- >>> decodeBase16Lenient "df" == "\223"+-- True+--+-- >>> decodeBase16Lenient "c39f" == "\223"+-- True+-- decodeBase16Lenient :: Text -> Text-decodeBase16Lenient = T.decodeUtf8 . B16.decodeBase16Lenient . T.encodeUtf8+decodeBase16Lenient = T.decodeUtf8With (\_ -> fmap (chr . fromIntegral))+  . B16.decodeBase16Lenient+  . T.encodeUtf8 {-# INLINE decodeBase16Lenient #-} --- | Tell whether a 'Text' value is Base16-encoded.+-- | Validate an untyped Base16 value and retain its encoded representation. ----- === __Examples__:+-- Accepts even-length ASCII hexadecimal input, including upper-case and+-- mixed-case digits. Empty input is valid. The original input is preserved,+-- and errors match 'decodeBase16Untyped'. ----- >>> isBase16 "666f6"--- False+-- This validates the Base16 encoding, not the UTF-8 validity of its decoded+-- payload. Use 'decodeBase16With' with a checked conversion for text decoding. ----- >>> isBase16 "666f"--- True+-- === __Examples__: ---isBase16 :: Text -> Bool-isBase16 = B16.isBase16 . T.encodeUtf8-{-# INLINE isBase16 #-}---- | Tell whether a 'Text' value is a valid Base16 format.+-- >>> parseBase16 "666f6"+-- Left "invalid bytestring size" ----- This will not tell you whether or not this is a correct Base16 representation,--- only that it conforms to the correct shape. To check whether it is a true--- Base16 encoded 'Text' value, use 'isBase16'.+-- >>> parseBase16 "666F"+-- Right "666F" ----- === __Examples__:+-- >>> fmap decodeBase16 (parseBase16 "666F")+-- Right "fo" ----- >>> isValidBase16 "666f+/6"--- False+-- >>> parseBase16 "6x"+-- Left "invalid character at offset: 1" ----- >>> isValidBase16 "666f6"--- True+-- @since 2.0 ---isValidBase16 :: Text -> Bool-isValidBase16 = B16.isValidBase16 . T.encodeUtf8-{-# INLINE isValidBase16 #-}+parseBase16 :: Text -> Either Text (Base16 Text)+parseBase16 bs = assertBase16 bs <$ B16.parseBase16 (T.encodeUtf8 bs)+{-# INLINE parseBase16 #-}

@@ -1,7 +1,7 @@ {-# LANGUAGE DeriveGeneric #-} -- | -- Module       : Data.Text.Encoding.Base16.Error--- Copyright    : (c) 2019-2023 Emily Pillmore+-- Copyright    : (c) 2019-2026 Emily Pillmore -- License      : BSD-style -- -- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>

@@ -1,6 +1,6 @@ -- |--- Module       : Data.Text.Encoding.Base16.Lazy--- Copyright    : (c) 2020-2023 Emily Pillmore+-- Module       : Data.Text.Lazy.Encoding.Base16+-- Copyright    : (c) 2020-2026 Emily Pillmore -- License      : BSD-style -- -- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>@@ -18,8 +18,7 @@ , decodeBase16Untyped , decodeBase16With , decodeBase16Lenient-, isBase16-, isValidBase16+, parseBase16 ) where  @@ -27,6 +26,7 @@ import Data.Bifunctor (first) import Data.ByteString.Lazy (ByteString) import qualified Data.ByteString.Lazy.Base16 as B16L+import Data.Char (chr) import qualified Data.Text as T import Data.Text.Encoding.Base16.Error (Base16Error(..)) import Data.Text.Lazy (Text)@@ -38,7 +38,7 @@ -- >>> :set -XOverloadedStrings -- --- | Encode a lazy 'Text' value in Base16 with padding.+-- | Encode a lazy 'Text' value in Base16. -- -- See: <https://tools.ietf.org/html/rfc4648#section-8 RFC-4648 section 8> --@@ -104,14 +104,15 @@     Right a -> first ConversionError $ f a {-# INLINE decodeBase16With #-} --- | Decode a Base16-encoded lazy 'Text' value leniently, using a--- strategy that never fails.+-- | Decode a Base16-encoded lazy 'Text' value leniently. ----- /Warning/: in the conversion to unicode text, exceptions may be thrown.--- Please use 'decodeBase16'' if you are unsure if you are working with--- base16-encoded values, or if you expect garbage.+-- Non-hexadecimal characters and a final unpaired hex digit are ignored.+-- Valid UTF-8 sequences are preserved, including across chunk boundaries.+-- Each invalid UTF-8 byte is interpreted as its Latin-1 character (U+0000+-- through U+00FF), so malformed input does not raise Unicode exceptions.+-- Decoding remains lazy in the input chunks. ----- N.B.: this is not RFC 4648-compliant. It may give you garbage if you're not careful!+-- N.B.: this is not RFC 4648-compliant. -- -- === __Examples__: --@@ -122,38 +123,39 @@ -- "f" -- decodeBase16Lenient :: Text -> Text-decodeBase16Lenient = TL.decodeUtf8 . B16L.decodeBase16Lenient . TL.encodeUtf8+decodeBase16Lenient = TL.decodeUtf8With (\_ -> fmap (chr . fromIntegral))+  . B16L.decodeBase16Lenient+  . TL.encodeUtf8 {-# INLINE decodeBase16Lenient #-} --- | Tell whether a lazy 'Text' value is Base16-encoded.+-- | Validate an untyped Base16 value and retain its encoded representation. ----- === __Examples__:+-- Accepts even-length ASCII hexadecimal input, including upper-case and+-- mixed-case digits. Empty input is valid. The original input is preserved,+-- and errors match 'decodeBase16Untyped'. ----- >>> isBase16 "666f6"--- False+-- The whole input is validated before returning 'Right', regardless of+-- chunk boundaries. The returned wrapper preserves those boundaries. ----- >>> isBase16 "666f"--- True+-- This validates the Base16 encoding, not the UTF-8 validity of its decoded+-- payload. Use 'decodeBase16With' with a checked conversion for text decoding. ---isBase16 :: Text -> Bool-isBase16 = B16L.isBase16 . TL.encodeUtf8--{-# INLINE isBase16 #-}---- | Tell whether a lazy 'Text' value is a valid Base16 format.+-- === __Examples__: ----- This will not tell you whether or not this is a correct Base16 representation,--- only that it conforms to the correct shape. To check whether it is a true--- Base16 encoded 'Text' value, use 'isBase16'.+-- >>> parseBase16 "666f6"+-- Left "invalid bytestring size" ----- === __Examples__:+-- >>> parseBase16 "666F"+-- Right "666F" ----- >>> isValidBase16 "666f+/6"--- False+-- >>> fmap decodeBase16 (parseBase16 "666F")+-- Right "fo" ----- >>> isValidBase16 "666f6"--- True+-- >>> parseBase16 "6x"+-- Left "invalid character at offset: 1" ---isValidBase16 :: Text -> Bool-isValidBase16 = B16L.isValidBase16 . TL.encodeUtf8-{-# INLINE isValidBase16 #-}+-- @since 2.0+--+parseBase16 :: Text -> Either T.Text (Base16 Text)+parseBase16 bs = assertBase16 bs <$ B16L.parseBase16 (TL.encodeUtf8 bs)+{-# INLINE parseBase16 #-}

@@ -1,6 +1,6 @@ -- | -- Module       : Data.Text.Short.Encoding.Base16--- Copyright    : (c) 2020-2023 Emily Pillmore+-- Copyright    : (c) 2020-2026 Emily Pillmore -- License      : BSD-style -- -- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>@@ -18,17 +18,16 @@ , decodeBase16Untyped , decodeBase16With , decodeBase16Lenient-, isBase16-, isValidBase16+, parseBase16 ) where   import Data.Base16.Types import Data.Bifunctor (first)-import qualified Data.ByteString.Base16 as B16 import Data.ByteString.Short import qualified Data.ByteString.Short.Base16 as BS16 import Data.Text (Text)+import qualified Data.Text.Encoding.Base16 as T16 import Data.Text.Encoding.Base16.Error import Data.Text.Short import Data.Text.Short.Unsafe@@ -39,7 +38,7 @@ -- >>> :set -XOverloadedStrings -- --- | Encode a 'ShortText' value in Base16 with padding.+-- | Encode a 'ShortText' value in Base16. -- -- See: <https://tools.ietf.org/html/rfc4648#section-8 RFC-4648 section 8> --@@ -112,10 +111,13 @@     Right a -> first ConversionError $ f a {-# INLINE decodeBase16With #-} --- | Decode an untyped Base16-encoded 'ShortText' value leniently, using a--- strategy that never fails, catching unicode exceptions raised in the--- process of converting to text values.+-- | Decode an untyped Base16-encoded 'ShortText' value leniently. --+-- Non-hexadecimal characters and a final unpaired hex digit are ignored.+-- Valid UTF-8 sequences are preserved. Each invalid UTF-8 byte is interpreted+-- as its Latin-1 character (U+0000 through U+00FF), ensuring the result is+-- valid 'ShortText' without raising Unicode exceptions.+-- -- N.B.: this is not RFC 4648-compliant. -- -- === __Examples__:@@ -127,39 +129,34 @@ -- "f" -- decodeBase16Lenient :: ShortText -> ShortText-decodeBase16Lenient = fromShortByteStringUnsafe-  . BS16.decodeBase16Lenient-  . toShortByteString+decodeBase16Lenient = fromText . T16.decodeBase16Lenient . toText {-# INLINE decodeBase16Lenient #-} --- | Tell whether a 'ShortText' value is Base16-encoded.+-- | Validate an untyped Base16 value and retain its encoded representation. ----- === __Examples__:+-- Accepts even-length ASCII hexadecimal input, including upper-case and+-- mixed-case digits. Empty input is valid. The original input is preserved,+-- and errors match 'decodeBase16Untyped'. ----- >>> isBase16 "666f6"--- False+-- This validates the Base16 encoding, not the UTF-8 validity of its decoded+-- payload. Use 'decodeBase16With' with a checked conversion for text decoding. ----- >>> isBase16 "666f"--- True+-- === __Examples__: ---isBase16 :: ShortText -> Bool-isBase16 = B16.isBase16 . toByteString-{-# INLINE isBase16 #-}---- | Tell whether a 'ShortText' value is a valid Base16 format.+-- >>> parseBase16 "666f6"+-- Left "invalid bytestring size" ----- This will not tell you whether or not this is a correct Base16 representation,--- only that it conforms to the correct shape. To check whether it is a true--- Base16 encoded 'ShortText' value, use 'isBase16'.+-- >>> parseBase16 "666F"+-- Right "666F" ----- === __Examples__:+-- >>> fmap decodeBase16 (parseBase16 "666F")+-- Right "fo" ----- >>> isValidBase16 "666f+/6"--- False+-- >>> parseBase16 "6x"+-- Left "invalid character at offset: 1" ----- >>> isValidBase16 "666f6"--- True+-- @since 2.0 ---isValidBase16 :: ShortText -> Bool-isValidBase16 = B16.isValidBase16 . toByteString-{-# INLINE isValidBase16 #-}+parseBase16 :: ShortText -> Either Text (Base16 ShortText)+parseBase16 bs = assertBase16 bs <$ BS16.parseBase16 (toShortByteString bs)+{-# INLINE parseBase16 #-}

@@ -6,7 +6,7 @@ {-# OPTIONS_GHC -fno-warn-orphans #-} -- | -- Module       : Main--- Copyright    : (c) 2020-2023 Emily Pillmore+-- Copyright    : (c) 2020-2026 Emily Pillmore -- License      : BSD-style -- -- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>@@ -87,10 +87,9 @@   decodeTyped :: Base16 bs -> bs   decode :: bs -> Either Text bs   lenient :: bs -> bs+  parse :: bs -> Either Text (Base16 bs)    lower :: bs -> bs-  correct :: bs -> Bool-  validate :: bs -> Bool   instance Harness 'B16 BS.ByteString where@@ -100,8 +99,7 @@   decodeTyped = B16.decodeBase16   decode = B16.decodeBase16Untyped   lenient = B16.decodeBase16Lenient-  correct = B16.isBase16-  validate = B16.isValidBase16+  parse = B16.parseBase16   lower = BS8.map toLower  instance Harness 'LB16 LBS.ByteString where@@ -111,8 +109,7 @@   decodeTyped = LB16.decodeBase16   decode = LB16.decodeBase16Untyped   lenient = LB16.decodeBase16Lenient-  correct = LB16.isBase16-  validate = LB16.isValidBase16+  parse = LB16.parseBase16   lower = LBS8.map toLower  instance Harness 'SB16 SBS.ShortByteString where@@ -122,8 +119,7 @@   decodeTyped = SB16.decodeBase16   decode = SB16.decodeBase16Untyped   lenient = SB16.decodeBase16Lenient-  correct = SB16.isBase16-  validate = SB16.isValidBase16+  parse = SB16.parseBase16   lower = SBS.toShort . BS8.map toLower . SBS.fromShort  instance Harness 'T16 Text where@@ -133,8 +129,7 @@   decodeTyped = T16.decodeBase16   decode = T16.decodeBase16Untyped   lenient = T16.decodeBase16Lenient-  correct = T16.isBase16-  validate = T16.isValidBase16+  parse = T16.parseBase16   lower = T.map toLower  instance Harness 'TL16 TL.Text where@@ -144,8 +139,7 @@   decodeTyped = TL16.decodeBase16   decode = TL16.decodeBase16Untyped   lenient = TL16.decodeBase16Lenient-  correct = TL16.isBase16-  validate = TL16.isValidBase16+  parse = TL16.parseBase16   lower = TL.map toLower  instance Harness 'TS16 TS.ShortText where@@ -155,8 +149,7 @@   decodeTyped = TS16.decodeBase16   decode = TS16.decodeBase16Untyped   lenient = TS16.decodeBase16Lenient-  correct = TS16.isBase16-  validate = TS16.isValidBase16+  parse = TS16.parseBase16   lower = TS.fromText . T.map toLower . TS.toText  class Harness a cs

@@ -5,7 +5,7 @@ {-# LANGUAGE TypeApplications #-} -- | -- Module       : Main--- Copyright    : (c) 2020-2023 Emily Pillmore+-- Copyright    : (c) 2020-2026 Emily Pillmore -- License      : BSD-style -- -- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>@@ -20,27 +20,37 @@ ) where  +import Control.Monad (forM_) import Data.Base16.Types import Data.Bifunctor (second) import qualified Data.ByteString as BS-import Data.ByteString.Internal (c2w) import qualified Data.ByteString.Lazy as LBS import qualified Data.ByteString.Short as SBS-import "base16" Data.ByteString.Base16 as B16+import qualified "base16" Data.ByteString.Base16 as B16+import qualified "base16" Data.ByteString.Lazy.Base16 as LB16+import qualified "base16" Data.ByteString.Short.Base16 as SB16 import qualified "base16-bytestring" Data.ByteString.Base16 as Bos+import Data.Char (chr)+import Data.Either (isLeft, isRight)+import qualified Data.List as L+import Data.String (fromString) import qualified Data.Text as T import qualified Data.Text.Encoding as T+import qualified "base16" Data.Text.Encoding.Base16 as T16 import Data.Text.Encoding.Base16.Error (Base16Error(..)) import qualified Data.Text.Lazy as TL import qualified Data.Text.Lazy.Encoding as TL+import qualified "base16" Data.Text.Lazy.Encoding.Base16 as TL16 import qualified Data.Text.Short as TS-import Data.Word+import qualified "base16" Data.Text.Short.Encoding.Base16 as TS16  import Internal+import qualified Simd  import Test.Tasty import Test.Tasty.HUnit import Test.Tasty.QuickCheck (testProperty)+import Test.QuickCheck (Positive(..))   main :: IO ()@@ -77,8 +87,32 @@     , mkDecodeTree       (second TS.fromText . T.decodeUtf8' . SBS.fromShort) sb16     ]+  , testCase "parseBase16 accepts exactly the ASCII hex alphabet" $+    assertBool "accepts invalid hex" $+      isRight (B16.parseBase16 valids) && all (isLeft . B16.parseBase16) invalids+  , prop_bos_coherence+  , parserRegressions+  , lenientTextRegressions+  , Simd.tests   ] +valids :: BS.ByteString+valids = fromString hexChars++hexChars :: String+hexChars = "0123456789abcdefABCDEF"++-- This is to verify EVERY non-hex char is "not valid"+-- The bytestrings have the same character twice, so that+-- the parser does not reject the input+-- because of the odd number of bytes.+invalids :: [BS.ByteString]+invalids = fromString <$> separate8s+  where+    allWord8 = chr <$> [0..255]+    nonHex = L.filter (`L.notElem` hexChars) allWord8+    separate8s = (\c -> [c,c]) <$> nonHex+ -- ---------------------------------------------------------------- -- -- Test tree generation @@ -110,7 +144,6 @@ mkPropTree = mkTests "Property Tests"   [ prop_roundtrip   , prop_untyped_correctness-  , const prop_bos_coherence   ]  -- | Make unit tests for a given harness instance@@ -122,6 +155,7 @@   -> TestTree mkUnitTree = mkTests "Unit tests"   [ rfcVectors+  , parserVectors   , lenientTests   ] @@ -157,10 +191,13 @@  prop_untyped_correctness :: forall a b proxy. Harness a b => proxy a -> TestTree prop_untyped_correctness _ = testGroup "prop_validity"-  [ testProperty "prop_std_valid" $ \(bs :: b) ->-    validate (extractBase16 $ encode bs)-  , testProperty "prop_std_correct" $ \(bs :: b) ->-    correct (extractBase16 $ encode bs)+  [ testProperty "parsing an encoding preserves the encoded representation" $ \(bs :: b) ->+      let encoded = encode bs+      in Right encoded == parse (extractBase16 encoded)+  , testProperty "successful parsing preserves its input" $ \(candidate :: b) ->+      case parse candidate of+        Left _ -> True+        Right encoded -> extractBase16 encoded == candidate   ]  -- | just a sanity check against `base16-bytestring`@@ -168,14 +205,160 @@ prop_bos_coherence :: TestTree prop_bos_coherence = testGroup "prop_bos_coherence"   [ testProperty "prop_std_bos_coherence" $ \bs ->-      Right bs == B16.decodeBase16Untyped (extractBase16 $ B16.encodeBase16' bs)-      && Right bs == Bos.decode (Bos.encode bs)-      && bs == B16.decodeBase16 (B16.encodeBase16' bs)+      let encoded = Bos.encode bs+      in encoded == extractBase16 (B16.encodeBase16' bs)+        && Right bs == B16.decodeBase16Untyped encoded+        && bs == B16.decodeBase16 (assertBase16 encoded)   ]  -- ---------------------------------------------------------------- -- -- Unit tests +parserVectors :: forall a b proxy. Harness a b => proxy a -> TestTree+parserVectors _ = testGroup "Parser contract"+  [ testCase "accepts empty, upper-case, lower-case and mixed-case encodings" $+      forM_ ["", "00", "0123456789aAbBcCdDeEfF", "ff", "FF"] $ \input ->+        Right input @=? fmap extractBase16 (parse @a input)+  , testCase "odd length takes precedence over invalid characters" $+      forM_ ["0", "z", "z00", "0z0"] $ \input ->+        Left "invalid bytestring size" @=? parse @a input+  , testCase "reports the first invalid character" $+      forM_ [("gg", 0 :: Int), ("0z", 1), ("00z0", 2), ("00zz", 2)] $+        \(input, offset) ->+          Left (invalidAt offset) @=? parse @a input+  ]++parserRegressions :: TestTree+parserRegressions = testGroup "Parser and lazy chunk regressions"+  [ testProperty "strict byte parsing and untyped decoding agree on arbitrary input" $+      \candidate ->+        fmap B16.decodeBase16 (B16.parseBase16 candidate)+          == B16.decodeBase16Untyped candidate+  , testProperty "short byte parsing and untyped decoding agree on arbitrary input" $+      \candidate ->+        fmap SB16.decodeBase16 (SB16.parseBase16 candidate)+          == SB16.decodeBase16Untyped candidate+  , testProperty "lazy errors and decoding are independent of arbitrary chunk boundaries" $+      \candidate sizes ->+        let chunked = LBS.fromChunks (chunksBy sizes candidate)+            parsed = fmap (LBS.toStrict . extractBase16) (LB16.parseBase16 chunked)+        in parsed == fmap extractBase16 (B16.parseBase16 candidate)+          && fmap LBS.toStrict (LB16.decodeBase16Untyped chunked)+            == B16.decodeBase16Untyped candidate+  , testProperty "parsing preserves lazy byte chunks and typed decoding joins split pairs" $+      \raw sizes ->+        let chunked = LBS.fromChunks (chunksBy sizes (Bos.encode raw))+        in fmap (LBS.toChunks . extractBase16) (LB16.parseBase16 chunked)+              == Right (LBS.toChunks chunked)+            && LB16.decodeBase16 (assertBase16 chunked) == LBS.fromStrict raw+  , testProperty "parsing preserves lazy text chunks and typed decoding joins split pairs" $+      \raw sizes ->+        let encoded = Bos.encode (T.encodeUtf8 raw)+            chunked = TL.fromChunks (map T.decodeUtf8 (chunksBy sizes encoded))+        in fmap (TL.toChunks . extractBase16) (TL16.parseBase16 chunked)+              == Right (TL.toChunks chunked)+            && TL16.decodeBase16 (assertBase16 chunked) == TL.fromStrict raw+  , testCase "every byte split preserves case and decodes pairs crossing a chunk" $+      forM_ ["", "fF", "aAbBcCdDeEfF", Bos.encode (BS.pack [0 .. 255])] $ \encoded ->+        forM_ [0 .. BS.length encoded] $ \split -> do+          let chunked = LBS.fromChunks [BS.take split encoded, BS.drop split encoded]+              note = "encoded=" ++ show encoded ++ ", split=" ++ show split+          assertEqual (note ++ ": parse") (Right (LBS.toChunks chunked))+            (fmap (LBS.toChunks . extractBase16) (LB16.parseBase16 chunked))+          expected <- either assertFailure pure (Bos.decode encoded)+          assertEqual (note ++ ": typed decode") expected+            (LBS.toStrict (LB16.decodeBase16 (assertBase16 chunked)))+          assertEqual (note ++ ": untyped decode") (Right expected)+            (fmap LBS.toStrict (LB16.decodeBase16Untyped chunked))+  , testCase "every text split preserves chunks and decodes Unicode payloads" $ do+      let raw = "a\x00e9\x03bb\x1f642"+          encoded = T.decodeUtf8 (Bos.encode (T.encodeUtf8 raw))+      forM_ [0 .. T.length encoded] $ \split -> do+        let chunked = TL.fromChunks [T.take split encoded, T.drop split encoded]+        Right (TL.toChunks chunked)+          @=? fmap (TL.toChunks . extractBase16) (TL16.parseBase16 chunked)+        TL.fromStrict raw @=? TL16.decodeBase16 (assertBase16 chunked)+        Right (TL.fromStrict raw) @=? TL16.decodeBase16Untyped chunked+  , testCase "lazy invalid offsets are global and odd total length has priority" $+      forM_ [("66z0", invalidAt 2), ("666f0zz0", invalidAt 5),+             ("z00", "invalid bytestring size")] $ \(input, expected) ->+        forM_ [0 .. BS.length input] $ \split -> do+          let chunked = LBS.fromChunks [BS.take split input, BS.drop split input]+              text = TL.fromChunks (map T.decodeUtf8 (LBS.toChunks chunked))+          Left expected @=? LB16.parseBase16 chunked+          Left expected @=? LB16.decodeBase16Untyped chunked+          Left expected @=? TL16.parseBase16 text+          Left expected @=? TL16.decodeBase16Untyped text+  , testCase "text parsing checks encoded bytes without decoding invalid UTF-8 payloads" $+      forM_ ["ff", "c0af", "eda080", "f4908080"] $ \input -> do+        Right input @=? fmap extractBase16 (T16.parseBase16 input)+        Right (TS.fromText input) @=? fmap extractBase16 (TS16.parseBase16 (TS.fromText input))+        let chunked = TL.fromChunks (T.chunksOf 1 input)+        Right (TL.toChunks chunked)+          @=? fmap (TL.toChunks . extractBase16) (TL16.parseBase16 chunked)+  , testCase "text diagnostics use UTF-8 byte lengths and reject non-ASCII hex lookalikes" $+      forM_ [("\x00e9", invalidAt 0), ("0\x20ac", invalidAt 1),+             ("00\x1f4a9", invalidAt 2), ("\xff10\xff11", invalidAt 0),+             ("f\x03bb", "invalid bytestring size"),+             ("\x00e9\&0", "invalid bytestring size")] $ \(input, expected) -> do+        Left expected @=? T16.parseBase16 input+        Left expected @=? TS16.parseBase16 (TS.fromText input)+        Left expected @=? TL16.parseBase16 (TL.fromChunks (T.chunksOf 1 input))+  ]++invalidAt :: Int -> T.Text+invalidAt offset = "invalid character at offset: " <> T.pack (show offset)++lenientTextRegressions :: TestTree+lenientTextRegressions = testGroup "Lenient text Unicode regressions"+  [ testCase "issue 32: Latin-1 byte df decodes to sharp s" $+      checkText "0014df1047454e53" "\NUL\DC4\xdf\DLEGENS"+  , testCase "each isolated byte has a defined character result" $+      forM_ [0 .. 255] $ \byte ->+        checkText (T.decodeUtf8 (Bos.encode (BS.singleton byte)))+          (T.singleton (chr (fromIntegral byte)))+  , testCase "valid UTF-8 survives beside invalid and truncated sequences" $+      forM_ [("c3a9dff09f9982", "\xe9\xdf\x1f642"),+             ("e282", "\xe2\x82"),+             ("e228a1", "\xe2(\xa1"),+             ("c0af", "\xc0\xaf"),+             ("eda080", "\xed\xa0\x80"),+             ("f4908080", "\xf4\x90\x80\x80"),+             ("c3a9#d!f0", "\xe9\xdf")] $ \(input, expected) ->+        checkText input expected+  , testCase "lazy decoding keeps UTF-8 sequences across every hex split" $+      forM_ [("0014df1047454e53", "\NUL\DC4\xdf\DLEGENS"),+             ("c3a9dff09f9982e282", "\xe9\xdf\x1f642\xe2\x82")] $ \(input, expected) ->+        forM_ [0 .. T.length input] $ \split -> do+          let chunked = TL.fromChunks [T.take split input, T.drop split input]+          expected @=? TL.toStrict (TL16.decodeBase16Lenient chunked)+  , testProperty "arbitrary decoded bytes yield valid UTF-8 in all text variants" $+      \raw ->+        let input = T.decodeUtf8 (Bos.encode raw)+            strict = T16.decodeBase16Lenient input+            lazy = TL.toStrict (TL16.decodeBase16Lenient (TL.fromChunks (T.chunksOf 1 input)))+            short = TS16.decodeBase16Lenient (TS.fromText input)+        in Right strict == T.decodeUtf8' (T.encodeUtf8 strict)+          && strict == lazy+          && Right strict == T.decodeUtf8' (TS.toByteString short)+  ]+  where+    checkText input expected = do+      expected @=? T16.decodeBase16Lenient input+      expected @=? TL.toStrict (TL16.decodeBase16Lenient (TL.fromStrict input))+      let short = TS16.decodeBase16Lenient (TS.fromText input)+      -- Validate the underlying bytes, not merely an unsafe text wrapper.+      Right expected @=? T.decodeUtf8' (TS.toByteString short)++-- Vary chunk boundaries independently of the payload, retaining a final chunk+-- when the generated sizes run out. The bound keeps generated chunks useful.+chunksBy :: [Positive Int] -> BS.ByteString -> [BS.ByteString]+chunksBy _ bytes | BS.null bytes = []+chunksBy [] bytes = [bytes]+chunksBy (Positive size : sizes) bytes =+  let (chunk, rest) = BS.splitAt (1 + size `mod` 97) bytes+  in chunk : chunksBy sizes rest+ rfcVectors :: forall a b proxy. Harness a b => proxy a -> TestTree rfcVectors _ = testGroup "RFC 4648 Test Vectors"     [ testGroup "lower-case"@@ -215,6 +398,9 @@          step "decode is sound"         s @=? decodeTyped (assertBase16 t)++        step "parse preserves the supplied case"+        Right t @=? fmap extractBase16 (parse @a t)          step "decodeUntyped is sound"         Right s @=? decode (extractBase16 $ encode s)

@@ -0,0 +1,215 @@+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeApplications #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE PackageImports #-}+{-# LANGUAGE MagicHash #-}+-- |+-- Module       : Simd+-- Copyright    : (c) 2026 Emily Pillmore+-- License      : BSD-style+--+-- Maintainer   : Emily Pillmore <emilypi@cohomolo.gy>+-- Stability    : Experimental+-- Portability  : portable+--+-- The same public API tests exercise scalar and SIMD builds. Expected bytes+-- come from an independent codec, so matching encoder and decoder mistakes+-- cannot cancel out as they could in roundtrip-only tests.+module Simd (tests) where++import Control.Concurrent (forkIO, killThread, threadDelay)+import Control.Exception (bracket)+import Control.Monad (forM_, forever)+import Data.Base16.Types (assertBase16, extractBase16)+import qualified Data.ByteString as BS+import qualified Data.ByteString.Char8 as BS8+import qualified Data.ByteString.Lazy as LBS+import qualified Data.ByteString.Short as SBS+import qualified Data.ByteString.Short.Internal as SBSI+import qualified "base16" Data.ByteString.Base16 as B16+import qualified "base16" Data.ByteString.Short.Base16 as S16+import qualified "base16-bytestring" Data.ByteString.Base16 as Bos+import Data.Char (toUpper)+import qualified Data.Text as T+import qualified Data.Text.Encoding as T+import qualified Data.Text.Lazy as TL+import qualified Data.Text.Short as TS+import Data.Word (Word8)+import GHC.Exts (isByteArrayPinned#, isTrue#)+import Internal (Harness(..), b16, lb16, sb16, t16, tl16, ts16)+import System.Mem (performGC)+import Test.QuickCheck (arbitrary, choose, conjoin, forAll, vectorOf, (===))+import Test.Tasty (TestTree, localOption, testGroup)+import Test.Tasty.HUnit (Assertion, assertEqual, testCase)+import Test.Tasty.QuickCheck (QuickCheckTests(..), testProperty)++tests :: TestTree+tests = testGroup "SIMD compatibility"+  [ testCase "vector and dispatch boundaries at every slice alignment" $+      forM_ sizes $ \n ->+        forM_ [0 .. 31] $ \offset ->+          checkHarness (context n offset) b16 (sliceAt offset) (payload n)+  , testCase "every input byte across multiple vector blocks" $+      forM_ [0 .. 255] $ \byte ->+        checkHarness ("repeated byte=" ++ show byte) b16+          (sliceAt (fromIntegral byte `mod` 32)) (BS.replicate 257 byte)+  , testCase "every non-hex byte is rejected before and after vector prefixes" $+      forM_ invalidBytes $ \byte ->+        forM_ [0, 1, 15, 16, 31, 32, 63, 64, 127, 128, 4096] $ \position ->+          checkInvalid (fromIntegral byte `mod` 32) position byte+  , testCase "every invalid lane reports its original offset after long prefixes" $+      forM_ [4096 .. 4159] $ \position ->+        forM_ [35, 128, 255] $ \byte ->+          forM_ [0, 1, 15, 31] $ \offset ->+            checkInvalid offset position byte+  , testCase "odd length takes precedence even after SIMD-sized input" $+      forM_ [0, 16, 32, 64, 128, 256, 4096] $ \prefixLength -> do+        let input = BS.replicate prefixLength 48 <> "#00"+        assertEqual "odd parser input" (Left "invalid bytestring size")+          (B16.parseBase16 input)+        assertEqual "odd decoder input" (Left "invalid bytestring size")+          (B16.decodeBase16Untyped input)+        assertEqual "odd short parser input" (Left "invalid bytestring size")+          (S16.parseBase16 (SBS.toShort input))+        assertEqual "odd short decoder input" (Left "invalid bytestring size")+          (S16.decodeBase16Untyped (SBS.toShort input))+  , testCase "lazy chunks split pairs across vector blocks" $+      forM_ [15, 16, 17, 31, 32, 33, 63, 64, 65, 127, 128, 129, 4096] $ \n ->+        forM_ [1, 2, 3, 7, 15, 16, 17, 31, 32, 63, 64, 127] $ \chunkSize ->+          checkHarness ("lazy length=" ++ show n ++ ", chunk=" ++ show chunkSize)+            lb16 (LBS.fromChunks . chunksOf chunkSize) (payload n)+  , testCase "short byte arrays remain compatible across boundaries and large inputs" $+      forM_ sizes $ \n ->+        checkHarness ("short length=" ++ show n) sb16 SBS.toShort (payload n)+  , testCase "short decoding reports errors in the final block and scalar tail" $+      forM_ [126, 128, 130, 254, 256, 258, 4096, 4098] $ \n ->+        forM_ [n - 33 .. n - 1] $ \position ->+          forM_ [35, 128, 255] $ \byte -> do+            let input = SBS.toShort $+                  BS.replicate position 48 <> BS.singleton byte+                    <> BS.replicate (n - position - 1) 70+                expected = Left ("invalid character at offset: " <> T.pack (show position))+                note = "short length=" ++ show n ++ ", position=" ++ show position+            assertEqual (note ++ ": decoder") expected (S16.decodeBase16Untyped input)+            assertEqual (note ++ ": parser") expected (S16.parseBase16 input)+  , testCase "short codecs retain unpinned arrays under GC pressure" $+      bracket (forkIO (forever (performGC >> threadDelay 1000))) killThread $ \_ ->+        forM_ [0 .. 31] $ \byte ->+          forM_ [64, 65, 127, 128, 129, 511, 1023] $ \n -> do+            let raw = BS.map (+ byte) (payload n)+                input = SBS.pack (BS.unpack raw)+                encoded = extractBase16 (S16.encodeBase16' input)+                decoded = S16.decodeBase16 (assertBase16 encoded)+                note = "unpinned length=" ++ show n ++ ", byte=" ++ show byte+            assertEqual (note ++ ": source is unpinned") False (isPinned input)+            assertEqual (note ++ ": encoded output is unpinned") False (isPinned encoded)+            assertEqual (note ++ ": decoded output is unpinned") False (isPinned decoded)+            checkHarness note sb16 (SBS.pack . BS.unpack) raw+            assertEqual (note ++ ": source unchanged") raw (SBS.fromShort input)+  , testCase "text wrappers preserve long Unicode payloads" textWrappers+  , localOption (QuickCheckTests 40) $+      testProperty "large random payloads agree with the independent codec" $+        forAll largePayload $ \raw ->+          let encoded = Bos.encode raw+              shortRaw = SBS.toShort raw+              shortEncoded = SBS.toShort encoded+          in conjoin+            [ extractBase16 (B16.encodeBase16' raw) === encoded+            , B16.decodeBase16 (assertBase16 encoded) === raw+            , B16.decodeBase16Untyped encoded === Right raw+            , fmap extractBase16 (B16.parseBase16 encoded) === Right encoded+            , extractBase16 (S16.encodeBase16' shortRaw) === shortEncoded+            , S16.decodeBase16 (assertBase16 shortEncoded) === shortRaw+            , S16.decodeBase16Untyped shortEncoded === Right shortRaw+            , fmap extractBase16 (S16.parseBase16 shortEncoded) === Right shortEncoded+            ]+  ]+  where+    sizes = [0 .. 5]+      ++ concatMap (\n -> [n - 2 .. n + 2]) [8, 16, 32, 64, 128, 256]+      ++ [4095, 4096, 4097, 65535, 65536, 65537]+    largePayload = do+      n <- choose (4096, 65536)+      BS.pack <$> vectorOf n arbitrary++isPinned :: SBS.ShortByteString -> Bool+isPinned (SBSI.SBS bytes) = isTrue# (isByteArrayPinned# bytes)++-- Invalid surrounding bytes ensure the codec respects the slice's bounds.+sliceAt :: Int -> BS.ByteString -> BS.ByteString+sliceAt offset bytes = BS.take (BS.length bytes) . BS.drop offset $+  BS.replicate offset 0xe5 <> bytes <> BS.replicate 32 0xff++payload :: Int -> BS.ByteString+payload n = BS.pack (take n (cycle [0 .. 255]))++context :: Int -> Int -> String+context n offset = "length=" ++ show n ++ ", slice offset=" ++ show offset++checkHarness+  :: forall a b proxy. Harness a b+  => String+  -> proxy a+  -> (BS.ByteString -> b)+  -> BS.ByteString+  -> Assertion+checkHarness note _ convert raw = do+  assertEqual (note ++ ": encode") (convert encoded)+    (extractBase16 (encode @a input))+  forM_ [("lower", encoded), ("upper", upper), ("mixed", mixed)] $+    \(name, representation) -> do+      let expected = convert representation+      assertEqual (note ++ ": " ++ name ++ " typed decode") input+        (decodeTyped @a (assertBase16 expected))+      assertEqual (note ++ ": " ++ name ++ " untyped decode") (Right input)+        (decode @a expected)+      assertEqual (note ++ ": " ++ name ++ " parser preserves case") (Right expected)+        (fmap extractBase16 (parse @a expected))+  where+    input = convert raw+    encoded = Bos.encode raw+    upper = BS8.map toUpper encoded+    mixed = BS.pack (zipWith chooseCase [0 :: Int ..] (BS.unpack encoded))+    chooseCase i byte+      | even i && byte >= 97 && byte <= 102 = byte - 32+      | otherwise = byte++invalidBytes :: [Word8]+invalidBytes = filter (`BS.notElem` "0123456789abcdefABCDEF") [0 .. 255]++checkInvalid :: Int -> Int -> Word8 -> Assertion+checkInvalid offset position byte = do+  assertEqual (note ++ ": parser") (Left expected) (B16.parseBase16 input)+  assertEqual (note ++ ": decoder") (Left expected) (B16.decodeBase16Untyped input)+  assertEqual (note ++ ": short parser") (Left expected)+    (S16.parseBase16 (SBS.toShort input))+  assertEqual (note ++ ": short decoder") (Left expected)+    (S16.decodeBase16Untyped (SBS.toShort input))+  where+    -- A second invalid byte ensures that the first bad lane wins. The valid+    -- suffix keeps the first error out of the scalar-only tail.+    input = sliceAt offset $+      BS.replicate position 48 <> BS.singleton byte+        <> BS.replicate 65 70 <> "#" <> BS.replicate suffixLength 97+    suffixLength = if even position then 257 else 256+    expected = "invalid character at offset: " <> T.pack (show position)+    note = context (BS.length input) offset+      ++ ", invalid byte=" ++ show byte ++ " at=" ++ show position++chunksOf :: Int -> BS.ByteString -> [BS.ByteString]+chunksOf n bytes+  | BS.null bytes = []+  | otherwise = let (chunk, rest) = BS.splitAt n bytes+                in sliceAt 1 chunk : chunksOf n rest++textWrappers :: Assertion+textWrappers =+  forM_ [0, 1, 15, 16, 31, 32, 127, 1024] $ \repetitions -> do+    let value = T.replicate repetitions "a\x00e9\x03bb\x1f642"+        raw = T.encodeUtf8 value+        note = "text repetitions=" ++ show repetitions+    checkHarness note t16 T.decodeUtf8 raw+    checkHarness note ts16 (TS.fromText . T.decodeUtf8) raw+    forM_ [1, 3, 15, 16, 63, 64] $ \chunkSize ->+      checkHarness (note ++ ", text chunk=" ++ show chunkSize) tl16+        (TL.fromChunks . T.chunksOf chunkSize . T.decodeUtf8) raw

@@ -0,0 +1,357 @@+/* Copyright (c) 2026, Emily Pillmore. BSD-3-Clause; see LICENSE. */++/*+ * Standalone differential/bounds checks; link with cbits/base16_simd.c.+ * Recommended: -std=c99 -O2 -Wall -Wextra -Werror+ *              -fsanitize=address,undefined -fno-omit-frame-pointer+ * Repeat with HS_BASE16_SIMD_FORCE_SCALAR and HS_BASE16_SIMD_FORCE_SSSE3.+ */+#ifndef _DEFAULT_SOURCE+#define _DEFAULT_SOURCE 1+#endif++#include <stdio.h>+#include <stdlib.h>+#include <string.h>++#if defined(__unix__) || defined(__APPLE__)+#include <sys/mman.h>+#include <unistd.h>+#define HS_BASE16_GUARD_PAGES 1+#endif++#include "base16_simd.h"++#define CHECK(condition) do { \+    if (!(condition)) { \+        fprintf(stderr, "%s:%d: check failed: %s\n", __FILE__, __LINE__, #condition); \+        exit(EXIT_FAILURE); \+    } \+} while (0)++static const uint8_t alphabet[] = "0123456789abcdef";+static size_t encode_block;+static size_t decode_block;+static size_t checks;++/* Deliberately scalar and independent of the kernels' vector classification. */+static int hex_value(uint8_t value)+{+    if (value >= '0' && value <= '9') {+        return value - '0';+    }+    if (value >= 'a' && value <= 'f') {+        return value - 'a' + 10;+    }+    if (value >= 'A' && value <= 'F') {+        return value - 'A' + 10;+    }+    return -1;+}++static size_t encode_prefix(size_t len)+{+    return encode_block == 0 ? 0 : len / encode_block * encode_block;+}++static size_t decode_prefix(const uint8_t *src, size_t len)+{+    size_t consumed = 0;++    if (decode_block == 0) {+        return 0;+    }+    while (len - consumed >= decode_block) {+        for (size_t i = 0; i < decode_block; i++) {+            if (hex_value(src[consumed + i]) < 0) {+                return consumed;+            }+        }+        consumed += decode_block;+    }+    return consumed;+}++static void scalar_encode(const uint8_t *src, size_t len, uint8_t *dst)+{+    for (size_t i = 0; i < len; i++) {+        *dst++ = alphabet[src[i] / 16];+        *dst++ = alphabet[src[i] % 16];+    }+}++static void scalar_decode(const uint8_t *src, size_t len, uint8_t *dst)+{+    for (size_t i = 0; i < len; i += 2) {+        *dst++ = (uint8_t)(hex_value(src[i]) * 16 + hex_value(src[i + 1]));+    }+}++static void check_canary(const uint8_t *data, size_t len)+{+    for (size_t i = 0; i < len; i++) {+        CHECK(data[i] == 0xa5);+    }+}++static void check_encode(const uint8_t *src, size_t len, size_t offset)+{+    const size_t expected = encode_prefix(len);+    const size_t output_len = expected * 2;+    uint8_t *output = malloc(offset + output_len + 32);+    uint8_t *reference = malloc(output_len + 1);+    size_t consumed;++    CHECK(output != NULL && reference != NULL);+    memset(output, 0xa5, offset + output_len + 32);+    scalar_encode(src, expected, reference);+    consumed = hs_base16_simd_encode(src, len, output + offset);+    CHECK(consumed == expected && consumed <= len);+    CHECK(memcmp(output + offset, reference, output_len) == 0);+    check_canary(output, offset);+    check_canary(output + offset + output_len, 32);+    free(reference);+    free(output);+    checks++;+}++static void check_decode(const uint8_t *src, size_t len, size_t offset)+{+    const size_t expected = decode_prefix(src, len);+    const size_t output_len = expected / 2;+    uint8_t *output = malloc(offset + output_len + 32);+    uint8_t *reference = malloc(output_len + 1);+    size_t consumed;++    CHECK(output != NULL && reference != NULL);+    memset(output, 0xa5, offset + output_len + 32);+    scalar_decode(src, expected, reference);+    consumed = hs_base16_simd_decode(src, len, output + offset);+    CHECK(consumed == expected);+    CHECK(consumed % 2 == 0 && consumed <= len);+    CHECK(memcmp(output + offset, reference, output_len) == 0);+    check_canary(output, offset);+    check_canary(output + offset + output_len, 32);+    free(reference);+    free(output);+    checks++;+}++static void test_lengths_and_alignment(void)+{+    uint8_t source[1056];+    uint8_t encoded[2112];++    for (size_t offset = 0; offset < 32; offset++) {+        for (size_t i = 0; i < sizeof(source); i++) {+            source[i] = (uint8_t)(i * 73 + offset * 19);+        }+        scalar_encode(source, sizeof(source), encoded);+        for (size_t len = 0; len <= 1024; len++) {+            check_encode(source + offset, len, 31 - offset);+            check_decode(encoded + offset, len, 31 - offset);+        }+    }+    /* Vary both alignments independently at every relevant vector boundary. */+    const size_t sizes[] = {0, 1, 15, 16, 17, 31, 32, 33, 63, 64, 65,+                            127, 128, 129, 255, 256, 257};+    for (size_t source_offset = 0; source_offset < 32; source_offset++) {+        for (size_t output_offset = 0; output_offset < 32; output_offset++) {+            for (size_t i = 0; i < sizeof(sizes) / sizeof(sizes[0]); i++) {+                check_encode(source + source_offset, sizes[i], output_offset);+                check_decode(encoded + source_offset, sizes[i], output_offset);+            }+        }+    }+}++static void test_all_byte_values(void)+{+    uint8_t source[257];+    uint8_t encoded[514];++    for (unsigned value = 0; value <= 255; value++) {+        memset(source, (int)value, sizeof(source));+        check_encode(source, sizeof(source), value % 32);+        scalar_encode(source, sizeof(source), encoded);+        check_decode(encoded, sizeof(encoded), value % 32);+        for (size_t i = 0; i < sizeof(encoded); i++) {+            if (encoded[i] >= 'a' && encoded[i] <= 'f' && i % 2 == 0) {+                encoded[i] = (uint8_t)(encoded[i] - 'a' + 'A');+            }+        }+        check_decode(encoded, sizeof(encoded), value % 32);+    }+}++static void test_invalid_block_stops(void)+{+    uint8_t source[580];++    for (size_t position = 0; position < sizeof(source); position++) {+        for (unsigned value = 0; value <= 255; value++) {+            memset(source, 'a', sizeof(source));+            source[position] = (uint8_t)value;+            check_decode(source, sizeof(source), position % 32);+        }+    }+}++static uint32_t next_random(uint32_t *state)+{+    uint32_t value = *state;+    value ^= value << 13;+    value ^= value >> 17;+    value ^= value << 5;+    *state = value;+    return value;+}++static void test_random_mixed_inputs(void)+{+    uint8_t source[4096];+    uint8_t encoded[8192];+    uint32_t state = UINT32_C(0x6d253fb9);++    for (size_t sample = 0; sample < 256; sample++) {+        const size_t len = next_random(&state) % (sizeof(source) + 1);+        const size_t encoded_len = len * 2;+        const size_t offset = next_random(&state) % 32;++        for (size_t i = 0; i < len; i++) {+            source[i] = (uint8_t)next_random(&state);+        }+        check_encode(source, len, offset);+        scalar_encode(source, len, encoded);+        for (size_t i = 0; i < encoded_len; i++) {+            if (encoded[i] >= 'a' && encoded[i] <= 'f' && next_random(&state) % 2) {+                encoded[i] = (uint8_t)(encoded[i] - 'a' + 'A');+            }+        }+        check_decode(encoded, encoded_len, offset);+        if (encoded_len > 0) {+            const size_t position = next_random(&state) % encoded_len;+            encoded[position] = (uint8_t)next_random(&state);+            check_decode(encoded, encoded_len, offset);+        }+    }+}++static void test_tight_allocations(void)+{+    for (size_t len = 1; len <= 1024; len++) {+        uint8_t *source = malloc(len);+        const size_t encode_len = encode_prefix(len) * 2;+        uint8_t *encoded = malloc(encode_len == 0 ? 1 : encode_len);+        size_t decode_len;+        uint8_t *decoded;++        CHECK(source != NULL && encoded != NULL);+        memset(source, 0xfb, len);+        CHECK(hs_base16_simd_encode(source, len, encoded) == encode_prefix(len));+        memset(source, 'F', len);+        decode_len = decode_prefix(source, len) / 2;+        decoded = malloc(decode_len == 0 ? 1 : decode_len);+        CHECK(decoded != NULL);+        CHECK(hs_base16_simd_decode(source, len, decoded) == decode_prefix(source, len));+        free(decoded);+        free(encoded);+        free(source);+        checks += 2;+    }+}++#if defined(HS_BASE16_GUARD_PAGES)+static uint8_t *guarded_buffer(size_t page_size)+{+    uint8_t *mapping = mmap(NULL, page_size * 3, PROT_NONE,+                            MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);+    CHECK(mapping != MAP_FAILED);+    CHECK(mprotect(mapping + page_size, page_size, PROT_READ | PROT_WRITE) == 0);+    return mapping;+}++static void test_guard_pages(void)+{+    const long raw_page_size = sysconf(_SC_PAGESIZE);+    size_t page_size;+    uint8_t *source_mapping;+    uint8_t *output_mapping;++    CHECK(raw_page_size >= 2048);+    page_size = (size_t)raw_page_size;+    source_mapping = guarded_buffer(page_size);+    output_mapping = guarded_buffer(page_size);++    for (size_t len = 0; len <= 1024; len++) {+        for (size_t at_end = 0; at_end < 2; at_end++) {+            uint8_t *source = source_mapping + page_size ++                              (at_end ? page_size - len : 0);+            const size_t encode_len = encode_prefix(len) * 2;+            uint8_t *output = output_mapping + page_size ++                              (at_end ? page_size - encode_len : 0);+            size_t expected;+            size_t decode_len;++            memset(source, 0xfb, len);+            CHECK(hs_base16_simd_encode(source, len, output) == encode_prefix(len));+            memset(source, 'F', len);+            expected = decode_prefix(source, len);+            decode_len = expected / 2;+            output = output_mapping + page_size + (at_end ? page_size - decode_len : 0);+            CHECK(hs_base16_simd_decode(source, len, output) == expected);+            checks += 2;+        }+    }+    CHECK(munmap(source_mapping, page_size * 3) == 0);+    CHECK(munmap(output_mapping, page_size * 3) == 0);+}+#endif++int main(int argc, char **argv)+{+    const int backend = hs_base16_simd_backend();++    if (argc != 1) {+        if (argc != 3 || strcmp(argv[1], "--require-backend") != 0 ||+            strlen(argv[2]) != 1 || argv[2][0] < '0' || argv[2][0] > '3') {+            fprintf(stderr, "usage: %s [--require-backend 0|1|2|3]\n", argv[0]);+            return EXIT_FAILURE;+        }+        if (backend != argv[2][0] - '0') {+            fprintf(stderr, "required SIMD backend %c, detected %d\n", argv[2][0], backend);+            return EXIT_FAILURE;+        }+    }+    CHECK(backend >= 0 && backend <= 3);+#if defined(HS_BASE16_SIMD_FORCE_SCALAR)+    CHECK(backend == 0);+#elif defined(HS_BASE16_SIMD_FORCE_SSSE3) && defined(__x86_64__)+    CHECK(backend == 0 || backend == 1);+#elif defined(__aarch64__) && defined(__ARM_NEON) && \+      (defined(__GNUC__) || defined(__clang__))+    CHECK(backend == 2);+#endif+    encode_block = backend == 3 ? 32 : backend == 0 ? 0 : 16;+    decode_block = backend == 1 ? 16 : backend == 0 ? 0 : 32;+    printf("SIMD backend: %d (%s)\n", backend,+           backend == 3 ? "AVX2" : backend == 2 ? "NEON" :+           backend == 1 ? "SSSE3" : "scalar fallback");+    /* Zero-length and unsupported calls must not dereference their pointers. */+    CHECK(hs_base16_simd_encode(NULL, 0, NULL) == 0);+    CHECK(hs_base16_simd_decode(NULL, 0, NULL) == 0);+    if (backend == 0) {+        CHECK(hs_base16_simd_encode(NULL, SIZE_MAX, NULL) == 0);+        CHECK(hs_base16_simd_decode(NULL, SIZE_MAX, NULL) == 0);+    }+    test_lengths_and_alignment();+    test_all_byte_values();+    test_invalid_block_stops();+    test_random_mixed_inputs();+    test_tight_allocations();+#if defined(HS_BASE16_GUARD_PAGES)+    test_guard_pages();+#endif+    printf("Passed %zu differential/bounds checks.\n", checks);+    return EXIT_SUCCESS;+}