typed-encoding 0.2.1.0 → 0.2.2.0
raw patch · 33 files changed
+961/−138 lines, 33 files
Files
- ChangeLog.md +32/−4
- README.md +29/−20
- src/Data/TypedEncoding.hs +19/−2
- src/Data/TypedEncoding/Combinators/Restriction/Bool.hs +2/−2
- src/Data/TypedEncoding/Combinators/Restriction/BoundedAlphaNums.hs +18/−13
- src/Data/TypedEncoding/Combinators/Restriction/Common.hs +5/−1
- src/Data/TypedEncoding/Conv/ByteString/Char8.hs +42/−0
- src/Data/TypedEncoding/Conv/ByteString/Lazy/Char8.hs +30/−0
- src/Data/TypedEncoding/Conv/Text.hs +37/−0
- src/Data/TypedEncoding/Conv/Text/Encoding.hs +79/−0
- src/Data/TypedEncoding/Conv/Text/Lazy.hs +25/−0
- src/Data/TypedEncoding/Conv/Text/Lazy/Encoding.hs +28/−0
- src/Data/TypedEncoding/Instances/Enc/Base64.hs +35/−5
- src/Data/TypedEncoding/Instances/Restriction/ASCII.hs +77/−3
- src/Data/TypedEncoding/Instances/Restriction/UTF8.hs +35/−20
- src/Data/TypedEncoding/Instances/Support.hs +3/−0
- src/Data/TypedEncoding/Internal/Class.hs +20/−11
- src/Data/TypedEncoding/Internal/Class/Encoder.hs +49/−0
- src/Data/TypedEncoding/Internal/Class/IsStringR.hs +4/−0
- src/Data/TypedEncoding/Internal/Class/Recreate.hs +1/−3
- src/Data/TypedEncoding/Internal/Class/Superset.hs +131/−0
- src/Data/TypedEncoding/Internal/Class/Util.hs +7/−1
- src/Data/TypedEncoding/Internal/Class/Util/StringConstraints.hs +6/−1
- src/Data/TypedEncoding/Internal/Combinators.hs +4/−0
- src/Data/TypedEncoding/Internal/Instances/Combinators.hs +18/−0
- src/Data/TypedEncoding/Internal/Types.hs +5/−2
- src/Data/TypedEncoding/Internal/Types/SomeEnc.hs +2/−2
- src/Data/TypedEncoding/Internal/Util/TypeLits.hs +34/−3
- src/Examples/TypedEncoding/Conversions.hs +161/−38
- src/Examples/TypedEncoding/DiySignEncoding.hs +1/−1
- src/Examples/TypedEncoding/Overview.hs +10/−0
- src/Examples/TypedEncoding/ToEncString.hs +2/−4
- typed-encoding.cabal +10/−2
ChangeLog.md view
@@ -1,19 +1,47 @@ # Changelog for typed-encoding -## Unreleased changes- ## Anticipated future breaking changes +- ByteString / Text conversion functions in `Data.TypedEncoding.Instances.Restriction.ASCII`, + `Data.TypedEncoding.Instances.Restriction.ASCII` and `Data.TypedEncoding.Instances.Enc.Base64`+ are now deprecated and will be removed.+- `EncodeFAll`, `DecodeFAll`, `RecreateFAll`, `EncodeF`, etc do not work well with more open + encoding annotation such as `"r-ban:soething"` they will be either changed or deprecated / replaced with constructions similar to `Encoder` in `Data.TypedEncoding.Internal.Class.Encoder`. - `Data.TypedEncoding.Internal.Class.IsStringR` expected to be be changed / replaced+- functions used to create encoding instances or encoding combinators (e.g. `implEncodeP`) will get more constraints. +- (never ending) rework of internal module stucture to make it easier to navigate +- Instance and Combinator modules will be merged.+- Displ String instance (used in examples, will be made consistent with Text and ByteString)+- (post 0.3) "enc-B64" will be moved to a different package (more distant goal) +## 0.2.2 ++- Next version (0.3) will have number of breaking changes, some rethinking and a lot of cleanup,+ this version preps for some of that (see section above)+- Fixes+ - Conversions type safety issues+ - new and corrected approach to conversions (all old conversion functions have been deprecated)+ - corrected documentation in `Data.TypedEncoding.Combinators.Restriction.BoundedAlphaNums`+- new functionality:+ - `Enc` versions for `pack`/ `unpack` for `Text` and `ByteString`. + - `Enc` versions of `decodeUtf8` / `encodeUtf8`+ - new and corrected approach to conversions (all old conversion functions have been deprecated)+ - `IsSuperset` type family with basic combinators deprecates `Superset` typeclass.+ - more modules exported from `Data.TypedEncoding.Instances.Support` for instance and combinator creation+ - more utility type families `Data.TypedEncoding.Internal.Util.TypeLits` + - more utility combinators for creating encoding instances and combinators.+ - String instance added in number of places, including for "r-ASCII" encoding+ - few more support convenience functions.+- deprecation warnings (see above) + ## 0.2.1.0 - new functionality: - bounded alpha-numeric restriction encodings (`r-ban`) - boolean algebra of encodings - minor improvements- - dropped IsString contraint from instances in `Data.TypedEncoding.Instances.Restriction.Common`- - added forall annotation to ecodeAll and decodeAll+ - dropped IsString constraint from instances in `Data.TypedEncoding.Instances.Restriction.Common`+ - added forall annotation to encodeAll and decodeAll ## 0.2.0.0
README.md view
@@ -22,13 +22,14 @@ ``` and provides ways for - - encoding- - decoding- - recreation (encoding validation)- - type conversions- - converting types to encoded strings- - typesafe conversion of encoded strings to types +- encoding+- decoding+- recreation (encoding validation)+- type conversions+- converting types to encoded strings+- typesafe conversion of encoded strings to types+ ... but this approach seems to be a bit more... ```Haskell@@ -40,12 +41,13 @@ It becomes a type directed, declarative approach to string transformations. Transformations can be- - used with parameters- - applied or undone partially (if encoding is reversible) -One of more intersting uses of this library are encoding restrictions. +- used with parameters+- applied or undone partially (if encoding is reversible)++One of more interesting uses of this library are encoding restrictions. (Arbitrary) bounded alpha-numeric (`r-ban`) restrictions -and a simple annotation boolean algebra are both provided.+and a simple annotation Boolean algebra are both provided. ```Haskell phone :: Enc '["r-ban:999-999-9999"] () T.Text@@ -57,25 +59,32 @@ ``` - ## Examples Please see `Examples.TypedEncoding` it the module list. -## Dependencies on other encoding libs -Currently it uses- - `base64-bytestring` because it was my driving example- - I will try to separate other deps like `servant`, specific encoding libraries, etc into separate libs if there is interest. I consider orphan instances to be OK in this context. (GHC will classify them as such despite use of unique symbols.)+## Other encoding packages +My approach will be to write specific encodings (e.g. _HTTP_) or wrap encodings from other packages using separate "bridge" projects.++Currently `typed-encoding` depends on++- `base64-bytestring` because it was my driving example, this is likely to move out to a separate bridge project at some point. ++Bridge work:++- [typed-encoding-encoding](https://github.com/rpeszek/typed-encoding-encoding) bridges [encoding](https://github.com/dmwit/encoding) package+ ## Plans, some TODOs- - lensifying conversions - - better implementation type safety +- lensifying conversions +- better implementation type safety+ ## Tested with- - stack (1.9.3) lts-14.27 (ghc-8.6.5)- - needs ghc >= 8.2.2, base >=4.10 for GHC.TypeLits support+- stack (1.9.3) lts-14.27 (ghc-8.6.5)+- needs ghc >= 8.2.2, base >=4.10 for GHC.TypeLits support ## Known issues- - running test suite: cabal has problems with doctest, use stack +- running test suite: cabal has problems with doctest, use stack https://github.com/haskell/cabal/issues/6087
src/Data/TypedEncoding.hs view
@@ -84,10 +84,11 @@ -- -- Examples: ----- @"boolOr:(r-ban:ffffffff-ffff-ffff-ffff-ffffffffffff)(r-ban:ffffffffffffffffffffffffffffffff)"@ +-- @"boolOr:(r-ban:FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF)(r-ban:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF)"@ -- -- "@boolNot:(r-ASCII)" --+-- -- = Usage -- -- To use this library import this module and one or more /instance/ or /combinator/ module.@@ -108,13 +109,26 @@ -- -- * "Data.TypedEncoding.Instances.Support" ----- Defining annotations with combinators is an alternative to using typeclass instances +-- Defining annotations with combinators is an alternative to using typeclass instances. --+-- Combinator modules with be merged with Instances modules in the future.+-- -- Included combinator modules: -- -- * "Data.TypedEncoding.Combinators.Restriction.Bool" -- * "Data.TypedEncoding.Combinators.Restriction.BoundedAlphaNums" --+-- Conversion combinator module structure is similar to one found in @text@ and @bytestring@ packages+-- And can be found (since 0.2.2) in+--+-- * "Data.TypedEncoding.Conv.Text"+-- * "Data.TypedEncoding.Conv.Text.Encoding"+-- * "Data.TypedEncoding.Conv.Text.Lazy" +-- * "Data.TypedEncoding.Conv.Text.Lazy.Encoding"+-- * "Data.TypedEncoding.Conv.ByteString.Char8"+-- * "Data.TypedEncoding.Conv.ByteString.Lazy.Char8"+--+-- -- = Examples -- -- Examples of how to use this library are included in@@ -124,6 +138,8 @@ module Data.TypedEncoding -- * Classes , module Data.TypedEncoding.Internal.Class+ -- * Encoding class and Encoder (replaces EncodeFAll)+ , module Data.TypedEncoding.Internal.Class.Encoder -- * Combinators , module Data.TypedEncoding.Internal.Combinators -- * Types@@ -157,3 +173,4 @@ import Data.TypedEncoding.Internal.Types.UncheckedEnc import Data.TypedEncoding.Internal.Class import Data.TypedEncoding.Internal.Combinators+import Data.TypedEncoding.Internal.Class.Encoder
src/Data/TypedEncoding/Combinators/Restriction/Bool.hs view
@@ -41,7 +41,7 @@ import Data.TypedEncoding import Data.TypedEncoding.Instances.Support-import Data.TypedEncoding.Internal.Util.TypeLits+-- import Data.TypedEncoding.Internal.Util.TypeLits import Data.TypedEncoding.Combinators.Restriction.Common -- import qualified Data.Text as T@@ -255,7 +255,7 @@ -- ... -- = 'True ----- :kind! NestedR "boolOr:(boolAnd:(r-ab)(ac))(boolNot:(r-cd))"+-- >>> :kind! NestedR "boolOr:(boolAnd:(r-ab)(ac))(boolNot:(r-cd))" -- ... -- ... (TypeError ...) -- ...
src/Data/TypedEncoding/Combinators/Restriction/BoundedAlphaNums.hs view
@@ -15,12 +15,12 @@ -- Restrictions @"r-ban:"@ cover commonly used fixed (short) size strings with restricted -- characters such as GUID, credit card numbers, etc. -- --- Alphanumeric chars are ordered: @0-9@ followed by --- @a-z@ followed by @A-Z@. Annotation specifies upper character bound. +-- Alphanumeric chars are ordered: @0-9@ followed by @A-Z@,+-- followed by @a-z@. Annotation specifies upper character bound. -- Any non alpha numeric characters are considered fixed delimiters -- and need to be present exactly as specified. -- For example @"r-ban:999-99-9999"@ could be used to describe SSN numbers,--- @"r-ban:ffff" would describe strings consisting of 4 hex digits.+-- @"r-ban:FFFF" would describe strings consisting of 4 hex digits. -- -- This is a simple implementation that converts to @String@, should be used -- only with short length data.@@ -54,12 +54,16 @@ -- better compilation errors? type family IsBan (s :: Symbol) :: Bool where- IsBan s = AcceptEq ('Text "Not ban restriction encoding " ':<>: ShowType s ) (CmpSymbol "r-ban:" (Take 6 s))+ IsBan s = AcceptEq ('Text "Not ban restriction encoding " ':<>: ShowType s ) (CmpSymbol (TakeUntil s ":") "r-ban") +type instance IsSupersetOpen "r-ASCII" "r-ban" xs = 'True +instance (KnownSymbol s, "r-ban" ~ TakeUntil s ":" , IsStringR str, Encodings (Either EncodeEx) xs grps c str) => Encodings (Either EncodeEx) (s ': xs) ("r-ban" ': grps) c str where+ encodings = AppendEnc encFBan encodings+ -- |--- >>> encFBan . toEncoding () $ "c59f9fb7-4621-44d9-9020-ce37bf6e2bd1" :: Either EncodeEx (Enc '["r-ban:ffffffff-ffff-ffff-ffff-ffffffffffff"] () T.Text)--- Right (MkEnc Proxy () "c59f9fb7-4621-44d9-9020-ce37bf6e2bd1")+-- >>> encFBan . toEncoding () $ "C59F9FB7-4621-44D9-9020-CE37BF6E2BD1" :: Either EncodeEx (Enc '["r-ban:FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF"] () T.Text)+-- Right (MkEnc Proxy () "C59F9FB7-4621-44D9-9020-CE37BF6E2BD1") -- -- >>> recWithEncR encFBan . toEncoding () $ "211-22-9934" :: Either RecreateEx (Enc '["r-ban:999-99-9999"] () T.Text) -- Right (MkEnc Proxy () "211-22-9934")@@ -73,16 +77,17 @@ Enc xs c str -> f (Enc (s ': xs) c str) encFBan = implEncodeF @s (verifyBoundedAlphaNum (Proxy :: Proxy s)) +-- TODO v0.3 remove f from forall in encFBan (slightly breaking chanage) -- |--- >>> verifyBoundedAlphaNum (Proxy :: Proxy "r-ban:ff-ff") (T.pack "12-3e")--- Right "12-3e"--- >>> verifyBoundedAlphaNum (Proxy :: Proxy "r-ban:ff-ff") (T.pack "1g-3e")--- Left "'g' not boulded by 'f'"--- >>> verifyBoundedAlphaNum (Proxy :: Proxy "r-ban:ff-ff") (T.pack "13g3e")--- Left "'g' not matching '-'"--- >>> verifyBoundedAlphaNum (Proxy :: Proxy "r-ban:ff-ff") (T.pack "13-234")+-- >>> verifyBoundedAlphaNum (Proxy :: Proxy "r-ban:FF-FF") (T.pack "12-3E")+-- Right "12-3E"+-- >>> verifyBoundedAlphaNum (Proxy :: Proxy "r-ban:FF-FF") (T.pack "1G-3E")+-- Left "'G' not boulded by 'F'"+-- >>> verifyBoundedAlphaNum (Proxy :: Proxy "r-ban:FF-FF") (T.pack "13G3E")+-- Left "'G' not matching '-'"+-- >>> verifyBoundedAlphaNum (Proxy :: Proxy "r-ban:FF-FF") (T.pack "13-234") -- Left "Input list has wrong size expecting 5 but length \"13-234\" == 6" verifyBoundedAlphaNum :: forall s a str . (KnownSymbol s, IsStringR str) => Proxy s -> str -> Either String str verifyBoundedAlphaNum p str =
src/Data/TypedEncoding/Combinators/Restriction/Common.hs view
@@ -31,7 +31,11 @@ -- | --- Manual recreate step combinator converting typical encode function to a recreate step+-- Manual recreate step combinator converting @"r-"@ encode function to a recreate step.+--+-- For "r-" encoding recreate and encode are the same other than the exception type used. +--+-- The convention in @typed-encoding@ is to implement encode and convert it to recreate. recWithEncR :: forall (s :: Symbol) xs c str . (IsR s ~ 'True) => (Enc xs c str -> Either EncodeEx (Enc (s ': xs) c str)) -> Enc xs c str -> Either RecreateEx (Enc (s ': xs) c str)
+ src/Data/TypedEncoding/Conv/ByteString/Char8.hs view
@@ -0,0 +1,42 @@++{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE PolyKinds #-} -- removes need to annotate kinds as [Symbol]++-- | Encoding safe version of "Data.ByteString.Char8"+-- @since 0.2.2.0+module Data.TypedEncoding.Conv.ByteString.Char8 where++import qualified Data.ByteString.Char8 as B8+import Data.TypedEncoding.Internal.Types.Enc (Enc, unsafeChangePayload)+import qualified Data.TypedEncoding.Internal.Util.TypeLits as Knds+import Data.TypedEncoding++-- $setup+-- >>> :set -XDataKinds -XTypeApplications -XOverloadedStrings++-- | +-- Type safer version of 'Data.ByteString.Char8.pack'.+-- +-- This assumes that each of the encodings in @xs@ work work equivalently in @String@ and @ByteString@.+--+-- Because of how @ByteString.Char8.pack@ works, the first encoding (last in the list) must restrict character set to a subset of @ASCII@. +--+-- >>> :t pack (undefined :: Enc '["r-bar", "r-foo"] () String)+-- ...+-- ... error:+-- ... Couldn't match type ...+-- ... "r-ASCII" "r-foo" ...+-- ...+--+-- >>> displ $ pack (unsafeSetPayload () "Hello" :: Enc '["r-bar", "r-ASCII"] () String)+-- "MkEnc '[r-bar,r-ASCII] () (ByteString Hello)"+pack :: (Knds.LLast xs ~ t, IsSuperset "r-ASCII" t ~ 'True) => Enc xs c String -> Enc xs c B8.ByteString+pack = unsafeChangePayload B8.pack++-- | @unpack@ on encoded strings.+--+-- See 'pack'+unpack :: (Knds.LLast xs ~ t, IsSuperset "r-ASCII" t ~ 'True) => Enc xs c B8.ByteString -> Enc xs c String+unpack = unsafeChangePayload B8.unpack
+ src/Data/TypedEncoding/Conv/ByteString/Lazy/Char8.hs view
@@ -0,0 +1,30 @@++{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE PolyKinds #-} -- removes need to annotate kinds as [Symbol]++-- | Lazy version of "Data.TypedEncoding.Conv.ByteString.Char8"+-- @since 0.2.2.0+module Data.TypedEncoding.Conv.ByteString.Lazy.Char8 where++import qualified Data.ByteString.Lazy.Char8 as BL8+import Data.TypedEncoding.Internal.Types.Enc (Enc, unsafeChangePayload)+import qualified Data.TypedEncoding.Internal.Util.TypeLits as Knds+import Data.TypedEncoding++-- $setup+-- >>> :set -XDataKinds -XTypeApplications -XOverloadedStrings++-- | +-- Type safe version of 'BL8.pack'.+--+-- :t pack (undefined :: Enc '["r-bar", "r-ASCII"] () String)+-- :t pack (undefined :: Enc '["r-bar", "r-foo"] () String)+pack :: (Knds.LLast xs ~ t, IsSuperset "r-ASCII" t ~ 'True) => Enc xs c String -> Enc xs c BL8.ByteString+pack = unsafeChangePayload BL8.pack++-- | +-- Type safe version of 'BL8.unpack'.+unpack :: (Knds.LLast xs ~ t, IsSuperset "r-ASCII" t ~ 'True) => Enc xs c BL8.ByteString -> Enc xs c String+unpack = unsafeChangePayload BL8.unpack
+ src/Data/TypedEncoding/Conv/Text.hs view
@@ -0,0 +1,37 @@+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE PolyKinds #-} -- removes need to annotate kinds as [Symbol]++-- | Text encoding combinators specific to 'T.Text'+-- @since 0.2.2.0+module Data.TypedEncoding.Conv.Text where++import qualified Data.Text as T+import Data.TypedEncoding.Instances.Support+++-- $setup+-- >>> :set -XDataKinds -XTypeFamilies -XTypeApplications+++-- | This assumes that each of the encodings in @xs@ work work equivalently in @String@ and @Text@. +pack :: Enc xs c String -> Enc xs c T.Text+pack = unsafeChangePayload T.pack++-- | This assumes that each of the encodings in @xs@ work work equivalently in @String@ and @Text@. +unpack :: Enc xs c T.Text -> Enc xs c String+unpack = unsafeChangePayload T.unpack ++-- | Text is automatically @"r-UTF8"@ encoded+-- +-- >>> displ $ utf8Promote $ toEncoding () ("text" :: T.Text)+-- "MkEnc '[r-UTF8] () (Text text)"+utf8Promote :: Enc xs c T.Text -> Enc (Snoc xs "r-UTF8") c T.Text+utf8Promote = withUnsafeCoerce id++-- | For 'T.Text' @"r-UTF8"@ is redundant+--+-- >>> displ . utf8Demote $ (unsafeSetPayload () "Hello" :: Enc '["r-UTF8"] () T.Text)+-- "MkEnc '[] () (Text Hello)"+utf8Demote :: (UnSnoc xs ~ '(,) ys "r-UTF8") => Enc xs c T.Text -> Enc ys c T.Text+utf8Demote = withUnsafeCoerce id
+ src/Data/TypedEncoding/Conv/Text/Encoding.hs view
@@ -0,0 +1,79 @@+++{-# LANGUAGE DataKinds #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE ScopedTypeVariables #-}++-- |+-- @since 0.2.2.0+module Data.TypedEncoding.Conv.Text.Encoding where++import qualified Data.ByteString as B+import qualified Data.Text as T+import qualified Data.Text.Encoding as TE ++import Data.TypedEncoding.Instances.Support+import Data.TypedEncoding.Instances.Restriction.UTF8 ()+import Data.TypedEncoding.Instances.Restriction.ASCII ()+import Data.TypedEncoding.Unsafe (withUnsafe)+++-- $setup+-- >>> :set -XScopedTypeVariables -XOverloadedStrings -XDataKinds -XFlexibleContexts -XTypeApplications+-- >>> import Test.QuickCheck+-- >>> import Test.QuickCheck.Instances.Text()+-- >>> import Test.QuickCheck.Instances.ByteString()+-- >>> import qualified Data.ByteString.Char8 as B8+-- >>> import Data.Char+-- >>> import Data.Either+-- >>> import Data.TypedEncoding.Conv.Text+-- >>> let emptyUTF8B = unsafeSetPayload () "" :: Enc '["r-UTF8"] () B.ByteString +-- >>> :{+-- instance Arbitrary (Enc '["r-UTF8"] () B.ByteString) where +-- arbitrary = fmap (fromRight emptyUTF8B) +-- . flip suchThat isRight +-- . fmap (encodeFAll @(Either EncodeEx) @'["r-UTF8"] @(). toEncoding ()) $ arbitrary +-- instance Arbitrary (Enc '["r-UTF8"] () T.Text) where +-- arbitrary = fmap (unsafeSetPayload ()) +-- arbitrary +-- instance Arbitrary (Enc '["r-ASCII"] () B.ByteString) where +-- arbitrary = fmap (unsafeSetPayload ()) +-- . flip suchThat (B8.all isAscii) +-- $ arbitrary +-- instance Arbitrary (Enc '["r-ASCII"] () T.Text) where +-- arbitrary = fmap (unsafeSetPayload ()) +-- . flip suchThat (T.all isAscii) +-- $ arbitrary +-- :}+++-- |+-- With given constraints 'decodeUtf8' and 'encodeUtf8' can be used on subsets of @"r-UTF8"@+--+-- >>> displ . decodeUtf8 $ (unsafeSetPayload () "Hello" :: Enc '["r-ASCII"] () B.ByteString)+-- "MkEnc '[r-ASCII] () (Text Hello)"+--+-- "r-UTF8" is redundant:+--+-- >>> displ . utf8Demote . decodeUtf8 $ (unsafeSetPayload () "Hello" :: Enc '["r-UTF8"] () B.ByteString)+-- "MkEnc '[] () (Text Hello)"+--+-- @decodeUtf8@ and @encodeUtf8@ form isomorphism+-- +-- prop> \x -> getPayload x == (getPayload . encodeUtf8 . decodeUtf8 @ '["r-UTF8"] @() $ x)+--+-- prop> \x -> getPayload x == (getPayload . decodeUtf8 . encodeUtf8 @ '["r-UTF8"] @() $ x)+--+-- These nicely work as iso's for "r-ASCII" subset+--+-- prop> \x -> getPayload x == (getPayload . encodeUtf8 . decodeUtf8 @ '["r-ASCII"] @() $ x)+-- prop> \x -> getPayload x == (getPayload . decodeUtf8 . encodeUtf8 @ '["r-ASCII"] @() $ x)+decodeUtf8 :: forall xs c t. (LLast xs ~ t, IsSuperset "r-UTF8" t ~ 'True) => Enc xs c B.ByteString -> Enc xs c T.Text +decodeUtf8 = withUnsafe (fmap TE.decodeUtf8)++-- |+-- >>> displ $ encodeUtf8 $ utf8Promote $ toEncoding () ("text" :: T.Text)+-- "MkEnc '[r-UTF8] () (ByteString text)"+encodeUtf8 :: forall xs c t. (LLast xs ~ t, IsSuperset "r-UTF8" t ~ 'True) => Enc xs c T.Text -> Enc xs c B.ByteString +encodeUtf8 = withUnsafe (fmap TE.encodeUtf8)
+ src/Data/TypedEncoding/Conv/Text/Lazy.hs view
@@ -0,0 +1,25 @@++-- | Lazy version of "Data.TypedEncoding.Conv.Text"+-- @since 0.2.2.0+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE PolyKinds #-} -- removes need to annotate kinds as [Symbol]++module Data.TypedEncoding.Conv.Text.Lazy where++import qualified Data.Text.Lazy as TL+import Data.TypedEncoding.Instances.Support++pack :: Enc xs c String -> Enc xs c TL.Text+pack = unsafeChangePayload TL.pack++unpack :: Enc xs c TL.Text -> Enc xs c String+unpack = unsafeChangePayload TL.unpack ++-- | Text is automatically @"r-UTF8"@ encoded+utf8Promote :: Enc xs c TL.Text -> Enc (Snoc xs "r-UTF8") c TL.Text+utf8Promote = withUnsafeCoerce id++-- | For 'T.Text' @"r-UTF8"@ is redundant+utf8Demote :: (UnSnoc xs ~ '(,) ys "r-UTF8") => Enc xs c TL.Text -> Enc ys c TL.Text+utf8Demote = withUnsafeCoerce id
+ src/Data/TypedEncoding/Conv/Text/Lazy/Encoding.hs view
@@ -0,0 +1,28 @@+++{-# LANGUAGE DataKinds #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE ScopedTypeVariables #-}++-- | Lazy version of "Data.TypedEncoding.Conv.Text.Encoding"+-- @since 0.2.2.0+module Data.TypedEncoding.Conv.Text.Lazy.Encoding where++import qualified Data.ByteString.Lazy as BL+import qualified Data.Text.Lazy as TL+import qualified Data.Text.Lazy.Encoding as TEL ++import Data.TypedEncoding.Instances.Support+import Data.TypedEncoding.Instances.Restriction.UTF8 ()+import Data.TypedEncoding.Instances.Restriction.ASCII ()+import Data.TypedEncoding.Unsafe (withUnsafe)++++decodeUtf8 :: forall xs c t. (LLast xs ~ t, IsSuperset "r-UTF8" t ~ 'True) => Enc xs c BL.ByteString -> Enc xs c TL.Text +decodeUtf8 = withUnsafe (fmap TEL.decodeUtf8)+++encodeUtf8 :: forall xs c t. (LLast xs ~ t, IsSuperset "r-UTF8" t ~ 'True) => Enc xs c TL.Text -> Enc xs c BL.ByteString +encodeUtf8 = withUnsafe (fmap TEL.encodeUtf8)
src/Data/TypedEncoding/Instances/Enc/Base64.hs view
@@ -34,20 +34,37 @@ -- Conversions -- ----------------- --- | Type-safer version of Byte-string to text conversion that prevent invalid UTF8 bytestrings--- to be conversted to B64 encoded Text.+-- | +-- DEPRECATED use 'Data.TypedEncoding.Conv.Text.Encoding.decodeUtf8'+-- and 'Data.TypedEncoding.Conv.Text.utf8Demote'+-- +-- Will be removed in 0.3.x.x+--+-- See warning in 'Data.TypedEncoding.Instances.Restriction.ASCII.byteString2TextS' byteString2TextS :: Enc ("enc-B64" ': "r-UTF8" ': ys) c B.ByteString -> Enc ("enc-B64" ': ys) c T.Text byteString2TextS = withUnsafeCoerce TE.decodeUtf8 +-- | +-- DEPRECATED use 'Data.TypedEncoding.Conv.Text.Lazy.Encoding.decodeUtf8'+-- and 'Data.TypedEncoding.Conv.Text.utf8Demote'+--+-- Will be removed in 0.3.x.x+--+-- See warning in 'Data.TypedEncoding.Instances.Restriction.ASCII.byteString2TextS' byteString2TextL :: Enc ("enc-B64" ': "r-UTF8" ': ys) c BL.ByteString -> Enc ("enc-B64" ': ys) c TL.Text byteString2TextL = withUnsafeCoerce TEL.decodeUtf8 --- | Converts encoded text to ByteString adding "r-UTF8" annotation.--- The question is why "r-UTF8", not for example, "r-UTF16"?--- No reason, there maybe a diffrent combinator for that in the future or one that accepts a proxy.+-- DEPRECATED use 'Data.TypedEncoding.Conv.Text.Encoding.encodeUtf8'+-- and 'Data.TypedEncoding.Conv.Text.utf8Promote'+-- +-- Will be removed in 0.3.x.x text2ByteStringS :: Enc ("enc-B64" ': ys) c T.Text -> Enc ("enc-B64" ': "r-UTF8" ': ys) c B.ByteString text2ByteStringS = withUnsafeCoerce TE.encodeUtf8 +-- DEPRECATED use 'Data.TypedEncoding.Conv.Text.Lazy.Encoding.encodeUtf8'+-- and 'Data.TypedEncoding.Conv.Text.utf8Promote'+-- +-- Will be removed in 0.3.x.x text2ByteStringL :: Enc ("enc-B64" ': ys) c TL.Text -> Enc ("enc-B64" ': "r-UTF8" ': ys) c BL.ByteString text2ByteStringL = withUnsafeCoerce TEL.encodeUtf8 @@ -57,15 +74,20 @@ byteString2TextS' :: Enc ("enc-B64" ': ys) c B.ByteString -> Enc ("enc-B64-nontext" ': ys) c T.Text byteString2TextS' = withUnsafeCoerce TE.decodeUtf8 +-- DEPRECATED byteString2TextL' :: Enc ("enc-B64" ': ys) c BL.ByteString -> Enc ("enc-B64-nontext" ': ys) c TL.Text byteString2TextL' = withUnsafeCoerce TEL.decodeUtf8 +-- DEPRECATED text2ByteStringS' :: Enc ("enc-B64-nontext" ': ys) c T.Text -> Enc ("enc-B64" ': ys) c B.ByteString text2ByteStringS' = withUnsafeCoerce TE.encodeUtf8 +-- DEPRECATED text2ByteStringL' :: Enc ("enc-B64-nontext" ': ys) c TL.Text -> Enc ("enc-B64" ': ys) c BL.ByteString text2ByteStringL' = withUnsafeCoerce TEL.encodeUtf8 ++ acceptLenientS :: Enc ("enc-B64-len" ': ys) c B.ByteString -> Enc ("enc-B64" ': ys) c B.ByteString acceptLenientS = withUnsafeCoerce (B64.encode . B64.decodeLenient) @@ -81,11 +103,19 @@ instance FlattenAs "r-ASCII" "enc-B64-nontext" where instance FlattenAs "r-ASCII" "enc-B64" where +-- DEPRECATED will be removed+--+-- dangerous, with new approach.+-- Supersets are for "r-" types only+instance Superset "r-ASCII" "enc-B64-nontext" where+instance Superset "r-ASCII" "enc-B64" where ----------------- -- Encodings -- ----------------- +instance Encodings (Either EncodeEx) xs grps c B.ByteString => Encodings (Either EncodeEx) ("enc-B64" ': xs) ("enc-B64" ': grps) c B.ByteString where+ encodings = encodeFEncoder @(Either EncodeEx) @"enc-B64" @"enc-B64" instance Applicative f => EncodeF f (Enc xs c B.ByteString) (Enc ("enc-B64" ': xs) c B.ByteString) where encodeF = implEncodeP B64.encode
src/Data/TypedEncoding/Instances/Restriction/ASCII.hs view
@@ -4,9 +4,15 @@ {-# LANGUAGE FlexibleInstances #-} {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE TypeApplications #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE ScopedTypeVariables #-} --- | Strings can move to 'Enc "r-ASCII' only if they contain only ascii characters.--- they always decode back+-- | Strings can be encoded as 'Enc "r-ASCII"@ only if they contain only ASCII characters (first 128 characters of the Unicode character set).+--+-- This is sometimes referred to as ASCII-7 and future versions of @type-encoding@ may change @"r-ASCII"@ symbol annotation to reflect this.+-- +-- prop> B8.all ((< 128) . ord) . getPayload @ '["r-ASCII"] @() @B.ByteString+-- -- >>> :set -XOverloadedStrings -XMultiParamTypeClasses -XDataKinds -- >>> encodeFAll . toEncoding () $ "Hello World" :: Either EncodeEx (Enc '["r-ASCII"] () T.Text) -- Right (MkEnc Proxy () "Hello World")@@ -25,6 +31,7 @@ import qualified Data.Text.Lazy as TL import qualified Data.Text.Encoding as TE import qualified Data.Text.Lazy.Encoding as TEL +import qualified Data.List as L import qualified Data.ByteString.Char8 as B8 import qualified Data.ByteString.Lazy.Char8 as BL8@@ -36,33 +43,77 @@ -- $setup -- >>> :set -XDataKinds -XTypeApplications+-- >>> import Test.QuickCheck+-- >>> import Test.QuickCheck.Instances.ByteString()+-- >>> :{+-- instance Arbitrary (Enc '["r-ASCII"] () B.ByteString) where +-- arbitrary = fmap (unsafeSetPayload ()) +-- . flip suchThat (B8.all isAscii) +-- $ arbitrary +-- :}+-- + ----------------- -- Conversions -- ----------------- +-- |+-- DEPRECATED use 'Data.TypedEncoding.Conv.Text.Encoding.decodeUtf8'+-- +-- Will be removed in 0.3.x.x+--+-- This is not type safe, for example, would allow converting+--+-- @Enc `["r-ASCII", "enc-B64"] c B.ByteString@ containing B64 encoded binary +-- to @Enc `["r-ASCII", "enc-B64"] c T.Text@ and which then could be decoded causing +-- unexpected error. + byteString2TextS :: Enc ("r-ASCII" ': ys) c B.ByteString -> Enc ("r-ASCII" ': ys) c T.Text byteString2TextS = withUnsafe (fmap TE.decodeUtf8) +-- | +-- DEPRECATED use 'Data.TypedEncoding.Conv.Text.Lazy.Encoding.decodeUtf8'+-- +-- Will be removed in 0.3.x.x+--+-- see 'byteString2TextS' byteString2TextL :: Enc ("r-ASCII" ': ys) c BL.ByteString -> Enc ("r-ASCII" ': ys) c TL.Text byteString2TextL = withUnsafe (fmap TEL.decodeUtf8) +-- | +-- DEPRECATED use 'Data.TypedEncoding.Conv.Text.Encoding.encodeUtf8'+-- +-- Will be removed in 0.3.x.x+-- text2ByteStringS :: Enc ("r-ASCII" ': ys) c T.Text -> Enc ("r-ASCII" ': ys) c B.ByteString text2ByteStringS = withUnsafe (fmap TE.encodeUtf8) +-- | +-- DEPRECATED use 'Data.TypedEncoding.Conv.Text.Lazy.Encoding.encodeUtf8'+-- +-- Will be removed in 0.3.x.x+-- text2ByteStringL :: Enc ("r-ASCII" ': ys) c TL.Text -> Enc ("r-ASCII" ': ys) c BL.ByteString text2ByteStringL = withUnsafe (fmap TEL.encodeUtf8) -- | allow to treat ASCII encodings as UTF8 forgetting about B64 encoding -- +-- UTF-8 is backward compatible on first 128 characters using just one byte to store it.+-- +-- Payload does not change when @ASCII@ only strings are encoded to @UTF8@ in types like @ByteString@.+-- -- >>> let Right tstAscii = encodeFAll . toEncoding () $ "Hello World" :: Either EncodeEx (Enc '["r-ASCII"] () T.Text) -- >>> displ (inject @ "r-UTF8" tstAscii) -- "MkEnc '[r-UTF8] () (Text Hello World)" instance Superset "r-UTF8" "r-ASCII" where +-- type instance IsSuperset "r-UTF8" "r-ASCII" = True +-- type instance IsSuperset "r-ASCII" "r-ASCII" = True + -------------------- Encondings --+-- Encodings -- ----------------- newtype NonAsciiChar = NonAsciiChar Char deriving (Eq, Show)@@ -74,6 +125,19 @@ instance Applicative f => DecodeF f (Enc ("r-ASCII" ': xs) c Char) (Enc xs c Char) where decodeF = implTranP id +instance Encodings (Either EncodeEx) xs grps c String => Encodings (Either EncodeEx) ("r-ASCII" ': xs) ("r-ASCII" ': grps) c String where+ encodings = encodeFEncoder @(Either EncodeEx) @"r-ASCII" @"r-ASCII"++instance EncodeF (Either EncodeEx) (Enc xs c String) (Enc ("r-ASCII" ': xs) c String) where+ encodeF = implEncodeF_ prxyAscii (encodeImpl L.partition L.head L.null)+instance (RecreateErr f, Applicative f) => RecreateF f (Enc xs c String) (Enc ("r-ASCII" ': xs) c String) where+ checkPrevF = implCheckPrevF (asRecreateErr @"r-ASCII" . encodeImpl L.partition L.head L.null)+instance Applicative f => DecodeF f (Enc ("r-ASCII" ': xs) c String) (Enc xs c String) where+ decodeF = implTranP id ++instance Encodings (Either EncodeEx) xs grps c T.Text => Encodings (Either EncodeEx) ("r-ASCII" ': xs) ("r-ASCII" ': grps) c T.Text where+ encodings = encodeFEncoder @(Either EncodeEx) @"r-ASCII" @"r-ASCII"+ instance EncodeF (Either EncodeEx) (Enc xs c T.Text) (Enc ("r-ASCII" ': xs) c T.Text) where encodeF = implEncodeF_ prxyAscii (encodeImpl T.partition T.head T.null) instance (RecreateErr f, Applicative f) => RecreateF f (Enc xs c T.Text) (Enc ("r-ASCII" ': xs) c T.Text) where@@ -81,6 +145,9 @@ instance Applicative f => DecodeF f (Enc ("r-ASCII" ': xs) c T.Text) (Enc xs c T.Text) where decodeF = implTranP id +instance Encodings (Either EncodeEx) xs grps c TL.Text => Encodings (Either EncodeEx) ("r-ASCII" ': xs) ("r-ASCII" ': grps) c TL.Text where+ encodings = encodeFEncoder @(Either EncodeEx) @"r-ASCII" @"r-ASCII"+ instance EncodeF (Either EncodeEx) (Enc xs c TL.Text) (Enc ("r-ASCII" ': xs) c TL.Text) where encodeF = implEncodeF_ prxyAscii (encodeImpl TL.partition TL.head TL.null) instance (RecreateErr f, Applicative f) => RecreateF f (Enc xs c TL.Text) (Enc ("r-ASCII" ': xs) c TL.Text) where @@ -88,6 +155,9 @@ instance Applicative f => DecodeF f (Enc ("r-ASCII" ': xs) c TL.Text) (Enc xs c TL.Text) where decodeF = implTranP id +instance Encodings (Either EncodeEx) xs grps c B.ByteString => Encodings (Either EncodeEx) ("r-ASCII" ': xs) ("r-ASCII" ': grps) c B.ByteString where+ encodings = encodeFEncoder @(Either EncodeEx) @"r-ASCII" @"r-ASCII"+ instance EncodeF (Either EncodeEx) (Enc xs c B.ByteString) (Enc ("r-ASCII" ': xs) c B.ByteString) where encodeF = implEncodeF_ prxyAscii (encodeImpl (\p -> B8.filter p &&& B8.filter (not . p)) B8.head B8.null) instance (RecreateErr f, Applicative f) => RecreateF f (Enc xs c B.ByteString) (Enc ("r-ASCII" ': xs) c B.ByteString) where@@ -95,12 +165,16 @@ instance Applicative f => DecodeF f (Enc ("r-ASCII" ': xs) c B.ByteString) (Enc xs c B.ByteString) where decodeF = implTranP id +instance Encodings (Either EncodeEx) xs grps c BL.ByteString => Encodings (Either EncodeEx) ("r-ASCII" ': xs) ("r-ASCII" ': grps) c BL.ByteString where+ encodings = encodeFEncoder @(Either EncodeEx) @"r-ASCII" @"r-ASCII"+ instance EncodeF (Either EncodeEx) (Enc xs c BL.ByteString) (Enc ("r-ASCII" ': xs) c BL.ByteString) where encodeF = implEncodeF_ prxyAscii (encodeImpl (\p -> BL8.filter p &&& BL8.filter (not . p)) BL8.head BL8.null) instance (RecreateErr f, Applicative f) => RecreateF f (Enc xs c BL.ByteString) (Enc ("r-ASCII" ': xs) c BL.ByteString) where checkPrevF = implCheckPrevF (asRecreateErr @"r-ASCII" . encodeImpl (\p -> BL8.filter p &&& BL8.filter (not . p)) BL8.head BL8.null) instance Applicative f => DecodeF f (Enc ("r-ASCII" ': xs) c BL.ByteString) (Enc xs c BL.ByteString) where decodeF = implTranP id + encodeImpl :: ((Char -> Bool) -> a -> (a, a))
src/Data/TypedEncoding/Instances/Restriction/UTF8.hs view
@@ -36,15 +36,15 @@ -- >>> import Test.QuickCheck.Instances.Text() -- >>> import Test.QuickCheck.Instances.ByteString() -- >>> import Data.TypedEncoding.Internal.Util (proxiedId)--- >>> let prxyArb = Proxy :: Proxy (Either EncodeEx (Enc '["r-UTF8"] () B.ByteString)) -- >>> :{ -- >>> instance Arbitrary (Enc '["r-UTF8"] () B.ByteString) where -- arbitrary = fmap (fromRight (emptyUTF8B ())) -- . flip suchThat isRight --- . fmap (proxiedId prxyArb . encodeFAll . toEncoding ()) $ arbitrary +-- . fmap (encodeFAll @(Either EncodeEx) @'["r-UTF8"] @(). toEncoding ()) $ arbitrary -- :} --- | empty string is valid utf8+-- | DEPRECATED will be removed in 0.3 +-- empty string is valid utf8 emptyUTF8B :: c -> Enc '["r-UTF8"] c B.ByteString emptyUTF8B c = unsafeSetPayload c "" @@ -52,43 +52,49 @@ -- Conversions -- ----------------- --- Right tst = encodeFAll . toEncoding () $ "Hello World" :: Either EncodeEx (Enc '["r-UTF8"] () B.ByteString)--- tstTxt = byteString2TextS tst---- | Type-safer version of @Data.Text.Encoding.encodeUtf8@------ >>> displ $ text2ByteStringS $ toEncoding () ("text" :: T.Text)--- "MkEnc '[r-UTF8] () (ByteString text)"+-- |+-- | DEPRECATED will be removed in 0.3 +-- +-- use 'Data.TypedEncoding.Conv.Text.Lazy.Encoding.encodeUtf8'+-- and 'Data.TypedEncoding.Conv.Text.utf8Promote' text2ByteStringS :: Enc ys c T.Text -> Enc ("r-UTF8" ': ys) c B.ByteString text2ByteStringS = withUnsafeCoerce TE.encodeUtf8 --- | Type-safer version of Data.Text.Encoding.decodeUtf8+-- | +-- DEPRECATED ----- >>> let Right tst = encodeFAll . toEncoding () $ "Hello World" :: Either EncodeEx (Enc '["r-UTF8"] () B.ByteString)--- >>> displ $ byteString2TextS tst--- "MkEnc '[] () (Text Hello World)"+-- | DEPRECATED will be removed in 0.3 +-- +-- use 'Data.TypedEncoding.Conv.Text.Lazy.Encoding.decodeUtf8'+-- and 'Data.TypedEncoding.Conv.Text.utf8Demote'+--+-- See warning in 'Data.TypedEncoding.Instances.Restriction.ASCII.byteString2TextS'+--+-- Type-safer version of Data.Text.Encoding.decodeUtf8+-- byteString2TextS :: Enc ("r-UTF8" ': ys) c B.ByteString -> Enc ys c T.Text byteString2TextS = withUnsafeCoerce TE.decodeUtf8 --- | Identity property "byteString2TextS . text2ByteStringS == id"--- prop> \t -> t == (fromEncoding . txtBsSIdProp (Proxy :: Proxy '[]) . toEncoding () $ t)+-- | To be removed txtBsSIdProp :: Proxy (ys :: [Symbol]) -> Enc ys c T.Text -> Enc ys c T.Text txtBsSIdProp _ = byteString2TextS . text2ByteStringS --- | Identity property "text2ByteStringS . byteString2TextS == id".------ prop> \(t :: Enc '["r-UTF8"] () B.ByteString) -> t == (bsTxtIdProp (Proxy :: Proxy '[]) $ t)+-- To be removed bsTxtIdProp :: Proxy (ys :: [Symbol]) -> Enc ("r-UTF8" ': ys) c B.ByteString -> Enc ("r-UTF8" ': ys) c B.ByteString bsTxtIdProp _ = text2ByteStringS . byteString2TextS +-- DEPRECATED see above text2ByteStringL :: Enc ys c TL.Text -> Enc ("r-UTF8" ': ys) c BL.ByteString text2ByteStringL = withUnsafeCoerce TEL.encodeUtf8 +-- DEPRECATED+--+-- See warning in 'Data.TypedEncoding.Instances.Restriction.ASCII.byteString2TextS' byteString2TextL :: Enc ("r-UTF8" ': ys) c BL.ByteString -> Enc ys c TL.Text byteString2TextL = withUnsafeCoerce TEL.decodeUtf8 -------------------- Encondings --+-- Encodings -- ----------------- prxyUtf8 = Proxy :: Proxy "r-UTF8"@@ -99,6 +105,7 @@ -- -- >>> encodeFAll . toEncoding () $ "\xc3\xb1" :: Either EncodeEx (Enc '["r-UTF8"] () B.ByteString) -- Right (MkEnc Proxy () "\195\177")+-- -- >>> encodeFAll . toEncoding () $ "\xc3\x28" :: Either EncodeEx (Enc '["r-UTF8"] () B.ByteString) -- Left (EncodeEx "r-UTF8" (Cannot decode byte '\xc3': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream)) --@@ -106,12 +113,20 @@ -- -- prop> \(b :: B.ByteString) -> verEncoding b (fmap (fromEncoding . decodeAll . proxiedId (Proxy :: Proxy (Enc '["r-UTF8"] _ _))) . (encodeFAll :: _ -> Either EncodeEx _). toEncoding () $ b) +instance Encodings (Either EncodeEx) xs grps c B.ByteString => Encodings (Either EncodeEx) ("r-UTF8" ': xs) ("r-UTF8" ': grps) c B.ByteString where+ encodings = encodeFEncoder @(Either EncodeEx) @"r-UTF8" @"r-UTF8"++ instance EncodeF (Either EncodeEx) (Enc xs c B.ByteString) (Enc ("r-UTF8" ': xs) c B.ByteString) where encodeF = implEncodeF_ prxyUtf8 (fmap TE.encodeUtf8 . TE.decodeUtf8') instance (RecreateErr f, Applicative f) => RecreateF f (Enc xs c B.ByteString) (Enc ("r-UTF8" ': xs) c B.ByteString) where checkPrevF = implCheckPrevF (asRecreateErr @"r-UTF8" . fmap TE.encodeUtf8 . TE.decodeUtf8') instance Applicative f => DecodeF f (Enc ("r-UTF8" ': xs) c B.ByteString) (Enc xs c B.ByteString) where decodeF = implTranP id +++instance Encodings (Either EncodeEx) xs grps c BL.ByteString => Encodings (Either EncodeEx) ("r-UTF8" ': xs) ("r-UTF8" ': grps) c BL.ByteString where+ encodings = encodeFEncoder @(Either EncodeEx) @"r-UTF8" @"r-UTF8" instance EncodeF (Either EncodeEx) (Enc xs c BL.ByteString) (Enc ("r-UTF8" ': xs) c BL.ByteString) where encodeF = implEncodeF_ prxyUtf8 (fmap TEL.encodeUtf8 . TEL.decodeUtf8')
src/Data/TypedEncoding/Instances/Support.hs view
@@ -10,7 +10,10 @@ , module Data.TypedEncoding.Internal.Class -- * Combinators , module Data.TypedEncoding.Internal.Instances.Combinators+ -- * Type level conveniences+ , module Data.TypedEncoding.Internal.Util.TypeLits ) where import Data.TypedEncoding.Internal.Types import Data.TypedEncoding.Internal.Class import Data.TypedEncoding.Internal.Instances.Combinators +import Data.TypedEncoding.Internal.Util.TypeLits
src/Data/TypedEncoding/Internal/Class.hs view
@@ -5,27 +5,34 @@ {-# LANGUAGE TypeOperators #-} {-# LANGUAGE FlexibleInstances #-} {-# LANGUAGE FlexibleContexts #-}--- {-# LANGUAGE UndecidableInstances #-}+{-# LANGUAGE UndecidableInstances #-} {-# LANGUAGE ScopedTypeVariables #-}--- {-# LANGUAGE TypeFamilies #-}--- {-# LANGUAGE TypeApplications #-}--- {-# LANGUAGE AllowAmbiguousTypes #-}+{-# LANGUAGE TypeFamilies #-}+{-# LANGUAGE TypeApplications #-}+{-# LANGUAGE AllowAmbiguousTypes #-} module Data.TypedEncoding.Internal.Class ( module Data.TypedEncoding.Internal.Class , module Data.TypedEncoding.Internal.Class.Util , module Data.TypedEncoding.Internal.Class.Encode , module Data.TypedEncoding.Internal.Class.Decode- , module Data.TypedEncoding.Internal.Class.Recreate + , module Data.TypedEncoding.Internal.Class.Recreate + , module Data.TypedEncoding.Internal.Class.Superset + -- * Encoder and Encoding replace EncodeFAll+ , module Data.TypedEncoding.Internal.Class.Encoder ) where import Data.TypedEncoding.Internal.Class.Util import Data.TypedEncoding.Internal.Class.Encode import Data.TypedEncoding.Internal.Class.Decode import Data.TypedEncoding.Internal.Class.Recreate+import Data.TypedEncoding.Internal.Class.Superset+import Data.TypedEncoding.Internal.Class.Encoder import Data.TypedEncoding.Internal.Types (Enc(..) - , withUnsafeCoerce)+ , withUnsafeCoerce+ -- , getPayload+ ) import Data.Functor.Identity import GHC.TypeLits @@ -52,11 +59,13 @@ -- Other classes -- --- | subsets are useful for restriction encodings--- like r-UFT8 but not for other encodings.-class Superset (y :: Symbol) (x :: Symbol) where- inject :: Enc (x ': xs) c str -> Enc (y ': xs) c str- inject = withUnsafeCoerce id+-- | Flatten is more permissive than 'Superset'+-- @+-- instance FlattenAs "r-ASCII" "enc-B64" where -- OK+-- @+-- +-- Now encoded data has form @Enc '["r-ASCII"] c str@ +-- and there is no danger of it begin incorrectly decoded. class FlattenAs (y :: Symbol) (x :: Symbol) where flattenAs :: Enc (x ': xs) c str -> Enc '[y] c str
+ src/Data/TypedEncoding/Internal/Class/Encoder.hs view
@@ -0,0 +1,49 @@+{-# LANGUAGE GADTs #-}+{-# LANGUAGE PolyKinds #-}+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE TypeOperators #-}+{-# LANGUAGE FlexibleInstances #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE MultiParamTypeClasses #-}+{-# LANGUAGE AllowAmbiguousTypes #-}+{-# LANGUAGE TypeApplications #-}+{-# LANGUAGE PartialTypeSignatures #-}+-- {-# LANGUAGE RankNTypes #-}+{-# OPTIONS_GHC -Wno-partial-type-signatures #-}++-- |+-- Internal definition of types+--+-- Possible replacement for EncodeFAll class that works with open definitions such as "r-ban"++module Data.TypedEncoding.Internal.Class.Encoder where++import Data.TypedEncoding.Internal.Types.Enc+-- import Data.TypedEncoding.Internal.Class.Util+import Data.TypedEncoding.Internal.Class.Encode+import Data.TypedEncoding.Internal.Util.TypeLits+import GHC.TypeLits+-- import Data.Symbol.Ascii+++data Encoder f (enc :: [Symbol]) (grps :: [Symbol]) conf str where+ -- | constructor is to be treated as Unsafe to Encode and Decode instance implementations+ -- particular encoding instances may expose smart constructors for limited data types+ ZeroEnc :: Encoder f '[] '[] conf str+ AppendEnc :: (Enc xs conf str -> f (Enc (x ': xs) conf str)) -> Encoder f xs grps conf str -> Encoder f (x ': xs) ((TakeUntil x ":") ': grps) conf str++runEncoder :: forall grps enc f c str . (Monad f) => Encoder f enc grps c str -> Enc ('[]::[Symbol]) c str -> f (Enc enc c str)+runEncoder ZeroEnc enc0 = pure enc0+runEncoder (AppendEnc fn enc) enc0 = + let re :: f (Enc _ c str) = runEncoder enc enc0+ in re >>= fn++encodeFEncoder :: forall f t tg xs gxs c str . (tg ~ TakeUntil t ":", Encodings f xs gxs c str, EncodeF f (Enc xs c str) (Enc (t ': xs) c str)) => Encoder f (t ': xs) (tg ': gxs) c str+encodeFEncoder = AppendEnc (encodeF @f @(Enc xs c str) @(Enc (t ': xs) c str)) encodings++class Encodings f (enc :: [Symbol]) (grps :: [Symbol]) c str where+ encodings :: Encoder f enc grps c str++instance Encodings f '[] '[] c str where+ encodings = ZeroEnc
src/Data/TypedEncoding/Internal/Class/IsStringR.hs view
@@ -3,6 +3,7 @@ {-# LANGUAGE ScopedTypeVariables #-} {-# LANGUAGE TypeApplications #-} {-# LANGUAGE AllowAmbiguousTypes #-}+{-# LANGUAGE FlexibleInstances #-} -- | This Module will be removed in 0.3.x.x in favor of -- "Data.TypedEncoding.Internal.Class.Util.StringConstraints"@@ -67,3 +68,6 @@ -- prop> prop_fromStringToString @TL.Text instance IsStringR TL.Text where toString = TL.unpack ++instance IsStringR [Char] where+ toString = id
src/Data/TypedEncoding/Internal/Class/Recreate.hs view
@@ -52,7 +52,7 @@ recreateAll = runIdentity . recreateFAll --- | Usefull for partially manual recreation+-- | Useful for partially manual recreation recreateFPart_ :: forall f xs xsf c str . (Functor f, RecreateFAll f xs c str) => Proxy xs -> Enc xsf c str -> f (Enc (Append xs xsf) c str) recreateFPart_ p (MkEnc _ conf str) = let re :: f (Enc xs c str) = recreateFAll $ MkEnc Proxy conf str@@ -73,8 +73,6 @@ Enc xsf c str -> Enc (Append xs xsf) c str recreatePart = recreatePart_ (Proxy :: Proxy xs) ---- TODO using RecreateErr typeclass is overkill -- | Recovery errors are expected unless Recovery allows Identity instance class RecreateErr f where
+ src/Data/TypedEncoding/Internal/Class/Superset.hs view
@@ -0,0 +1,131 @@++{-# LANGUAGE MultiParamTypeClasses #-}+{-# LANGUAGE PolyKinds #-}+{-# LANGUAGE DataKinds #-}+{-# LANGUAGE TypeOperators #-}+{-# LANGUAGE FlexibleInstances #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE UndecidableInstances #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeFamilies #-}+-- {-# LANGUAGE TypeApplications #-}+{-# LANGUAGE GADTs #-}+{-# LANGUAGE AllowAmbiguousTypes #-}++module Data.TypedEncoding.Internal.Class.Superset where++import Data.TypedEncoding.Internal.Util.TypeLits+--import Data.TypedEncoding.Internal.Class.Util (displ)++import Data.TypedEncoding.Internal.Types (Enc(..) + , withUnsafeCoerce+ --, unsafeSetPayload+ )+import GHC.TypeLits+import Data.Symbol.Ascii+-- import Data.Proxy++-- $setup+-- >>> :set -XOverloadedStrings -XMultiParamTypeClasses -XDataKinds -XTypeApplications+-- >>> import Data.TypedEncoding.Internal.Class.Util (displ)+-- >>> import Data.TypedEncoding.Internal.Types (unsafeSetPayload)+-- >>> import Data.Text as T+++-- |+-- DEPRECATED see 'IsSuperset'+-- +-- Subsets are useful for restriction encodings+-- like r-UFT8 but should not be used for other encodings.+--+-- This would be dangerous, it would, for example, permit converting encoded binary +-- @"Enc '["enc-"] c ByteString@ to @"Enc '["enc-"] c Text@, decoding which+-- could result in runtime errors.+--+-- The requirement is that that the decoding in the superset+-- can replace the decoding from injected subset.+--+-- @+-- instance Superset "r-ASCII" "enc-B64" where -- DANGEROUS+-- @+--+-- 'inject' is identity on payloads+--+-- @Superset bigger smaller@ reads as @bigger@ is a superset of @smaller@+class Superset (y :: Symbol) (x :: Symbol) where+ inject :: Enc (x ': xs) c str -> Enc (y ': xs) c str+ inject = withUnsafeCoerce id++instance Superset x x where++-- | more permissive than class+type family IsSuperset (y :: Symbol) (x :: Symbol) :: Bool where+ IsSuperset "r-ASCII" "r-ASCII" = 'True+ IsSuperset "r-UTF8" "r-ASCII" = 'True+ IsSuperset "r-UTF8" "r-UTF8" = 'True+ IsSuperset y x = IsSupersetOpen y (TakeUntil x ":") (ToList x)++type family IsSupersetOpen (y :: Symbol) (x :: Symbol) (xs :: [Symbol]) :: Bool++injectInto :: forall y x xs c str . (IsSuperset y x ~ 'True) => Enc (x ': xs) c str -> Enc (y ': xs) c str+injectInto = withUnsafeCoerce id++-- | remove redundant superset right after the top (at second last encoding position)+--+-- >>> displ $ demoteFlattenTop (unsafeSetPayload () "" :: Enc '["r-ASCII", "r-UTF8", "r-boo"] () T.Text)+-- "MkEnc '[r-ASCII,r-boo] () (Text )"+demoteFlattenTop :: forall y x xs c str . (IsSuperset y x ~ 'True) => Enc (x ': y ': xs) c str -> Enc (x ': xs) c str+demoteFlattenTop = withUnsafeCoerce id++-- | add redundant superset right after+--+-- >>> displ $ promoteUnFlattenTop @"r-UTF8" (unsafeSetPayload () "" :: Enc '["r-ASCII", "r-boo"] () T.Text)+-- "MkEnc '[r-ASCII,r-UTF8,r-boo] () (Text )"+promoteUnFlattenTop :: forall y x xs c str . (IsSuperset y x ~ 'True) => Enc (x ': xs) c str -> Enc (x ': y ': xs) c str+promoteUnFlattenTop = withUnsafeCoerce id++-- | remove redunant superset from the top (at last applied encoding position)+--+-- >>> displ $ demoteRemoveTop (unsafeSetPayload () "" :: Enc '["r-UTF8", "r-ASCII", "r-boo"] () T.Text)+-- "MkEnc '[r-ASCII,r-boo] () (Text )"+demoteRemoveTop :: forall y x xs c str . (IsSuperset y x ~ 'True) => Enc (y ': x ' : xs) c str -> Enc (x ': xs) c str+demoteRemoveTop = withUnsafeCoerce id++-- | add redundant superset at the top+--+-- >>> displ $ promoteAddTop @"r-UTF8" (unsafeSetPayload () "" :: Enc '["r-ASCII", "r-boo"] () T.Text)+-- "MkEnc '[r-UTF8,r-ASCII,r-boo] () (Text )"+promoteAddTop :: forall y x xs c str . (IsSuperset y x ~ 'True) => Enc (x ': xs) c str -> Enc (y ': x ' : xs) c str +promoteAddTop = withUnsafeCoerce id++-- | remove redundant superset at bottom (first encoding) position+--+-- >>> displ $ demoteRemoveBot (unsafeSetPayload () "" :: Enc '["r-boo", "r-ASCII", "r-UTF8"] () T.Text)+-- "MkEnc '[r-boo,r-ASCII] () (Text )"+demoteRemoveBot :: (UnSnoc xs ~ '(,) ys y, UnSnoc ys ~ '(,) zs x, IsSuperset y x ~ 'True) => Enc xs c str -> Enc ys c str+demoteRemoveBot = withUnsafeCoerce id++-- | add redundant superset at bottom (first encoding) position+--+-- >>> displ $ promoteAddBot @"r-UTF8" (unsafeSetPayload () "" :: Enc '["r-boo", "r-ASCII"] () T.Text)+-- "MkEnc '[r-boo,r-ASCII,r-UTF8] () (Text )"+promoteAddBot :: forall y x xs c str ys . (UnSnoc xs ~ '(,) ys x, IsSuperset y x ~ 'True) => Enc xs c str -> Enc (Snoc xs y) c str+promoteAddBot = withUnsafeCoerce id++-- | remove redundant superset at second bottom (second encoding) position+--+-- >>> displ $ demoteFlattenBot (unsafeSetPayload () "" :: Enc '["r-boo", "r-UTF8", "r-ASCII"] () T.Text)+-- "MkEnc '[r-boo,r-ASCII] () (Text )"+demoteFlattenBot :: (UnSnoc xs ~ '(,) ys x, UnSnoc ys ~ '(,) zs y, IsSuperset y x ~ 'True) => Enc xs c str -> Enc (Snoc zs x) c str+demoteFlattenBot = withUnsafeCoerce id++-- | add redundant superset at second bottom (second encoding) position+--+-- >>> displ $ promoteUnFlattenBot @"r-UTF8" (unsafeSetPayload () "" :: Enc '["r-boo", "r-ASCII"] () T.Text)+-- "MkEnc '[r-boo,r-UTF8,r-ASCII] () (Text )"+promoteUnFlattenBot :: forall y x xs c str ys . (UnSnoc xs ~ '(,) ys x, IsSuperset y x ~ 'True) => Enc xs c str -> Enc (Snoc (Snoc ys y) x) c str+promoteUnFlattenBot = withUnsafeCoerce id++-- prop_Superset :: forall y x xs c str . (Superset y x, Eq str) => Enc (x ': xs) c str -> Bool+-- prop_Superset x = getPayload x == (getPayload . inject @y @x $ x)+
src/Data/TypedEncoding/Internal/Class/Util.hs view
@@ -50,10 +50,12 @@ class Displ x where displ :: x -> String +-- | TODO 0.3 TO BE REMOVED instance Displ EncAnn where displ = id + instance Displ [EncAnn] where - displ x = "[" ++ L.intercalate "," (map displ x) ++ "]"+ displ x = "[" ++ L.intercalate "," x ++ "]" instance Displ T.Text where displ x = "(Text " ++ T.unpack x ++ ")" instance Displ TL.Text where@@ -62,6 +64,10 @@ displ x = "(ByteString " ++ B.unpack x ++ ")" instance Displ BL.ByteString where displ x = "(ByteString " ++ BL.unpack x ++ ")" ++-- TODO 0.3 replaces: instance Displ EncAnn +-- instance Displ String where+-- displ x = "(String " ++ x ++ ")"
src/Data/TypedEncoding/Internal/Class/Util/StringConstraints.hs view
@@ -63,6 +63,9 @@ toString = TL.unpack +instance ToStrInj String String where+ toString = id+ -- will not work! -- prop> prop_toStringFromString (Proxy :: Proxy B.ByteString) -- instance ToStrInj String B.ByteString where@@ -93,6 +96,8 @@ -- prop> prop_fromStringToString @TL.Text instance ToStrIso String TL.Text where +instance ToStrIso String String where+ -- | -- Used to find exceptions that violated "r-" encoding -- Expected to be used to check encoding of ASCII-7 so Text and ByteString are compatible.@@ -118,7 +123,7 @@ -- >>> B.length $ B8.pack "\160582" -- 1 ----- This instance allows to check elementes of ByteString interpreting them as Char.+-- This instance allows to check elements of ByteString interpreting them as Char. -- -- This may or may not work with UTF8 conversions. -- Safe if restricting to 7bit code points.s
src/Data/TypedEncoding/Internal/Combinators.hs view
@@ -7,12 +7,14 @@ {-# LANGUAGE AllowAmbiguousTypes #-} {-# LANGUAGE FlexibleContexts #-} +-- | Combinators reexported in Data.TypedEncoding module Data.TypedEncoding.Internal.Combinators where import Data.TypedEncoding.Internal.Types import Data.TypedEncoding.Internal.Class.Recreate import Data.TypedEncoding.Internal.Class.Util (SymbolList) import GHC.TypeLits+-- import Data.Proxy -- $setup -- >>> :set -XTypeApplications@@ -48,3 +50,5 @@ UncheckedEnc c str -> Maybe (Either RecreateEx (Enc xs c str)) verifyUncheckedEnc' = verifyUncheckedEnc++
src/Data/TypedEncoding/Internal/Instances/Combinators.hs view
@@ -77,6 +77,7 @@ -- -- >>> verifyWithRead @Word8 "Word8-decimal" (T.pack "256") -- Left "Payload does not satisfy format Word8-decimal: 256"+-- -- >>> verifyWithRead @Word8 "Word8-decimal" (T.pack "123") -- Right "123" verifyWithRead :: forall a str . (IsStringR str, Read a, Show a) => String -> str -> Either String str@@ -87,3 +88,20 @@ in if check then Right x else Left $ "Payload does not satisfy format " ++ msg ++ ": " ++ s +++-- | Convenience function for checking if @str@ decodes properly+-- using @enc@ encoding markers and decoders that can pick decoder based+-- on that marker+verifyDynEnc :: forall s str err1 err2 enc a. (KnownSymbol s, Show err1, Show err2) => + Proxy s -- ^ proxy defining encoding annotation+ -> (Proxy s -> Either err1 enc) -- ^ finds encoding marker @enc@ for given annotation or fails+ -> (enc -> str -> Either err2 a) -- ^ decoder based on @enc@ marker+ -> str -- ^ input+ -> Either EncodeEx str+verifyDynEnc p findenc decoder str = + do+ enc <- asEncodeEx p . findenc $ p+ case decoder enc str of+ Left err -> Left $ EncodeEx p err+ Right r -> Right str
src/Data/TypedEncoding/Internal/Types.hs view
@@ -28,7 +28,6 @@ import Data.TypedEncoding.Internal.Types.UncheckedEnc import Data.TypedEncoding.Internal.Types.Common - import Data.Proxy -- import Data.Functor.Identity import GHC.TypeLits@@ -63,7 +62,11 @@ instance Show EncodeEx where show (EncodeEx prxy a) = "(EncodeEx \"" ++ symbolVal prxy ++ "\" (" ++ show a ++ "))" --- | usefull when manually recreating using recovery+asEncodeEx :: (Show a, KnownSymbol x) => Proxy x -> Either a b -> Either EncodeEx b+asEncodeEx p = either (Left . EncodeEx p) Right +++-- | Useful when manually recreating using recovery encToRecrEx :: EncodeEx -> RecreateEx encToRecrEx (EncodeEx p a) = RecreateEx p a
src/Data/TypedEncoding/Internal/Types/SomeEnc.hs view
@@ -4,8 +4,8 @@ -- {-# LANGUAGE DataKinds #-} -- {-# LANGUAGE TypeOperators #-} -- {-# LANGUAGE FlexibleInstances #-}-{-# LANGUAGE StandaloneDeriving #-}-{-# LANGUAGE TypeApplications #-}+-- {-# LANGUAGE StandaloneDeriving #-}+-- {-# LANGUAGE TypeApplications #-} {-# LANGUAGE AllowAmbiguousTypes #-} {-# LANGUAGE RankNTypes #-}
src/Data/TypedEncoding/Internal/Util/TypeLits.hs view
@@ -24,7 +24,7 @@ -- * "Data.TypedEncoding.Internal.Class.Util" -- * "Data.TypedEncoding.Internal.Types.SomeAnnotation" ----- TODO these will need to get consolidated here+-- (TODO) these will need to get consolidated. module Data.TypedEncoding.Internal.Util.TypeLits where import GHC.TypeLits@@ -66,7 +66,8 @@ type family Drop (n :: Nat) (s :: Symbol) :: Symbol where Drop n s = Concat (LDrop n (ToList s)) --- TODO create TypeList.List+-- TODO create TypeList.List module ?+ -- | :kind! LDrop 6 (ToList "bool: \"r-ban:ff-ff\" | \"r-ban:ffff\"") type family LDrop (n :: Nat) (s :: [k]) :: [k] where LDrop 0 s = s@@ -85,7 +86,6 @@ type family Take (n :: Nat) (s :: Symbol) :: Symbol where Take n s = Concat (LTake n (ToList s)) --- TODO create TypeList.List -- | :kind! LTake 3 (ToList "123456") type family LTake (n :: Nat) (s :: [k]) :: [k] where LTake 0 s = '[]@@ -113,3 +113,34 @@ type family LLengh (s :: [k]) :: Nat where LLengh '[] = 0 LLengh (x ': xs) = 1 + LLengh xs+++-- |+-- >>> :kind! LLast '["1","2","3"]+-- ...+-- = "3"+type family LLast (s :: [Symbol]) :: Symbol where+ LLast '[] = TypeError ('Text "Empty Symbol list not allowed")+ LLast '[x] = x+ LLast (_ ': xs) = LLast xs+++-- |+-- Concat (Snoc '["1","2","3"] "4") +-- ...+-- = "1234"+type family Snoc (s :: [k]) (t :: k) :: [k] where+ Snoc '[] x = '[x]+ Snoc (x ': xs) y = x ': Snoc xs y+++-- |+-- :kind! UnSnoc '["1","2","3"] +type family UnSnoc (s :: [k]) :: ([k], k) where+ UnSnoc '[] = TypeError ('Text "Empty list, no last element") + UnSnoc '[x] = '(,) '[] x+ UnSnoc (x ': xs) = UnSnocHelper x (UnSnoc xs)+++type family UnSnocHelper (s :: k) (t :: ([k], k)) :: ([k], k) where + UnSnocHelper y ('(,) xs x) = '(,) (y ': xs) x
src/Examples/TypedEncoding/Conversions.hs view
@@ -1,31 +1,83 @@ {-# LANGUAGE DataKinds #-} {-# LANGUAGE MultiParamTypeClasses #-}-{-# LANGUAGE TypeOperators #-}+{-# LANGUAGE KindSignatures #-} {-# LANGUAGE FlexibleInstances #-} {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE OverloadedStrings #-} {-# LANGUAGE TypeApplications #-}+-- {-# LANGUAGE PartialTypeSignatures #-}+-- {-# OPTIONS_GHC -Wno-partial-type-signatures #-} -- | Examples or moving between type annotated encodings ----- Modules that define encoding and decoding instances also provide conversion functions.+-- Haskell programs typically make these imports to do String, ByteString, and Text conversions:+--+-- @+-- import qualified Data.Text as T (pack, unpack)+-- import qualified Data.ByteString.Char8 as B8 (pack, unpack)+-- import Data.Text.Encoding (decodeUtf8, encodeUtf8)+-- @+--+-- or corresponding @Lazy@ imports (not shown).+--+-- Enc-specific equivalents can be found in:+--+-- @+-- import qualified Data.TypedEncoding.Conv.Text as EncT (pack, unpack)+-- import qualified Data.TypedEncoding.Conv.ByteString.Char8 as EncB8 (pack, unpack)+-- import Data.TypedEncoding.Conv.Text.Encoding (decodeUtf8, encodeUtf8)+-- @ +--+-- Conversions aim at providing type safety when moving between encoded string-like types.+--+-- __The assumption__ made by `typed-encoding` is that encodings work in equivalent way independently of the payload type.+-- For example, if the following instances exist:+--+-- @+-- EncodeF SomeErr (Enc xs () String) (Enc ("enc-B64" ': xs) () String) +-- EncodeF SomeErr (Enc xs () Text) (Enc ("enc-B64" ': xs) () Text) +-- @ -- --- Currently, these are separate functions, generalization of conversions seems hard.+-- Then @typed-encoding@ expects @pack@ @encodeF@ to commute:+-- +-- @+-- str -- EncT.pack --> txt+-- | |+-- encodeF encodeF+-- | | +-- v v+-- estr -- fmap EncT.pack --> etxt+-- @ ----- These examples discuss handling of __subsets__ (for character sets), __leniency__, and __flattening__. +-- (@unpack@ and $decode$ are expected to satisfy similar diagrams, not shown)+--+-- Basically, it should not matter which type we run the encoding on (other than performance cost).+--+--+-- This module also discusses concepts of __Superset__ (for @"r-"@ encodings), __leniency__, and __flattening__. module Examples.TypedEncoding.Conversions where import Data.TypedEncoding-import qualified Data.TypedEncoding.Instances.Enc.Base64 as EnB64-import qualified Data.TypedEncoding.Instances.Restriction.ASCII as EnASCII--- import qualified Data.TypedEncoding.Instances.Restriction.UTF8 as EnUTF8+import Data.TypedEncoding.Instances.Enc.Base64 () +import Data.TypedEncoding.Instances.Restriction.ASCII ()+import Data.TypedEncoding.Instances.Restriction.UTF8 () +import qualified Data.TypedEncoding.Conv.Text as EncT +import qualified Data.TypedEncoding.Conv.Text.Encoding as EncTe (decodeUtf8)+ import qualified Data.Text as T import qualified Data.ByteString as B+import GHC.TypeLits +import qualified Data.TypedEncoding.Conv.ByteString.Char8 as EncB8+import Data.TypedEncoding.Combinators.Restriction.BoundedAlphaNums ()+ -- $setup--- >>> :set -XOverloadedStrings -XMultiParamTypeClasses -XDataKinds -XTypeApplications--- >>> import Data.TypedEncoding.Instances.Restriction.UTF8 ()+-- >>> :set -XDataKinds -XMultiParamTypeClasses -XKindSignatures -XFlexibleInstances -XFlexibleContexts -XOverloadedStrings -XTypeApplications -XScopedTypeVariables+-- >>> import qualified Data.TypedEncoding.Instances.Enc.Base64 as EnB64 (acceptLenientS)+-- >>> import qualified Data.TypedEncoding.Conv.Text as EncT (pack, utf8Promote, utf8Demote)+-- >>> import qualified Data.TypedEncoding.Conv.ByteString.Char8 as EncB8 (pack, unpack)+-- >>> import qualified Data.TypedEncoding.Conv.Text.Encoding as EncTe (decodeUtf8, encodeUtf8) -- >>> import Data.Proxy -- -- This module contains some ghci friendly values to play with.@@ -36,7 +88,6 @@ -- * Moving between Text and ByteString - eHelloAsciiB :: Either EncodeEx (Enc '["r-ASCII"] () B.ByteString) eHelloAsciiB = encodeFAll . toEncoding () $ "HeLlo world" -- ^ Example value to play with@@ -48,32 +99,85 @@ -- ^ above with either removed helloAsciiT :: Enc '["r-ASCII"] () T.Text-helloAsciiT = EnASCII.byteString2TextS helloAsciiB--- ^ When converted to Text the annotation is preserved.+helloAsciiT = EncTe.decodeUtf8 helloAsciiB+-- ^ +-- We use a tween function to the popular 'Data.Text.Encoding.decodeUtf8' +-- from the @test@ package. ----- Currently separate function is defined for each allowed conversion. +-- Notice the encoding annotation is preserved. ----- >>> displ $ EnASCII.byteString2TextS helloAsciiB+-- >>> displ $ EncTe.decodeUtf8 helloAsciiB -- "MkEnc '[r-ASCII] () (Text HeLlo world)" --- * Subsets +-- * pack and unpack +helloZero :: Enc ('[] :: [Symbol]) () String+helloZero = toEncoding () "Hello"+-- ^ Consider 0-encoding of a 'String', to move it to @Enc '[] () String@ one could try:+--+-- >>> displ . EncT.pack $ helloZero+-- "MkEnc '[] () (Text Hello)"+--+-- this works, but:+-- +-- >>> EncB8.pack helloZero+-- ...+-- ... error: +-- ... Empty Symbol list not allowed+-- ...+--+-- this does not compile. And it should not. @pack@ from "Data.ByteString.Char8" is error prone.+-- It is not an injection as it only considers first 7 bits of information from each 'Char'. +-- I doubt that there are any code examples of its intentional use on a String that is not ASCII. +-- +-- @EncB8.pack@ will not compile unless the encoding is ASCII restricted, this works:+-- +-- >>> fmap (displ . EncB8.pack) . encodeFAll @(Either EncodeEx) @'["r-ASCII"] $ helloZero+-- Right "MkEnc '[r-ASCII] () (ByteString Hello)"+--+-- And the result is a @ByteString@ with bonus annotation describing its content.+++helloRestricted :: Either EncodeEx (Enc '["r-ban:zzzzz"] () B.ByteString)+helloRestricted = fmap EncB8.pack . runEncoder @'["r-ban"] encodings $ toEncoding () "Hello"+-- ^ more interstingly @EncB8.pack@ works fine on "r-" encodings that are subsets of "r-ASCII"+-- this example @"r-ban:zzzzz"@ restricts to 5 alapha-numeric charters all < @'z'@+-- +-- >>> displ <$> helloRestricted+-- Right "MkEnc '[r-ban:zzzzz] () (ByteString Hello)"+--+-- Adding @"r-ASCII"@ annotation on this ByteString would have been redundant since @"r-ban:zzzzz"@ is more+-- restrictive (see Supersets below).+--+-- @unpack@, as expected will put us back in a String keeping the annotation+--+-- >>> fmap (displ . EncB8.unpack) helloRestricted+-- Right "MkEnc '[r-ban:zzzzz] () Hello"+-- +++-- * Supersets+ helloUtf8B :: Enc '["r-UTF8"] () B.ByteString-helloUtf8B = inject helloAsciiB+helloUtf8B = injectInto helloAsciiB -- ^ To get UTF8 annotation, instead of doing this: -- -- >>> encodeFAll . toEncoding () $ "HeLlo world" :: Either EncodeEx (Enc '["r-UTF8"] () B.ByteString) -- Right (MkEnc Proxy () "HeLlo world") -- --- We should be able to convert the ASCII version.+-- We should be able to convert the ASCII version we already have. -- -- This is done using 'Superset' typeclass. ----- @inject@ method accepts proxy to specify superset to use.+-- @injectInto@ method accepts proxy to specify superset to use. ----- >>> displ $ inject @ "r-UTF8" helloAsciiB+-- >>> displ $ injectInto @ "r-UTF8" helloAsciiB -- "MkEnc '[r-UTF8] () (ByteString HeLlo world)"+--+-- Superset is intended for @"r-"@ annotations only, should not be used+-- with general encodings like @"enc-B64"@, it assumes that decoding in the superset+-- can replace the decoding from injected subset. @@ -81,42 +185,61 @@ helloUtf8B64B :: Enc '["enc-B64", "r-UTF8"] () B.ByteString helloUtf8B64B = encodePart @'["enc-B64"] helloUtf8B --- ^ We put Base64 on the UFT8 ByteString+-- ^ We put Base64 on a ByteString which adheres to UTF8 layout -- -- >>> displ $ encodePart_ (Proxy :: Proxy '["enc-B64"]) helloUtf8B -- "MkEnc '[enc-B64,r-UTF8] () (ByteString SGVMbG8gd29ybGQ=)" helloUtf8B64T :: Enc '["enc-B64"] () T.Text-helloUtf8B64T = EnB64.byteString2TextS helloUtf8B64B +helloUtf8B64T = EncT.utf8Demote . EncTe.decodeUtf8 $ helloUtf8B64B -- ^ .. and copy it over to Text.--- but UTF8 would be redundant in Text so the "r-UTF8" is dropped ----- >>> :t EnB64.byteString2TextS helloUtf8B64B--- EnB64.byteString2TextS helloUtf8B64B :: Enc '["enc-B64"] () T.Text+-- >>> displ $ EncTe.decodeUtf8 helloUtf8B64B+-- "MkEnc '[enc-B64,r-UTF8] () (Text SGVMbG8gd29ybGQ=)" ----- Conversely moving back to ByteString recovers the annotation.--- (there could be a choice of a UTF annotation to recover in the future)+-- but UTF8 would be redundant in Text so the "r-UTF8" can be dropped:+--+-- >>> displ . EncT.utf8Demote . EncTe.decodeUtf8 $ helloUtf8B64B+-- "MkEnc '[enc-B64] () (Text SGVMbG8gd29ybGQ=)"+--+-- Conversely moving back to ByteString we need to recover the annotation -- --- >>> :t EnB64.text2ByteStringS helloUtf8B64T--- EnB64.text2ByteStringS helloUtf8B64T--- ... :: Enc '["enc-B64", "r-UTF8"] () B.ByteString+-- >>> :t EncTe.encodeUtf8 helloUtf8B64T+-- ...+-- ... Couldn't match type ‘IsSupersetOpen+-- ... "r-UTF8" "enc-B64" ...+-- ...+--+-- This is not allowed! We need to add the redundant "r-UTF8" back:+--+-- >>> displ . EncTe.encodeUtf8 . EncT.utf8Promote $ helloUtf8B64T+-- "MkEnc '[enc-B64,r-UTF8] () (ByteString SGVMbG8gd29ybGQ=)"+--+-- To achieve type safety, our @encodeUtf8@ and @decodeUtf8@ require "r-UTF8" annotation. +-- But since @Text@ values can always emit @UTF8@ layout, we can simply add and remove+-- these annotations on @Text@ encodings. This approach gives us type level safety over UTF8 encoding/decoding errors. notTextB :: Enc '["enc-B64"] () B.ByteString notTextB = encodeAll . toEncoding () $ "\195\177"--- ^ 'notTextB' a binary, one that does not even represent valid UTF8.+-- ^ 'notTextB' a binary, one that does not even represent a valid UTF8. -- -- >>> encodeAll . toEncoding () $ "\195\177" :: Enc '["enc-B64"] () B.ByteString -- MkEnc Proxy () "w7E=" ----- 'EnB64.byteString2TextS'' is a fuction that allows to convert Base 64 ByteString that is not UTF8.--- --- >>> :t EnB64.byteString2TextS' notTextB--- EnB64.byteString2TextS' notTextB--- ... :: Enc '["enc-B64-nontext"] () T.Text+-- Decoding it to Text is prevented by the compiler ----- The result is annotated as "enc-B64-nontext" which prevents decoding it within 'T.Text' type.--- We can only move it back to ByteString as "enc-B64".-+-- >>> :t EncTe.decodeUtf8 notTextB+-- ...+-- ... error:+-- ... Couldn't match type ...+-- ... "r-UTF8" "enc-B64" ...+-- ...+--+-- This is good because having the payload inside of @Enc '["enc-B64"] () Text@ would allow us+-- to try to decode it to Text (causing runtime errors).+-- +-- We can move it to Text but to do that we will need to forget the "enc-B64" annotation.+-- This can be done, for example, using flattening (see below). -- * Lenient recovery
src/Examples/TypedEncoding/DiySignEncoding.hs view
@@ -73,7 +73,7 @@ helloSigned :: Enc '["my-sign"] () T.Text helloSigned = encodeAll . toEncoding () $ "Hello World" --- | property checks that 'T.Text' values are exected to decode +-- | property checks that 'T.Text' values are expected to decode -- without error after encoding. -- -- prop> \t -> propEncDec
src/Examples/TypedEncoding/Overview.hs view
@@ -3,6 +3,9 @@ {-# LANGUAGE DataKinds #-} {-# LANGUAGE TypeApplications #-} +-- {-# LANGUAGE PartialTypeSignatures #-}+-- {-# OPTIONS_GHC -Wno-partial-type-signatures #-}+ -- | type-encoding overview examples. -- -- This library is concerned with 3 main operations done on strings:@@ -47,6 +50,13 @@ -- -- >>> encodeAll . toEncoding () $ "Hello World" :: Enc '["enc-B64"] () B.ByteString -- MkEnc Proxy () "SGVsbG8gV29ybGQ="+--+-- There is currently an alternative polymorphic way (prototype) to create encodings by /running/ 'Encoder' on 'encodings'.+-- This approach works better with more open / dynamic encoding setup it also provides first class @Encoder@. +-- In future versions encodeAll will either be improved to provide similar flexibility or will be deprecated and removed.+-- +-- >>> displ <$> (runEncoder @'["enc-B64"] encodings $ toEncoding () "Hello" :: Either EncodeEx (Enc '["enc-B64"] () B.ByteString))+-- Right "MkEnc '[enc-B64] () (ByteString SGVsbG8=)" helloB64 :: Enc '["enc-B64"] () B.ByteString helloB64 = encodeAll . toEncoding () $ "Hello World"
src/Examples/TypedEncoding/ToEncString.hs view
@@ -21,7 +21,7 @@ -- @Show@ and @Read@ classes use a very permissive String type. This often results in -- read errors. type-encoding approach provides type safety over decoding process. ----- This module includes a simplified email example. This is a non-homogenious case, +-- This module includes a simplified email example. This is a non-homogeneous case, -- email parts do not have the same encoding. -- -- Examples here could be made more type safe with use of dependently typed@@ -92,7 +92,7 @@ -- >>> let fn a b = if b == "" then a else a <> "." <> b -- >>> let reduce = EnT.foldEncStr @'["r-IPv4"] @'["r-Word8-decimal"] () fn -- >>> displ . reduce . fmap toEncString $ tstIp--- "MkEnc '[r-IPv4] () 128.1.1.10" +-- "MkEnc '[r-IPv4] () 128.1.1.10" -- -- Note lack of type safety here, the same code would work just fine if we added -- 5th field to 'IpV4F' constructor. @@ -175,8 +175,6 @@ type SimplifiedEmailEncB = SimplifiedEmailF (CheckedEnc () B.ByteString) --- TODO--- type SimplifiedEmailEncT = SimplifiedEmailF (CheckedEnc () T.Text) -- | @tstEmail@ contains some simple data to play with tstEmail :: SimplifiedEmail
typed-encoding.cabal view
@@ -4,10 +4,10 @@ -- -- see: https://github.com/sol/hpack ----- hash: 229ee0dd2c77a44f82646983a5b80e9fd45ab517a25feec0482cc0b301b29ee6+-- hash: 2bc32a22f6f7e7c49333ee187e608cb1c101c6711ae2fa7e0ee92c5d524e97ed name: typed-encoding-version: 0.2.1.0+version: 0.2.2.0 synopsis: Type safe string transformations description: See README.md in the project github repository. category: Data, Text@@ -33,6 +33,12 @@ Data.TypedEncoding.Combinators.Restriction.Bool Data.TypedEncoding.Combinators.Restriction.BoundedAlphaNums Data.TypedEncoding.Combinators.Restriction.Common+ Data.TypedEncoding.Conv.ByteString.Char8+ Data.TypedEncoding.Conv.ByteString.Lazy.Char8+ Data.TypedEncoding.Conv.Text+ Data.TypedEncoding.Conv.Text.Encoding+ Data.TypedEncoding.Conv.Text.Lazy+ Data.TypedEncoding.Conv.Text.Lazy.Encoding Data.TypedEncoding.Instances.Do.Sample Data.TypedEncoding.Instances.Enc.Base64 Data.TypedEncoding.Instances.Restriction.ASCII@@ -43,8 +49,10 @@ Data.TypedEncoding.Internal.Class Data.TypedEncoding.Internal.Class.Decode Data.TypedEncoding.Internal.Class.Encode+ Data.TypedEncoding.Internal.Class.Encoder Data.TypedEncoding.Internal.Class.IsStringR Data.TypedEncoding.Internal.Class.Recreate+ Data.TypedEncoding.Internal.Class.Superset Data.TypedEncoding.Internal.Class.Util Data.TypedEncoding.Internal.Class.Util.StringConstraints Data.TypedEncoding.Internal.Combinators