packages feed

typed-encoding 0.3.0.1 → 0.3.0.2

raw patch · 9 files changed

+141/−31 lines, 9 files

Files

ChangeLog.md view
@@ -4,6 +4,13 @@  - `Data.TypedEncoding.Internal.Class.IsStringR` expected to be be changed / replaced - (post 0.3) "enc-B64" will be moved to a different package (more distant goal)+- Improved constrains in `Data.TypedEncoding.Conv` modules++## 0.3.0.2++- Added documentation to `Data.TypedEncoding.Conv` outlining current limitations, challenges of conversions.+- Improved readme+  ## 0.3.0.1
README.md view
@@ -58,21 +58,19 @@ phone' = ... ``` +## Goals and limitations -## Examples +The main goal is to provide improved type safety for programs that use string encodings and +transformations.  _Not to provide encoding implementation type safety_. +_Encoding and string manipulation libraries are typically well established and tested, type safety is really needed at the usage site, not at the implementation site_. -Here are some code examples:+This library approach is to fight issues with (value level) strings using (type level) strings. Using `Symbol`-s effectively forces us to play the orphan instances game.   +One of the long term goals is for this library to provide combinator alternatives to typeclass polymorphism so that the orphan instances are more of a convenience and not the necessity.   -- [Overview](src/Examples/TypedEncoding/Overview.hs)-- [Conversions between encodings](src/Examples/TypedEncoding/Conversions.hs)-- [Adding a new encoding, error handling](src/Examples/TypedEncoding/DiySignEncoding.hs)-- [To and from string conversions](src/Examples/TypedEncoding/ToEncString.hs)-- [Unsafe - working inside encodings](src/Examples/TypedEncoding/Unsafe.hs)-  -## Hackage+## Examples  -https://hackage.haskell.org/package/typed-encoding+Please see `Examples.TypedEncoding` it the module list.   ## Other encoding packages
src/Data/TypedEncoding.hs view
@@ -104,16 +104,9 @@ -- ... and needed conversions.  -- -- Conversion combinator module structure is similar to one found in /text/ and /bytestring/ packages--- And can be found (since 0.2.2) in------ * "Data.TypedEncoding.Conv.Text"--- * "Data.TypedEncoding.Conv.Text.Encoding"--- * "Data.TypedEncoding.Conv.Text.Lazy"    --- * "Data.TypedEncoding.Conv.Text.Lazy.Encoding"--- * "Data.TypedEncoding.Conv.ByteString.Char8"--- * "Data.TypedEncoding.Conv.ByteString.Lazy.Char8"+-- Please see comments in "Data.TypedEncoding.Conv" for more information. ----- This list is not intended to be exhaustive, rather separate libraries+-- The instance list is not intended to be exhaustive, rather separate libraries -- can provide instances for other encodings and transformations. -- -- = New encoding instance creation
+ src/Data/TypedEncoding/Conv.hs view
@@ -0,0 +1,81 @@+++-- |+-- Conversion combinator module structure is similar to one found in /text/ and /bytestring/ packages+-- And can be found nested under this module:+--+-- * "Data.TypedEncoding.Conv.Text"+-- * "Data.TypedEncoding.Conv.Text.Encoding"+-- * "Data.TypedEncoding.Conv.Text.Lazy"    +-- * "Data.TypedEncoding.Conv.Text.Lazy.Encoding"+-- * "Data.TypedEncoding.Conv.ByteString.Char8"+-- * "Data.TypedEncoding.Conv.ByteString.Lazy.Char8"+--+-- Two goals of conversions are:+--+-- * provide a way to easily convert encoded data directly between /text/ and /bytestring/ types.+-- * provide added type safety for string conversions+--+-- == Enc conversions +-- +-- Consider defining a conversion function @:: Enc xs c str1 -> f (Enc xs c str2)@.+--+-- One challenge is how do we know that @xs@ is a valid encoding stack also for @str2@?+-- Should we constrain that?      +--+-- This is made even more difficult because this library plays (has to) games with orphan instances.+--+-- The other challenge is how to ensure that if the destination decides to partially or fully decode, then+-- it will do so without errors and the decoding will be meaningful.+--+-- Current definition is not optimal, it was selected because it works with a wide range of+-- encodings (all @"r-"@ encodings, all non-@"r-"@ encodings available in this version of the library).+-- However, future versions should try to improve on this.    +--+-- == Type Safety+--+-- Consider the following diagram(s) of popular /text/ and /bytestring/ conversion functions:+--+-- @+--  String -> B8.pack ->   ByteString+--   ^                    ^     |+--   |                    | encodeUtf8+--  id                    |     |+--   |               decodeUtf8 |+--   v                    |     v+--  String -> T.pack ->     Text+-- @+--+-- and the reverse of these:+--+-- @+--  String <- B8.unpack <- ByteString+--   ^                      ^     |+--   |                      | encodeUtf8+--  id                      |     |+--   |                 decodeUtf8 |+--   v                      |     v+--  String <- T.unpack  <-    Text+-- @+--+-- These diagrams actually do not commute.  This makes it easy to code bugs+-- that are hard to find and hard to troubleshoot.+--+-- Well, they actually do commute on a subset of @String@ / @Text@ values:+--+-- @+-- Enc '["r-ASCII"] c String+-- Enc '["r-ASCII"] c ByteString+-- Enc '["r-ASCII"] c Text+-- @ +--+-- This is because /UTF8/ is backward compatible with /ASCII(-7)/ and we speak /UTF8/ when+-- converting and and from @Text@.+-- +-- This is the reason why this version of /typed-encoding/ decided on using+-- "r-ASCII" to constrain when wrapping @B8.pack@ and @B8.unpack@ in +-- "Data.TypedEncoding.Conv.ByteString.Char8"+--+-- This approach seems to be limiting and future versions will work on relaxing it.++module Data.TypedEncoding.Conv where
src/Data/TypedEncoding/Conv/ByteString/Char8.hs view
@@ -4,7 +4,7 @@ {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE PolyKinds #-} -- removes need to annotate kinds as [Symbol] --- | Encoding safe version of "Data.ByteString.Char8"+-- | Encoding safe(r) version of "Data.ByteString.Char8" -- @since 0.2.2.0 module Data.TypedEncoding.Conv.ByteString.Char8 where @@ -20,10 +20,19 @@ --  -- This assumes that each of the encodings in @xs@ work work equivalently in @String@ and @ByteString@. ----- Because of how @ByteString.Char8.pack@ works, the first encoding (last in the list) must restrict character set to a subset of @ASCII@. +-- This function assumes that encoding stack @xs@ does not shift characters outside of Char8 range.  +-- This is obviously true for all "r-" types+-- it is also true for any "enc-" and "do-" encodings currently available in this package. However, is possible to define a "do-" or "enc-" encodings that violate that. +-- Future versions of /type-encoding/ are likely +-- to introduce constraints to guard this aspect of the type safety better.  ----- Currently this uses (an over-conservative) @"r-ASCII"@ superset constraint, in the future, this could be relaxed to a superset of /ASCII/, e.g. /r-CHAR8/ when one is in place.+-- This function also (currently) does not insist that @xs@ is a valid encoding stack for @ByteString@. +-- This will be reexamined in the future, possibly offering alternatives with different safety levels. --+-- Currently this uses (an over-conservative) @"r-ASCII"@ superset constraint.+--+-- See "Data.TypedEncoding.Conv" for more detailed discussion.+-- -- >>> :t pack (undefined :: Enc '["r-bar", "r-foo"] () String) -- ... -- ... error:@@ -33,12 +42,21 @@ -- -- >>> displ $ pack (unsafeSetPayload () "Hello" :: Enc '["r-bar", "r-ASCII"] () String) -- "Enc '[r-bar,r-ASCII] () (ByteString Hello)"+--+-- @since 0.2.2.0 pack :: (Knds.LLast xs ~ t, IsSuperset "r-ASCII" t ~ 'True) => Enc xs c String -> Enc xs c B8.ByteString pack = unsafeChangePayload B8.pack  -- | @unpack@ on encoded strings. -- -- See 'pack'+--+-- Similarly to 'pack' this makes assumptions on what the encoding stack is allowed to do. These are not type checked.+-- Again, this is safe with any stack that uses "r-" only encodings. +-- Future versions of /type-encoding/ are likely +-- to introduce constraints to guard this aspect of the type safety better. +--+-- @since 0.2.2.0 unpack :: (Knds.LLast xs ~ t, IsSuperset "r-ASCII" t ~ 'True) => Enc xs c B8.ByteString -> Enc xs c String unpack = unsafeChangePayload B8.unpack
src/Data/TypedEncoding/Conv/ByteString/Lazy/Char8.hs view
@@ -16,14 +16,11 @@ -- >>> :set -XDataKinds -XTypeApplications -XOverloadedStrings  -- | --- Type safe version of 'BL8.pack'.------ :t pack (undefined :: Enc '["r-bar", "r-ASCII"] () String)--- :t pack (undefined :: Enc '["r-bar", "r-foo"] () String)+-- Lazy version of 'Data.TypedEncoding.Conv.ByteString.Char8.pack'. pack :: (Knds.LLast xs ~ t, IsSuperset "r-ASCII" t ~ 'True) => Enc xs c String -> Enc xs c BL8.ByteString pack = unsafeChangePayload BL8.pack  -- | --- Type safe version of 'BL8.unpack'.+-- Lazy version of 'Data.TypedEncoding.Conv.ByteString.Char8.unpack'. unpack :: (Knds.LLast xs ~ t, IsSuperset "r-ASCII" t ~ 'True) => Enc xs c BL8.ByteString -> Enc xs c String unpack = unsafeChangePayload BL8.unpack
src/Data/TypedEncoding/Conv/Text/Encoding.hs view
@@ -70,11 +70,26 @@ -- -- prop> \x -> getPayload x == (getPayload . encodeUtf8 . decodeUtf8 @ '["r-ASCII"] @() $ x) -- prop> \x -> getPayload x == (getPayload . decodeUtf8 . encodeUtf8 @ '["r-ASCII"] @() $ x)+--+-- Similarly to 'Data.TypedEncoding.Conv.ByteString.Char8.pack' this function makes unverified assumption+-- that the encoding stack @xs@ does invalidate UTF8 byte layout.  This is safe for any "r-" encoding as well+-- as any of the "enc-" and "do-" encodings that can be currently found in this library. +-- Future versions of this method are likely to introduce constraints that guarantee better type safety.+--+-- See "Data.TypedEncoding.Conv" for more detailed discussion.+--+-- @since 0.2.2.0 decodeUtf8 :: forall xs c t. (LLast xs ~ t, IsSuperset "r-UTF8" t ~ 'True) => Enc xs c B.ByteString -> Enc xs c T.Text  decodeUtf8 = withUnsafe (fmap TE.decodeUtf8)  -- | -- >>> displ $ encodeUtf8 $ utf8Promote $ toEncoding () ("text" :: T.Text) -- "Enc '[r-UTF8] () (ByteString text)"+--+-- See 'decodeUtf8'.  Similar type safety concerns apply.+--+-- See "Data.TypedEncoding.Conv" for more detailed discussion.+--+-- @since 0.2.2.0 encodeUtf8 :: forall xs c t.  (LLast xs ~ t, IsSuperset "r-UTF8" t ~ 'True) => Enc xs c T.Text -> Enc xs c B.ByteString  encodeUtf8 = withUnsafe (fmap TE.encodeUtf8)
src/Data/TypedEncoding/Conv/Text/Lazy/Encoding.hs view
@@ -19,10 +19,10 @@ import           Data.TypedEncoding.Unsafe (withUnsafe)  -+-- | Lazy version of 'Data.TypedEncoding.Conv.Text.Encoding.decodeUtf8' decodeUtf8 :: forall xs c t. (LLast xs ~ t, IsSuperset "r-UTF8" t ~ 'True) => Enc xs c BL.ByteString -> Enc xs c TL.Text  decodeUtf8 = withUnsafe (fmap TEL.decodeUtf8) -+-- | Lazy version of 'Data.TypedEncoding.Conv.Text.Encoding.encodeUtf8' encodeUtf8 :: forall xs c t.  (LLast xs ~ t, IsSuperset "r-UTF8" t ~ 'True) => Enc xs c TL.Text -> Enc xs c BL.ByteString  encodeUtf8 = withUnsafe (fmap TEL.encodeUtf8)
typed-encoding.cabal view
@@ -4,10 +4,10 @@ -- -- see: https://github.com/sol/hpack ----- hash: b3491bbd06ee3c10f1a7f4c04cd6ab96fe78edacd8d4e8023e47e303ce1b12b0+-- hash: 004e70cb6bb533b5bf1a14ddf696fa82800c2c9416d608734605d3e6103f521e  name:           typed-encoding-version:        0.3.0.1+version:        0.3.0.2 synopsis:       Type safe string transformations description:    See README.md in the project github repository. category:       Data, Text@@ -58,6 +58,7 @@       Data.TypedEncoding.Common.Types.Unsafe       Data.TypedEncoding.Common.Types.Validation       Data.TypedEncoding.Common.Util.TypeLits+      Data.TypedEncoding.Conv       Data.TypedEncoding.Conv.ByteString.Char8       Data.TypedEncoding.Conv.ByteString.Lazy.Char8       Data.TypedEncoding.Conv.Text