diff --git a/README.md b/README.md new file mode 100644 --- /dev/null +++ b/README.md @@ -0,0 +1,177 @@ + + +[![Build Status](https://travis-ci.org/qfpl/waargonaut.svg?branch=master)](https://travis-ci.org/qfpl/waargonaut) + +# Waargonaut + +Flexible, precise, and efficient JSON decoding/encoding library. This package +provides a plethora of tools for decoding, encoding, and manipulating JSON data. + +## Features + +* Fully RFC compliant, with property based testing used to ensure the desired + invariants are preserved. + +* Encoders and Decoders are values, they are not tied to a typeclass and as such + you are not tied to a single interpretation of how a particular type + "_should_" be handled. + +* No information is discarded on parsing. Trailing whitespace, and any + formatting whitespace (carriage returns etc) are all preserved. + +* A history keeping zipper is used for Decoding, providing precise control of + how _you_ decode _your_ JSON data. With informative error messages if things + don't go according to plan. + +* Flexible and expressive Decoder & Encoder functions let you parse and build + the JSON structures _you_ require, with no surprises. + +* BYO parsing library, the parser built into Waargonaut does not tie you to a + particular parsing library. With the caveat that your parsing library must + have an instance of `CharParsing` from the [parsers](https://hackage.haskell.org/package/parsers) package. + +* Generic functions are provided to make the creation of Encoders and Decoders + are bit easier. However these _are_ tied to typeclasses, so they do come with + some assumptions. + +* Lenses, Prisms, and Traversals are provided to allow you to investigate and + manipulate the JSON data structures to your hearts content, without breaking + the invariants. + +* The awesome work on succinct data structures by John Ky and [Haskell Works](https://github.com/haskell-works/) + is used to power the decoder. Providing the same zipper capabilities and + property based guarantees, but with all the speed and efficiency capabilities + that succinct data structures have to offer. + +## Example + +- Data Structure: +```haskell +data Image = Image + { _imageWidth :: Int + , _imageHeight :: Int + , _imageTitle :: Text + , _imageAnimated :: Bool + , _imageIDs :: [Int] + } +``` + +- Encoder: +```haskell +encodeImage :: Applicative f => Encoder f Image +encodeImage = E.mapLikeObj $ \img -> + E.intAt "Width" (_imageWidth img) + . E.intAt "Height" (_imageHeight img) + . E.textAt "Title" (_imageTitle img) + . E.boolAt "Animated" (_imageAnimated img) + . E.listAt E.int "IDs" (_imageIDs img) +``` + +- Decoder: +```haskell +imageDecoder :: Monad f => D.Decoder f Image +imageDecoder = D.withCursor $ \curs -> do + -- Move down into the JSON object. + io <- D.down curs + -- We need individual values off of our object, + Image + <$> D.fromKey "Width" D.int io + <*> D.fromKey "Height" D.int io + <*> D.fromKey "Title" D.text io + <*> D.fromKey "Animated" D.bool io + <*> D.fromKey "IDs" (D.list D.int) io +``` + +### Zippers + +Waargonaut uses zippers for its decoding which allows for precise control in +how you interrogate your JSON input. Take JSON structures and decode them +precisely as you require: + +##### Input: + +```JSON +["a","fred",1,2,3,4] +``` + +##### Data Structure: + +```haskell +data Foo = Foo (Char,String,[Int]) +``` + +##### Decoder: + +The zipper starts the very root of the JSON input, we tell it to move 'down' +into the first element. +```haskell +fooDecoder :: Monad f => Decoder f Foo +fooDecoder = D.withCursor $ \cursor -> do + fstElem <- D.down cursor +``` +From the first element we can then decode the focus of the zipper using a +specific decoder: +```haskell + aChar <- D.focus D.unboundedChar fstElem +``` +The next thing we want to decode is the second element of the array, so we +move right one step or tooth, and then attempt to decode a string at the +focus. +```haskell + aString <- D.moveRight1 fstElem >>= D.focus D.string +``` +Finally we want to take everything else in the list and combine them into a +single list of Int values. Starting from the first element, we move right +two positions (over the char and the string elements), then we use one of +the provided decoder functions that will repeatedly move in a direction and +combine all of the elements it can until it can no longer move. +```haskell + aIntList <- D.moveRightN 2 fstElem >>= D.rightwardSnoc [] D.int +``` +Lastly, we build the Foo using the decoded values. +```haskell + pure $ Foo (aChar, aString, aIntList) +``` + +The zipper stores the history of your movements, so any errors provide +information about the path they took prior to encountering an error. Making +debugging precise and straight-forward. + +### Property Driven Development + +This library is built to parse and produce JSON in accordance with the [RFC +8259](https://tools.ietf.org/html/rfc8259) standard. The data structures, +parser, and printer are built to satify the [Round Trip Property](https://teh.id.au/posts/2017/06/07/round-trip-property/): + +Which may be expressed using the following pseudocode: + +``` +parse . print = id +``` +This indicates that any JSON produced by this library will be parsed back in as +the exact data structure that produced it. This includes whitespace such as +carriage returns and trailing whitespace. There is no loss of information. + +There is also this property, again in pseudocode: + +``` +print . parse . print = print +``` +This states that the printed form of the JSON will not change will be identical +after parsing and then re-printing. There is no loss of information. + +This provides a solid foundation to build upon. + +**NB:** The actual code will of course return values that account for the +possibility of failure. Computers being what they are. + +### TODO(s) + +In no particular order... + +- [ ] improve/bikeshed encoding object api +- [ ] gather feedback on tests/benchmarks that matter +- [ ] provide testing functions so users can be more confident in their Encoder/Decoder construction +- [x] (feedback required) documentation in the various modules to explain any weirdness or things that users may consider to be 'missing' or 'wrong'. +- [x] (mostly) provide greater rationale behind lack of reliance in typeclasses for encoding/decoding +- [ ] provide functions to add preset whitespace layouts to encoded json. diff --git a/changelog.md b/changelog.md --- a/changelog.md +++ b/changelog.md @@ -1,5 +1,11 @@ # Revision history for waargonaut -## 0.1.0.0 -- YYYY-mm-dd +## 0.2.0.0 -- 2018-11-06 + +* Provide more precise errors from Decoder for missing or invalid keys +* Removed a parameter from `KeyDecodeFailed` error constructor +* Fix issue where printing the zipper movements had left and right movement arrows swapped. + +## 0.1.0.0 -- 2018-11-01 * First version. Released on an unsuspecting world. diff --git a/src/Waargonaut.hs b/src/Waargonaut.hs --- a/src/Waargonaut.hs +++ b/src/Waargonaut.hs @@ -121,10 +121,10 @@ -- @ -- personDecoder2 :: Monad f => Decoder f Person -- personDecoder2 = Person --- <$> D.atKey "name" D.text --- <*> D.atKey "age" D.int --- <*> D.atKey "address" D.text --- <*> D.atKey "numbers" (D.list D.int) +-- \<$> D.atKey "name" D.text +-- \<*> D.atKey "age" D.int +-- \<*> D.atKey "address" D.text +-- \<*> D.atKey "numbers" (D.list D.int) -- @ -- -- Using the 'Waargonaut.Decode.atKey' function which tries to handle those basic movements for us diff --git a/src/Waargonaut/Decode.hs b/src/Waargonaut/Decode.hs --- a/src/Waargonaut/Decode.hs +++ b/src/Waargonaut/Decode.hs @@ -86,7 +86,7 @@ Snoc, cons, lens, modifying, preview, snoc, traverseOf, - view, (.~), (^.), + view, (.~), (^.), (#), _Wrapped) import Prelude (Bool, Bounded, Char, @@ -108,7 +108,7 @@ import Control.Monad.State (MonadState) import Control.Error.Util (note) -import Control.Monad.Error.Hoist (()) +import Control.Monad.Error.Hoist ((),()) import Data.Either (Either (..)) import Data.Foldable (foldl) @@ -148,7 +148,7 @@ import qualified HaskellWorks.Data.Json.Cursor as JC -import Waargonaut.Decode.Error (DecodeError (..), +import Waargonaut.Decode.Error (DecodeError (..), AsDecodeError (..), Err (..)) import Waargonaut.Decode.ZipperMove (ZipperMove (..)) @@ -423,13 +423,16 @@ => Text -> JCurs -> DecodeResult f JCurs -moveToKey k c = +moveToKey k c = do -- Tease out the key - focus text c >>= \k' -> if k' == k -- Are we at the key we want to be at ? - -- if we are, then move into the THING at the key - then moveRight1 c - -- if not, then jump to the next key index, the adjacent sibling is opening of the value of the current key - else moveRightN 2 c >>= moveToKey k + k' <- DI.try (focus text c) (_KeyDecodeFailed # ()) + + -- Are we at the key we want to be at ? + if k' == k + -- Then move into the THING at the key + then moveRight1 c + -- Try jump to the next key index + else ( DI.try (moveRightN 2 c) (_KeyNotFound # k) ) >>= moveToKey k -- | Move to the first occurence of this key, as per 'moveToKey' and then -- attempt to run the given 'Decoder' on that value, returning the result. diff --git a/src/Waargonaut/Decode/Error.hs b/src/Waargonaut/Decode/Error.hs --- a/src/Waargonaut/Decode/Error.hs +++ b/src/Waargonaut/Decode/Error.hs @@ -29,7 +29,7 @@ -- data DecodeError = ConversionFailure Text - | KeyDecodeFailed Text + | KeyDecodeFailed | KeyNotFound Text | FailedToMove ZipperMove | NumberOutOfBounds JNumber @@ -41,7 +41,7 @@ class AsDecodeError r where _DecodeError :: Prism' r DecodeError _ConversionFailure :: Prism' r Text - _KeyDecodeFailed :: Prism' r Text + _KeyDecodeFailed :: Prism' r () _KeyNotFound :: Prism' r Text _FailedToMove :: Prism' r ZipperMove _NumberOutOfBounds :: Prism' r JNumber @@ -67,10 +67,10 @@ ) _KeyDecodeFailed - = L.prism KeyDecodeFailed + = L.prism (const KeyDecodeFailed) (\x -> case x of - KeyDecodeFailed y -> Right y - _ -> Left x + KeyDecodeFailed -> Right () + _ -> Left x ) _KeyNotFound diff --git a/src/Waargonaut/Decode/ZipperMove.hs b/src/Waargonaut/Decode/ZipperMove.hs --- a/src/Waargonaut/Decode/ZipperMove.hs +++ b/src/Waargonaut/Decode/ZipperMove.hs @@ -34,10 +34,10 @@ ppZipperMove :: ZipperMove -> Doc a ppZipperMove m = case m of U -> WL.text "up/" - D -> WL.text "into\\" + D -> WL.text "down\\" - (L n) -> WL.text "->-" <+> ntxt n - (R n) -> WL.text "-<-" <+> ntxt n + (L n) -> WL.text "-<-" <+> ntxt n + (R n) -> WL.text "->-" <+> ntxt n (DAt k) -> WL.text "into\\" <+> itxt "key" k (Item t) -> WL.text "-::" <+> itxt "item" t diff --git a/src/Waargonaut/Generic.hs b/src/Waargonaut/Generic.hs --- a/src/Waargonaut/Generic.hs +++ b/src/Waargonaut/Generic.hs @@ -232,14 +232,14 @@ -- Becomes: -- -- @ --- (proxy mkDecoder @GWaarg) :: Monad f => Decoder f Image +-- (proxy mkDecoder \@GWaarg) :: Monad f => Decoder f Image -- @ -- -- You can also use the @TypeApplications@ directly on the 'mkEncoder' or 'mkDecoder' function: -- -- @ --- mkEncoder @GWaarg :: Applicative f => Tagged GWaarg (Encoder f Image) --- mkDecoder @GWaarg :: Monad f => Tagged GWaarg (Decoder f Image) +-- mkEncoder \@GWaarg :: Applicative f => Tagged GWaarg (Encoder f Image) +-- mkDecoder \@GWaarg :: Monad f => Tagged GWaarg (Decoder f Image) -- @ -- diff --git a/test/Decoder.hs b/test/Decoder.hs --- a/test/Decoder.hs +++ b/test/Decoder.hs @@ -4,7 +4,7 @@ ( decoderTests ) where -import Prelude (Char, Int, String, print) +import Prelude (Char, Int, String, print, (==)) import Control.Applicative (liftA3, (<$>)) import Control.Category ((.)) @@ -24,6 +24,7 @@ import Waargonaut.Decode.Internal (ppCursorHistory) import qualified Waargonaut.Decode as D +import qualified Waargonaut.Decode.Error as D import Types.Common (imageDecodeSuccinct, parseBS, testImageDataType) @@ -33,7 +34,37 @@ [ testCase "Decode Image (test1.json)" decodeTest1Json , testCase "Decode [Int]" decodeTest2Json , testCase "Decode (Char,String,[Int])" decodeTest3Json + , testCase "Decode Fail with Bad Key" decodeTestBadObjKey + , testCase "Decode Fail with Missing Key" decodeTestMissingObjKey ] + +decodeTestMissingObjKey :: Assertion +decodeTestMissingObjKey = do + let + j = "{\"foo\":33}" + + d = D.withCursor $ D.down >=> D.fromKey "bar" D.int + + r <- D.runDecode d parseBS (D.mkCursor j) + + Either.either + (\(e, _) -> assertBool "Incorrect Error - Expected KeyDecodeFailed" (e == D.KeyNotFound "bar") ) + (\_ -> assertFailure "Expected Error!") + r + +decodeTestBadObjKey :: Assertion +decodeTestBadObjKey = do + let + j = "{33:33}" + + d = D.withCursor $ D.down >=> D.fromKey "foo" D.int + + r <- D.runDecode d parseBS (D.mkCursor j) + + Either.either + (\(e, _) -> assertBool "Incorrect Error - Expected KeyDecodeFailed" (e == D.KeyDecodeFailed) ) + (\_ -> assertFailure "Expected Error!") + r decodeTest1Json :: Assertion decodeTest1Json = D.runPureDecode imageDecodeSuccinct parseBS . D.mkCursor diff --git a/waargonaut.cabal b/waargonaut.cabal --- a/waargonaut.cabal +++ b/waargonaut.cabal @@ -10,7 +10,7 @@ -- PVP summary: +-+------- breaking API changes -- | | +----- non-breaking API additions -- | | | +--- code changes with no API change -version: 0.1.0.0 +version: 0.2.0.0 -- A short (one-line) description of the package. synopsis: JSON wrangling @@ -32,7 +32,7 @@ maintainer: oᴉ˙ldɟb@uɐǝs -- A copyright notice. -copyright: Copyright (C) 2017 Commonwealth Scientific and Industrial Research Organisation (CSIRO) +copyright: Copyright (C) 2018 Commonwealth Scientific and Industrial Research Organisation (CSIRO) category: Parser, Web, JSON @@ -41,6 +41,7 @@ -- Extra files to be distributed with the package, such as examples or a -- README. extra-source-files: changelog.md + , README.md , test/json-data/numbers.json , test/json-data/test1.json , test/json-data/test2.json