diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -6,8 +6,8 @@ The API consists of just a few functions and achieves performance that gets up to **3 times** better than that of "aeson" in typical use-cases. -In cases where we deal with really large documents (60MB) the performance -of "aeson" becomes more comparable. +In cases where we deal with very large documents the performance difference +becomes less drastic. # Performance @@ -16,35 +16,42 @@ Following are the benchmark results comparing the performance of encoding typical documents using this library, "aeson" and "buffer-builder". Every approach is measured on Twitter API data of sizes ranging from roughly 1kB to 60MB. -"lazy-aeson" stands for "aeson" producing a lazy bytestring, -otherwise it's strict. +"aeson" stands for "aeson" producing a strict bytestring, +"lazy-aeson" - lazy bytestring, +"lazy-aeson-untrimmed-32k" - lazy bytestring using an untrimmed builder strategy with allocation of 32k. "buffer-builder" is another library providing an alternative JSON encoder. ``` -jsonifier/1kB mean 2.087 μs ( +- 260.0 ns ) -jsonifier/6kB mean 12.33 μs ( +- 222.2 ns ) -jsonifier/60kB mean 118.3 μs ( +- 1.991 μs ) -jsonifier/600kB mean 1.270 ms ( +- 38.92 μs ) -jsonifier/6MB mean 20.53 ms ( +- 1.042 ms ) -jsonifier/60MB mean 194.9 ms ( +- 15.04 ms ) -aeson/1kB mean 6.542 μs ( +- 199.2 ns ) -aeson/6kB mean 31.25 μs ( +- 494.5 ns ) -aeson/60kB mean 261.7 μs ( +- 8.044 μs ) -aeson/600kB mean 3.395 ms ( +- 114.6 μs ) -aeson/6MB mean 30.71 ms ( +- 701.0 μs ) -aeson/60MB mean 277.1 ms ( +- 4.776 ms ) -lazy-aeson/1kB mean 6.423 μs ( +- 83.69 ns ) -lazy-aeson/6kB mean 30.74 μs ( +- 607.0 ns ) -lazy-aeson/60kB mean 259.1 μs ( +- 4.890 μs ) -lazy-aeson/600kB mean 2.511 ms ( +- 18.71 μs ) -lazy-aeson/6MB mean 24.92 ms ( +- 95.36 μs ) -lazy-aeson/60MB mean 248.6 ms ( +- 736.6 μs ) -buffer-builder/1kB mean 5.512 μs ( +- 77.39 ns ) -buffer-builder/6kB mean 30.29 μs ( +- 459.9 ns ) -buffer-builder/60kB mean 307.0 μs ( +- 3.640 μs ) -buffer-builder/600kB mean 3.001 ms ( +- 75.72 μs ) -buffer-builder/6MB mean 33.05 ms ( +- 336.3 μs ) -buffer-builder/60MB mean 308.5 ms ( +- 3.489 ms ) +jsonifier/1kB mean 2.037 μs ( +- 15.93 ns ) +jsonifier/6kB mean 12.68 μs ( +- 272.7 ns ) +jsonifier/60kB mean 122.7 μs ( +- 3.081 μs ) +jsonifier/600kB mean 1.304 ms ( +- 16.41 μs ) +jsonifier/6MB mean 20.98 ms ( +- 825.8 μs ) +jsonifier/60MB mean 197.1 ms ( +- 14.81 ms ) +aeson/1kB mean 6.470 μs ( +- 118.5 ns ) +aeson/6kB mean 31.42 μs ( +- 680.3 ns ) +aeson/60kB mean 265.0 μs ( +- 5.558 μs ) +aeson/600kB mean 3.435 ms ( +- 99.90 μs ) +aeson/6MB mean 30.57 ms ( +- 470.7 μs ) +aeson/60MB mean 278.5 ms ( +- 6.307 ms ) +lazy-aeson/1kB mean 6.419 μs ( +- 183.5 ns ) +lazy-aeson/6kB mean 30.72 μs ( +- 501.1 ns ) +lazy-aeson/60kB mean 257.0 μs ( +- 4.227 μs ) +lazy-aeson/600kB mean 2.533 ms ( +- 61.61 μs ) +lazy-aeson/6MB mean 25.08 ms ( +- 263.9 μs ) +lazy-aeson/60MB mean 249.5 ms ( +- 1.333 ms ) +lazy-aeson-untrimmed-32k/1kB mean 6.952 μs ( +- 427.0 ns ) +lazy-aeson-untrimmed-32k/6kB mean 29.68 μs ( +- 656.5 ns ) +lazy-aeson-untrimmed-32k/60kB mean 259.8 μs ( +- 4.344 μs ) +lazy-aeson-untrimmed-32k/600kB mean 2.521 ms ( +- 21.90 μs ) +lazy-aeson-untrimmed-32k/6MB mean 25.25 ms ( +- 295.5 μs ) +lazy-aeson-untrimmed-32k/60MB mean 250.8 ms ( +- 3.536 ms ) +buffer-builder/1kB mean 5.573 μs ( +- 151.5 ns ) +buffer-builder/6kB mean 30.40 μs ( +- 457.2 ns ) +buffer-builder/60kB mean 308.9 μs ( +- 4.601 μs ) +buffer-builder/600kB mean 3.020 ms ( +- 54.79 μs ) +buffer-builder/6MB mean 33.55 ms ( +- 497.8 μs ) +buffer-builder/60MB mean 316.1 ms ( +- 3.747 ms ) ``` The benchmark suite is bundled with the package. diff --git a/bench/Main.hs b/bench/Main.hs --- a/bench/Main.hs +++ b/bench/Main.hs @@ -67,6 +67,7 @@ benchLib "jsonifier" encodeWithJsonifier, benchLib "aeson" encodeWithAeson, benchLib "lazy-aeson" encodeWithLazyAeson, + benchLib "lazy-aeson-untrimmed-32k" Main.Aeson.resultToLazyByteStringWithUntrimmedStrategy, benchLib "buffer-builder" BufferBuilder.encodeResult ] diff --git a/bench/Main/Aeson.hs b/bench/Main/Aeson.hs --- a/bench/Main/Aeson.hs +++ b/bench/Main/Aeson.hs @@ -3,7 +3,21 @@ import Prelude import Main.Model import Data.Aeson hiding (Result) +import qualified Data.ByteString.Lazy as Lbs +import qualified Data.ByteString.Builder as ByteStringBuilder +import qualified Data.ByteString.Builder.Extra as ByteStringBuilder + +resultToLazyByteStringWithUntrimmedStrategy :: Result -> Lbs.ByteString +resultToLazyByteStringWithUntrimmedStrategy = + ByteStringBuilder.toLazyByteStringWith + (ByteStringBuilder.untrimmedStrategy 32000 32000) + Lbs.empty + . resultToByteStringBuilder + +resultToByteStringBuilder :: Result -> ByteStringBuilder.Builder +resultToByteStringBuilder = + fromEncoding . toEncoding instance ToJSON Metadata where toJSON Metadata{..} = object [ diff --git a/jsonifier.cabal b/jsonifier.cabal --- a/jsonifier.cabal +++ b/jsonifier.cabal @@ -1,17 +1,11 @@ name: jsonifier -version: 0.1.0.3 +version: 0.1.0.4 synopsis: Fast and simple JSON encoding toolkit description: - Minimalistic library for encoding JSON directly to strict bytestring. - . - The library focuses on 2 aspects: simplicity and performance. - The API consists of just a few functions and - achieves performance that is 3 times better than that of \"aeson\" - in typical use-cases. - In cases where we deal with really large documents (60MB) the performance - of \"aeson\" becomes more comparable. + Minimalistic library for encoding JSON directly to strict bytestring, + which is up to 3x faster than \"aeson\". . - For further details please refer to README. + For introduction, benchmark results and demo please skip to Readme. category: JSON homepage: https://github.com/nikita-volkov/jsonifier bug-reports: https://github.com/nikita-volkov/jsonifier/issues @@ -37,7 +31,7 @@ exposed-modules: Jsonifier other-modules: - Jsonifier.Allocation + Jsonifier.Size Jsonifier.Ffi Jsonifier.Poke Jsonifier.Prelude diff --git a/library/Jsonifier.hs b/library/Jsonifier.hs --- a/library/Jsonifier.hs +++ b/library/Jsonifier.hs @@ -1,3 +1,7 @@ +{-| +Simple DSL for mapping Haskell values into JSON representation and +rendering it into 'ByteString'. +-} module Jsonifier ( -- * ByteString @@ -24,7 +28,7 @@ import Jsonifier.Prelude hiding (null, bool) import PtrPoker.Poke (Poke) import PtrPoker.Write (Write) -import qualified Jsonifier.Allocation as Allocation +import qualified Jsonifier.Size as Size import qualified Jsonifier.Poke as Poke import qualified Jsonifier.Write as Write import qualified PtrPoker.Poke as Poke @@ -47,10 +51,11 @@ {-| Specification of how to render a JSON value to 'ByteString'. -A sort of a JSON-specialized string 'ByteString' builder. +Sort of a JSON-specialized 'ByteString' builder. -You can construct it by using the specialized -conversion functions from Haskell types. +You can construct it from Haskell types +using the specialized conversion functions +like 'intNumber', 'textString' or 'object'. After constructing, you can convert to strict 'ByteString' using the 'toByteString' function. -} @@ -124,11 +129,11 @@ textString :: Text -> Json textString text = let - allocation = - 2 + Allocation.stringBody text + size = + 2 + Size.stringBody text poke = Poke.string text - in write allocation poke + in write size poke {-| JSON String literal from @Scientific@. @@ -158,7 +163,7 @@ step (Json (Write.Write writeSize _)) next !count !size = next (succ count) (writeSize + size) finalize count size = - Allocation.array count size + Size.array count size poke = Poke.Poke $ Poke.pokePtr Poke.openingSquareBracket >=> @@ -187,23 +192,23 @@ object f = foldr step finalize f True 0 0 mempty where - step (key, Json (Write.Write{..})) next first !size !allocation !poke = + step (key, Json (Write.Write{..})) next first !count !size !poke = if first then - next False 1 rowAllocation rowPoke + next False 1 rowSize rowPoke else - next False (succ size) (allocation + rowAllocation) + next False (succ count) (size + rowSize) (poke <> Poke.comma <> rowPoke) where - rowAllocation = - Allocation.stringBody key + + rowSize = + Size.stringBody key + writeSize rowPoke = Poke.objectRow key writePoke - finalize _ size contentsAllocation bodyPoke = - write allocation poke + finalize _ count contentsSize bodyPoke = + write size poke where - allocation = - Allocation.object size contentsAllocation + size = + Size.object count contentsSize poke = Poke.openingCurlyBracket <> bodyPoke <> Poke.closingCurlyBracket diff --git a/library/Jsonifier/Allocation.hs b/library/Jsonifier/Allocation.hs deleted file mode 100644 --- a/library/Jsonifier/Allocation.hs +++ /dev/null @@ -1,45 +0,0 @@ -{-# LANGUAGE UnliftedFFITypes #-} -module Jsonifier.Allocation -where - -import Jsonifier.Prelude -import qualified Data.Text.Internal as Text -import qualified Data.Text.Array as TextArray -import qualified Jsonifier.Ffi as Ffi - - -{-# INLINE object #-} -object :: Int -> Int -> Int -object rowsAmount contentsAllocation = - curlies + commas rowsAmount + colonsAndQuotes + contentsAllocation - where - curlies = - 2 - colonsAndQuotes = - rowsAmount * 3 - -{-# INLINE array #-} -array :: Int -> Int -> Int -array elementsAmount contentsAllocation = - brackets + commas elementsAmount + contentsAllocation - where - brackets = - 2 - -{-# INLINE commas #-} -commas rowsAmount = - if rowsAmount <= 1 - then 0 - else pred rowsAmount - -{-| -Amount of bytes required for an escaped JSON string value without quotes. - -https://hackage.haskell.org/package/text-1.2.4.0/docs/src/Data.Text.Encoding.html#encodeUtf8BuilderEscaped --} -stringBody :: Text -> Int -stringBody (Text.Text arr off len) = - Ffi.countStringAllocationSize - (TextArray.aBA arr) (fromIntegral off) (fromIntegral len) - & unsafeDupablePerformIO - & fromIntegral diff --git a/library/Jsonifier/Size.hs b/library/Jsonifier/Size.hs new file mode 100644 --- /dev/null +++ b/library/Jsonifier/Size.hs @@ -0,0 +1,44 @@ +{-# LANGUAGE UnliftedFFITypes #-} +module Jsonifier.Size +where + +import Jsonifier.Prelude +import qualified Data.Text.Internal as Text +import qualified Data.Text.Array as TextArray +import qualified Jsonifier.Ffi as Ffi + + +{-# INLINE object #-} +object :: Int -> Int -> Int +object rowsAmount contentsSize = + curlies + commas rowsAmount + colonsAndQuotes + contentsSize + where + curlies = + 2 + colonsAndQuotes = + rowsAmount * 3 + +{-# INLINE array #-} +array :: Int -> Int -> Int +array elementsAmount contentsSize = + brackets + commas elementsAmount + contentsSize + where + brackets = + 2 + +{-# INLINE commas #-} +commas :: Int -> Int +commas rowsAmount = + if rowsAmount <= 1 + then 0 + else pred rowsAmount + +{-| +Amount of bytes required for an escaped JSON string value without quotes. +-} +stringBody :: Text -> Int +stringBody (Text.Text arr off len) = + Ffi.countStringAllocationSize + (TextArray.aBA arr) (fromIntegral off) (fromIntegral len) + & unsafeDupablePerformIO + & fromIntegral