linnet-conduit (empty) → 0.2.0.0
raw patch · 6 files changed
+465/−0 lines, 6 filesdep +QuickCheckdep +basedep +bytestring
Dependencies added: QuickCheck, base, bytestring, conduit, hspec, http-types, linnet, linnet-conduit, quickcheck-classes, quickcheck-instances, wai, warp
Files
- LICENSE +201/−0
- README.md +15/−0
- linnet-conduit.cabal +61/−0
- src/Linnet/Conduit.hs +90/−0
- test/ConduitSpec.hs +97/−0
- test/Spec.hs +1/−0
+ LICENSE view
@@ -0,0 +1,201 @@+ Apache License+ Version 2.0, January 2004+ http://www.apache.org/licenses/++ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION++ 1. Definitions.++ "License" shall mean the terms and conditions for use, reproduction,+ and distribution as defined by Sections 1 through 9 of this document.++ "Licensor" shall mean the copyright owner or entity authorized by+ the copyright owner that is granting the License.++ "Legal Entity" shall mean the union of the acting entity and all+ other entities that control, are controlled by, or are under common+ control with that entity. For the purposes of this definition,+ "control" means (i) the power, direct or indirect, to cause the+ direction or management of such entity, whether by contract or+ otherwise, or (ii) ownership of fifty percent (50%) or more of the+ outstanding shares, or (iii) beneficial ownership of such entity.++ "You" (or "Your") shall mean an individual or Legal Entity+ exercising permissions granted by this License.++ "Source" form shall mean the preferred form for making modifications,+ including but not limited to software source code, documentation+ source, and configuration files.++ "Object" form shall mean any form resulting from mechanical+ transformation or translation of a Source form, including but+ not limited to compiled object code, generated documentation,+ and conversions to other media types.++ "Work" shall mean the work of authorship, whether in Source or+ Object form, made available under the License, as indicated by a+ copyright notice that is included in or attached to the work+ (an example is provided in the Appendix below).++ "Derivative Works" shall mean any work, whether in Source or Object+ form, that is based on (or derived from) the Work and for which the+ editorial revisions, annotations, elaborations, or other modifications+ represent, as a whole, an original work of authorship. For the purposes+ of this License, Derivative Works shall not include works that remain+ separable from, or merely link (or bind by name) to the interfaces of,+ the Work and Derivative Works thereof.++ "Contribution" shall mean any work of authorship, including+ the original version of the Work and any modifications or additions+ to that Work or Derivative Works thereof, that is intentionally+ submitted to Licensor for inclusion in the Work by the copyright owner+ or by an individual or Legal Entity authorized to submit on behalf of+ the copyright owner. For the purposes of this definition, "submitted"+ means any form of electronic, verbal, or written communication sent+ to the Licensor or its representatives, including but not limited to+ communication on electronic mailing lists, source code control systems,+ and issue tracking systems that are managed by, or on behalf of, the+ Licensor for the purpose of discussing and improving the Work, but+ excluding communication that is conspicuously marked or otherwise+ designated in writing by the copyright owner as "Not a Contribution."++ "Contributor" shall mean Licensor and any individual or Legal Entity+ on behalf of whom a Contribution has been received by Licensor and+ subsequently incorporated within the Work.++ 2. Grant of Copyright License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ copyright license to reproduce, prepare Derivative Works of,+ publicly display, publicly perform, sublicense, and distribute the+ Work and such Derivative Works in Source or Object form.++ 3. Grant of Patent License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ (except as stated in this section) patent license to make, have made,+ use, offer to sell, sell, import, and otherwise transfer the Work,+ where such license applies only to those patent claims licensable+ by such Contributor that are necessarily infringed by their+ Contribution(s) alone or by combination of their Contribution(s)+ with the Work to which such Contribution(s) was submitted. If You+ institute patent litigation against any entity (including a+ cross-claim or counterclaim in a lawsuit) alleging that the Work+ or a Contribution incorporated within the Work constitutes direct+ or contributory patent infringement, then any patent licenses+ granted to You under this License for that Work shall terminate+ as of the date such litigation is filed.++ 4. Redistribution. You may reproduce and distribute copies of the+ Work or Derivative Works thereof in any medium, with or without+ modifications, and in Source or Object form, provided that You+ meet the following conditions:++ (a) You must give any other recipients of the Work or+ Derivative Works a copy of this License; and++ (b) You must cause any modified files to carry prominent notices+ stating that You changed the files; and++ (c) You must retain, in the Source form of any Derivative Works+ that You distribute, all copyright, patent, trademark, and+ attribution notices from the Source form of the Work,+ excluding those notices that do not pertain to any part of+ the Derivative Works; and++ (d) If the Work includes a "NOTICE" text file as part of its+ distribution, then any Derivative Works that You distribute must+ include a readable copy of the attribution notices contained+ within such NOTICE file, excluding those notices that do not+ pertain to any part of the Derivative Works, in at least one+ of the following places: within a NOTICE text file distributed+ as part of the Derivative Works; within the Source form or+ documentation, if provided along with the Derivative Works; or,+ within a display generated by the Derivative Works, if and+ wherever such third-party notices normally appear. The contents+ of the NOTICE file are for informational purposes only and+ do not modify the License. You may add Your own attribution+ notices within Derivative Works that You distribute, alongside+ or as an addendum to the NOTICE text from the Work, provided+ that such additional attribution notices cannot be construed+ as modifying the License.++ You may add Your own copyright statement to Your modifications and+ may provide additional or different license terms and conditions+ for use, reproduction, or distribution of Your modifications, or+ for any such Derivative Works as a whole, provided Your use,+ reproduction, and distribution of the Work otherwise complies with+ the conditions stated in this License.++ 5. Submission of Contributions. Unless You explicitly state otherwise,+ any Contribution intentionally submitted for inclusion in the Work+ by You to the Licensor shall be under the terms and conditions of+ this License, without any additional terms or conditions.+ Notwithstanding the above, nothing herein shall supersede or modify+ the terms of any separate license agreement you may have executed+ with Licensor regarding such Contributions.++ 6. Trademarks. This License does not grant permission to use the trade+ names, trademarks, service marks, or product names of the Licensor,+ except as required for reasonable and customary use in describing the+ origin of the Work and reproducing the content of the NOTICE file.++ 7. Disclaimer of Warranty. Unless required by applicable law or+ agreed to in writing, Licensor provides the Work (and each+ Contributor provides its Contributions) on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or+ implied, including, without limitation, any warranties or conditions+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A+ PARTICULAR PURPOSE. You are solely responsible for determining the+ appropriateness of using or redistributing the Work and assume any+ risks associated with Your exercise of permissions under this License.++ 8. Limitation of Liability. In no event and under no legal theory,+ whether in tort (including negligence), contract, or otherwise,+ unless required by applicable law (such as deliberate and grossly+ negligent acts) or agreed to in writing, shall any Contributor be+ liable to You for damages, including any direct, indirect, special,+ incidental, or consequential damages of any character arising as a+ result of this License or out of the use or inability to use the+ Work (including but not limited to damages for loss of goodwill,+ work stoppage, computer failure or malfunction, or any and all+ other commercial damages or losses), even if such Contributor+ has been advised of the possibility of such damages.++ 9. Accepting Warranty or Additional Liability. While redistributing+ the Work or Derivative Works thereof, You may choose to offer,+ and charge a fee for, acceptance of support, warranty, indemnity,+ or other liability obligations and/or rights consistent with this+ License. However, in accepting such obligations, You may act only+ on Your own behalf and on Your sole responsibility, not on behalf+ of any other Contributor, and only if You agree to indemnify,+ defend, and hold each Contributor harmless for any liability+ incurred by, or claims asserted against, such Contributor by reason+ of your accepting any such warranty or additional liability.++ END OF TERMS AND CONDITIONS++ APPENDIX: How to apply the Apache License to your work.++ To apply the Apache License to your work, attach the following+ boilerplate notice, with the fields enclosed by brackets "[]"+ replaced with your own identifying information. (Don't include+ the brackets!) The text should be enclosed in the appropriate+ comment syntax for the file format. We also recommend that a+ file or class name and description of purpose be included on the+ same "printed page" as the copyright notice for easier+ identification within third-party archives.++ Copyright 2019 Sergey Kolbasov++ Licensed under the Apache License, Version 2.0 (the "License");+ you may not use this file except in compliance with the License.+ You may obtain a copy of the License at++ http://www.apache.org/licenses/LICENSE-2.0++ Unless required by applicable law or agreed to in writing, software+ distributed under the License is distributed on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ See the License for the specific language governing permissions and+ limitations under the License.
+ README.md view
@@ -0,0 +1,15 @@+# Linnet Conduit++This package adds support of streaming requests & responses in Linnet library+using [conduit](http://hackage.haskell.org/package/conduit).+++```haskell+import Linnet.Conduit+```++Import above exposes:+- orphan instance of `ToResponse ct (ConduitT () a m ())`+- orphan instance of `ToResponse ApplicationJson (ConduitT () a m ())` that intersperse JSON chunks+with newline symbol+- `streamBody :: Endpoint m (ConduitT BL.ByteString BL.ByteString m ())` endpoint
+ linnet-conduit.cabal view
@@ -0,0 +1,61 @@+cabal-version: 1.12+name: linnet-conduit+version: 0.2.0.0+license: Apache+license-file: LICENSE+copyright: 2019 Sergey Kolbasov+maintainer: whoisliar@gmail.com+author: Sergey Kolbasov+homepage: https://github.com/haskell-linnet/linnet#readme+bug-reports: https://github.com/haskell-linnet/linnet/issues+synopsis: Conduit-backed support for streaming in Linnet+description:+ Please see the documentation at <http://linnet.io>+category: linnet+build-type: Simple+extra-source-files:+ README.md+ LICENSE++source-repository head+ type: git+ location: https://github.com/haskell-linnet/linnet++library+ exposed-modules:+ Linnet.Conduit+ hs-source-dirs: src+ other-modules:+ Paths_linnet_conduit+ default-language: Haskell2010+ build-depends:+ base >=4.7 && <5,+ bytestring >=0.10.8.2,+ conduit >=1.3.1.1,+ http-types >=0.12.3,+ linnet >=0.2.0.0,+ wai >=3.2.2.1,+ warp >=3.2.28++test-suite linnet-conduit-test+ type: exitcode-stdio-1.0+ main-is: Spec.hs+ hs-source-dirs: test+ other-modules:+ ConduitSpec+ Paths_linnet_conduit+ default-language: Haskell2010+ ghc-options: -threaded -rtsopts -with-rtsopts=-N+ build-depends:+ QuickCheck >=2.13.2,+ base >=4.7 && <5,+ bytestring >=0.10.8.2,+ conduit >=1.3.1.1,+ hspec >=2.7.1,+ http-types >=0.12.3,+ linnet >=0.2.0.0,+ linnet-conduit -any,+ quickcheck-classes >=0.6.1.0,+ quickcheck-instances >=0.3.22,+ wai >=3.2.2.1,+ warp >=3.2.28
+ src/Linnet/Conduit.hs view
@@ -0,0 +1,90 @@+{-# LANGUAGE AllowAmbiguousTypes #-}+{-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE FlexibleInstances #-}+{-# LANGUAGE MultiParamTypeClasses #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeApplications #-}+{-# LANGUAGE TypeSynonymInstances #-}++{-|+Module: Linnet Conduit+Description: Streaming support for Linnet+Copyright: (c) Sergey Kolbasov, 2019+License: Apache License 2.0++This package adds support of streaming requests & responses in Linnet library using <http://hackage.haskell.org/package/conduit conduit>.++See the detailed documentation on <http://linnet.io linnet.io>.++-}+module Linnet.Conduit+ ( streamBody+ ) where++import Conduit (ConduitT, mapM_C, runConduit,+ yield, (.|))+import Control.Monad (unless)+import Control.Monad.IO.Class (MonadIO (..))+import qualified Data.ByteString.Builder as Builder+import qualified Data.ByteString.Char8 as C8+import qualified Data.ByteString.Lazy as BL+import Data.Data (Proxy (..))+import Data.Void (Void)+import GHC.TypeLits (KnownSymbol, symbolVal)+import Linnet+import Linnet.Endpoint (EndpointResult (..))+import Linnet.Input (request)+import Linnet.NaturalTransformation (NaturalTransformation (..))+import Linnet.ToResponse+import Network.HTTP.Types (status200)+import Network.Wai (RequestBodyLength (..),+ lazyRequestBody,+ requestBodyLength,+ responseStream)+import Network.Wai.Handler.Warp (pauseTimeout)++fromLazyBody :: (MonadIO m) => IO BL.ByteString -> ConduitT i BL.ByteString m ()+fromLazyBody reader = do+ bs <- liftIO reader+ unless (bs == BL.empty) $ yield bs >> fromLazyBody reader++-- | Endpoint that reads chunked request body as 'ConduitT' stream of lazy @ByteString@. Matches if body is chunked.+-- Beware that it also pauses slowloris timeout in Warp as it reads request body using lazy IO.+streamBody ::+ forall m i. (MonadIO m)+ => Endpoint m (ConduitT i BL.ByteString m ())+streamBody =+ Endpoint+ { runEndpoint =+ \input ->+ let req = request input+ in case requestBodyLength req of+ ChunkedBody ->+ Matched+ { matchedReminder = input+ , matchedOutput =+ do liftIO $ pauseTimeout req+ pure $ ok $ (fromLazyBody . lazyRequestBody) req+ }+ KnownLength _ -> NotMatched+ , toString = "streamBody"+ }++instance {-# OVERLAPS #-} (Encode ApplicationJson a, NaturalTransformation m IO, MonadIO m) =>+ ToResponse ApplicationJson (ConduitT () a m ()) where+ toResponse stream =+ responseStream status200 [("Content-Type", "application/json")] $ \write flush ->+ let push :: ConduitT a Void m ()+ push =+ mapM_C+ (\chunk -> liftIO $ write (Builder.lazyByteString (encode @ApplicationJson chunk)) >> write "\n" >> flush)+ in mapK (runConduit $ stream .| push)++instance {-# OVERLAPS #-} (Encode ct a, KnownSymbol ct, NaturalTransformation m IO, MonadIO m) =>+ ToResponse ct (ConduitT () a m ()) where+ toResponse stream =+ responseStream status200 [("Content-Type", C8.pack $ symbolVal (Proxy :: Proxy ct))] $ \write flush ->+ let push :: ConduitT a Void m ()+ push = mapM_C (\chunk -> liftIO $ write (Builder.lazyByteString (encode @ct chunk)) >> flush)+ in mapK (runConduit $ stream .| push)
+ test/ConduitSpec.hs view
@@ -0,0 +1,97 @@+{-# LANGUAGE FlexibleInstances #-}+{-# LANGUAGE MultiParamTypeClasses #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE ScopedTypeVariables #-}+{-# LANGUAGE TypeApplications #-}+{-# LANGUAGE TypeSynonymInstances #-}++module ConduitSpec where++import Conduit (ConduitT, runConduit, yield, (.|))+import Control.Monad.IO.Class (liftIO)+import qualified Data.ByteString as BS+import qualified Data.ByteString.Builder as Builder+import Data.ByteString.Char8 (unpack)+import qualified Data.ByteString.Lazy as BL+import Data.Conduit.Combinators (sinkList)+import Data.IORef (atomicModifyIORef, modifyIORef,+ newIORef, readIORef)+import Data.List (intersperse)+import Linnet+import Linnet.Conduit+import Linnet.Endpoint (EndpointResult (..), isMatched,+ runEndpoint)+import Linnet.Input+import Linnet.ToResponse+import Network.Wai (RequestBodyLength (..),+ defaultRequest, requestBody,+ requestBodyLength, responseStream,+ responseToStream)+import Test.Hspec+import Test.QuickCheck (property)+import Test.QuickCheck.Instances+import Test.QuickCheck.Monadic (assert, monadicIO)++spec :: Spec+spec =+ describe "Conduit streaming" $ do+ describe "streamBody" $ do+ it "should match only Chunked requests" $ do+ let streamed = inputFromRequest $ defaultRequest {requestBodyLength = ChunkedBody}+ let fixed = inputFromRequest defaultRequest+ let endpoint = streamBody @IO+ isMatched (runEndpoint endpoint streamed) `shouldBe` True+ isMatched (runEndpoint endpoint fixed) `shouldBe` False+ it "streams request body as conduit stream" $+ property $ \(bytes :: [BS.ByteString]) ->+ let nonEmpty = filter (/= mempty) bytes+ getChunk :: [BS.ByteString] -> ([BS.ByteString], BS.ByteString)+ getChunk [] = ([], BS.empty)+ getChunk (h:t) = (t, h)+ in monadicIO $ do+ ref <- liftIO $ newIORef nonEmpty+ let streamed =+ inputFromRequest $+ defaultRequest {requestBodyLength = ChunkedBody, requestBody = atomicModifyIORef ref getChunk}+ let endpoint = streamBody @IO ~> (\stream -> ok <$> runConduit (stream .| sinkList))+ let (Matched _ mOut) = runEndpoint endpoint streamed+ output <- liftIO mOut+ assert $ output == ok (map BL.fromStrict nonEmpty)+ describe "ToResponse conduit" $ do+ it "should create chunked response" $+ property $ \(bytes :: [BS.ByteString]) ->+ let stream :: [BS.ByteString] -> ConduitT () BS.ByteString IO ()+ stream = foldr ((>>) . yield) (return ())+ in monadicIO $ do+ final <- liftIO $ newIORef ([] @BS.ByteString)+ buffer <- liftIO $ newIORef ([] @BS.ByteString)+ let response = toResponse @TextPlain $ stream bytes+ let (_, _, responseBody) = responseToStream response+ let write builder = modifyIORef buffer (++ [BL.toStrict $ Builder.toLazyByteString builder])+ let flush = do+ buf <- readIORef buffer+ _ <- modifyIORef final (++ buf)+ _ <- modifyIORef buffer (const [])+ return ()+ result <- liftIO $ responseBody (\fn -> fn write flush >> readIORef final)+ assert (result == bytes)+ it "should intersperse newline symbol in JSON stream" $+ property $ \(bytes :: [BS.ByteString]) ->+ let stream :: [BS.ByteString] -> ConduitT () BS.ByteString IO ()+ stream = foldr ((>>) . yield) (return ())+ in monadicIO $ do+ final <- liftIO $ newIORef ([] @BS.ByteString)+ buffer <- liftIO $ newIORef ([] @BS.ByteString)+ let response = toResponse @ApplicationJson $ stream bytes+ let (_, _, responseBody) = responseToStream response+ let write builder = modifyIORef buffer (++ [BL.toStrict $ Builder.toLazyByteString builder])+ let flush = do+ buf <- readIORef buffer+ _ <- modifyIORef final (++ buf)+ _ <- modifyIORef buffer (const [])+ return ()+ result <- liftIO $ responseBody (\fn -> fn write flush >> readIORef final)+ assert (result == if null bytes then [] else intersperse "\n" bytes ++ ["\n"])++instance Encode ApplicationJson BS.ByteString where+ encode = BL.fromStrict
+ test/Spec.hs view
@@ -0,0 +1,1 @@+{-# OPTIONS_GHC -F -pgmF hspec-discover #-}