wai-handler-hal 0.2.0.0 → 0.3.0.0

raw patch · 5 files changed

+277/−47 lines, 5 filesdep +tasty-hunitdep ~halPVP ok

version bump matches the API change (PVP)

Dependencies added: tasty-hunit

Dependency ranges changed: hal

API changes (from Hackage documentation)

Files

• CHANGELOG.md +8/−0
• README.md +157/−0
• src/Network/Wai/Handler/Hal.hs +70/−34
• test/Network/Wai/Handler/HalTest.hs +25/−3
• wai-handler-hal.cabal +17/−10

@@ -1,5 +1,13 @@ # Revision history for wai-handler-hal +## 0.3.0.0 -- 2023-12-17++- Breaking change: add `Options` record parameter to `runWithOptions`,+  `toWaiRequest` and `fromWaiResponse`.+- Provide a `defaultOptions`.+- Make whether or not to run base64-encoding on the response body customizable+  through `Options.binaryMimeType`.+ ## 0.2.0.0 -- 2023-03-17  - Breaking change: `toWaiRequest` now sorts request headers and query string

@@ -17,3 +17,160 @@ CloudFormation). If you try to use such a Lambda with an API Gateway **HTTP API** (`AWS::ApiGatewayV2::Api` in CloudFormation), it will return 500s.+++For code examples, see [this repository](https://github.com/bellroy/wai-handler-hal-example),+which contains:++* [`wai-handler-hal-example`](https://github.com/bellroy/wai-handler-hal-example): A simple `servant` API set up to run on both `hal` and `warp`; and+* [`wai-handler-hal-cdk`](https://github.com/bellroy/wai-handler-hal-example/tree/master/wai-handler-hal-cdk): A small [AWS+  CDK](https://docs.aws.amazon.com/cdk/latest/guide/home.html)+  application that deploys an API backed by `wai-handler-hal-example`.++## Developing++Instead of serving your application with something like+[`warp`](https://hackage.haskell.org/package/warp), you will set up an+executable that serves your application with+[`hal`](https://hackage.haskell.org/package/hal). The necessary glue+code looks something like this:++```haskell+import AWS.Lambda.Runtime (mRuntime)+import Network.Wai (Application)+import qualified Network.Wai.Handler.Hal as WaiHandler++app :: Application+app = undefined -- From Servant or wherever else++main :: IO ()+main = mRuntime $ WaiHandler.run app+```++### Local testing++Compiled Lambda functions can be awkward to test, especially if you're+relying on an API Gateway integration to translate HTTP+requests. Consider defining a second executable target that serves+your `Application` using `warp`, which you can use for local+testing.++### Caveats++* The Lambda is never told the port that the API Gateway is listening+  on. Most APIs will listen on `443` (HTTPS), so that's what the+  library reports by default. See `runWithContext` if you need to+  change this.++* The Lambda is never told the HTTP version that the client uses when+  talking with the API Gateway. We assume HTTP 1.1.++## Packaging++Lambda functions are packaged in one of two ways: as `.zip` files or+as Docker container images. The [`wai-handler-hal-cdk` example](https://github.com/bellroy/wai-handler-hal-example/tree/master/wai-handler-hal-cdk) uses+`.zip` files for simplicity. CDK doesn't give us an easy way to build+container images using `nix build`, so a container-based deployment+using CDK would need to push to ECR first and then reference the image+by name.++### `.zip` files++You will need to create a `.zip` file containing ([at+least](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html#runtimes-custom-build))+an executable named `bootstrap`. This executable needs to run in+Amazon's runtime environment, and there are multiple ways to ensure+this.++#### Static linking++IOHK's [`haskell.nix`](https://github.com/input-output-hk/haskell.nix)+can build static Haskell binaries by cross-compiling against musl+libc. This is convenient, but consider copyleft implications (of+`libgmp`; `wai-handler-hal` is BSD-3-Clause) if you are distributing+the binaries to other people.++#### Dynamic linking++The other option is to compile the executable in an environment with+packages that match the [Lambda runtime+environment](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html). Some+ideas:++* Build the executable in an "imitation environment" like the [`lambci/lambda`](https://hub.docker.com/r/lambci/lambda) Docker container; or+* (Untested) build the executable on an Amazon Linux 2 EC2 instance.++### Docker container images++It is possible to package Lambdas into Docker containers for+deployment. Amazon provide base containers for custom runtimes in the+repository+[`amazon/aws-lambda-provided`](https://hub.docker.com/r/amazon/aws-lambda-provided/). One+nice feature of these images is that they provide a [runtime interface+emulator](https://docs.aws.amazon.com/lambda/latest/dg/images-test.html#images-test-AWSbase),+which they fall back to when not running "for real". This makes it+possible to directly invoke the lambda and see how it behaves. (This+is less important here because we can serve our application off+`warp`, but for writing Lambdas not invoked by an API Gateway Proxy+Integration, it's handy.)++At the time of writing (2021-04-06), the+`amazon/aws-lambda-provided:al2` (Amazon Linux 2 tag) is the newest+base image that Amazon recommends for custom runtimes. You can use a+`Dockerfile` to build your images; there are also commented .nix files+in the [example repository](https://github.com/bellroy/wai-handler-hal-example),+showing a couple of different approaches to building container images using Nix.++## Integrating++Actually calling the Lambda is done with a [Lambda proxy+integration](https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html)+from an [API Gateway REST+API](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-rest-api.html).++The simplest possible integration sends every request to the Lambda+(where the `wai` `Application` can return 404 or whatever if the+endpoint doesn't match). This is done by mapping the paths `/` and+`/{proxy+}` for the HTTP method `ANY`. We do this in our [CDK example](https://github.com/bellroy/wai-handler-hal-example/tree/master/wai-handler-hal-cdk),+and [CDK provides a Construct](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_apigateway.LambdaRestApi.html) which encapsulates this pattern, making+it extremely simple to deploy.++### Splitting Servant applications++While it's simplest to back the entire API with a single Lambda, it+does mean that you risk one Lambda carrying an ever-expanding set of+IAM permissions. Servant's combinators make it easy to break the API+into parts that are served by individual Lambdas, which can each have+the minimal set of permissions attached to their role. If your Servant+service had endpoints `foo`, `bar`, and `baz`, you'd serve them all+from Warp with an expression like `Warp.runEnv 8080 . Servant.serve $+foo :<|> bar :<|> baz`. But `Servant.serve foo`, `Servant.serve bar`,+and `Servant.serve baz` are also all WAI `Application`s, and can each+be bundled into distinct Lambda functions using `wai-handler-hal`.++### Other caveats++* The stage name in a path segment is not passed to the Lambda, so it+  is not passed to the `wai` `Application`. An invoke URL like+  `https://abcde12345.execute-api.us-east-1.amazonaws.com/prod/hoot`+  will send `pathInfo = ["hoot"]` to the `Application`.++## Formatters++The formatters used in this repo are provided by `shell.nix`:++* `*.hs`: [`ormolu`](https://github.com/tweag/ormolu)+* `*.cabal`:+  [`cabal-fmt`](https://hackage.haskell.org/package/cabal-fmt)+  (`cabal-fmt --inplace wai-handler-hal.cabal`)+* `*.nix`:+  [`nixpkgs-fmt`](https://github.com/nix-community/nixpkgs-fmt)+  (`nixpkgs-fmt *.nix`)++## Regenerate CI++This repo uses `haskell-ci`, which is provided by `flake.nix`:++```shell+haskell-ci regenerate+```

@@ -32,6 +32,8 @@ module Network.Wai.Handler.Hal   ( run,     runWithContext,+    Options (..),+    defaultOptions,     toWaiRequest,     fromWaiResponse,   )@@ -99,31 +101,64 @@ -- form, and probably all that you'll need. See 'runWithContext' if -- you have more complex needs. run ::-  MonadIO m =>+  (MonadIO m) =>   Wai.Application ->   HalRequest.ProxyRequest HalRequest.NoAuthorizer ->   m HalResponse.ProxyResponse run app req = liftIO $ do-  waiReq <- toWaiRequest Vault.empty 443 req+  waiReq <- toWaiRequest defaultOptions req   responseRef <- IORef.newIORef Nothing   Wai.ResponseReceived <- app waiReq $ \waiResp ->     Wai.ResponseReceived <$ IORef.writeIORef responseRef (Just waiResp)   Just waiResp <- IORef.readIORef responseRef-  fromWaiResponse waiResp+  fromWaiResponse defaultOptions waiResp +-- | Options that can be used to customize the behaviour of 'runWithContext'.+-- 'defaultOptions' provides sensible defaults.+data Options = Options+  { -- | Vault of values to share between the application and any+    -- middleware. You can pass in @Data.Vault.Lazy.'Vault.empty'@, or+    -- 'mempty' if you don't want to depend on @vault@ directly.+    vault :: Vault,+    -- | API Gateway doesn't tell us the port it's listening on, so you+    -- have to tell it yourself. This is almost always going to be 443+    -- (HTTPS).+    portNumber :: PortNumber,+    -- | Binary responses need to be encoded as base64. This option lets you+    -- customize which mime types are considered binary data.+    --+    -- The following mime types are __not__ considered binary by default:+    --+    -- * @application/json@+    -- * @application/xml@+    -- * anything starting with @text/@+    -- * anything ending with @+json@+    -- * anything ending with @+xml@+    binaryMimeType :: Text -> Bool+  }++-- | Default options for running 'Wai.Application's on Lambda.+defaultOptions :: Options+defaultOptions =+  Options+    { vault = Vault.empty,+      portNumber = 443,+      binaryMimeType = \mime -> case mime of+        "application/json" -> False+        "application/xml" -> False+        _ | "text/" `T.isPrefixOf` mime -> False+        _ | "+json" `T.isSuffixOf` mime -> False+        _ | "+xml" `T.isSuffixOf` mime -> False+        _ -> True+    }+ -- | Convert a WAI 'Wai.Application' into a function that can -- be run by hal's 'AWS.Lambda.Runtime.mRuntimeWithContext''. This -- function exposes all the configurable knobs. runWithContext ::-  MonadIO m =>-  -- | Vault of values to share between the application and any-  -- middleware. You can pass in @Data.Vault.Lazy.'Vault.empty'@, or-  -- 'mempty' if you don't want to depend on @vault@ directly.-  Vault ->-  -- | API Gateway doesn't tell us the port it's listening on, so you-  -- have to tell it yourself. This is almost always going to be 443-  -- (HTTPS).-  PortNumber ->+  (MonadIO m) =>+  -- | Configuration options. 'defaultOptions' provides sensible defaults.+  Options ->   -- | We pass two 'Vault' keys to the callback that provides the   -- 'Wai.Application'. This allows the application to look into the   -- 'Vault' part of each request and read @hal@ data structures, if@@ -146,19 +181,20 @@   -- an "ambiguous type variable" error at the use site.   HalRequest.ProxyRequest HalRequest.NoAuthorizer ->   m HalResponse.ProxyResponse-runWithContext vault port app ctx req = liftIO $ do+runWithContext opts app ctx req = liftIO $ do   contextKey <- Vault.newKey   requestKey <- Vault.newKey   let vault' =-        vault+        vault opts           & Vault.insert contextKey ctx           & Vault.insert requestKey req-  waiReq <- toWaiRequest vault' port req+      opts' = opts {vault = vault'}+  waiReq <- toWaiRequest opts' req   responseRef <- IORef.newIORef Nothing   Wai.ResponseReceived <- app contextKey requestKey waiReq $ \waiResp ->     Wai.ResponseReceived <$ IORef.writeIORef responseRef (Just waiResp)   Just waiResp <- IORef.readIORef responseRef-  fromWaiResponse waiResp+  fromWaiResponse opts' waiResp  -- | Convert the request sent to a Lambda serving an API Gateway proxy -- integration into a WAI request.@@ -166,12 +202,12 @@ -- __Note:__ We aren't told the HTTP version the client is using, so -- we assume HTTP 1.1. toWaiRequest ::-  Vault ->-  PortNumber ->+  Options ->   HalRequest.ProxyRequest a ->   IO Wai.Request-toWaiRequest vault port req = do-  let pathSegments = T.splitOn "/" . T.dropWhile (== '/') $ HalRequest.path req+toWaiRequest opts req = do+  let port = portNumber opts+      pathSegments = T.splitOn "/" . T.dropWhile (== '/') $ HalRequest.path req       query = sort . constructQuery $ HalRequest.multiValueQueryStringParameters req       hints =         NS.defaultHints@@ -226,7 +262,7 @@             Wai.pathInfo = pathSegments,             Wai.queryString = query,             Wai.requestBody = body,-            Wai.vault = vault,+            Wai.vault = vault opts,             Wai.requestBodyLength =               Wai.KnownLength . fromIntegral . BL.length $ HalRequest.body req,             Wai.requestHeaderHost = getHeader hHost req,@@ -262,29 +298,29 @@  -- | Convert a WAI 'Wai.Response' into a hal -- 'HalResponse.ProxyResponse'.-fromWaiResponse :: Wai.Response -> IO HalResponse.ProxyResponse-fromWaiResponse (Wai.ResponseFile status headers path mFilePart) = do+fromWaiResponse :: Options -> Wai.Response -> IO HalResponse.ProxyResponse+fromWaiResponse opts (Wai.ResponseFile status headers path mFilePart) = do   fileData <- readFilePart path mFilePart   pure     . addHeaders headers     . HalResponse.response status-    . createProxyBody (getContentType headers)+    . createProxyBody opts (getContentType headers)     $ fileData-fromWaiResponse (Wai.ResponseBuilder status headers builder) =+fromWaiResponse opts (Wai.ResponseBuilder status headers builder) =   pure     . addHeaders headers     . HalResponse.response status-    . createProxyBody (getContentType headers)+    . createProxyBody opts (getContentType headers)     . BL.toStrict     $ Builder.toLazyByteString builder-fromWaiResponse (Wai.ResponseStream status headers stream) = do+fromWaiResponse opts (Wai.ResponseStream status headers stream) = do   builderRef <- IORef.newIORef mempty   let addChunk chunk = IORef.modifyIORef builderRef (<> chunk)       flush = IORef.modifyIORef builderRef (<> Builder.flush)   stream addChunk flush   builder <- IORef.readIORef builderRef-  fromWaiResponse (Wai.ResponseBuilder status headers builder)-fromWaiResponse (Wai.ResponseRaw _ resp) = fromWaiResponse resp+  fromWaiResponse opts (Wai.ResponseBuilder status headers builder)+fromWaiResponse opts (Wai.ResponseRaw _ resp) = fromWaiResponse opts resp  readFilePart :: FilePath -> Maybe Wai.FilePart -> IO ByteString readFilePart path mPart = withFile path ReadMode $ \h -> do@@ -294,12 +330,12 @@       hSeek h AbsoluteSeek offset       B.hGet h $ fromIntegral count -createProxyBody :: Text -> ByteString -> HalResponse.ProxyBody-createProxyBody contentType body-  | any (`T.isPrefixOf` contentType) ["text/plain", "application/json"] =-    HalResponse.ProxyBody contentType (T.decodeUtf8 body) False+createProxyBody :: Options -> Text -> ByteString -> HalResponse.ProxyBody+createProxyBody opts contentType body+  | binaryMimeType opts contentType =+      HalResponse.ProxyBody contentType (T.decodeUtf8 $ B64.encode body) True   | otherwise =-    HalResponse.ProxyBody contentType (T.decodeUtf8 $ B64.encode body) True+      HalResponse.ProxyBody contentType (T.decodeUtf8 body) False  addHeaders ::   ResponseHeaders -> HalResponse.ProxyResponse -> HalResponse.ProxyResponse

@@ -4,11 +4,13 @@  import AWS.Lambda.Events.ApiGateway.ProxyRequest import Data.Aeson (eitherDecodeFileStrict')-import qualified Data.Text.Lazy.Encoding as T+import qualified Data.Text as T+import qualified Data.Text.Lazy.Encoding as TL import Data.Void (Void) import Network.Wai.Handler.Hal import Test.Tasty import Test.Tasty.Golden+import Test.Tasty.HUnit (assertEqual, testCase) import Text.Pretty.Simple  test_ConvertProxyRequest :: TestTree@@ -17,5 +19,25 @@     proxyRequest :: ProxyRequest Void <-       eitherDecodeFileStrict' "test/data/ProxyRequest.json"         >>= either fail pure-    waiRequest <- toWaiRequest mempty 443 proxyRequest-    pure . T.encodeUtf8 $ pShowNoColor waiRequest+    waiRequest <- toWaiRequest defaultOptions proxyRequest+    pure . TL.encodeUtf8 $ pShowNoColor waiRequest++test_DefaultBinaryMimeTypes :: TestTree+test_DefaultBinaryMimeTypes = testCase "default binary MIME types" $ do+  assertBinary False "text/plain"+  assertBinary False "text/html"+  assertBinary False "application/json"+  assertBinary False "application/xml"+  assertBinary False "application/vnd.api+json"+  assertBinary False "application/vnd.api+xml"+  assertBinary False "image/svg+xml"++  assertBinary True "application/octet-stream"+  assertBinary True "audio/vorbis"+  assertBinary True "image/png"+  where+    assertBinary expected mime =+      assertEqual+        mime+        (binaryMimeType defaultOptions (T.pack mime))+        expected

@@ -1,6 +1,6 @@ cabal-version:      2.2 name:               wai-handler-hal-version:            0.2.0.0+version:            0.3.0.0 synopsis:           Wrap WAI applications to run on AWS Lambda description:   This library provides a function 'Network.Wai.Handler.Hal.run' to@@ -28,7 +28,13 @@   test/golden/WaiRequest.txt  tested-with:-  GHC ==8.6.5 || ==8.8.4 || ==8.10.7 || ==9.0.2 || ==9.2.2+  GHC ==8.6.5+   || ==8.8.4+   || ==8.10.7+   || ==9.0.2+   || ==9.2.4+   || ==9.4.5+   || ==9.6.2  common opts   default-language: Haskell2010@@ -41,14 +47,14 @@  common deps   build-depends:-    , base                  >=4.12     && <4.18-    , base64-bytestring     >=1.0.0.0  && <1.3-    , bytestring            >=0.10.8   && <0.12+    , base                  >=4.12      && <4.19+    , base64-bytestring     >=1.0.0.0   && <1.3+    , bytestring            >=0.10.8    && <0.12.1     , case-insensitive      ^>=1.2.0.0-    , hal                   >=0.4.7    && <0.4.11 || >=1.0.0 && <1.1+    , hal                   >=0.4.7     && <0.4.11 || >=1.0.0 && <1.1     , http-types            ^>=0.12.3-    , network               >=2.8.0.0  && <3.2-    , text                  ^>=1.2.3 || ^>=2.0+    , network               >=2.8.0.0   && <3.2+    , text                  ^>=1.2.3    || >=2.0   && <2.1.1     , unordered-containers  ^>=0.2.10.0     , vault                 ^>=0.3.1.0     , wai                   ^>=3.2.2@@ -67,10 +73,11 @@   other-modules:      Network.Wai.Handler.HalTest   build-tool-depends: tasty-discover:tasty-discover ^>=4.2.2   build-depends:-    , aeson            >=1.5.6.0 && <1.6 || >=2.0 && <2.2+    , aeson            >=1.5.6.0  && <1.6  || >=2.0 && <2.3     , pretty-simple    ^>=4.1.0.0-    , tasty            >=1.3     && <1.5+    , tasty            >=1.3      && <1.6     , tasty-golden     ^>=2.3+    , tasty-hunit      >=0.9      && <0.11     , text     , wai-handler-hal