instana-haskell-trace-sdk 0.8.0.0 → 0.9.0.0

raw patch · 9 files changed

+296/−93 lines, 9 filesPVP ok

version bump matches the API change (PVP)

API changes (from Hackage documentation)

Files

• CHANGELOG.md +11/−0
• README.md +2/−2
• instana-haskell-trace-sdk.cabal +1/−2
• src/Instana/SDK/SDK.hs +146/−39
• src/Instana/SDK/Span/Span.hs +14/−8
• test/apps/wai/Main.hs +5/−5
• test/integration/Instana/SDK/IntegrationTest/ExtraHttpHeaders.hs +8/−8
• test/integration/Instana/SDK/IntegrationTest/Metrics.hs +1/−1
• test/unit/Instana/SDK/SpanTest.hs +108/−28

@@ -1,5 +1,16 @@ # Changelog for instana-haskell-trace-sdk +## 0.9.0.0+- Fix format of annotation span.data.http.header - it is now a JSON object instead of an array of arrays.+- BREAKING: Rename confusingly named `Instana.SDK.addAnnotationAt` to `Instana.SDK.addJsonValueAt`, because it actually takes a JSON value. `Instana.SDK.addAnnotationAt` still exists, but takes an actual `Instana.SDK.Span.SpanData.Annotation` value as its argument now, see below.+- Provide additional functions to add annotations:+  - `Instana.SDK.addAnnotationAt` which takes a path and an `Instana.SDK.Span.SpanData.Annotation` value+  - `Instana.SDK.addAnnotationToEntrySpan`, which has the same signature as `addAnnotation` but always unconditionally adds the annotation to the entry span, even if an intermediate or exit span is currently active+  - `Instana.SDK.addAnnotationToEntrySpanAt`, which has the same signature as `addAnnotationAt` but always unconditionally adds the annotation to the entry span, even if an intermediate or exit span is currently active+  - `Instana.SDK.addAnnotationValueToEntrySpanAt`, which has the same signature as `addAnnotationValueAt` but always unconditionally adds the annotation to the entry span, even if an intermediate or exit span is currently active+  - `Instana.SDK.addJsonValueToEntrySpanAt`, which has the same signature as `addAnnotationJsonValueAt` but always unconditionally adds the annotation to the entry span, even if an intermediate or exit span is currently active.+- Remove stability=experimental from cabal file.+ ## 0.8.0.0 - Fix: Apply secrets config from agent configuration. - Fix: Redact secrets from query parameters instead of removing them.

@@ -15,7 +15,7 @@  ``` extra-deps:-- instana-haskell-trace-sdk-0.8.0.0+- instana-haskell-trace-sdk-0.9.0.0 ```  Depending on the stack resolver you use, you might also need to add `aeson-extra` to your extra-deps:@@ -155,7 +155,7 @@  #### Best Practices -Make sure you have read Instana's [docs on custom tracing](https://docs.instana.io/quick_start/custom_tracing/#tips--best-practices) and in particular the [best practices section](https://docs.instana.io/quick_start/custom_tracing/#tips--best-practices). This documentation contains a lot of useful info for integrating Instana tracing into your code; among other things, it explains which [annotations](https://docs.instana.io/quick_start/custom_tracing/#list-of-processed-tags) can be added to spans (via `InstanaSDK.addAnnotation`, `InstanaSDK.addAnnotationAt`, and `InstanaSDK.addAnnotationValueAt`).+Make sure you have read Instana's [docs on custom tracing](https://docs.instana.io/quick_start/custom_tracing/#tips--best-practices) and in particular the [best practices section](https://docs.instana.io/quick_start/custom_tracing/#tips--best-practices). This documentation contains a lot of useful info for integrating Instana tracing into your code; among other things, it explains which [annotations](https://docs.instana.io/quick_start/custom_tracing/#list-of-processed-tags) can be added to spans (via `InstanaSDK.addAnnotation`, `InstanaSDK.addAnnotationAt`, `InstanaSDK.addAnnotationValueAt`, and `InstanaSDK.addJsonValueAt`).  Instana differentiates between so-called registered spans and SDK spans. Registered spans are usually created by automatic tracing and there is specialized handling for each registered in Instana's processing pipeline. SDK spans, in contrast, are the type of spans created by using a trace SDK (like the Haskell trace SDK or other, similar SDKs for other runtime platforms). SDK span are processed in a more generic fashion by Instana's processing pipeline. 

@@ -1,5 +1,5 @@ name:           instana-haskell-trace-sdk-version:        0.8.0.0+version:        0.9.0.0 synopsis:       SDK for adding custom Instana tracing support to Haskell applications. description:    Please also see the README on Github at <https://github.com/instana/haskell-trace-sdk#readme> homepage:       https://www.instana.com/@@ -11,7 +11,6 @@ license:        MIT license-file:   LICENSE build-type:     Simple-stability:      experimental cabal-version:  2.0  extra-source-files:

@@ -13,8 +13,13 @@     , InstanaContext     , addAnnotation     , addAnnotationAt+    , addAnnotationToEntrySpan+    , addAnnotationToEntrySpanAt     , addAnnotationValueAt+    , addAnnotationValueToEntrySpanAt     , addHttpTracingHeaders+    , addJsonValueAt+    , addJsonValueToEntrySpanAt     , addToErrorCount     , addWebsiteMonitoringBackEndCorrelation     , agentHost@@ -848,7 +853,7 @@     httpAnnotations' =       case capturedHeaders of         Just headers ->-          httpAnnotations ++ [SpanData.listAnnotation "header" headers]+          (SpanData.objectAnnotation "header" headers) : httpAnnotations         Nothing ->           httpAnnotations @@ -859,17 +864,22 @@ collectHeaders ::   [CI BSC8.ByteString]   -> [HTTPTypes.Header]-  -> Maybe [(String, String)]+  -> Maybe [Annotation] collectHeaders extraHeadersConfig allHeaders =   let-    all2 = allHeaders-    filtered = filterHeaders extraHeadersConfig all2-    serialized =+    filtered = filterHeaders extraHeadersConfig allHeaders+    listOfStringTuples =       fmap         (\(name, value) -> (BSC8.unpack $ CI.original name, BSC8.unpack value))         filtered+    headersAsAnnotations =+      fmap+        (\(name, value) -> SpanData.simpleAnnotation (T.pack name) value)+        listOfStringTuples   in-  if null serialized then Nothing else Just serialized+  if null headersAsAnnotations+    then Nothing+    else Just $ headersAsAnnotations   filterHeaders :: [CI BSC8.ByteString] -> [HTTPTypes.Header] -> [HTTPTypes.Header]@@ -970,11 +980,11 @@   let     (HTTPTypes.Status statusCode statusMessage) =       Wai.responseStatus response-  addAnnotationToEntrySpanAt context "http.status" $+  addAnnotationValueToEntrySpanAt context "http.status" $     SpanData.simpleValue statusCode   when     (statusCode >= 500 )-    (addAnnotationAt context "http.message" $+    (addAnnotationValueToEntrySpanAt context "http.message" $       SpanData.simpleValue $ BSC8.unpack statusMessage     ) @@ -1015,8 +1025,8 @@     (do        let          Just headers = capturedHeaders-       addAnnotationValueToEntrySpanAt context "http.header" $-         SpanData.listValue headers+       addAnnotationToEntrySpanAt context "http" $+         SpanData.objectAnnotation "header" headers     )  @@ -1163,16 +1173,16 @@                 collectHeaders extraHeadersConfig $                   HTTP.responseHeaders res -            addAnnotationAt context "http.status" $+            addAnnotationValueAt context "http.status" $               SpanData.simpleValue status              when               (Maybe.isJust capturedResponseHeaders)               (do                  let-                   Just responseHeaders = capturedResponseHeaders-                 addAnnotationValueAt context "http.header" $-                   SpanData.listValue responseHeaders+                   Just headers = capturedResponseHeaders+                 addAnnotationAt context "http" $+                     SpanData.objectAnnotation "header" headers               )              originalCheckResponse req res@@ -1192,8 +1202,8 @@       ]     httpAnnotations' =       case capturedRequestHeaders of-        Just requestHeaders ->-          httpAnnotations ++ [SpanData.listAnnotation "header" requestHeaders]+        Just headers ->+          (SpanData.objectAnnotation "header" headers) : httpAnnotations         Nothing ->           httpAnnotations @@ -1374,37 +1384,39 @@  -- |Adds an annotation to the currently active span. Call this between -- startEntry/startRootEntry/startExit and completeEntry/completeExit or+-- inside the IO action given to with withEntry/withExit/withRootEntry. Can be+-- called multiple times, data from multiple calls will be merged.+addAnnotation :: MonadIO m => InstanaContext -> Annotation -> m ()+addAnnotation context annotation =+  liftIO $ modifyCurrentSpan context+    (\span_ -> Span.addAnnotation annotation span_)+++-- |Adds an annotation to the currently active span. Call this between+-- startEntry/startRootEntry/startExit and completeEntry/completeExit or -- inside the IO action given to withEntry/withExit/withRootEntry. -- The given path can be a nested path, with path fragments separated by dots, -- like "http.url". This will result in -- "data": { --   ... --   "http": {---     "url": "..."+--     "url": {+--       "key_from_the_provided_annotation_value": ...+--     } --   }, --   ... -- } addAnnotationAt ::-  (MonadIO m, ToJSON a) =>+  MonadIO m =>   InstanaContext   -> Text-  -> a+  -> Annotation   -> m () addAnnotationAt context path value =   liftIO $ modifyCurrentSpan context     (\span_ -> Span.addAnnotationAt path value span_)  --- |Adds an annotation to the currently active span. Call this between--- startEntry/startRootEntry/startExit and completeEntry/completeExit or--- inside the IO action given to with withEntry/withExit/withRootEntry. Can be--- called multiple times, data from multiple calls will be merged.-addAnnotation :: MonadIO m => InstanaContext -> Annotation -> m ()-addAnnotation context annotation =-  liftIO $ modifyCurrentSpan context-    (\span_ -> Span.addAnnotation annotation span_)-- -- |Adds an annotation with the given value to the currently active span. Call -- this between startEntry/startRootEntry/startExit and -- completeEntry/completeExit or inside the IO action given to with@@ -1428,23 +1440,90 @@     (\span_ -> Span.addAnnotationValueAt path value span_)  --- |Adds an additional annotation to the currently active entry span, even if--- the currently active span is an exit child of that entry span.-addAnnotationToEntrySpanAt ::+-- |Adds a simple value (string, boolean, number) to the currently active span's+-- data section. Should not be used for objects or lists in case you intend to+-- merge them with additional values at the same path later. Call this between+-- startEntry/startRootEntry/startExit and completeEntry/completeExit or+-- inside the IO action given to withEntry/withExit/withRootEntry.+-- The given path can be a nested path, with path fragments separated by dots,+-- like "http.url". This will result in+-- "data": {+--   ...+--   "http": {+--     "url": "..."+--   },+--   ...+-- }+addJsonValueAt ::   (MonadIO m, ToJSON a) =>   InstanaContext   -> Text   -> a   -> m ()-addAnnotationToEntrySpanAt context path value =+addJsonValueAt context path value =+  liftIO $ modifyCurrentSpan context+    (\span_ -> Span.addJsonValueAt path value span_)+++-- |Adds an additional annotation to the currently active entry span, even if+-- the currently active span is an intermediate or exit child of that entry+-- span. Call this between startEntry/startRootEntry/startExit and+-- completeEntry/completeExit or inside the IO action given to with+-- withEntry/withExit/withRootEntry. Can be called multiple times, data from+-- multiple calls will be merged.+addAnnotationToEntrySpan ::+  MonadIO m =>+  InstanaContext+  -> Annotation+  -> m ()+addAnnotationToEntrySpan context annotation =   liftIO $ modifyCurrentEntrySpan context-    (\span_ -> Span.addAnnotationAt path value span_)+    (\span_ -> Span.addAnnotation annotation span_)   -- |Adds an additional annotation to the currently active entry span, even if--- the currently active span is an exit child of that entry span.+-- the currently active span is an intermediate or exit child of that entry+-- span. Call this between startEntry/startRootEntry/startExit and+-- completeEntry/completeExit or inside the IO action given to with+-- withEntry/withExit/withRootEntry. Can be called multiple times, data from+-- multiple calls will be merged. The given path can be a nested path, with path+-- fragments separated by dots, like "http.url". This will result in+-- "data": {+--   ...+--   "http": {+--     "url": {+--       "key_from_the_provided_annotation_value": ...+--     }+--   },+--   ...+-- }+addAnnotationToEntrySpanAt ::+  MonadIO m =>+  InstanaContext+  -> Text+  -> Annotation+  -> m ()+addAnnotationToEntrySpanAt context path annotation =+  liftIO $ modifyCurrentEntrySpan context+    (\span_ -> Span.addAnnotationAt path annotation span_)+++-- |Adds an additional annotation value to the currently active entry span, even+-- if the currently active span is an intermediate or exit child of that entry+-- span. Call this between startEntry/startRootEntry/startExit and+-- completeEntry/completeExit or inside the IO action given to with+-- withEntry/withExit/withRootEntry. Can be called multiple times, data from+-- multiple calls will be merged. The given path can be a nested path, with path+-- fragments separated by dots, like "http.url". This will result in+-- "data": {+--   ...+--   "http": {+--     "url": "..."+--   },+--   ...+-- } addAnnotationValueToEntrySpanAt ::-  (MonadIO m) =>+  MonadIO m =>   InstanaContext   -> Text   -> AnnotationValue@@ -1454,6 +1533,34 @@     (\span_ -> Span.addAnnotationValueAt path value span_)  +-- |Adds an additional annotation from a simple JSON value (string, number,+-- boolean) to the currently active entry span, even if+-- the currently active span is an intermediate or exit child of that entry+-- span. Should not be used for objects or lists in case you intend to merge+-- them with additional values at the same path later. Call this between+-- startEntry/startRootEntry/startExit and completeEntry/completeExit or inside+-- the IO action given to with withEntry/withExit/withRootEntry. Can be called+-- multiple times, data from  multiple calls will be merged.+-- The given path can be a nested path, with path fragments separated by dots,+-- like "http.url". This will result in+-- "data": {+--   ...+--   "http": {+--     "url": "..."+--   },+--   ...+-- }+addJsonValueToEntrySpanAt ::+  (MonadIO m, ToJSON a) =>+  InstanaContext+  -> Text+  -> a+  -> m ()+addJsonValueToEntrySpanAt context path value =+  liftIO $ modifyCurrentEntrySpan context+    (\span_ -> Span.addJsonValueAt path value span_)++ -- |Adds the Instana tracing headers -- (https://docs.instana.io/core_concepts/tracing/#http-tracing-headers) -- from the currently active span to the given HTTP client request.@@ -1729,8 +1836,8 @@   -- |Applies the given function to the currently active entry span, even if the--- currently active span is an exit child of that entry span. The entry span--- will be replaced with the result of the given function.+-- currently active span is an intermediate or exit child of that entry span.+-- The entry span will be replaced with the result of the given function. modifyCurrentEntrySpan ::   InstanaContext   -> (Span -> Span)@@ -1749,7 +1856,7 @@   -- |Applies the given function to the entry span (if present) on the given span--- stack, even if there is already an exit span on top of it.+-- stack, even if there is already an intermediate or exit span on top of it. mapCurrentEntrySpan :: (Span -> Span) -> Maybe SpanStack -> SpanStack mapCurrentEntrySpan fn stack =   Maybe.fromMaybe

@@ -10,6 +10,7 @@   , addAnnotation   , addAnnotationAt   , addAnnotationValueAt+  , addJsonValueAt   , addToErrorCount   , correlationId   , correlationType@@ -286,19 +287,16 @@     Exit exit   -> Exit $ ExitSpan.addAnnotation wrappedAnnotation exit  --- |Add a simple value (string, boolean, number) at the given path to the span's--- data section.-addAnnotationAt :: ToJSON a => Text -> a -> Span -> Span-addAnnotationAt path value span_ =+-- |Add an annotation at the given path to the span's data section.+addAnnotationAt :: Text -> Annotation -> Span -> Span+addAnnotationAt path annotation span_ =   let-    pathSegments = T.splitOn "." path-    lastPathSegment = List.last pathSegments-    pathPrefix = List.take (List.length pathSegments - 1) pathSegments+    pathPrefix = T.splitOn "." path     newData = List.foldr       (\nextPathSegment accumulated ->         SpanData.objectAnnotation nextPathSegment [accumulated]       )-      (SpanData.simpleAnnotation lastPathSegment value)+      annotation       pathPrefix   in   addAnnotation newData span_@@ -322,6 +320,14 @@       pathPrefix   in   addAnnotation newData span_+++-- |Add a simple value (string, boolean, number) at the given path to the span's+-- data section. Should not be used for objects or lists in case you intend to+-- merge them with additional values at the same path later.+addJsonValueAt :: ToJSON a => Text -> a -> Span -> Span+addJsonValueAt path value span_ =+  addAnnotationValueAt path (SpanData.simpleValue value) span_   wrapAnnotationIfNecessary :: Span -> Annotation -> Annotation

@@ -163,9 +163,9 @@   threadDelay $ 10 * 1000   InstanaSDK.addToErrorCount instana 2   addAnnotations instana (moreSpanData "exit")-  InstanaSDK.addAnnotationAt instana "nested.key1" ("nested.text.value1" :: String)-  InstanaSDK.addAnnotationAt instana "nested.key2" ("nested.text.value2" :: String)-  InstanaSDK.addAnnotationAt instana "nested.key3" (1604 :: Int)+  InstanaSDK.addJsonValueAt instana "nested.key1" ("nested.text.value1" :: String)+  InstanaSDK.addJsonValueAt instana "nested.key2" ("nested.text.value2" :: String)+  InstanaSDK.addJsonValueAt instana "nested.key3" (1604 :: Int)   return "exit done"  @@ -274,7 +274,7 @@   result <- doExitCallWithTags instana   InstanaSDK.incrementErrorCount instana   addAnnotations instana (moreSpanData "entry")-  InstanaSDK.addAnnotationAt+  InstanaSDK.addJsonValueAt     instana "nested.entry.key" ("nested.entry.value" :: String)   InstanaSDK.completeEntry instana   respondWithPlainText respond result@@ -289,7 +289,7 @@   result <- simulateExitCall   InstanaSDK.incrementErrorCount instana   addAnnotations instana (moreSpanData "exit")-  InstanaSDK.addAnnotationAt instana "nested.exit.key" ("nested.exit.value" :: String)+  InstanaSDK.addJsonValueAt instana "nested.exit.key" ("nested.exit.value" :: String)   InstanaSDK.completeExit instana   return result 

@@ -45,10 +45,10 @@           , "host"   .= ("127.0.0.1:1207" :: String)           , "url"    .= ("/http/bracket/api" :: String)           , "status" .= (200 :: Int)-          , "header" .= (-            [ ("X-Response-Header-On-Entry", "response header on entry value")-            , ("X-Request-Header-On-Entry", "request header on entry value")-            ] :: [(String, String)])+          , "header" .= (Aeson.object+            [ "X-Response-Header-On-Entry" .= ("response header on entry value" :: String)+            , "X-Request-Header-On-Entry"  .= ("request header on entry value" :: String)+            ])           ]         )       ]@@ -66,10 +66,10 @@           , "url"    .= ("http://127.0.0.1:1208/echo" :: String)           , "status" .= (200 :: Int)           , "params" .= ("some=query&parameters=2&pass=<redacted>" :: String)-          , "header" .= (-            [ ("X-Request-Header-On-Exit", "request header on exit value")-            , ("X-Response-Header-On-Exit", "response header on exit value")-            ] :: [(String, String)])+          , "header" .= (Aeson.object+            [ "X-Request-Header-On-Exit"  .= ("request header on exit value" :: String)+            , "X-Response-Header-On-Exit" .= ("response header on exit value" :: String)+            ])           ]         )       ]

@@ -51,7 +51,7 @@               (EntityDataRequest.arguments entityData)           , assertLabelIs               "sensorVersion"-              "0.8.0.0"+              "0.9.0.0"               (EntityDataRequest.sensorVersion entityData)           , assertCounterSatisfies               "startTime"

@@ -32,15 +32,26 @@     , TestLabel "shouldAddTwoAtSameNestedPath" shouldAddTwoAtSameNestedPath     , TestLabel "shouldAddMultipleRegistered" shouldAddMultipleRegistered     , TestLabel "shouldMergeListValues" shouldMergeListValues-    , TestLabel "shouldWrapAnnotationsInSdkCustomTagsForSdkSpans" shouldWrapAnnotationsInSdkCustomTagsForSdkSpans-    , TestLabel "shouldConvertToSimpleSpanFormat" shouldConvertToSimpleSpanFormat+    , TestLabel+        "shouldWrapAnnotationsInSdkCustomTagsForSdkSpans"+        shouldWrapAnnotationsInSdkCustomTagsForSdkSpans+    , TestLabel+        "shouldConvertToSimpleSpanFormat"+        shouldConvertToSimpleSpanFormat+    , TestLabel "shouldAddAnnotationAt" shouldAddAnnotationAt+    , TestLabel+        "shouldAddAnnotationAtToExistingKey"+        shouldAddAnnotationAtToExistingKey+    , TestLabel+        "shouldAddAnnotationAtToDeeplyNestedExistingKey"+        shouldAddAnnotationAtToDeeplyNestedExistingKey     ]   shouldAddStringNotNested :: Test shouldAddStringNotNested =   let-    span_ = Span.addAnnotationAt "path" ("value" :: String) $ registeredEntrySpan+    span_ = Span.addJsonValueAt "path" ("value" :: String) $ registeredEntrySpan     spanData = Span.spanData span_   in   TestCase $@@ -55,7 +66,7 @@ shouldAddStringNested =   let     span_ =-      Span.addAnnotationAt+      Span.addJsonValueAt         "nested.path"         ("value" :: String) $           registeredEntrySpan@@ -75,7 +86,7 @@ shouldAddStringDeeplyNested =   let     span_ =-      Span.addAnnotationAt+      Span.addJsonValueAt         "really.deeply.nested.path"         ("deeplyNestedValue" :: String)         registeredEntrySpan@@ -99,7 +110,7 @@ shouldAddIntDeeplyNested =   let     span_ =-      Span.addAnnotationAt+      Span.addJsonValueAt         "really.deeply.nested.path"         (42 :: Int) $           registeredEntrySpan@@ -123,7 +134,7 @@ shouldAddDoubleDeeplyNested =   let     span_ =-      Span.addAnnotationAt+      Span.addJsonValueAt         "really.deeply.nested.path"         (28.08 :: Double) $           registeredEntrySpan@@ -146,7 +157,7 @@ shouldAddBooleanDeeplyNested :: Test shouldAddBooleanDeeplyNested =   let-    span_ = Span.addAnnotationAt "really.deeply.nested.path" True $ registeredEntrySpan+    span_ = Span.addJsonValueAt "really.deeply.nested.path" True $ registeredEntrySpan     spanData = Span.spanData span_   in   TestCase $@@ -167,8 +178,8 @@ shouldAddTwoAtDifferentPaths =   let     span_ =-      Span.addAnnotationAt "nested2.key" (2 :: Int) $-      Span.addAnnotationAt "nested1.key" (1 :: Int) $+      Span.addJsonValueAt "nested2.key" (2 :: Int) $+      Span.addJsonValueAt "nested1.key" (1 :: Int) $       registeredEntrySpan     spanData = Span.spanData span_   in@@ -188,8 +199,8 @@ shouldAddTwoAtSamePath =   let     span_ =-      Span.addAnnotationAt "nested.key2" (2 :: Int) $-      Span.addAnnotationAt "nested.key1" (1 :: Int) $+      Span.addJsonValueAt "nested.key2" (2 :: Int) $+      Span.addJsonValueAt "nested.key1" (1 :: Int) $       registeredEntrySpan     spanData = Span.spanData span_   in@@ -209,8 +220,8 @@ shouldAddTwoAtSameNestedPath =   let     span_ =-      Span.addAnnotationAt "nested.deeper.key2" (2 :: Int) $-      Span.addAnnotationAt "nested.deeper.key1" (1 :: Int) $+      Span.addJsonValueAt "nested.deeper.key2" (2 :: Int) $+      Span.addJsonValueAt "nested.deeper.key1" (1 :: Int) $       registeredEntrySpan     spanData = Span.spanData span_   in@@ -232,13 +243,13 @@ shouldAddMultipleRegistered =   let     span_ =-      Span.addAnnotationAt "nested.key3" (12.07 :: Double) $-      Span.addAnnotationAt "nested.key2" (2 :: Int) $-      Span.addAnnotationAt "nested.key1" ("value.n.1" :: String) $-      Span.addAnnotationAt "list" (["one", "two", "three"] :: [String]) $-      Span.addAnnotationAt "key3" (16.04 :: Double) $-      Span.addAnnotationAt "key2" (13 :: Int) $-      Span.addAnnotationAt "key1" ("value1" :: String) $+      Span.addJsonValueAt "nested.key3" (12.07 :: Double) $+      Span.addJsonValueAt "nested.key2" (2 :: Int) $+      Span.addJsonValueAt "nested.key1" ("value.n.1" :: String) $+      Span.addJsonValueAt "list" (["one", "two", "three"] :: [String]) $+      Span.addJsonValueAt "key3" (16.04 :: Double) $+      Span.addJsonValueAt "key2" (13 :: Int) $+      Span.addJsonValueAt "key1" ("value1" :: String) $       registeredEntrySpan     spanData = Span.spanData span_   in@@ -285,12 +296,12 @@ shouldWrapAnnotationsInSdkCustomTagsForSdkSpans =   let     span_ =-      Span.addAnnotationAt "nested.key3" (12.07 :: Double) $-      Span.addAnnotationAt "nested.key2" (2 :: Int) $-      Span.addAnnotationAt "nested.key1" ("value.n.1" :: String) $-      Span.addAnnotationAt "key3" (16.04 :: Double) $-      Span.addAnnotationAt "key2" (13 :: Int) $-      Span.addAnnotationAt "key1" ("value1" :: String) $+      Span.addJsonValueAt "nested.key3" (12.07 :: Double) $+      Span.addJsonValueAt "nested.key2" (2 :: Int) $+      Span.addJsonValueAt "nested.key1" ("value.n.1" :: String) $+      Span.addJsonValueAt "key3" (16.04 :: Double) $+      Span.addJsonValueAt "key2" (13 :: Int) $+      Span.addJsonValueAt "key1" ("value1" :: String) $       sdkEntrySpan     spanData = Span.spanData span_   in@@ -320,7 +331,7 @@ shouldConvertToSimpleSpanFormat =   let     span_ =-      Span.addAnnotationAt "really.deeply.nested.path" True registeredEntrySpan+      Span.addJsonValueAt "really.deeply.nested.path" True registeredEntrySpan     simple = SimpleSpan.convert span_     traceId = SimpleSpan.traceId simple     spanId = SimpleSpan.spanId simple@@ -351,6 +362,75 @@           ]         ]       )+      spanData+++shouldAddAnnotationAt :: Test+shouldAddAnnotationAt =+  let+    span_ =+      Span.addAnnotationAt+        "parent"+        (SpanData.simpleAnnotation "key" ("value" :: String)) $+      registeredEntrySpan+    spanData = Span.spanData span_+  in+  TestCase $+    assertEqual "shouldAddAnnotationAt"+      (SpanData [+        SpanData.objectAnnotation "parent" [+          SpanData.simpleAnnotation "key" ("value" :: String)+        ]+      ])+      spanData+++shouldAddAnnotationAtToExistingKey :: Test+shouldAddAnnotationAtToExistingKey =+  let+    span_ =+      Span.addAnnotationAt+        "nested"+        (SpanData.simpleAnnotation "key3" ("value" :: String)) $+      Span.addJsonValueAt "nested.key2" (2 :: Int) $+      Span.addJsonValueAt "nested.key1" (1 :: Int) $+      registeredEntrySpan+    spanData = Span.spanData span_+  in+  TestCase $+    assertEqual "shouldAddAnnotationAtToExistingKey"+      (SpanData [+        SpanData.objectAnnotation "nested"+          [ SpanData.simpleAnnotation "key1" (1 :: Int)+          , SpanData.simpleAnnotation "key2" (2 :: Int)+          , SpanData.simpleAnnotation "key3" ("value" :: String)+          ]+      ])+      spanData+++shouldAddAnnotationAtToDeeplyNestedExistingKey :: Test+shouldAddAnnotationAtToDeeplyNestedExistingKey =+  let+    span_ =+      Span.addAnnotationAt+        "nested.deeper"+        (SpanData.simpleAnnotation "key3" ("value" :: String)) $+      Span.addJsonValueAt "nested.deeper.key2" (2 :: Int) $+      Span.addJsonValueAt "nested.deeper.key1" (1 :: Int) $+      registeredEntrySpan+    spanData = Span.spanData span_+  in+  TestCase $+    assertEqual "shouldAddAnnotationAtToExistingKey"+      (SpanData [+        SpanData.objectAnnotation "nested" [+          SpanData.objectAnnotation "deeper"+            [ SpanData.simpleAnnotation "key1" (1 :: Int)+            , SpanData.simpleAnnotation "key2" (2 :: Int)+            , SpanData.simpleAnnotation "key3" ("value" :: String) ]+        ]+      ])       spanData