eventlog-live 0.9.0.0 → 0.10.0.0

raw patch · 17 files changed

+634/−124 lines, 17 filesdep ~ghc-stack-profiler-corePVP ok

version bump matches the API change (PVP)

Dependency ranges changed: ghc-stack-profiler-core

API changes (from Hackage documentation)

Files

• CHANGELOG.md +16/−0
• README.md +66/−41
• data/config.schema.json +5/−1
• data/default.yaml +35/−15
• data/help.progDesc.txt +1/−0
• eventlog-live.cabal +3/−3
• src-app/GHC/Eventlog/Live/Otlp/Config.hs +20/−0
• src-app/GHC/Eventlog/Live/Otlp/Config/Types.hs +104/−12
• src-app/GHC/Eventlog/Live/Otlp/Processor/Heap.hs +55/−4
• src-app/GHC/Eventlog/Live/Otlp/Processor/Threads.hs +33/−21
• src-lib/GHC/Eventlog/Live/Data/Capability.hs +1/−1
• src-lib/GHC/Eventlog/Live/Data/Metric.hs +35/−2
• src-lib/GHC/Eventlog/Live/Data/Thread.hs +8/−0
• src-lib/GHC/Eventlog/Live/Machine/Analysis/Capability.hs +171/−15
• src-lib/GHC/Eventlog/Live/Machine/Analysis/Heap.hs +44/−0
• src-lib/GHC/Eventlog/Live/Machine/Analysis/Profile.hs +9/−8
• src-lib/GHC/Eventlog/Live/Machine/Core.hs +28/−1

@@ -1,3 +1,19 @@+### 0.10.0.0++- Add support for `ghc-stack-profiler-0.5.0.0` and drop support for previous versions.++### 0.9.1.0++- Add support for cumulative `Productivity` metric computed by Eventlog Live,+  rather than from `CapabilityUsage` in Grafana.++- Fix bug where `HeapAllocated` metric was erroneously treated as a delta+  metric and aggregated by summation. From this version onwards, it is treated+  as a cumulative metric and aggregated by taking the most recent measurement.++- Add workaround where `CapabilityUsage` metric is exported as a cumulative+  metric, as deltas are not supported by most collectors (Alloy, Prometheus).+ ### 0.9.0.0  - Add support for the following OTLP exporter configuration options:

@@ -11,9 +11,9 @@  Eventlog Live analyses the [eventlog](https://ghc.gitlab.haskell.org/ghc/doc/users_guide/runtime_control.html#rts-eventlog) of any Haskell program and streams the resulting telemetry data to any observability platform that supports the OpenTelemetry protocol, such as [Grafana Cloud](https://grafana.com), [HoneyComb](https://www.honeycomb.io/), or [Prometheus](https://prometheus.io/docs/guides/opentelemetry/). -The following shows the Grafana Heap Profiles dashboard for [`oddball`](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/examples/oddball/oddball-with-pipe.sh) running with _zero instrumentation_.+The following shows the Grafana Heap Profiles dashboard for [`oddball`](examples/oddball/oddball-with-pipe.sh) running with _zero instrumentation_. -![A screen recording of the Grafana Heap Profiles dashboard for the oddball example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/assets/oddball-with-pipe-2026-07-31.gif?raw=true)+![A screen recording of the Grafana Heap Profiles dashboard for the oddball example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.10.0.0/assets/oddball-with-pipe-2026-07-31.gif?raw=true)  Eventlog Live is designed to be _lightweight_, running alongside your application using only in a few megabytes of memory, and _highly configurable_, so that you only send the telemetry data you are interested in. While Eventlog Live works with zero instrumentation, it has support for several Haskell profiling packages that can enable new features. @@ -21,9 +21,9 @@    The [`eventlog-socket`](https://github.com/well-typed/eventlog-socket) package adds two features. First, it lets you to stream the eventlog over Unix domain and TCP/IP sockets. Secondly, it lets you control your program from the observability dashboard. The `eventlog-socket` instrumentation has builtin support that lets you toggle RTS features such as heap and stack profiling at runtime, but its control protocol has an easy-to-use plugin mechanism that lets you integrate other actions into your telemetry platform. -  The following shows dynamic control of heap profiling from the Grafana Heap Profiles dashboard for [`oddball`](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/examples/oddball/oddball-with-hT.sh) instrumented with `eventlog-socket`. When the _Stop_ button is pressed, the heap profiling is stopped, and the heap profile flatlines. When the _Start_ button is pressed, heap profiling is restarted.+  The following shows dynamic control of heap profiling from the Grafana Heap Profiles dashboard for [`oddball`](examples/oddball/oddball-with-hT.sh) instrumented with `eventlog-socket`. When the _Stop_ button is pressed, the heap profiling is stopped, and the heap profile flatlines. When the _Start_ button is pressed, heap profiling is restarted. -  ![A screen recording of the Grafana Heap Profiles dashboard for the oddball example program that shows dynamic control of heap profiling.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/assets/oddball-control-2026-07-31.gif?raw=true)+  ![A screen recording of the Grafana Heap Profiles dashboard for the oddball example program that shows dynamic control of heap profiling.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.10.0.0/assets/oddball-control-2026-07-31.gif?raw=true)  - **GHC Stack Profiler – Lightweight Call-Stack Profiles** @@ -31,13 +31,27 @@    The lightweight call-stack sampler has an approximate 5-10% overhead while running with a sampling interval of 10ms, whereas [GHC's builtin cost-centre profiling](https/downloads.haskell.org/ghc/latest/docs/users_guide/profiling.html#time-and-allocation-profiling) has an approximate 50% overhead for the instrumentation alone, i.e., prior to any cost-centres and without sampling. -  The following shows the Grafana Call-Stack Profiles dashboard for [`jumpy-jump`](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/examples/jumpy-jump/jumpy-jump-with-ghc-stack-profiler.sh) instrumented with `ghc-stack-profiler`.+  The following shows the Grafana Call-Stack Profiles dashboard for [`jumpy-jump`](examples/jumpy-jump/jumpy-jump-with-ghc-stack-profiler.sh) instrumented with `ghc-stack-profiler`. -  ![A screen recording of the Grafana Call-Stack Profiles dashboard for the jumpy-jump example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/assets/jumpy-jump-with-ghc-stack-profiler-2026-07-31.gif?raw=true)+  ![A screen recording of the Grafana Call-Stack Profiles dashboard for the jumpy-jump example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.10.0.0/assets/jumpy-jump-with-ghc-stack-profiler-2026-07-31.gif?raw=true) +## Table of Contents++- [Demos](#demos)+- [Getting Started](#getting-started)+  - [Eventlog Live – The Basic Version](#eventlog-live-the-basic-version)+  - [Eventlog Live with Eventlog Socket](#eventlog-live-with-eventlog-socket)+  - [Eventlog Live with Cost-Centre Stack Profiling](#eventlog-live-with-cost-centre-stack-profiling)+  - [Eventlog Live with Heap Profiling by Info Table](#eventlog-live-with-heap-profiling-by-info-table)+  - [Eventlog Live with GHC Stack Profiler](#eventlog-live-with-ghc-stack-profiler)+- [Fine-Tuning Eventlog Live](#fine-tuning-eventlog-live)+  - [Configuration Files](#configuration-files)+  - [Restricted Event Classes](#restricted-event-classes)+  - [Info Table and Cost-Centre Databases](#info-table-and-cost-centre-databases)+ ## Demos -The [`demo`](demo/) directory contains a [Docker Compose](https://docs.docker.com/compose/) configuration that runs Grafana and all required data sources and opens an OTLP gRPC receiver on port 4317. This configuration is used in [Getting Started](#getting-started) and by the various scripts under the [`examples`](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/examples/) directory.+The [`demo`](demo/) directory contains a [Docker Compose](https://docs.docker.com/compose/) configuration that runs Grafana and all required data sources and opens an OTLP gRPC receiver on port 4317. This configuration is used in [Getting Started](#getting-started) and by the various scripts under the [`examples`](examples/) directory. You can import the dashboards under [`demo/config/grafana-dashboards`](demo/config/grafana-dashboards/) into your Grafana setup, but beware that you'll have to change the variables and visualisations to use your data sources and change the buttons to use your control server. The dashboards under [`demo-grafana-cloud`](demo-grafana-cloud/grafana-dashboards/) were prepared to be shared with other Grafana instances. @@ -111,6 +125,7 @@ # OpenTelemetry Configuration export OTEL_LOG_LEVEL="debug" export OTEL_SERVICE_NAME="your-application"+export OTEL_RESOURCE_ATTRIBUTES="service.instance.id=$(uuidgen)" export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"  # Create a pipe for the eventlog@@ -143,17 +158,17 @@    You should be greeted by a dashboard that looks something like this: -  ![A screen recording of the Grafana Heap Profiles dashboard for the oddball example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/assets/oddball-with-pipe-2026-07-31.gif?raw=true)+  ![A screen recording of the Grafana Heap Profiles dashboard for the oddball example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.10.0.0/assets/oddball-with-pipe-2026-07-31.gif?raw=true)  - Select '☰ > Dashboards > Logs'    You should be greeted by a dashboard that looks something like this: -  ![A screen capture of the Grafana Logs dashboard for the oddball example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/assets?raw=true/oddball-with-pipe-logs-2026-07-31.png)+  ![A screen capture of the Grafana Logs dashboard for the oddball example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.10.0.0/assets/oddball-with-pipe-logs-2026-07-31.png?raw=true)    This dashboard will include a separate tab that shows the logs for `eventlog-live-otlp`: -  ![A screen capture of the Grafana Logs dashboard for the eventlog-live-otlp program for oddball.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/assets/eventlog-live-otlp-for-oddball-with-pipe-logs-2026-07-31.png?raw=true)+  ![A screen capture of the Grafana Logs dashboard for the eventlog-live-otlp program for oddball.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.10.0.0/assets/eventlog-live-otlp-for-oddball-with-pipe-logs-2026-07-31.png?raw=true)  Let's briefly discuss what this script does: @@ -162,6 +177,7 @@   The Eventlog Live exporter is configured using the [OpenTelemetry environment variables](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables).   - `OTEL_LOG_LEVEL` sets Eventlog Live's log level.   - `OTEL_SERVICE_NAME` sets the service name under which your application's telemetry shows up on your dashboard.+  - `OTEL_RESOURCE_ATTRIBUTES` sets other resource attributes. In this case, it's used to set `service.instance.id` to a unique UUID that identifies the specific instance of your application.   - `OTEL_EXPORTER_OTLP_PROTOCOL` selects the exporter's protocol. The default is `http/protobuf`, but the docker setup uses gRPC. If the protocol is gRPC, the default endpoint is `http://localhost:4317`, which is what our docker setup uses.  - **The Eventlog.**@@ -295,6 +311,7 @@ # OpenTelemetry Configuration export OTEL_LOG_LEVEL="debug" export OTEL_SERVICE_NAME="your-application"+export OTEL_RESOURCE_ATTRIBUTES="service.instance.id=$(uuidgen)" export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"  # Start your application@@ -319,7 +336,7 @@  If you run these commands, you should be greeted by a dashboard that looks something like this: -![A screen recording of the Grafana Heap Profiles dashboard for the oddball example program that shows dynamic control of heap profiling.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/assets/oddball-control-2026-07-31.gif?raw=true)+![A screen recording of the Grafana Heap Profiles dashboard for the oddball example program that shows dynamic control of heap profiling.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.10.0.0/assets/oddball-control-2026-07-31.gif?raw=true)  The Start and Stop buttons can now be used to enable/disable Heap Profiling and the Census button can now be used to request a single heap census. @@ -384,6 +401,7 @@ # OpenTelemetry Configuration export OTEL_LOG_LEVEL="debug" export OTEL_SERVICE_NAME="your-application"+export OTEL_RESOURCE_ATTRIBUTES="service.instance.id=$(uuidgen)" export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"  # Start your application@@ -403,7 +421,6 @@   --control-cors-ignore-failure  # See: examples/jumpy-jump/jumpy-jump-with-cost-centre-profiler.sh-#      examples/jumpy-jump/jumpy-jump-with-cost-centre-profiler-with-pipe.sh ```  If you run these commands, you should start seeing profiles show up on your dashboard.@@ -414,7 +431,7 @@    You should be greeted by a dashboard that looks something like this: -  ![A screen capture of the Grafana Cost-Centre Stack Profiles dashboard for the jumpy-jump example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/assets/jumpy-jump-with-cost-centre-profiler-2026-07-31.png?raw=true)+  ![A screen capture of the Grafana Cost-Centre Stack Profiles dashboard for the jumpy-jump example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.10.0.0/assets/jumpy-jump-with-cost-centre-profiler-2026-07-31.png?raw=true)  If your application is instrumented with Eventlog Socket, the Start and Stop buttons should enable/disable cost-centre stack profiling. @@ -478,6 +495,7 @@ # OpenTelemetry Configuration export OTEL_LOG_LEVEL="debug" export OTEL_SERVICE_NAME="your-application"+export OTEL_RESOURCE_ATTRIBUTES="service.instance.id=$(uuidgen)" export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"  # Start your application@@ -502,7 +520,7 @@  If you run these commands, you should be greeted by a dashboard that looks something like this: -![A screen recording of the Grafana Heap Profiles dashboard for the oddball example program using Heap Profiling by Info Table.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/assets/oddball-with-hi-2026-07-31.gif?raw=true)+![A screen recording of the Grafana Heap Profiles dashboard for the oddball example program using Heap Profiling by Info Table.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.10.0.0/assets/oddball-with-hi-2026-07-31.gif?raw=true)  Notably, your Heap Profile should show Info Table information. If you ran Heap Profiles with different breakdowns, you can select the appropriate breakdown in the Heap Profile Breakdown dropdown menu. @@ -514,45 +532,51 @@  ### Eventlog Live with GHC Stack Profiler -[GHC Stack Profiler](https://github.com/well-typed/ghc-stack-profiler) is a lightweight profiler that samples the runtime call-stack and writes these to the eventlog. It only has an 5-10% overhead while running. When compiled with the `+control` feature flag, it supports Eventlog Socket's command protocol, which lets you enable/disable samples at runtime.+[GHC Stack Profiler](https://github.com/well-typed/ghc-stack-profiler) is a light-weight profiler that samples the runtime call-stack and writes these to the eventlog. The instrumentation alone has zero overhead and, when it's running, it has [about 2-8% overhead](https://github.com/well-typed/ghc-stack-profiler/#benchmarks). When compiled with the `+control` feature flag, it supports Eventlog Socket's command protocol, which lets you enable/disable samples at runtime.  To instrument your application with GHC Stack Profiler, you need to make three changes: -1. Add `ghc-stack-profiler` to the `build-depends` for your application:+1.  Add `ghc-stack-profiler` to the `build-depends` for your application: -   ```diff-   executable your-application-     ...+    ```diff+      executable your-application+        ... -     build-depends:-       ...-   +   , ghc-stack-profiler >=0.4 && <0.5-   ```+        build-depends:+          ...+    +     , ghc-stack-profiler ==0.5.0.0+    ``` -2. Instrument your main function:+    > ⚠️ **Warning:** If you're using `ghc-stack-profiler-speedscope`, `eventlog-live-otlp`, or any other program that processes the eventlog produced by `ghc-stack-profiler`, it is important that both are built with the same version of `ghc-stack-profiler-core`. -   ```diff-     module Main where-     ...+2.  Instrument your main function: -   + import qualified GHC.Stack.Profiler as GSP+    ```diff+      module Main where+      ... -     main :: IO ()-     main = do-   +   GSP.withRootStackProfiler True $ \manager ->-   +     GSP.withStackProfiler manager (GSP.SampleIntervalMs 100) $-           ...-   ```+    + import GHC.Stack.Profiler (withProfilerFromEnv) -   The current version of GHC Stack Profiler requires you to set the sampling interval at compile-time.+      main :: IO ()+      main =+    +   withProfilerFromEnv $+          ...+    ``` -3. Build your application and its dependencies with info table maps.+    > ℹ️ **Tip:**+    > If you prefer not to configure your program from the environment, the [GHC.Stack.Profiler](https://hackage.haskell.org/package/ghc-stack-profiler/docs/GHC-Stack-Profiler.html) exposes a variety of function that instrument your program. -   For detailed instructions, see [Eventlog Live with Heap Profiling by Info Table](#eventlog-live-with-heap-profiling-by-info-table).+3.  Build your application and its dependencies with info table maps. +    For detailed instructions, see [Eventlog Live with Heap Profiling by Info Table](#eventlog-live-with-heap-profiling-by-info-table).+ To start monitoring your application, run your application and Evenlog Live. The following script builds on the example from [Eventlog Live with Heap Profiling by Info Table](#eventlog-live-with-heap-profiling-by-info-table), but using GHC Stack Profiler is independent from Eventlog Socket and while it needs info table maps, it does not require _running_ an Info Table Profile (`-hi`).  ```sh+# GHC Stack Profiler Configuration+export GHC_STACK_PROFILER="ON" # or any other non-empty value+export GHC_STACK_PROFILER_SAMPLE_INTERVAL="10" # milliseconds+ # Eventlog Socket Configuration export GHC_EVENTLOG_UNIX_PATH="/tmp/eventlog.sock" export GHC_EVENTLOG_WAIT="true"@@ -560,6 +584,7 @@ # OpenTelemetry Configuration export OTEL_LOG_LEVEL="debug" export OTEL_SERVICE_NAME="your-application"+export OTEL_RESOURCE_ATTRIBUTES="service.instance.id=$(uuidgen)" export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"  # Start your application@@ -590,18 +615,18 @@    You should be greeted by a dashboard that looks something like this: -  ![A screen recording of the Grafana Call-Stack Profiles dashboard for the jumpy-jump example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.9.0.0/assets/jumpy-jump-with-ghc-stack-profiler-2026-07-31.gif?raw=true)+  ![A screen recording of the Grafana Call-Stack Profiles dashboard for the jumpy-jump example program.](https://github.com/well-typed/eventlog-live/blob/eventlog-live-v0.10.0.0/assets/jumpy-jump-with-ghc-stack-profiler-2026-07-31.gif?raw=true)  If your application is instrumented with Eventlog Socket, the Start and Stop buttons should enable/disable call-stack profiling.  > ℹ️ **Tip:**-> You can use the [`annotateStackIO`](https://hackage-content.haskell.org/package/ghc-experimental-9.1401.0/docs/GHC-Stack-Annotation-Experimental.html#v:annotateStackIO) functions from `ghc-experimental` to push annotation frames onto the call-stack at runtime.+> You can use the [`annotateStackIO`](https://hackage-content.haskell.org/package/ghc-stack-annotations/docs/GHC-Stack-Annotation.html#v:annotateStackIO) functions from [`ghc-stack-annotations`](https://hackage-content.haskell.org/package/ghc-stack-annotations) to push annotation frames onto the call-stack at runtime. > These annotation frames are visible in call-stack profiles captured by GHC Stack Profiler. > See [Better Haskell stack traces via user annotations](https://www.well-typed.com/blog/2025/09/better-haskell-stack-traces/).  > ⚠️ **Warning:**-> Due to a bug in GHC, copying the call-stack may cause a segfault at runtime in applications built with GHC 9.14 and older.-> If you use GHC Stack Profiler in production, you should build your application with GHC 10 or later.+> Due to a bug in GHC, copying the call-stack may cause a segfault at runtime in applications built with GHC 9.14.1 and older.+> If you use GHC Stack Profiler in production, you should build your application with GHC 9.14.2 or newer.  ## Fine-Tuning Eventlog Live 

@@ -28,7 +28,11 @@             "heap_size": { "$ref": "#/definitions/metric" },             "mem_current": { "$ref": "#/definitions/metric" },             "mem_needed": { "$ref": "#/definitions/metric" },-            "mem_returned": { "$ref": "#/definitions/metric" }+            "mem_returned": { "$ref": "#/definitions/metric" },+            "gc_copied": { "$ref": "#/definitions/metric" },+            "gc_slop": { "$ref": "#/definitions/metric" },+            "gc_fragmentation": { "$ref": "#/definitions/metric" },+            "productivity": { "$ref": "#/definitions/metric" }           },           "additionalProperties": false         },

@@ -65,6 +65,10 @@ # default to the values in this file. Otherwise, the processor is _disabled_. processors:   logs:+    internal_log_message:+      name: eventlog_live_InternalLogMessage+      description: An internal eventlog-live log message.+      export: 30s     thread_label:       name: ghc_eventlog_ThreadLabel       description: A thread label.@@ -77,10 +81,6 @@       name: ghc_eventlog_UserMessage       description: A user log message.       export: 30s-    internal_log_message:-      name: eventlog_live_InternalLogMessage-      description: An internal eventlog-live log message.-      export: 30s   metrics:     blocks_size:       name: ghc_eventlog_BlocksSize@@ -91,12 +91,27 @@       name: ghc_eventlog_CapabilityUsageDuration       description: The duration of each capability usage span.       aggregate: 1s-      export: false # The current format of this metric breaks various OTLP receivers+      export: 30s+    gc_copied:+      name: ghc_eventlog_GcCopied+      description: The number of bytes copied in one garbage collection pass.+      aggregate: 1s+      export: 30s+    gc_fragmentation:+      name: ghc_eventlog_GcFragmentation+      description: The number of bytes of fragmentation found in one garbage collection pass.+      aggregate: 1s+      export: 30s+    gc_slop:+      name: ghc_eventlog_GcSlop+      description: The number of slop bytes found in one garbage collection pass.+      aggregate: 1s+      export: 30s     heap_allocated:       name: ghc_eventlog_HeapAllocated       description: The size of a newly allocated chunk of heap.       aggregate: 1s-      export: false # The current format of this metric breaks various OTLP receivers+      export: 30s     heap_live:       name: ghc_eventlog_HeapLive       description: The current heap size, calculated by the allocated number of megablocks.@@ -127,15 +142,11 @@       description: The number of megablocks currently being returned to the OS.       aggregate: 1s       export: 30s-  traces:-    capability_usage:-      name: ghc_eventlog_CapabilityUsage-      description: A trace of capability usage (either mutator thread or garbage collection).-      export: false # These spans easily overwhelm most OTLP receivers-    thread_state:-      name: ghc_eventlog_ThreadState-      description: A trace of thread state changes (either running or stopped).-      export: false # These spans easily overwhelm most OTLP receivers+    productivity:+      name: ghc_eventlog_Productivity+      description: The ratio between GC and mutator usage.+      aggregate: 1s+      export: 30s   profiles:     call_stack_profile:       name: ghc_eventlog_CallStackProfile@@ -145,3 +156,12 @@       name: ghc_eventlog_CostCentreStackProfile       description: A GHC cost-centre stack profile.       export: 30s+  traces:+    capability_usage:+      name: ghc_eventlog_CapabilityUsage+      description: A trace of capability usage (either mutator thread or garbage collection).+      export: false # These spans easily overwhelm most OTLP receivers+    thread_state:+      name: ghc_eventlog_ThreadState+      description: A trace of thread state changes (either running or stopped).+      export: false # These spans easily overwhelm most OTLP receivers

@@ -4,6 +4,7 @@    # Configure OpenTelemetry exporter   export OTEL_SERVICE_NAME="your-application"+  export OTEL_RESOURCE_ATTRIBUTES="service.instance.id=$(uuidgen)"   export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"   export OTEL_EXPORTER_OTLP_ENDPOINT="localhost:4317" 

@@ -1,6 +1,6 @@ cabal-version: 3.0 name: eventlog-live-version: 0.9.0.0+version: 0.10.0.0 synopsis: Real-time monitoring for any Haskell application with little to no instrumentation description: Real-time monitoring for any Haskell application with little to no instrumentation! license: BSD-3-Clause@@ -32,7 +32,7 @@ source-repository this   type: git   location: https://github.com/well-typed/eventlog-live.git-  tag: eventlog-live-v0.9.0.0+  tag: eventlog-live-v0.10.0.0   subdir: eventlog-live  -- This flag enables the control command server, which is an HTTP endpoint@@ -174,7 +174,7 @@     containers >=0.6 && <0.9,     dlist >=1.0 && <1.1,     ghc-events >=0.20 && <0.21,-    ghc-stack-profiler-core >=0.3 && <0.4,+    ghc-stack-profiler-core ==0.5.0.0,     hashable >=1.4 && <1.6,     ipedb >=0.2 && <0.3,     machines >=0.7.4 && <0.8,

@@ -43,8 +43,12 @@   MemCurrentMetric (..),   MemNeededMetric (..),   MemReturnedMetric (..),+  GcCopiedMetric (..),+  GcSlopMetric (..),+  GcFragmentationMetric (..),   HeapProfSampleMetric (..),   CapabilityUsageMetric (..),+  ProductivityMetric (..),    -- *** Trace processor configuration types   Traces (..),@@ -227,6 +231,18 @@   def :: MemReturnedMetric   def = $(getDefault @'["processors", "metrics", "memReturned"] defaultConfig) +instance Default GcCopiedMetric where+  def :: GcCopiedMetric+  def = $(getDefault @'["processors", "metrics", "gcCopied"] defaultConfig)++instance Default GcSlopMetric where+  def :: GcSlopMetric+  def = $(getDefault @'["processors", "metrics", "gcSlop"] defaultConfig)++instance Default GcFragmentationMetric where+  def :: GcFragmentationMetric+  def = $(getDefault @'["processors", "metrics", "gcFragmentation"] defaultConfig)+ instance Default HeapProfSampleMetric where   def :: HeapProfSampleMetric   def = $(getDefault @'["processors", "metrics", "heapProfSample"] defaultConfig)@@ -234,6 +250,10 @@ instance Default CapabilityUsageMetric where   def :: CapabilityUsageMetric   def = $(getDefault @'["processors", "metrics", "capabilityUsage"] defaultConfig)++instance Default ProductivityMetric where+  def :: ProductivityMetric+  def = $(getDefault @'["processors", "metrics", "productivity"] defaultConfig)  -- NOTE: This should be kept in sync with the list of traces. --       Specifically, there should be a `Default` instance for every trace.

@@ -35,7 +35,11 @@   MemNeededMetric (..),   MemReturnedMetric (..),   HeapProfSampleMetric (..),+  GcCopiedMetric (..),+  GcSlopMetric (..),+  GcFragmentationMetric (..),   CapabilityUsageMetric (..),+  ProductivityMetric (..),    -- *** Trace processor configuration types   Traces (..),@@ -191,8 +195,12 @@   , memCurrent :: Maybe MemCurrentMetric   , memNeeded :: Maybe MemNeededMetric   , memReturned :: Maybe MemReturnedMetric+  , gcCopied :: Maybe GcCopiedMetric+  , gcSlop :: Maybe GcSlopMetric+  , gcFragmentation :: Maybe GcFragmentationMetric   , heapProfSample :: Maybe HeapProfSampleMetric   , capabilityUsage :: Maybe CapabilityUsageMetric+  , productivity :: Maybe ProductivityMetric   }   deriving (Lift, Show) @@ -200,17 +208,21 @@   parseYAML :: YAML.Node YAML.Pos -> YAML.Parser Metrics   parseYAML =     -- NOTE: This should be kept in sync with the list of metrics.-    YAML.withMap "Metrics" $ \m ->-      Metrics-        <$> m .:? "heap_allocated"-        <*> m .:? "blocks_size"-        <*> m .:? "heap_size"-        <*> m .:? "heap_live"-        <*> m .:? "mem_current"-        <*> m .:? "mem_needed"-        <*> m .:? "mem_returned"-        <*> m .:? "heap_prof_sample"-        <*> m .:? "capability_usage"+    YAML.withMap "Metrics" $ \m -> do+      heapAllocated <- m .:? "heap_allocated"+      blocksSize <- m .:? "blocks_size"+      heapSize <- m .:? "heap_size"+      heapLive <- m .:? "heap_live"+      memCurrent <- m .:? "mem_current"+      memNeeded <- m .:? "mem_needed"+      memReturned <- m .:? "mem_returned"+      gcCopied <- m .:? "gc_copied"+      gcSlop <- m .:? "gc_slop"+      gcFragmentation <- m .:? "gc_fragmentation"+      heapProfSample <- m .:? "heap_prof_sample"+      capabilityUsage <- m .:? "capability_usage"+      productivity <- m .:? "productivity"+      pure Metrics{..}  instance ToYAML Metrics where   toYAML :: Metrics -> YAML.Node ()@@ -224,8 +236,12 @@       , "mem_current" .= metrics.memCurrent       , "mem_needed" .= metrics.memNeeded       , "mem_returned" .= metrics.memReturned+      , "gc_copied" .= metrics.gcCopied+      , "gc_slop" .= metrics.gcSlop+      , "gc_fragmentation" .= metrics.gcFragmentation       , "heap_prof_sample" .= metrics.heapProfSample       , "capability_usage" .= metrics.capabilityUsage+      , "productivity" .= metrics.productivity       ]  {- |@@ -522,8 +538,65 @@   toYAML = genericToYAMLMetricProcessorConfig  {- |-The configuration options for `GHC.Eventlog.Live.Machine.Analysis.Capability.processCapabilityUsageMetrics`.+The configuration options for the @copied@ field for `GHC.Eventlog.Live.Machine.Analysis.Heap.processGcStats`. -}+data GcCopiedMetric = GcCopiedMetric+  { name :: Maybe Text+  , description :: Maybe Text+  , aggregate :: Maybe AggregationStrategy+  , export :: Maybe ExportStrategy+  }+  deriving (Lift, Show)++instance FromYAML GcCopiedMetric where+  parseYAML :: YAML.Node YAML.Pos -> YAML.Parser GcCopiedMetric+  parseYAML = genericParseYAMLMetricProcessorConfig "GcCopiedMetric" GcCopiedMetric++instance ToYAML GcCopiedMetric where+  toYAML :: GcCopiedMetric -> YAML.Node ()+  toYAML = genericToYAMLMetricProcessorConfig++{- |+The configuration options for the @copied@ field for `GHC.Eventlog.Live.Machine.Analysis.Heap.processGcStats`.+-}+data GcSlopMetric = GcSlopMetric+  { name :: Maybe Text+  , description :: Maybe Text+  , aggregate :: Maybe AggregationStrategy+  , export :: Maybe ExportStrategy+  }+  deriving (Lift, Show)++instance FromYAML GcSlopMetric where+  parseYAML :: YAML.Node YAML.Pos -> YAML.Parser GcSlopMetric+  parseYAML = genericParseYAMLMetricProcessorConfig "GcSlopMetric" GcSlopMetric++instance ToYAML GcSlopMetric where+  toYAML :: GcSlopMetric -> YAML.Node ()+  toYAML = genericToYAMLMetricProcessorConfig++{- |+The configuration options for the @copied@ field for `GHC.Eventlog.Live.Machine.Analysis.Heap.processGcStats`.+-}+data GcFragmentationMetric = GcFragmentationMetric+  { name :: Maybe Text+  , description :: Maybe Text+  , aggregate :: Maybe AggregationStrategy+  , export :: Maybe ExportStrategy+  }+  deriving (Lift, Show)++instance FromYAML GcFragmentationMetric where+  parseYAML :: YAML.Node YAML.Pos -> YAML.Parser GcFragmentationMetric+  parseYAML = genericParseYAMLMetricProcessorConfig "GcFragmentationMetric" GcFragmentationMetric++instance ToYAML GcFragmentationMetric where+  toYAML :: GcFragmentationMetric -> YAML.Node ()+  toYAML = genericToYAMLMetricProcessorConfig++{- |+The configuration options for `GHC.Eventlog.Live.Machine.Analysis.Capability.processCapabilityUsageDurationData`.+-} data CapabilityUsageMetric = CapabilityUsageMetric   { name :: Maybe Text   , description :: Maybe Text@@ -538,6 +611,25 @@  instance ToYAML CapabilityUsageMetric where   toYAML :: CapabilityUsageMetric -> YAML.Node ()+  toYAML = genericToYAMLMetricProcessorConfig++{- |+The configuration options for `GHC.Eventlog.Live.Machine.Analysis.Capability.processProductivityData`.+-}+data ProductivityMetric = ProductivityMetric+  { name :: Maybe Text+  , description :: Maybe Text+  , aggregate :: Maybe AggregationStrategy+  , export :: Maybe ExportStrategy+  }+  deriving (Lift, Show)++instance FromYAML ProductivityMetric where+  parseYAML :: YAML.Node YAML.Pos -> YAML.Parser ProductivityMetric+  parseYAML = genericParseYAMLMetricProcessorConfig "ProductivityMetric" ProductivityMetric++instance ToYAML ProductivityMetric where+  toYAML :: ProductivityMetric -> YAML.Node ()   toYAML = genericToYAMLMetricProcessorConfig  -------------------------------------------------------------------------------

@@ -16,7 +16,7 @@ import Data.Machine (Process, ProcessT, asParts, echo, mapping, (~>)) import Data.Proxy (Proxy (..)) import GHC.Eventlog.Live.Logger (Logger)-import GHC.Eventlog.Live.Machine.Analysis.Heap (MemReturnData (..))+import GHC.Eventlog.Live.Machine.Analysis.Heap (GcStatsData (..), MemReturnData (..)) import GHC.Eventlog.Live.Machine.Analysis.Heap qualified as M import GHC.Eventlog.Live.Machine.Core (Tick) import GHC.Eventlog.Live.Machine.Core qualified as M@@ -24,7 +24,7 @@ import GHC.Eventlog.Live.Otlp.Config qualified as C import GHC.Eventlog.Live.Otlp.Config.Types (FullConfig (..)) import GHC.Eventlog.Live.Otlp.Processor.Common.Core (runIf)-import GHC.Eventlog.Live.Otlp.Processor.Common.Metrics (MetricProcessor (..), asGauge, asSum, runMetricProcessor, viaLast, viaSum)+import GHC.Eventlog.Live.Otlp.Processor.Common.Metrics (MetricProcessor (..), asGauge, asSum, runMetricProcessor, viaLast) import GHC.RTS.Events (Event (..), HeapProfBreakdown (..)) import IpeDB.Database qualified as DB import IpeDB.Types.InfoProv qualified as IP@@ -50,6 +50,7 @@     , processHeapSize fullConfig     , processHeapLive fullConfig     , processMemReturn fullConfig+    , processGcStats fullConfig     , processHeapProfSample verbosity maybeInfoProvTable maybeHeapProfBreakdown fullConfig     ] @@ -62,12 +63,12 @@     MetricProcessor       { metricProcessorProxy = Proxy @"heapAllocated"       , dataProcessor = M.processHeapAllocatedData-      , aggregators = viaSum+      , aggregators = viaLast       , postProcessor = echo       , unit = "By"       , asMetric'Data =           asSum-            [ OM.aggregationTemporality .~ OM.AGGREGATION_TEMPORALITY_DELTA+            [ OM.aggregationTemporality .~ OM.AGGREGATION_TEMPORALITY_CUMULATIVE             , OM.isMonotonic .~ True             ]       }@@ -166,6 +167,56 @@   C.processorEnabled (.metrics) (.memCurrent) fullConfig     || C.processorEnabled (.metrics) (.memNeeded) fullConfig     || C.processorEnabled (.metrics) (.memReturned) fullConfig++--------------------------------------------------------------------------------+-- GcStats++processGcStats :: FullConfig -> Process (Tick (WithStartTime Event)) (Tick (DList OM.Metric))+processGcStats fullConfig =+  runIf (shouldComputeGcStats fullConfig) $+    M.liftTick M.processGcStatsData+      ~> M.fanoutTick+        [ runMetricProcessor+            MetricProcessor+              { metricProcessorProxy = Proxy @"gcCopied"+              , dataProcessor = mapping (fmap (.copied))+              , aggregators = viaLast+              , postProcessor = echo+              , unit = "By"+              , asMetric'Data = asGauge+              }+            fullConfig+        , runMetricProcessor+            MetricProcessor+              { metricProcessorProxy = Proxy @"gcSlop"+              , dataProcessor = mapping (fmap (.slop))+              , aggregators = viaLast+              , postProcessor = echo+              , unit = "By"+              , asMetric'Data = asGauge+              }+            fullConfig+        , runMetricProcessor+            MetricProcessor+              { metricProcessorProxy = Proxy @"gcFragmentation"+              , dataProcessor = mapping (fmap (.fragmentation))+              , aggregators = viaLast+              , postProcessor = echo+              , unit = "By"+              , asMetric'Data = asGauge+              }+            fullConfig+        ]++{- |+Internal helper.+Determine whether the MemReturn data should be computed.+-}+shouldComputeGcStats :: FullConfig -> Bool+shouldComputeGcStats fullConfig =+  C.processorEnabled (.metrics) (.gcCopied) fullConfig+    || C.processorEnabled (.metrics) (.gcSlop) fullConfig+    || C.processorEnabled (.metrics) (.gcFragmentation) fullConfig  -------------------------------------------------------------------------------- -- HeapProfSample

@@ -17,6 +17,7 @@ import Data.Machine (ProcessT, asParts, echo, mapping, (~>)) import Data.Machine.Fanout (fanout) import Data.Proxy (Proxy (..))+import GHC.Eventlog.Live.Data.Metric qualified as M import GHC.Eventlog.Live.Logger (Logger) import GHC.Eventlog.Live.Machine.Analysis.Capability qualified as M import GHC.Eventlog.Live.Machine.Analysis.Thread qualified as M@@ -27,7 +28,7 @@ import GHC.Eventlog.Live.Otlp.Config qualified as C import GHC.Eventlog.Live.Otlp.Config.Types (FullConfig (..)) import GHC.Eventlog.Live.Otlp.Processor.Common.Core (runIf)-import GHC.Eventlog.Live.Otlp.Processor.Common.Metrics (MetricProcessor (..), asSum, runMetricProcessor, viaSum)+import GHC.Eventlog.Live.Otlp.Processor.Common.Metrics (MetricProcessor (..), asGauge, asSum, runMetricProcessor, viaLast) import GHC.Eventlog.Live.Otlp.Processor.Common.Traces (asSpan) import GHC.RTS.Events (Event (..)) import Lens.Family2 ((.~))@@ -70,20 +71,33 @@                 ~> asParts             )             ~> M.fanoutTick-              [ runMetricProcessor-                  MetricProcessor-                    { metricProcessorProxy = Proxy @"capabilityUsage"-                    , dataProcessor = M.processCapabilityUsageMetrics-                    , aggregators = viaSum-                    , postProcessor = echo-                    , unit = "ns"-                    , asMetric'Data =-                        asSum-                          [ OM.aggregationTemporality .~ OM.AGGREGATION_TEMPORALITY_DELTA-                          , OM.isMonotonic .~ True-                          ]-                    }-                  fullConfig+              [ M.liftTick M.processCapabilityUsageDurationData+                  ~> M.fanoutTick+                    [ runMetricProcessor+                        MetricProcessor+                          { metricProcessorProxy = Proxy @"capabilityUsage"+                          , dataProcessor = mapping M.toMetric+                          , aggregators = viaLast+                          , postProcessor = echo+                          , unit = "ns"+                          , asMetric'Data =+                              asSum+                                [ OM.aggregationTemporality .~ OM.AGGREGATION_TEMPORALITY_CUMULATIVE+                                , OM.isMonotonic .~ True+                                ]+                          }+                        fullConfig+                    , runMetricProcessor+                        MetricProcessor+                          { metricProcessorProxy = Proxy @"productivity"+                          , dataProcessor = M.processProductivity ~> mapping (fmap (* 100.0) . M.toMetric)+                          , aggregators = viaLast+                          , postProcessor = echo+                          , unit = "%"+                          , asMetric'Data = asGauge+                          }+                        fullConfig+                    ]                   ~> mapping (fmap (fmap Left))               , runIf (C.processorEnabled (.traces) (.capabilityUsage) fullConfig) $                   M.liftTick@@ -127,10 +141,7 @@ Determine whether or not any thread events should be processed at all. -} shouldProcessThreadEvents :: FullConfig -> Bool-shouldProcessThreadEvents fullConfig =-  C.processorEnabled (.metrics) (.capabilityUsage) fullConfig-    || C.processorEnabled (.traces) (.capabilityUsage) fullConfig-    || C.processorEnabled (.traces) (.threadState) fullConfig+shouldProcessThreadEvents = shouldComputeThreadStateSpan  {- | Internal helper.@@ -138,8 +149,9 @@ -} shouldComputeCapabilityUsageSpan :: FullConfig -> Bool shouldComputeCapabilityUsageSpan fullConfig =-  C.processorEnabled (.traces) (.capabilityUsage) fullConfig-    || C.processorEnabled (.metrics) (.capabilityUsage) fullConfig+  C.processorEnabled (.metrics) (.capabilityUsage) fullConfig+    || C.processorEnabled (.metrics) (.productivity) fullConfig+    || C.processorEnabled (.traces) (.capabilityUsage) fullConfig  {- | Internal helper.

@@ -13,7 +13,7 @@ import Data.Word (Word16) import GHC.Eventlog.Live.Data.Attribute (IsAttrValue) import GHC.RTS.Events (Event (..))-import GHC.Stack.Profiler.Core.Eventlog (CapabilityId (..))+import GHC.Stack.Profiler.Core (CapabilityId (..))  {- | A capability number.

@@ -5,9 +5,14 @@ Portability : portable -} module GHC.Eventlog.Live.Data.Metric (+  -- * Metric superclass+  IsMetric,+  toMetric,++  -- * Generic metric type   Metric (..), -  -- * Existential wrapper+  -- ** Existential wrapper   SomeMetric (..),   SMetricType (..),   KnownMetricType (..),@@ -21,8 +26,36 @@ import GHC.Eventlog.Live.Data.Attribute (Attrs) import GHC.Eventlog.Live.Data.Group (GroupBy (..)) import GHC.RTS.Events (Timestamp)+import GHC.Records (HasField) +--------------------------------------------------------------------------------+-- Superclass for metric types+--------------------------------------------------------------------------------+ {- |+A metric is any type that has the fields of the generic metric type.+-}+type IsMetric ma a =+  ( HasField "value" ma a+  , HasField "maybeTimeUnixNano" ma (Maybe Timestamp)+  , HasField "maybeStartTimeUnixNano" ma (Maybe Timestamp)+  , HasField "attrs" ma Attrs+  )++toMetric :: (IsMetric ma a) => ma -> Metric a+toMetric ma =+  Metric+    { value = ma.value+    , maybeTimeUnixNano = ma.maybeTimeUnixNano+    , maybeStartTimeUnixNano = ma.maybeStartTimeUnixNano+    , attrs = ma.attrs+    }++--------------------------------------------------------------------------------+-- Generic Metric type+--------------------------------------------------------------------------------++{- | Metrics combine a measurement with a timestamp representing the time of the measurement, a timestamp representing the earliest possible measurement, and a list of attributes.@@ -39,7 +72,7 @@   , attrs :: Attrs   -- ^ A set of attributes.   }-  deriving (Functor, Show)+  deriving (Functor, Foldable, Traversable, Show)  instance GroupBy (Metric a) where   type Key (Metric a) = Attrs

@@ -6,10 +6,12 @@ -} module GHC.Eventlog.Live.Data.Thread (   ThreadId (..),+  fromThreadId, ) where  import Data.Word (Word64) import GHC.Eventlog.Live.Data.Attribute (IsAttrValue)+import GHC.Stack.Profiler.Core qualified as GSP  {- | A thread ID.@@ -26,3 +28,9 @@   }   deriving stock (Show, Eq, Ord)   deriving newtype (IsAttrValue)++{- |+Get the thread ID from a @ghc-stack-profiler@ `ThreadId`.+-}+fromThreadId :: GSP.ThreadId -> ThreadId+fromThreadId (GSP.MkThreadId value) = ThreadId value

@@ -11,8 +11,15 @@ module GHC.Eventlog.Live.Machine.Analysis.Capability (   -- * Capability Usage +  -- ** Productivity Metrics+  Productivity (..),+  processProductivity,+   -- ** Capability Usage Metrics-  processCapabilityUsageMetrics,+  processCapabilityUsageDurationData,+  CapabilityUsageDuration (..),+  processCapabilityUsageDuration'Delta,+  processCapabilityUsageDuration'DeltaToCumulative,    -- ** Capability Usage Spans   CapabilityUsageSpan,@@ -35,52 +42,197 @@   processMutatorSpans', ) where +import Control.Exception (assert) import Control.Monad (when) import Control.Monad.Trans.Class (MonadTrans (..)) import Data.Char (isSpace) import Data.Foldable (for_)+import Data.Hashable (Hashable) import Data.Machine (Is (..), PlanT, ProcessT, asParts, await, construct, mapping, repeatedly, yield, (~>)) import Data.Machine.Fanout (fanout)+import Data.Semigroup (Max (..)) import Data.Text (Text) import Data.Text qualified as T import Data.Void (Void)-import GHC.Eventlog.Live.Data.Attribute (AttrValue, IsAttrValue (..), (~=))-import GHC.Eventlog.Live.Data.Metric (Metric (..))+import GHC.Eventlog.Live.Data.Attribute (AttrValue, Attrs, IsAttrValue (..), (~=)) import GHC.Eventlog.Live.Data.Severity (Severity (..)) import GHC.Eventlog.Live.Data.Span (duration) import GHC.Eventlog.Live.Logger (Logger, writeLog) import GHC.Eventlog.Live.Machine.Analysis.Thread (ThreadState (..), ThreadStateSpan (..), processThreadStateSpans')-import GHC.Eventlog.Live.Machine.Core (liftRouter)+import GHC.Eventlog.Live.Machine.Core (deltaToCumulative, liftRouter) import GHC.Eventlog.Live.Machine.WithStartTime (WithStartTime (..), setWithStartTime'value, tryGetTimeUnixNano)+import GHC.Generics (Generic) import GHC.RTS.Events (Event (..), EventInfo, ThreadId, Timestamp) import GHC.RTS.Events qualified as E import GHC.Records (HasField (..)) import Text.Printf (printf)  ---------------------------------------------------------------------------------- Capability Usage Metrics+-- Productivity Metrics  {- |+The productivity measure.++This holds the cumulative elapsed time for both GC and mutator threads.+-}+data Productivity = Productivity+  { gc :: !Timestamp+  , mutator :: !Timestamp+  , maybeTimeUnixNano :: !(Maybe Timestamp)+  , maybeStartTimeUnixNano :: !(Maybe Timestamp)+  , cap :: Int+  }++instance HasField "value" Productivity Double where+  getField :: Productivity -> Double+  getField Productivity{..} =+    realToFrac mutator / realToFrac (mutator + gc)++instance HasField "attrs" Productivity Attrs where+  getField :: Productivity -> Attrs+  getField Productivity{..} =+    ["cap" ~= cap]++instance Semigroup Productivity where+  (<>) :: Productivity -> Productivity -> Productivity+  x <> y =+    Productivity+      { gc = max x.gc y.gc+      , mutator = max x.mutator y.mutator+      , maybeTimeUnixNano = getMax <$> (Max <$> x.maybeTimeUnixNano) <> (Max <$> y.maybeTimeUnixNano)+      , maybeStartTimeUnixNano = getMax <$> (Max <$> x.maybeStartTimeUnixNano) <> (Max <$> y.maybeStartTimeUnixNano)+      , cap = assert (x.cap == y.cap) x.cap+      }++{- |+Convert a t`CapabilityUsageDuration` to a partial t`Productivity` that only+represents the category corresponding to this t`CapabilityUsageDuration`'s+usage category (GC or mutator).+-}+toProductivity :: CapabilityUsageDuration Timestamp -> Maybe Productivity+toProductivity CapabilityUsageDuration{..} = do+  usage >>= \case+    GC -> pure Productivity{gc = value, mutator = 0, ..}+    Mutator{} -> pure Productivity{gc = 0, mutator = value, ..}++{- |+This machine processes t`CapabilityUsageDuration` with the cumulative elapsed+time for each category and produces metrics that contain productivity.+-}+processProductivity ::+  forall m.+  (Monad m) =>+  ProcessT m (CapabilityUsageDuration Timestamp) Productivity+processProductivity =+  liftRouter measure spawn+ where+  -- This measure splits the input by capability.+  measure :: CapabilityUsageDuration Timestamp -> Maybe Int+  measure cud = Just cud.cap++  spawn :: Int -> ProcessT m (CapabilityUsageDuration Timestamp) Productivity+  spawn _cap = construct $ go Nothing+   where+    go ::+      Maybe Productivity ->+      PlanT (Is (CapabilityUsageDuration Timestamp)) Productivity m Void+    go maybeProductivity =+      await >>= \case+        cud+          -- If this usage duration yields a productivity update,+          -- yield an updated productivity.+          | productivityUpdate@Just{} <- toProductivity cud -> do+              let maybeProductivity' = maybeProductivity <> productivityUpdate+              for_ maybeProductivity' yield+              go maybeProductivity'+          -- Otherwise, ignore it.+          | otherwise ->+              go maybeProductivity++-------------------------------------------------------------------------------+-- Capability Usage Duration - Cumulative++{- |+This machine processes t`CapabilityUsageSpan` data and produces metrics+that contain the cumulative elapsed time for each category (idle, GC, mutator).+-}+processCapabilityUsageDurationData ::+  forall m.+  (Monad m) =>+  ProcessT m (WithStartTime CapabilityUsageSpan) (CapabilityUsageDuration Timestamp)+processCapabilityUsageDurationData =+  processCapabilityUsageDuration'Delta+    ~> processCapabilityUsageDuration'DeltaToCumulative++-------------------------------------------------------------------------------+-- Capability Usage Duration++{- |+The delta capability usage duration.+-}+data CapabilityUsageDuration a+  = CapabilityUsageDuration+  { value :: !a+  , maybeTimeUnixNano :: !(Maybe Timestamp)+  , maybeStartTimeUnixNano :: !(Maybe Timestamp)+  , cap :: !Int+  , usage :: !(Maybe CapabilityUser)+  {- ^+  If the capability is actively used, this value is `Just` a `CapabilityUser`.+  If the capability is idle, this value is `Nothing`.+  -}+  }+  deriving (Functor, Foldable, Traversable)++instance HasField "attrs" (CapabilityUsageDuration a) Attrs where+  getField :: CapabilityUsageDuration a -> Attrs+  getField CapabilityUsageDuration{..} =+    [ "cap" ~= cap+    , "category" ~= maybe "Idle" showCapabilityUserCategory usage+    , "user" ~= usage+    ]++{- |+This machine processes t`CapabilityUsageDuration` with the delta of elapsed+time for each category and produces metrics that contain the cumulative elapsed+time for each category (idle, GC, mutator).+-}+processCapabilityUsageDuration'DeltaToCumulative ::+  forall m.+  (Monad m) =>+  ProcessT m (CapabilityUsageDuration Timestamp) (CapabilityUsageDuration Timestamp)+processCapabilityUsageDuration'DeltaToCumulative =+  liftRouter measure (const deltaToCumulative)+ where+  -- This measure splits the input by capability _and_ usage category:+  --+  -- 1. The `Int` represents the capability.+  -- 2. The `Maybe CapabilityUser` represents the usage category.+  --+  measure :: CapabilityUsageDuration Timestamp -> Maybe (Int, Maybe CapabilityUser)+  measure cud = Just (cud.cap, cud.usage)++{- | This machine processes t`CapabilityUsageSpan` spans and produces metrics that contain the duration and category of each such span and each idle period in between. -}-processCapabilityUsageMetrics ::+processCapabilityUsageDuration'Delta ::   forall m.   (Monad m) =>-  ProcessT m (WithStartTime CapabilityUsageSpan) (Metric Timestamp)-processCapabilityUsageMetrics =+  ProcessT m (WithStartTime CapabilityUsageSpan) (CapabilityUsageDuration Timestamp)+processCapabilityUsageDuration'Delta =   liftRouter measure spawn  where   measure :: WithStartTime CapabilityUsageSpan -> Maybe Int   measure = Just . (.value.cap) -  spawn :: Int -> ProcessT m (WithStartTime CapabilityUsageSpan) (Metric Timestamp)+  spawn :: Int -> ProcessT m (WithStartTime CapabilityUsageSpan) (CapabilityUsageDuration Timestamp)   spawn cap = construct $ go Nothing    where     go ::       Maybe CapabilityUsageSpan ->-      PlanT (Is (WithStartTime CapabilityUsageSpan)) (Metric Timestamp) m Void+      PlanT (Is (WithStartTime CapabilityUsageSpan)) (CapabilityUsageDuration Timestamp) m Void     go mi =       await >>= \j -> do         -- If there is a previous span, and...@@ -89,20 +241,21 @@           when (i.endTimeUnixNano < j.value.startTimeUnixNano) $             -- ...yield an idle duration metric.             yield-              Metric+              CapabilityUsageDuration                 { value = j.value.startTimeUnixNano - i.endTimeUnixNano                 , maybeTimeUnixNano = Just i.endTimeUnixNano                 , maybeStartTimeUnixNano = j.maybeStartTimeUnixNano-                , attrs = ["cap" ~= cap, "category" ~= ("Idle" :: Text)]+                , cap = cap+                , usage = Nothing -- Idle                 }         -- Yield a duration metric for the current span.-        let user = capabilityUser j.value         yield-          Metric+          CapabilityUsageDuration             { value = duration j.value             , maybeTimeUnixNano = Just j.value.startTimeUnixNano             , maybeStartTimeUnixNano = j.maybeStartTimeUnixNano-            , attrs = ["cap" ~= cap, "category" ~= showCapabilityUserCategory user, "user" ~= user]+            , cap = cap+            , usage = Just $! capabilityUser j.value             }         go (Just j.value) @@ -113,6 +266,9 @@ data CapabilityUser   = GC   | Mutator {thread :: !ThreadId}+  deriving stock (Eq, Generic)++instance Hashable CapabilityUser  instance Show CapabilityUser where   show :: CapabilityUser -> String

@@ -15,6 +15,8 @@   processHeapLiveData,   MemReturnData (..),   processMemReturnData,+  GcStatsData (..),+  processGcStatsData,   HeapProfSampleData,   heapProfSamples,   processHeapProfSampleData,@@ -56,6 +58,8 @@  {- | This machine processes `E.HeapAllocated` events into metrics.++This metric is the total bytes allocated over the whole run by the heap capability set. -} processHeapAllocatedData :: Process (WithStartTime Event) (Metric Word64) processHeapAllocatedData =@@ -75,6 +79,8 @@  {- | This machine processes `E.HeapSize` events into metrics.++This metric is the current bytes allocated from the OS to use for the heap. -} processHeapSizeData :: Process (WithStartTime Event) (Metric Word64) processHeapSizeData = repeatedly go@@ -114,6 +120,8 @@  {- | This machine processes `E.HeapLive` events into metrics.++This metric is the current amount of live/reachable data in the heap. -} processHeapLiveData :: Process (WithStartTime Event) (Metric Word64) processHeapLiveData =@@ -156,6 +164,42 @@               metric i MemReturnData{..} $                 [ "evCap" ~= i.value.evCap                 , "heapCapset" ~= heapCapset+                ]+        | otherwise -> pure ()++-------------------------------------------------------------------------------+-- GcStats++data GcStatsData = GcStatsData+  { copied :: !Word64+  -- ^ Number of bytes copied.+  , slop :: !Word64+  {- ^ Amount of slop in bytes.++  Slop is unused memory between objects in the heap.+  -}+  , fragmentation :: !Word64+  {- ^ Amount of fragmentation in bytes.++  The difference between total mblock size and total block size.+  If all mblocks are full of blocks, this number is 0.+  -}+  }++{- |+This machine processes `E.GCStatsGHC` events into metrics.+-}+processGcStatsData :: Process (WithStartTime Event) (Metric GcStatsData)+processGcStatsData =+  repeatedly $+    await >>= \case+      i+        | E.GCStatsGHC{..} <- i.value.evSpec -> do+            yield $+              metric i GcStatsData{copied, slop, fragmentation = frag} $+                [ "evCap" ~= i.value.evCap+                , "heapCapset" ~= heapCapset+                , "gen" ~= gen                 ]         | otherwise -> pure () 

@@ -35,14 +35,12 @@ import GHC.Eventlog.Live.Data.Attribute (Attrs, HasAttrs (..), (~=)) import GHC.Eventlog.Live.Data.Capability (CapNo (..), fromCapabilityId) import GHC.Eventlog.Live.Data.Severity (Severity (..))-import GHC.Eventlog.Live.Data.Thread (ThreadId (..))+import GHC.Eventlog.Live.Data.Thread (ThreadId (..), fromThreadId) import GHC.Eventlog.Live.Logger (Logger, writeLog) import GHC.Eventlog.Live.Machine.WithStartTime (WithStartTime (..), tryGetTimeUnixNano) import GHC.RTS.Events (Event (..), Timestamp) import GHC.RTS.Events qualified as E-import GHC.Stack.Profiler.Core.Eventlog qualified as GSP-import GHC.Stack.Profiler.Core.SymbolTable qualified as GSP-import GHC.Stack.Profiler.Core.ThreadSample qualified as GSP+import GHC.Stack.Profiler.Core qualified as GSP import IpeDB.Database qualified as DB import IpeDB.Types.CostCentre (CostCentre (..), CostCentreId (..)) import IpeDB.Types.InfoProv (InfoProv (..), InfoProvId (..))@@ -85,7 +83,7 @@ -} data GhcStackProfilerState = GhcStackProfilerState   { warnOnDeserializeError :: !Bool-  , callStackChunksRev :: ![GSP.BinaryCallStackMessage]+  , callStackChunksRev :: ![GSP.CallStackChunk]   , maybeTimeUnixNano :: !(Maybe Timestamp)   , symbolTable :: !GSP.IntMapTable   }@@ -132,6 +130,9 @@                     ]                   go st{warnOnDeserializeError = False}               | otherwise -> go st+            -- If we receive a protocol version, do nothing.+            Right (GSP.ProtocolVersion _protocolVersion) -> do+              go st             -- If we receive the final call-stack chunk, decode and yield the call-stack, the restart...             Right (GSP.CallStackFinal callStackChunk) -> do               let symbolTableReader = GSP.mkIntMapSymbolTableReader st.symbolTable@@ -171,11 +172,11 @@   decodeCallStack ::     Maybe Timestamp ->     GSP.SymbolTableReader ->-    NonEmpty GSP.BinaryCallStackMessage ->+    NonEmpty GSP.CallStackChunk ->     m CallStack   decodeCallStack maybeTimeUnixNano symbolTableReader callStackChunks = do     -- Concatenate the chunks into a full binary call-stack message.-    let !gspBinaryCallStack = GSP.catCallStackMessage callStackChunks+    let !gspBinaryCallStack = GSP.joinCallStackChunks callStackChunks      -- Decode the binary call-stack and log any decoding errors.     let !(gspCallStackMessage, decodeErrors) =@@ -210,7 +211,7 @@         <$> mapAccumM toCallStackFrame (V.toList maybeInfoProvs) gspCallStack      let !capNo = fromCapabilityId . GSP.callCapabilityId $ gspCallStackMessage-    let !threadId = ThreadId . GSP.callThreadId $ gspCallStackMessage+    let !threadId = fromThreadId . GSP.callThreadId $ gspCallStackMessage     pure CallStack{..}  {- |

@@ -35,6 +35,9 @@   sortByTick,   sortByTicks, +  -- * Aggregation Temporality+  deltaToCumulative,+   -- * Delimiting   delimit,   betweenEach,@@ -49,6 +52,7 @@ import Control.Monad (when) import Control.Monad.Trans.Class (MonadTrans (..)) import Control.Monad.Trans.Control (MonadBaseControl)+import Control.Monad.Trans.State.Strict (get, put, runState) import Data.DList qualified as D import Data.Foldable (Foldable (..), for_) import Data.Function (on)@@ -58,13 +62,16 @@ import Data.Hashable (Hashable (..)) import Data.Kind (Constraint) import Data.List qualified as L-import Data.Machine (Is (..), MachineT (..), Moore (..), PlanT, Process, ProcessT, SourceT, Step (..), asParts, await, construct, encased, mapping, repeatedly, starve, stopped, yield, (~>))+import Data.Machine (Is (..), MachineT (..), Moore (..), Plan, PlanT, Process, ProcessT, SourceT, Step (..), asParts, await, construct, encased, mapping, repeatedly, starve, stopped, yield, (~>)) import Data.Machine.Concurrent qualified as CC import Data.Machine.Fanout (fanout) import Data.Maybe (fromMaybe) import Data.Semigroup (Max (..)) import Data.Text qualified as T+import Data.Traversable (for) import Data.Void (Void)+import Data.Word (Word64)+import GHC.Eventlog.Live.Data.Metric (Metric) import GHC.Eventlog.Live.Data.Severity (Severity (..)) import GHC.Eventlog.Live.Logger (Logger, writeLog) import Text.Printf (printf)@@ -651,6 +658,26 @@     ~> batchByTicks ticks     ~> mapping (fmap D.toList)     ~> liftTick (sortByBatch key ~> asParts)++-------------------------------------------------------------------------------+-- Aggregation+-------------------------------------------------------------------------------++deltaToCumulative :: forall f a. (Traversable f, Num a) => Process (f a) (f a)+deltaToCumulative = construct $ go 0+ where+  go :: a -> Plan (Is (f a)) (f a) ()+  go acc =+    await >>= \fa -> do+      -- Add the cumulative sum to the value in the item.+      let (!fa', !acc') = flip runState acc . for fa $ \a -> do+            a' <- (a +) <$> get; put a'; pure a'+      -- Yield the updated item.+      yield fa'+      -- Continue with the new cumulative sum.+      go acc'+{-# SPECIALIZE deltaToCumulative :: Process (Metric Double) (Metric Double) #-}+{-# SPECIALIZE deltaToCumulative :: Process (Metric Word64) (Metric Word64) #-}  ------------------------------------------------------------------------------- -- Filtering semaphores