link-canonical-0.1.0.0: docs/architecture/v1.md

# Haskell Link Normalizer – Technical Specification v1

## 1. Problem Statement

Modern applications ingest URLs from many sources (browsers, mobile apps, email clients, social media, short-link services). These URLs often differ syntactically while referring to the same underlying resource.

Examples of instability:

* Tracking parameters (`utm_*`, `gclid`, `fbclid`)
* Redirect chains (URL shorteners)
* Domain-specific variants (YouTube, Amazon, GitHub, Twitter/X)
* Encoding variations (`%2F` vs `/`)
* Port and scheme inconsistencies

**Goal**: Build a reusable Haskell library that converts arbitrary URLs into *canonical, semantically stable identifiers*.

---

## 2. Non-Goals

* Crawling page content
* SEO-style canonical discovery (`<link rel="canonical">`)
* Browser automation or JavaScript execution
* Meta-refresh redirect handling

The library operates **only on URLs and HTTP metadata**.

---

## 3. Semantic Model: What is "Canonical"?

This library defines canonicalization as **resource identity**, not content identity or experience identity.

| Level | Example | Same? |
|-------|---------|-------|
| Resource identity | `youtube.com/watch?v=X` and `youtube.com/watch?v=X&t=30` | Yes |
| Content identity | Same video at different timestamps | No |
| Experience identity | `youtube.com/watch?v=X` and `youtube.com/embed/X` | No |

**This library targets resource identity.** Two URLs are canonical equivalents if they reference the same underlying resource, even if the presentation or position within that resource differs.

Implications:
* Timestamps (`t=`, `start=`) are stripped
* Embed variants are normalized to watch URLs
* Mobile/desktop variants are unified

---

## 4. Core Design Principles

1. **Separation of concerns**
   * Redirect resolution (effectful)
   * URL normalization (pure)
   * Domain semantics (pure, pluggable)

2. **Determinism**
   * Same input URL → same canonical output
   * Query parameters are sorted alphabetically

3. **Bounded effects**
   * Redirect depth limits (default: 10)
   * Loop detection
   * Timeout enforcement

4. **Extensibility**
   * Domain rules as independent modules
   * Configuration-driven rules

5. **Security by default**
   * Private IP rejection
   * Scheme restrictions
   * No downgrade attacks

---

## 5. Technology Choices

### URI Library: `modern-uri`

We use `Text.URI` from the `modern-uri` package:

* Megaparsec-based parsing with good error messages
* Text-based (fits domain rule implementations)
* Well-designed API for URI manipulation
* Active maintenance

```haskell
import Text.URI (URI)
import qualified Text.URI as URI
```

### HTTP Client

The library is HTTP-client agnostic through a typeclass abstraction, but provides a default implementation using `http-client-tls`.

### GHC and Extensions

**Language**: GHC2024

**Default Extensions** (in cabal `common` stanza):
```cabal
default-language: GHC2024
default-extensions:
  DeriveAnyClass
  DuplicateRecordFields
  NoFieldSelectors
  OverloadedLabels
  OverloadedStrings
  StrictData
```

These extensions enable:
- `DuplicateRecordFields`: No field name prefixes required
- `NoFieldSelectors`: Prevents auto-generation of field accessor functions (use lenses instead)
- `OverloadedLabels`: Enables `#fieldName` syntax for generic-lens
- `StrictData`: All fields strict by default

---

## 6. High-Level Pipeline

```
Raw URL (Text)
  │
  ├─ 1. Parse (Text → URI)
  │
  ├─ 2. Validate scheme (http/https only)
  │
  ├─ 3. Resolve redirects (IO, bounded)
  │     ├─ Loop detection
  │     ├─ Private IP rejection
  │     └─ Max depth enforcement
  │
  ├─ 4. Generic normalization (pure)
  │     ├─ Lowercase scheme/host
  │     ├─ Remove default ports
  │     ├─ Normalize percent-encoding
  │     ├─ Normalize path
  │     ├─ Sort query parameters
  │     ├─ Strip tracking parameters
  │     └─ Remove fragments
  │
  ├─ 5. Domain-specific normalization (pure)
  │     └─ First matching rule wins
  │
  └─ 6. Final generic pass (pure)
        └─ Re-apply tracking param removal
              ↓
      Canonical URI
```

---

## 7. Public API Surface

### Layered API Design

```haskell
-- Layer 1: Pure normalization (no IO)
normalizeUri :: NormConfig -> URI -> URI

-- Layer 2: With redirect resolution
normalizeLink
  :: (MonadIO m, MonadCatch m)
  => NormConfig
  -> HttpClient m
  -> URI
  -> m (Either NormError NormResult)

-- Layer 3: Convenient defaults
normalizeWithDefaults
  :: (MonadIO m, MonadCatch m)
  => URI
  -> m (Either NormError URI)
```

### Supporting APIs

```haskell
-- Pure operations
applyDomainRules  :: [DomainRule] -> URI -> URI
stripTracking     :: TrackingConfig -> URI -> URI
normalizeEncoding :: URI -> URI

-- Effectful operations
resolveFinalUri
  :: (MonadIO m, MonadCatch m)
  => HttpClient m
  -> RedirectConfig
  -> URI
  -> m (Either RedirectError URI)
```

---

## 8. Package Prelude

The package provides a custom prelude at `Link.Canonical.Prelude` that:
- Re-exports common types and functions
- Enables `#fieldName` syntax via `generic-lens`
- Avoids repetitive imports across modules

```haskell
module Link.Canonical.Prelude
  ( module X
  , module Control.Lens
  )
where

-- Enable #label syntax for Generic records
import "generic-lens" Data.Generics.Labels ()

-- Re-export lens operators
import "lens" Control.Lens

-- Common re-exports
import Control.Monad as X (forM, forM_, unless, when)
import Control.Monad.IO.Class as X (MonadIO, liftIO)
import Data.Maybe as X (fromMaybe, isJust, isNothing, mapMaybe)
import Data.Set as X (Set)
import Data.Text as X (Text)
import Data.Time as X (NominalDiffTime, UTCTime)
import GHC.Generics as X (Generic)
import Text.URI as X (URI)
```

**Usage**: Internal modules import only the prelude:
```haskell
module Link.Canonical.Normalize where

import Link.Canonical.Prelude
```

---

## 9. Configuration

### Main Configuration Record

```haskell
data NormConfig = NormConfig
  { redirects     :: RedirectConfig
  , tracking      :: TrackingConfig
  , domainRules   :: [DomainRule]
  , stripFragment :: Bool          -- default: True
  , sortParams    :: Bool          -- default: True
  , trailingSlash :: TrailingSlash -- default: Strip
  }
  deriving stock (Generic)

data TrailingSlash = Keep | Strip | Add
  deriving stock (Generic, Eq, Show)

defaultConfig :: NormConfig
```

### Redirect Configuration

```haskell
data RedirectConfig = RedirectConfig
  { maxRedirects    :: Int              -- default: 10
  , timeout         :: NominalDiffTime  -- default: 10s
  , allowDowngrade  :: Bool             -- default: False (https→http)
  , blockPrivateIPs :: Bool             -- default: True
  , userAgent       :: Text             -- default: "link-canonical/1.0"
  }
  deriving stock (Generic)
```

### Tracking Parameter Configuration

```haskell
data TrackingConfig = TrackingConfig
  { denyPatterns :: [Text]    -- e.g., ["utm_*", "gclid"]
  , allowlist    :: Set Text  -- params to never strip
  }
  deriving stock (Generic)

defaultTrackingParams :: [Text]
defaultTrackingParams =
  [ "utm_*"        -- Google Analytics
  , "gclid"        -- Google Ads
  , "fbclid"       -- Facebook
  , "mc_*"         -- Mailchimp
  , "oly_*"        -- Omeda
  , "_ga"          -- Google Analytics
  , "_gl"          -- Google Linker
  , "ref"          -- Generic referral (configurable)
  , "source"       -- Generic source (configurable)
  , "msclkid"      -- Microsoft Ads
  , "dclid"        -- DoubleClick
  , "zanpid"       -- Zanox
  , "igshid"       -- Instagram
  , "si"           -- Spotify
  ]
```

### Configuration with Lenses

All configuration updates use generic-lens:

```haskell
-- Customize redirect settings
customConfig :: NormConfig
customConfig = defaultConfig
  & #redirects . #maxRedirects .~ 5
  & #redirects . #timeout .~ 30
  & #tracking . #allowlist .~ Set.fromList ["ref"]
```

---

## 10. Error Model

### Error Types

```haskell
data NormError
  = ParseError Text
  | UnsupportedScheme Text
  | RedirectError RedirectError
  | DomainRuleError Text
  deriving stock (Generic, Eq, Show)

data RedirectError
  = TooManyRedirects Int
  | RedirectLoop [URI]
  | RedirectTimeout
  | PrivateIPBlocked URI
  | SchemeDowngrade URI URI
  | ConnectionFailed Text
  | InvalidLocation Text
  | HttpError Status
  deriving stock (Generic, Eq, Show)
```

### Error Handling Philosophy

* Parse errors: Unrecoverable, return `Left`
* Network errors: Unrecoverable, return `Left`
* Invalid redirects: Stop following, use last valid URI
* Domain rule failures: Log and continue (best-effort)

---

## 11. HTTP Client Abstraction

### Typeclass Definition

```haskell
class Monad m => HttpClient m where
  -- HEAD request for efficiency (no body needed for redirects)
  headRequest :: URI -> m (Either HttpClientError HttpResponse)

data HttpResponse = HttpResponse
  { status   :: Status
  , location :: Maybe URI  -- Parsed Location header
  , headers  :: [(Text, Text)]
  }
  deriving stock (Generic, Eq, Show)

data HttpClientError
  = ConnectionError Text
  | TimeoutError
  | TlsError Text
  | InvalidUri Text
  deriving stock (Generic, Eq, Show)
```

### Default Implementation

```haskell
-- Using http-client-tls
newtype HttpClientIO a = HttpClientIO { runHttpClientIO :: Manager -> IO a }
  deriving stock (Generic)

instance HttpClient HttpClientIO where
  headRequest = httpClientIOHead
```

### Why HEAD Requests?

* Redirect resolution only needs headers
* Saves bandwidth (no response body)
* Faster for redirect chains
* Politer to servers

---

## 12. Redirect Resolution

### Algorithm

```haskell
resolveFinalUri :: HttpClient m => RedirectConfig -> URI -> m (Either RedirectError URI)
resolveFinalUri config uri = go Set.empty 0 uri
  where
    go visited depth current
      | depth > config ^. #maxRedirects = Left (TooManyRedirects depth)
      | current `Set.member` visited    = Left (RedirectLoop $ Set.toList visited)
      | isPrivateIP current && config ^. #blockPrivateIPs = Left (PrivateIPBlocked current)
      | otherwise = do
          response <- headRequest current
          case response of
            Left err -> Left (ConnectionFailed $ show err)
            Right hr -> case hr ^. #status of
              s | isRedirect s -> case hr ^. #location of
                  Nothing  -> Left (InvalidLocation "Missing Location header")
                  Just loc ->
                    let next = resolveRelative current loc
                    in if isDowngrade current next && not (config ^. #allowDowngrade)
                       then Left (SchemeDowngrade current next)
                       else go (Set.insert current visited) (depth + 1) next
              s | isSuccess s -> Right current
              s -> Left (HttpError s)
```

### Private IP Detection

Block redirects to:
* `127.0.0.0/8` (loopback)
* `10.0.0.0/8` (private)
* `172.16.0.0/12` (private)
* `192.168.0.0/16` (private)
* `169.254.0.0/16` (link-local)
* `::1` (IPv6 loopback)
* `fc00::/7` (IPv6 private)

---

## 13. Generic URL Normalization (Pure)

### Operations (in order)

1. **Scheme normalization**
   * Lowercase: `HTTP` → `http`
   * Only `http` and `https` allowed

2. **Host normalization**
   * Lowercase: `Example.COM` → `example.com`
   * IDN to ASCII (punycode): `münchen.de` → `xn--mnchen-3ya.de`
   * Remove trailing dot: `example.com.` → `example.com`

3. **Port normalization**
   * Remove default ports: `http://example.com:80` → `http://example.com`
   * Remove default ports: `https://example.com:443` → `https://example.com`

4. **Path normalization**
   * Remove dot segments: `/a/./b/../c` → `/a/c`
   * Ensure leading slash: `example.com` → `example.com/`
   * Trailing slash policy (configurable)

5. **Percent-encoding normalization**
   * Uppercase hex digits: `%2f` → `%2F`
   * Decode unreserved characters: `%41` → `A`
   * Encode reserved characters consistently

6. **Query parameter normalization**
   * Sort alphabetically by key (for determinism)
   * Remove empty `?`: `example.com?` → `example.com`
   * Strip tracking parameters

7. **Fragment removal**
   * `example.com/page#section` → `example.com/page`

---

## 14. Tracking Parameter Policy

### Pattern Matching

```haskell
matchesTrackingPattern :: Text -> Text -> Bool
matchesTrackingPattern pattern param
  | "*" `T.isSuffixOf` pattern = T.init pattern `T.isPrefixOf` param
  | otherwise = pattern == param
```

### Configurable Allowlist

Some parameters look like tracking but are semantically meaningful:

```haskell
-- Example: preserve 'ref' for Amazon affiliate links in some contexts
config :: NormConfig
config = defaultConfig
  & #tracking . #allowlist .~ Set.fromList ["ref"]
```

### Update Strategy

The tracking parameter list is:
1. Hardcoded defaults in the library
2. Overridable via configuration
3. Documented in a central location
4. Versioned with semantic versioning (additions are minor, removals are major)

---

## 15. Domain-Specific Normalization

### Domain Rule Model

```haskell
data DomainRule = DomainRule
  { name  :: Text
  , match :: URI -> Bool
  , apply :: URI -> URI
  }
  deriving stock (Generic)

-- Convenience constructor
domainRule :: Text -> (URI -> Bool) -> (URI -> URI) -> DomainRule
domainRule n m a = DomainRule { name = n, match = m, apply = a }

-- Application: first match wins
applyDomainRules :: [DomainRule] -> URI -> URI
applyDomainRules rules uri =
  case find (\r -> (r ^. #match) uri) rules of
    Nothing -> uri
    Just r  -> (r ^. #apply) uri
```

### Rule Composition

Rules are **not** composed—first match wins. However, generic normalization runs both **before** and **after** domain rules:

```
Generic normalize → Domain rule → Generic normalize (tracking only)
```

This ensures domain rules can introduce URLs that still need tracking param removal.

### Required Documentation per Rule

Each domain rule module must document:

1. **Canonical form**: The target URL structure
2. **Handled variants**: What URL patterns are normalized
3. **Preserved parameters**: Semantically meaningful params kept
4. **Stripped parameters**: Params removed as noise
5. **Examples**: Before/after pairs

---

## 16. Built-in Domain Rules

### YouTube

**Canonical form:**
```
https://www.youtube.com/watch?v={VIDEO_ID}
```

**Handled variants:**
* `youtu.be/{VIDEO_ID}`
* `youtube.com/watch?v={VIDEO_ID}`
* `www.youtube.com/watch?v={VIDEO_ID}`
* `m.youtube.com/watch?v={VIDEO_ID}`
* `youtube.com/embed/{VIDEO_ID}`
* `youtube.com/v/{VIDEO_ID}`
* `youtube.com/shorts/{VIDEO_ID}`

**Stripped parameters:** `t`, `start`, `end`, `feature`, `list`, `index`, `si`

**Note:** Timestamps are stripped per our resource-identity semantic model.

---

### Amazon

**Canonical form:**
```
https://www.amazon.com/dp/{ASIN}
```

**Handled variants:**
* `amazon.com/dp/{ASIN}`
* `amazon.com/gp/product/{ASIN}`
* `amazon.com/{product-name}/dp/{ASIN}`
* `amzn.to/{SHORT_CODE}` (requires redirect resolution)

**Stripped parameters:** `tag`, `ref`, `psc`, `keywords`, `qid`, `sr`

**Regional handling:** Normalize to `.com` or preserve regional TLD (configurable).

---

### GitHub

**Canonical form:**
```
https://github.com/{owner}/{repo}[/path]
```

**Handled variants:**
* `www.github.com` → `github.com`
* Fragments preserved for file links: `#L123` (exception to fragment stripping)

**Stripped parameters:** `tab`, `ref_src`, `ref_source`

---

### Twitter / X

**Canonical form:**
```
https://x.com/{user}/status/{id}
```

**Handled variants:**
* `twitter.com` → `x.com`
* `mobile.twitter.com` → `x.com`
* `x.com/{user}/status/{id}/photo/1` → `x.com/{user}/status/{id}`
* `x.com/{user}/status/{id}/video/1` → `x.com/{user}/status/{id}`

**Stripped parameters:** `s`, `t`, `ref_src`

---

### Instagram

**Canonical form:**
```
https://www.instagram.com/p/{POST_ID}/
https://www.instagram.com/reel/{REEL_ID}/
https://www.instagram.com/{username}/
```

**Handled variants:**
* `instagram.com` → `www.instagram.com`

**Stripped parameters:** `igshid`, `utm_*`

---

### Reddit

**Canonical form:**
```
https://www.reddit.com/r/{subreddit}/comments/{id}/{slug}/
```

**Handled variants:**
* `old.reddit.com` → `www.reddit.com`
* `np.reddit.com` → `www.reddit.com`
* `redd.it/{id}` (requires redirect resolution)

**Stripped parameters:** `utm_*`, `ref`, `ref_source`

---

### LinkedIn

**Canonical form:**
```
https://www.linkedin.com/posts/{id}
https://www.linkedin.com/in/{username}/
```

**Stripped parameters:** `miniProfileUrn`, `lipi`, `lici`

---

### URL Shorteners

These require redirect resolution and produce no direct canonical form:

* `bit.ly`
* `t.co`
* `goo.gl`
* `tinyurl.com`
* `ow.ly`
* `is.gd`
* `buff.ly`

After redirect resolution, the target URL is then normalized.

---

## 17. Rule Ordering Strategy

```haskell
defaultDomainRules :: [DomainRule]
defaultDomainRules =
  [ youtubeRule      -- High volume, complex variants
  , amazonRule       -- High volume, complex paths
  , twitterRule      -- Domain migration (twitter→x)
  , instagramRule
  , redditRule
  , linkedInRule
  , githubRule
  ]
```

Order rationale:
1. **Most specific first**: Complex extraction rules before simple ones
2. **Highest volume**: Optimize for common cases
3. **Recently changed**: Platforms with recent migrations (Twitter→X)

---

## 18. Configuration-Driven Rules

### Declarative Rule Format

```haskell
data DeclRule = DeclRule
  { domains       :: [Text]           -- e.g., ["youtube.com", "youtu.be"]
  , canonicalHost :: Maybe Text       -- e.g., Just "www.youtube.com"
  , keepParams    :: [Text]           -- e.g., ["v"]
  , stripParams   :: [Text]           -- e.g., ["t", "feature"]
  , pathPatterns  :: [PathPattern]    -- Pattern matching for paths
  , stripFragment :: Bool
  }
  deriving stock (Generic)

data PathPattern = PathPattern
  { match   :: Text  -- e.g., "/embed/{id}"
  , rewrite :: Text  -- e.g., "/watch?v={id}"
  }
  deriving stock (Generic, Eq, Show)
```

### Dhall Configuration Example

```dhall
let Rule = { domains : List Text
           , canonicalHost : Optional Text
           , keepParams : List Text
           , stripParams : List Text
           , pathPatterns : List { match : Text, rewrite : Text }
           , stripFragment : Bool
           }

let youtube : Rule =
  { domains = ["youtube.com", "www.youtube.com", "m.youtube.com", "youtu.be"]
  , canonicalHost = Some "www.youtube.com"
  , keepParams = ["v"]
  , stripParams = ["t", "feature", "list", "index", "si"]
  , pathPatterns =
      [ { match = "/embed/{id}", rewrite = "/watch?v={id}" }
      , { match = "/v/{id}", rewrite = "/watch?v={id}" }
      , { match = "/shorts/{id}", rewrite = "/watch?v={id}" }
      ]
  , stripFragment = True
  }

in [youtube]
```

### Runtime Interpretation

```haskell
interpretRule :: DeclRule -> DomainRule
loadRulesFromDhall :: FilePath -> IO [DomainRule]
```

---

## 19. Result Types

### Rich Result

```haskell
data NormResult = NormResult
  { canonical      :: URI
  , original       :: URI
  , redirectsCount :: Int
  , redirectChain  :: [URI]      -- For debugging
  , ruleApplied    :: Maybe Text -- Name of domain rule, if any
  , paramsStripped :: [Text]     -- Tracking params removed
  }
  deriving stock (Generic, Eq, Show)
```

### Accessing Results with Lenses

```haskell
processResult :: NormResult -> IO ()
processResult result = do
  print $ result ^. #canonical
  when (result ^. #redirectsCount > 5) $
    putStrLn "Warning: long redirect chain"
  forM_ (result ^. #ruleApplied) $ \rule ->
    putStrLn $ "Applied rule: " <> unpack rule
```

### Observability Trace

```haskell
data NormTrace = NormTrace
  { input          :: URI
  , afterRedirects :: URI
  , afterGeneric   :: URI
  , afterDomain    :: URI
  , final          :: URI
  , redirectChain  :: [URI]
  , ruleApplied    :: Maybe Text
  , duration       :: NominalDiffTime
  }
  deriving stock (Generic, Eq, Show)
```

---

## 20. Observability Hooks

```haskell
data NormHooks m = NormHooks
  { onRedirectStep  :: URI -> URI -> m ()
  , onRuleApplied   :: Text -> URI -> URI -> m ()
  , onNormComplete  :: NormTrace -> m ()
  }
  deriving stock (Generic)

defaultHooks :: Applicative m => NormHooks m
defaultHooks = NormHooks
  { onRedirectStep  = \_ _ -> pure ()
  , onRuleApplied   = \_ _ _ -> pure ()
  , onNormComplete  = \_ -> pure ()
  }
```

---

## 21. Security Considerations

### SSRF Prevention

* **Private IP blocking**: Enabled by default
* **Scheme restriction**: Only `http`/`https`
* **No file:// or other schemes**

### Downgrade Attack Prevention

* **https→http rejected by default**
* Configurable for legacy systems

### Resource Exhaustion Prevention

* **Max redirects**: Default 10
* **Timeout**: Default 10 seconds per request
* **Loop detection**: Set-based visited tracking

### Input Validation

* Invalid URIs rejected early with `ParseError`
* Malformed percent-encoding handled gracefully

---

## 22. Testing Strategy

### Property Tests

```haskell
-- Idempotence
prop_idempotent :: URI -> Property
prop_idempotent uri =
  normalizeUri config uri === normalizeUri config (normalizeUri config uri)

-- Determinism
prop_deterministic :: URI -> Property
prop_deterministic uri =
  normalizeUri config uri === normalizeUri config uri

-- Query param order independence
prop_paramOrderIndependent :: URI -> [(Text, Text)] -> Property
prop_paramOrderIndependent uri params =
  let uri1 = setParams params uri
      uri2 = setParams (reverse params) uri
  in normalizeUri config uri1 === normalizeUri config uri2
```

### Golden Tests

Each domain rule has a fixture file:

```yaml
# test/fixtures/youtube.yaml
- input: "https://youtu.be/dQw4w9WgXcQ"
  expected: "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

- input: "https://www.youtube.com/watch?v=dQw4w9WgXcQ&t=30&feature=share"
  expected: "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

- input: "https://youtube.com/embed/dQw4w9WgXcQ"
  expected: "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
```

### Effectful Tests

```haskell
-- Stubbed HttpClient for deterministic testing
newtype MockHttp a = MockHttp (Reader RedirectMap a)
  deriving stock (Generic)

instance HttpClient MockHttp where
  headRequest uri = MockHttp $ asks (lookup uri)

-- Test redirect chains
test_redirectChain :: TestTree
test_redirectChain = testCase "follows redirect chain" $ do
  let redirects = Map.fromList
        [ (uri "http://a.com", redirect "http://b.com")
        , (uri "http://b.com", redirect "http://c.com")
        , (uri "http://c.com", success)
        ]
  result <- runMockHttp redirects $ resolveFinalUri config (uri "http://a.com")
  result @?= Right (uri "http://c.com")

-- Test loop detection
test_loopDetection :: TestTree
test_loopDetection = testCase "detects redirect loop" $ do
  let redirects = Map.fromList
        [ (uri "http://a.com", redirect "http://b.com")
        , (uri "http://b.com", redirect "http://a.com")
        ]
  result <- runMockHttp redirects $ resolveFinalUri config (uri "http://a.com")
  assertBool "should detect loop" (isRedirectLoop result)
```

### Integration Tests (Optional)

Live tests against real URLs, run manually or in CI with rate limiting:

```haskell
-- Only run with --integration flag
test_liveYouTube :: TestTree
test_liveYouTube = testCase "live YouTube normalization" $ do
  result <- normalizeWithDefaults (uri "https://youtu.be/dQw4w9WgXcQ")
  fmap (^. #canonical) result @?= Right (uri "https://www.youtube.com/watch?v=dQw4w9WgXcQ")
```

---

## 23. Performance Considerations

### Connection Pooling

```haskell
-- Reuse Manager across calls
withHttpClient :: (HttpClient IO -> IO a) -> IO a
withHttpClient action = do
  manager <- newTlsManager
  action (httpClientFromManager manager)
```

### Caching

Optional redirect cache for repeated normalization:

```haskell
data CacheConfig = CacheConfig
  { maxSize :: Int
  , ttl     :: NominalDiffTime
  }
  deriving stock (Generic)

newtype CachedHttpClient m a = CachedHttpClient
  { runCached :: StateT (Map URI (URI, UTCTime)) m a }
  deriving stock (Generic)
```

### Batch Processing

```haskell
normalizeLinks
  :: (MonadIO m, MonadCatch m)
  => NormConfig
  -> HttpClient m
  -> [URI]
  -> m [Either NormError NormResult]
```

With configurable concurrency limits.

---

## 24. Versioning Strategy

### Semantic Versioning

* **Major**: Breaking changes to API, removal of domain rules, removal of tracking params
* **Minor**: New domain rules, new tracking params, new config options
* **Patch**: Bug fixes, documentation

### Domain Rule Versioning

Rules are versioned independently:

```haskell
youtubeRule :: DomainRule
youtubeRule = DomainRule
  { name  = "youtube-v2"
  , match = matchYouTube
  , apply = applyYouTube
  }
```

Breaking changes to a rule bump its internal version.

### Deprecation Policy

* Deprecated rules/params remain for 2 major versions
* Deprecation warnings via compile-time or runtime logging

---

## 25. Module Structure

```
link-canonical/
├── src/
│   └── Link/
│       ├── Canonical.hs              -- Main entry point, re-exports
│       └── Canonical/
│           ├── Prelude.hs            -- Package prelude with lens re-exports
│           ├── Types.hs              -- Core types (NormConfig, NormResult, etc.)
│           ├── Config.hs             -- Configuration defaults and builders
│           ├── Error.hs              -- Error types
│           ├── Normalize.hs          -- Generic normalization
│           ├── Redirect.hs           -- Redirect resolution
│           ├── Http.hs               -- HttpClient abstraction
│           ├── Tracking.hs           -- Tracking parameter handling
│           └── Rules/
│               ├── Types.hs          -- DomainRule type
│               ├── YouTube.hs
│               ├── Amazon.hs
│               ├── Twitter.hs
│               ├── GitHub.hs
│               ├── Instagram.hs
│               ├── Reddit.hs
│               ├── LinkedIn.hs
│               └── Default.hs        -- Default rule set
├── test/
│   ├── Main.hs
│   ├── Link/
│   │   └── Canonical/
│   │       ├── NormalizeSpec.hs
│   │       ├── RedirectSpec.hs
│   │       └── Rules/
│   │           ├── YouTubeSpec.hs
│   │           └── ...
│   └── fixtures/
│       ├── youtube.yaml
│       └── ...
└── link-canonical.cabal
```

### Cabal Configuration

```cabal
cabal-version: 3.0
name:          link-canonical
version:       0.1.0.0

common common
  ghc-options:        -Wall -Wunused-packages
  default-language:   GHC2024
  default-extensions:
    DeriveAnyClass
    DuplicateRecordFields
    NoFieldSelectors
    OverloadedLabels
    OverloadedStrings
    StrictData

library
  import:          common
  hs-source-dirs:  src
  exposed-modules:
    Link.Canonical
    Link.Canonical.Types
    Link.Canonical.Config
    Link.Canonical.Rules.YouTube
    -- etc.

  other-modules:
    Link.Canonical.Prelude
    Link.Canonical.Normalize
    Link.Canonical.Redirect
    Link.Canonical.Http
    Link.Canonical.Tracking
    Link.Canonical.Rules.Types
    -- etc.

  build-depends:
    , base            >=4.18 && <5
    , bytestring
    , containers
    , generic-lens    >=2.2
    , http-client
    , http-client-tls
    , http-types
    , lens            >=5.2
    , modern-uri      >=0.3
    , mtl
    , text
    , time

test-suite link-canonical-test
  import:         common
  type:           exitcode-stdio-1.0
  hs-source-dirs: test
  main-is:        Main.hs
  build-depends:
    , base
    , hedgehog
    , link-canonical
    , tasty
    , tasty-hedgehog
    , tasty-hunit
```

---

## 26. Future Considerations

### Potential Additions

* **Google AMP unwrapping**: `google.com/amp/...` → original URL
* **Archive.org handling**: Extract original URL from Wayback Machine links
* **Medium**: Normalize custom domains to medium.com
* **TikTok**: Complex mobile/web variants
* **Hacker News**: Handle ycombinator short links

### Out of Scope (Revisit Later)

* Machine learning for unknown tracking parameters
* Headless browser for JavaScript redirects
* Rate limiting / politeness policies

---

## 27. Summary

This library treats URLs as *semantic identifiers*, not strings.

Key design decisions:

| Decision | Rationale |
|----------|-----------|
| `modern-uri` | Best ergonomics for Text-based manipulation |
| HEAD requests | Efficient redirect resolution |
| First-match rules | Simple, predictable semantics |
| Two-pass generic normalization | Domain rules can introduce trackable params |
| Resource identity semantics | Clear definition of "canonical" |
| Layered API | Flexibility for different use cases |
| Security by default | SSRF prevention, scheme restrictions |
| `DuplicateRecordFields` + `OverloadedLabels` | Clean field names, ergonomic access |
| `generic-lens` | Uniform field access via `#fieldName` |
| Package prelude | Reduced boilerplate, consistent imports |

The library provides:

* Stable link identity
* Deduplication support
* Cache correctness
* Event correlation capabilities

In event-driven systems, canonical URLs function as **entity identifiers**.