clod 0.1.16 → 0.1.18
raw patch · 16 files changed
+69/−3093 lines, 16 filesPVP ok
version bump matches the API change (PVP)
API changes (from Hackage documentation)
Files
- CAPABILITY_SECURITY.md +0/−203
- CHANGELOG.md +8/−0
- CONTRIBUTING.md +0/−102
- HASKELL_PATTERNS.md +0/−1680
- INSTALLING.md +0/−128
- MAN_PAGES.md +0/−45
- PROJECT_ARCHITECTURE.md +0/−298
- README.md +45/−36
- RELEASING.md +0/−101
- SERIALIZATION.md +0/−343
- TEST_SAFETY.md +0/−99
- bin/release +12/−45
- clod.cabal +1/−10
- man/clod.1.md +1/−1
- man/clod.7.md +1/−1
- man/clod.8.md +1/−1
− CAPABILITY_SECURITY.md
@@ -1,203 +0,0 @@-# Capability-Based Security--This document details the capability-based security patterns used in Clod for secure file and resource access management.--## Core Concept--Capability-based security restricts operations based on explicit capability tokens rather than implicit permissions. A capability is an unforgeable token that grants specific permissions to perform operations.--The key principles are:-1. **Default Denial**: By default, no file access is permitted-2. **Explicit Capabilities**: Each operation requires an explicit capability token-3. **Restricted Paths**: Capabilities only allow operations within specific directory trees-4. **Capability Verification**: All operations verify capabilities before execution--## Why This Matters--When an AI assistant like Claude interacts with a filesystem, strict security boundaries are essential. The capability system in Clod ensures that:--1. Operations can only access files in explicitly allowed directories-2. Path traversal attacks (using `../` or symbolic links) are prevented-3. Capabilities must be explicitly passed to functions that need them-4. The security model is made visible in the code through explicit tokens--This approach significantly reduces the risk of accidentally exposing sensitive files or allowing unauthorized modifications.--## Implementation Approaches--Clod implements two capability-based security approaches:--### 1. Standard Runtime Capability System--The standard system is implemented in `Clod.Types` and used throughout the application:--```haskell--- Grant permission to read from specific directories-data FileReadCap = FileReadCap - { allowedReadDirs :: [FilePath] -- Directories where reading is permitted- } deriving (Show, Eq)---- Grant permission to write to specific directories-data FileWriteCap = FileWriteCap - { allowedWriteDirs :: [FilePath] -- Directories where writing is permitted- } deriving (Show, Eq)---- Create capabilities-fileReadCap :: [FilePath] -> FileReadCap-fileReadCap dirs = FileReadCap { allowedReadDirs = dirs }--fileWriteCap :: [FilePath] -> FileWriteCap-fileWriteCap dirs = FileWriteCap { allowedWriteDirs = dirs }-```--### 2. Advanced Type-Level Capability System--The advanced system in `Clod.AdvancedCapability` uses type-level programming to enforce permissions at compile-time:--```haskell--- Permission types for capabilities-data Permission = Read | Write | Execute | All---- A path with type-level permission information-data TypedPath (p :: Permission) where- TypedPath :: FilePath -> TypedPath p---- Capability token that grants permissions-data Capability (p :: Permission) = Capability - { allowedDirs :: [FilePath] -- Directories this capability grants access to- }---- Create a capability token for the given permission and directories-createCapability :: forall p. [FilePath] -> Capability p-createCapability dirs = Capability { allowedDirs = dirs }-```--## Path Validation--Both implementations use similar path validation logic to prevent path traversal attacks:--```haskell--- Check if a path is within allowed directories-isPathAllowed :: [FilePath] -> FilePath -> IO Bool-isPathAllowed allowedDirs path = do- -- Get canonical paths to resolve any `.`, `..`, or symlinks- canonicalPath <- canonicalizePath path- -- Check if the canonical path is within any of the allowed directories- checks <- mapM (\dir -> do- canonicalDir <- canonicalizePath dir- -- A path is allowed if:- -- 1. It equals an allowed directory exactly, or- -- 2. It's a proper subdirectory (dir is a prefix and has a path separator)- let isAllowed = canonicalDir == canonicalPath || - (canonicalDir `isPrefixOf` canonicalPath && - length canonicalPath > length canonicalDir &&- isPathSeparator (canonicalPath !! length canonicalDir))- return isAllowed) allowedDirs- -- Return result- return (or checks)- where- isPathSeparator c = c == '/' || c == '\\'-```--## Secure Operations--All file system operations require appropriate capabilities:--```haskell--- Safe file reading that checks capabilities-safeReadFile :: FileReadCap -> FilePath -> ClodM BS.ByteString-safeReadFile cap path = do- allowed <- liftIO $ isPathAllowed (allowedReadDirs cap) path- if allowed- then liftIO $ BS.readFile path- else do- canonicalPath <- liftIO $ canonicalizePath path- throwError $ CapabilityError $ - "Access denied: Cannot read file outside allowed directories: " ++ canonicalPath---- Safe file writing that checks capabilities-safeWriteFile :: FileWriteCap -> FilePath -> BS.ByteString -> ClodM ()-safeWriteFile cap path content = do- allowed <- liftIO $ isPathAllowed (allowedWriteDirs cap) path- if allowed- then liftIO $ BS.writeFile path content- else do- canonicalPath <- liftIO $ canonicalizePath path- throwError $ CapabilityError $ - "Access denied: Cannot write file outside allowed directories: " ++ canonicalPath---- Safe file copying that checks capabilities for both read and write-safeCopyFile :: FileReadCap -> FileWriteCap -> FilePath -> FilePath -> ClodM ()-safeCopyFile readCap writeCap src dest = do- srcAllowed <- liftIO $ isPathAllowed (allowedReadDirs readCap) src- destAllowed <- liftIO $ isPathAllowed (allowedWriteDirs writeCap) dest- if srcAllowed && destAllowed- then liftIO $ copyFile src dest- else throwError $ CapabilityError $ - "Access denied: Path restrictions violated"-```--## Type-Level Operations (Advanced System)--The advanced system provides type-safe file operations:--```haskell--- Read a file with the given capability-readFile :: forall p m. (MonadIO m, PermissionFor 'Read p) - => Capability p -> TypedPath p -> m BS.ByteString-readFile _ (TypedPath path) = liftIO $ BS.readFile path---- Write to a file with the given capability-writeFile :: forall p m. (MonadIO m, PermissionFor 'Write p) - => Capability p -> TypedPath p -> BS.ByteString -> m ()-writeFile _ (TypedPath path) content = liftIO $ BS.writeFile path content---- Check if a path is allowed by this capability and create a typed path if it is-withPath :: forall p m a. (MonadIO m) - => Capability p -> FilePath -> (Maybe (TypedPath p) -> m a) -> m a-withPath cap path f = do- allowed <- liftIO $ isPathAllowed (allowedDirs cap) path- f $ if allowed then Just (TypedPath path) else Nothing-```--## Using Capabilities in the Application--Capabilities are typically created at the application entry point and passed down to functions that need them:--```haskell-runApp :: Config -> IO ()-runApp config = do- -- Create capabilities with appropriate permissions- let readCap = fileReadCap [config.sourceDir]- let writeCap = fileWriteCap [config.outputDir]- - -- Use capabilities in operations- result <- runClodM config $ do- processFiles readCap writeCap config.files- - case result of- Left err -> putStrLn $ "Error: " ++ show err- Right _ -> putStrLn "Processing complete"-```--## Security Benefits--This capability-based approach provides several important security benefits:--1. **Path Traversal Prevention**: Files outside allowed directories cannot be accessed, even with path traversal attacks-2. **Explicit Permission Model**: The code clearly indicates which operations are permitted and where-3. **Principle of Least Privilege**: Components only get access to the specific directories they need-4. **Transparent Intentions**: Code that needs file access must explicitly request capabilities-5. **Compile-Time Checks**: The advanced system catches permission errors at compile time with type-level constraints-6. **Composable Security**: Capabilities can be restricted and combined-7. **Testable**: Security restrictions can be verified through automated tests--## Future Directions--For future versions of Clod, we're considering:--1. **More Granular Capabilities**: Adding more specialized capabilities (e.g., for specific operations)-2. **Enhanced Type-Level Guarantees**: Extending the type-level verification of capabilities-3. **Better Error Messages**: Improving error messages for capability violations-4. **Capability Composition**: Making it easier to compose and transform capabilities-5. **Effect Integration**: Deeper integration with algebraic effects systems
CHANGELOG.md view
@@ -1,5 +1,13 @@ # Changelog +## [0.1.18] - 2025-04-03++Test release process again++## [0.1.17] - 2025-04-03++Update README, remove crufty documentation files, test release again+ ## [0.1.16] - 2025-04-03 Run the tests before the rest of the release process
− CONTRIBUTING.md
@@ -1,102 +0,0 @@-# Contributing to Clod--Thank you for considering contributing to Clod! This document outlines the process for contributing to the project.--## Code of Conduct--This project and everyone participating in it is governed by our Code of Conduct. By participating, you are expected to uphold this code.--## How Can I Contribute?--### Reporting Bugs--- Check if the bug has already been reported-- Use the bug report template if available-- Include as much detail as possible-- Include steps to reproduce the issue-- Include the version of Clod you're using-- Include your OS and environment details--### Suggesting Enhancements--- Check if the enhancement has already been suggested-- Provide a clear description of the enhancement-- Explain why this enhancement would be useful-- Consider how the enhancement fits with the project's goals--### Pull Requests--1. Fork the repository-2. Create a branch for your changes-3. Make your changes-4. Add or update tests as necessary-5. Update documentation if needed-6. Run the test suite: `cabal test`-7. Ensure your changes meet the project's code style-8. Submit the pull request--## Development Setup--Ensure you have the following installed:--- GHC (Glasgow Haskell Compiler) 9.0 or newer-- Cabal 3.0 or newer-- Git--Clone the repository:--```bash-git clone https://github.com/fuzz/clod.git-cd clod-```--Build the project:--```bash-cabal build-```--Run the tests:--```bash-cabal test-```--## Style Guidelines--This project follows idiomatic Haskell style:--- Use 2 spaces for indentation (no tabs)-- Provide Haddock documentation for public functions-- Use meaningful variable names-- Follow the [Haskell Style Guide](https://github.com/tibbe/haskell-style-guide/blob/master/haskell-style.md)--## Commit Messages--- Use the present tense ("Add feature" not "Added feature")-- Use the imperative mood ("Move cursor to..." not "Moves cursor to...")-- Reference issues and pull requests where appropriate-- Consider starting with a tag:- - `feat:` for new features- - `fix:` for bug fixes- - `docs:` for documentation changes- - `test:` for test changes- - `refactor:` for code refactoring--## Testing--- Add tests for every new feature-- Update tests for bug fixes-- Run the test suite before submitting a pull request-- Aim for high test coverage--## Documentation--- Update the README.md if necessary-- Add Haddock documentation for new functions-- Update example code if needed-- Consider updating the CHANGELOG.md for significant changes--## Questions?--If you have any questions about contributing, feel free to open an issue or reach out to the maintainers directly.
− HASKELL_PATTERNS.md
@@ -1,1680 +0,0 @@-# Haskell Patterns and Best Practices--This document contains common Haskell patterns and best practices for efficient and idiomatic Haskell development. These patterns are particularly useful for human-AI collaboration, where clear communication and code understanding are essential.--## Reflections on the Clod Project--Working on Clod provided valuable insights into developing Haskell applications collaboratively between humans and AI. Some lessons we learned:--### What Worked Well--1. **Capability-based security**: Using explicit capability tokens (`FileReadCap`, `FileWriteCap`) proved to be an elegant approach for enforcing access restrictions. The type system effectively prevented accidental security violations.--2. **Newtype wrappers**: Wrapping primitive types (like `String`) in newtypes (`IgnorePattern`, `OptimizedName`, `Checksum`) helped prevent confusion and enforce type safety throughout the application.--3. **Clear monad stack**: The `ClodM` monad combined `ReaderT` and `ExceptT` to provide a clean approach to dependency injection and error handling. This simplified the application compared to a more complex effects system.--4. **Modular architecture**: Breaking functionality into focused modules with clear responsibilities made the codebase easier to extend and maintain.--### Challenges Encountered--1. **Advanced type-level programming**: While the `AdvancedCapability` module showcases sophisticated type-level techniques, there's a point of diminishing returns where complexity can outweigh benefits in a pragmatic application.--2. **Effects system complexity**: Our early attempts at using more advanced effects systems (like Polysemy) introduced additional complexity without proportional benefits for this specific use case.--3. **Testing stateful code**: Testing code that interacts with the filesystem required careful setup and teardown of test environments.--4. **Cross-platform considerations**: Ensuring consistent behavior across operating systems required attention to path normalization and platform-specific conventions.--## Functional Programming Patterns--### Pure Functions and Side Effects--- **Pure Functions**: Prefer pure functions over impure operations whenever possible. Pure functions are easier to reason about, test, and compose.- ```haskell- -- Pure function- calculateTotal :: [Item] -> Price- calculateTotal items = sum (map itemPrice items)- - -- Instead of this impure approach- calculateTotal :: [Item] -> IO Price- calculateTotal items = do- forM items $ \item -> do- logItemProcess item -- Side effect!- return (itemPrice item)- ...- ```--- **Effect Localization**: When side effects are necessary, localize them to the boundaries of your application. Keep your core logic pure.- ```haskell- -- Good: Centralized effects at the edge- main :: IO ()- main = do- input <- readInput -- IO at the boundary- let result = process input -- Pure core logic- writeOutput result -- IO at the boundary- ```--- **Resource Management**: Use higher-order functions like `bracket`, `withFile`, or `ResourceT` to ensure resources are properly acquired and released.- ```haskell- -- Ensures file is closed even if an exception occurs- withConfigFile :: FilePath -> (Handle -> IO a) -> IO a- withConfigFile path action = bracket - (openFile path ReadMode) -- acquire- hClose -- release- action -- use- ```--- **Error Handling**: Use types to represent errors rather than exceptions. Wrap impure code with `try`/`catch` and convert exceptions to domain-specific error types.- ```haskell- -- Domain-specific error type- data AppError = FileNotFound FilePath | ParseError String | NetworkError- - -- Convert IO exceptions to domain errors- readConfig :: FilePath -> IO (Either AppError Config)- readConfig path = do- result <- try (readFile path)- case result of- Left e -> return $ Left $ FileNotFound path- Right content -> - case parseConfig content of- Nothing -> return $ Left $ ParseError "Invalid config"- Just config -> return $ Right config- ```--### Type-Driven Development--- **Newtype Wrappers**: Use `newtype` to create distinct types for values that might otherwise be confused.- ```haskell- -- Without newtypes- processUser :: String -> Int -> String -> IO () -- What do these mean?- - -- With newtypes- newtype UserId = UserId String- newtype Age = Age Int- newtype Email = Email String- - processUser :: UserId -> Age -> Email -> IO () -- Much clearer!- ```- -- **Smart Constructors**: Use smart constructors to enforce invariants and hide implementation details.- ```haskell- module Email (Email, mkEmail, emailToText) where- - newtype Email = Email { _unEmail :: Text } -- Private constructor- - -- Smart constructor with validation- mkEmail :: Text -> Either String Email- mkEmail txt- | "@" `isInfixOf` txt = Right (Email txt)- | otherwise = Left "Email must contain @"- - -- Accessor function- emailToText :: Email -> Text- emailToText (Email t) = t- ```--- **Phantom Types**: Use phantom types to encode additional information in the type.- ```haskell- -- Phantom type for file access permissions- data Permission = Read | Write | ReadWrite- - newtype File (p :: Permission) = File FilePath- - -- Operations that respect permissions- readFile :: File p -> IO String- readFile (File path) = -- ...- - writeFile :: File 'Write -> String -> IO ()- writeFile (File path) content = -- ...- - withWritableFile :: File 'Read -> (File 'Write -> IO a) -> IO a- ```--## Module Organization and API Design--### Clean Module Structure--- **Hierarchical Module Structure**: Organize modules hierarchically (e.g., `App.Module.Submodule`) to make the codebase easier to navigate.- ```- MyApp/- Core.hs -- Core functionality- Core/ -- Implementation details- Types.hs- Operations.hs- Database.hs -- Database facade- Database/ -- Database implementations- MySQL.hs- PostgreSQL.hs- ```--- **Facade Modules**: Create facade modules that re-export functionality from specialized modules. This allows implementation changes without affecting users of your API.- ```haskell- -- Database.hs (facade module)- module Database - ( Connection- , QueryResult- , connect- , disconnect- , query- ) where- - import Database.Internal.Types- import Database.Internal.Connection- import Database.Internal.Query- ```--- **Re-export Pattern**: Use selective re-exports to create a clean, focused API while hiding implementation details.- ```haskell- -- Clear separation between public API and internal details- module MyLib- ( -- * Core Types- Widget(..)- , WidgetId- -- * Widget Creation- , createWidget- , defaultWidget- -- * Widget Operations- , updateWidget- , renderWidget- ) where- - import MyLib.Internal.Types- import MyLib.Internal.Operations- ```--### API Design for Human Understanding--- **Module Documentation**: Begin each module with a comprehensive Haddock comment that explains its purpose, main concepts, and usage examples.- ```haskell- {-|- Module : Data.Parser- Description : Parser combinators for structured data- - This module provides parser combinators for processing structured data.- It supports:- - * Basic parsers for primitive types- * Combinators for sequence and choice- * Error reporting with context- - Example usage:- - @- parseJSON :: String -> Either ParseError JSONValue- parseJSON input = runParser jsonValue input- @- -}- ```--- **Function Grouping**: Group related functions together and use Haddock section headers to organize the module documentation.- ```haskell- -- | Core data types- - -- | @Widget@ represents a UI element- data Widget = ...- - -- | Operations on widgets- - -- | Create a new widget- createWidget :: WidgetConfig -> Widget- - -- | Update widget properties- updateWidget :: Widget -> WidgetUpdate -> Widget- ```--- **Type Signatures as Documentation**: Write expressive type signatures that communicate intent. Use meaningful type and function names.- ```haskell- -- Less clear- process :: [a] -> [(a, b)] -> [b] -> [c]- - -- More clear- reconcileInventory :: [Product] -> [(Product, Quantity)] -> [Adjustment] -> [StockChange]- ```--## Advanced Type System Features for Safety and Clarity--### Type Classes and Constraints--- **Type Class Constraints**: Use type class constraints to make requirements explicit and enable polymorphism.- ```haskell- -- Generic function that works with any monoid- combineAll :: Monoid a => [a] -> a- combineAll = foldr (<>) mempty- - -- Use with different monoid instances- sumAll :: [Int] -> Int- sumAll = getSum . combineAll . map Sum- - concatAll :: [[a]] -> [a]- concatAll = combineAll -- Works because lists are monoids- ```--- **Constraint Type Aliases**: Use ConstraintKinds to create aliases for common constraint combinations.- ```haskell- {-# LANGUAGE ConstraintKinds #-}- - -- Alias for common constraint combination- type Serializable a = (ToJSON a, FromJSON a, Show a, Eq a)- - -- Simplified type signature- storeEntity :: Serializable a => Connection -> a -> IO ()- storeEntity conn entity = do- let json = toJSON entity- -- Store the entity...- ```--- **Multi-Parameter Type Classes**: Use MPTCs with functional dependencies or associated types to express relationships between types.- ```haskell- {-# LANGUAGE MultiParamTypeClasses, FunctionalDependencies #-}- - -- Repository pattern with type safety- class Repository r e | r -> e where- save :: e -> r -> IO r- findById :: Id e -> r -> IO (Maybe e)- delete :: Id e -> r -> IO r- - -- Implementation for specific entity type- instance Repository UserRepo User where- save user repo = -- Implementation- findById userId repo = -- Implementation- delete userId repo = -- Implementation- ```--### Advanced Type Safety Features--- **GADTs**: Use Generalized Algebraic Data Types to enforce invariants at the type level.- ```haskell- {-# LANGUAGE GADTs, DataKinds #-}- - -- Status for a request- data Status = Pending | Approved | Rejected- - -- GADT that ensures status-specific operations- data Request s where- PendingRequest :: RequestId -> UserData -> Request 'Pending- ApprovedRequest :: RequestId -> UserData -> ApproverInfo -> Request 'Approved- RejectedRequest :: RequestId -> UserData -> RejectReason -> Request 'Rejected- - -- Type-safe operations- approve :: ApproverInfo -> Request 'Pending -> Request 'Approved- approve approver (PendingRequest id userData) = - ApprovedRequest id userData approver- - -- Won't compile:- -- approve :: ApproverInfo -> Request 'Rejected -> Request 'Approved- ```--- **Phantom Types**: Use phantom types to add type-level tags without runtime overhead.- ```haskell- {-# LANGUAGE RankNTypes, KindSignatures #-}- - -- Phantom type for validation state- data Validated- data Unvalidated- - -- Email with validation state in the type- newtype Email (s :: Type) = Email Text- - -- Smart constructor that returns validated email- validateEmail :: Email Unvalidated -> Either String (Email Validated)- validateEmail (Email txt)- | "@" `isInfixOf` txt = Right (Email txt)- | otherwise = Left "Invalid email address"- - -- Only validated emails can be sent- sendEmail :: Email Validated -> Message -> IO ()- sendEmail (Email addr) msg = -- Implementation- ```--- **Type Families**: Use type families to compute types based on other types.- ```haskell- {-# LANGUAGE TypeFamilies #-}- - -- Type family for result of an operation based on input type- type family ResultOf a where- ResultOf String = Int- ResultOf Int = Double- ResultOf (Maybe a) = Maybe (ResultOf a)- - -- Function with type that depends on input- process :: a -> ResultOf a- process = -- Implementation- ```--### Expressive Deriving Mechanisms--- **DerivingVia**: Use DerivingVia for zero-boilerplate reuse of implementations.- ```haskell- {-# LANGUAGE DerivingVia, DerivingStrategies #-}- - -- Newtype wrapper for JSON serialization customization- newtype UserName = UserName Text- deriving stock (Show, Eq)- deriving newtype (Semigroup, Monoid)- deriving (ToJSON, FromJSON) via Text- - -- Composition of deriving strategies- newtype UserId = UserId Int- deriving stock (Show, Eq, Ord)- deriving (ToJSON, FromJSON) via (Tagged "id" Int)- ```--- **DerivingStrategies**: Be explicit about deriving mechanisms for clarity.- ```haskell- -- Explicitly specify deriving strategy- data User = User- { userId :: UserId- , userName :: UserName- , userEmail :: Email Validated- }- deriving stock (Show, Eq)- deriving anyclass (ToJSON, FromJSON)- deriving (Semigroup) via (GenericSemigroup User)- ```--## Composition Patterns for Readability--### Kleisli Composition for Monadic Pipelines--Kleisli composition elegantly chains monadic operations, improving readability for complex workflows.--```haskell-import Control.Arrow ((>>>), (<<<), Kleisli(..), runKleisli)---- Monadic functions (error handling, IO, etc.)-validateInput :: Input -> Either Error ValidInput-processData :: ValidInput -> Either Error ProcessedData-generateReport :: ProcessedData -> Either Error Report---- Create Kleisli arrows for these functions-validateK = Kleisli validateInput-processK = Kleisli processData-reportK = Kleisli generateReport---- Compose them into a clean pipeline-pipeline :: Kleisli (Either Error) Input Report-pipeline = validateK >>> processK >>> reportK---- Run the pipeline-processBatch :: [Input] -> [Either Error Report]-processBatch inputs = map (runKleisli pipeline) inputs---- Compare to nested approach:-processManually :: Input -> Either Error Report-processManually input = do- validInput <- validateInput input- processed <- processData validInput - generateReport processed -- Less clear for complex pipelines-```--### Function Composition for Pure Pipelines--When working with pure functions, standard function composition offers clarity.--```haskell--- Pure data transformations-normalize :: RawData -> NormalizedData-analyze :: NormalizedData -> AnalysisResult -format :: AnalysisResult -> FormattedOutput---- Direct composition-pipeline :: RawData -> FormattedOutput-pipeline = format . analyze . normalize---- Data flows from right to left, which can be counterintuitive---- Alternative: Forward composition with Data.Function-import Data.Function ((&))--pipeline' :: RawData -> FormattedOutput-pipeline' data = data - & normalize -- First step- & analyze -- Second step- & format -- Final step-```--## Error Handling Patterns--### Typed Errors with Monad Transformers--Use explicit error types and monad transformers for comprehensive error handling.--```haskell--- Define a clear error hierarchy-data AppError- = FileSystemError FilePath IOError- | ConfigError String- | NetworkError ConnectionInfo String- | ValidationError [String]- | PermissionError UserId Resource- deriving (Show, Eq)---- Application monad with built-in error handling-type AppM a = ReaderT AppConfig (ExceptT AppError IO) a---- Helper for running the monad stack-runAppM :: AppConfig -> AppM a -> IO (Either AppError a)-runAppM config action = runExceptT (runReaderT action config)---- Convert IO exceptions to domain-specific errors-safeFileOperation :: FilePath -> AppM ByteString-safeFileOperation path = do- result <- liftIO $ try $ readFile path- case result of- Left e -> throwError $ FileSystemError path e- Right content -> return content-```--### Railway-Oriented Programming with Either--Use Either for explicit error handling in pure code without the complexity of monad transformers.--```haskell--- Define error types-data ValidationError = - MissingField String - | InvalidFormat String String- | OutOfRange String Int Int Int- deriving (Show, Eq)---- Input validation function returning Either-validateInput :: UserInput -> Either ValidationError ValidatedInput-validateInput input = do- name <- validateName (inputName input)- age <- validateAge (inputAge input)- email <- validateEmail (inputEmail input)- pure ValidatedInput- { validName = name- , validAge = age- , validEmail = email- }---- Simple validation function-validateAge :: Maybe Int -> Either ValidationError Int-validateAge Nothing = Left (MissingField "age")-validateAge (Just age)- | age < 18 = Left (OutOfRange "age" age 18 120)- | age > 120 = Left (OutOfRange "age" age 18 120)- | otherwise = Right age-```--### Smart Constructors for Validation--Use smart constructors to ensure valid data at the type level.--```haskell--- Define a newtype with private constructor-module Email (Email, mkEmail, emailToText) where--newtype Email = Email { _unEmail :: Text } -- Private constructor---- Smart constructor returns Either for explicit error handling-mkEmail :: Text -> Either EmailError Email-mkEmail txt- | T.null txt = Left EmailEmpty- | not ("@" `T.isInfixOf` txt) = Left EmailMissingAt- | not (hasDomainPart txt) = Left EmailInvalidDomain- | otherwise = Right (Email $ T.toLower txt)- --- Safe access functions -emailToText :: Email -> Text-emailToText (Email t) = t---- Because Email constructor is not exported, all Email values in your--- program are guaranteed to be valid-```--### Nested Error Handling with MonadError--Use MonadError for cleaner nested error handling.--```haskell-import Control.Monad.Except---- Function signatures are cleaner with constraints instead of concrete types-processTransaction :: (MonadError AppError m, MonadIO m) => Transaction -> m Receipt-processTransaction tx = do- -- validate will throw an error on invalid transaction- validTx <- validate tx- - -- attempt to process, may throw network error- result <- processPayment validTx `catchError` \e -> - -- Add context to the error- throwError $ PaymentError (transactionId tx) e- - -- generate receipt if successful- generateReceipt tx result-```--## Resource Management and Safety Patterns--### Bracket Pattern for Resource Safety--Use the bracket pattern to ensure resources are properly acquired and released even when exceptions occur.--```haskell-import Control.Exception (bracket)-import System.IO---- Generic template for resource handling-withResource :: IO a -- acquire resource- -> (a -> IO ()) -- release resource- -> (a -> IO b) -- use resource- -> IO b-withResource acquire release use = bracket acquire release use---- Example: File handling with automatic cleanup-withFile' :: FilePath -> IOMode -> (Handle -> IO a) -> IO a-withFile' path mode = bracket - (openFile path mode) -- acquire- hClose -- release- --- Example: Database connection with transaction support-withTransaction :: Connection -> (Connection -> IO a) -> IO a-withTransaction conn action = bracket- (do beginTransaction conn; return conn) -- start transaction- (\c -> do rollback c; return ()) -- rollback on exception- (\c -> do result <- action c -- run action- commit c -- commit on success- return result)-```--### Resource Management with ResourceT--For complex resource management scenarios, ResourceT from the resourcet package provides more flexibility.--```haskell-import Control.Monad.Trans.Resource---- Create a computation that allocates and automatically frees resources-complexOperation :: ResourceT IO Result-complexOperation = do- -- Register resources with cleanup actions- (dbReleaseKey, dbConn) <- allocate - (connectDB "database.db") -- acquire- disconnectDB -- release- - (fileReleaseKey, fileHandle) <- allocate- (openFile "output.txt" WriteMode) -- acquire- hClose -- release- - -- Early release if needed- release dbReleaseKey - - -- Resources automatically released when ResourceT exits- liftIO $ processWithResources dbConn fileHandle---- Run the ResourceT computation-runResourceOperation :: IO Result-runResourceOperation = runResourceT complexOperation-```--### Capability-Based Security--Use the capability pattern to restrict access to sensitive operations.--```haskell--- Define capability tokens-newtype FileReadCap = FileReadCap { allowedDirs :: [FilePath] } -newtype FileWriteCap = FileWriteCap { writeDirs :: [FilePath] }---- Operations require explicit capabilities-readFile' :: FileReadCap -> FilePath -> IO String-readFile' cap path = do- -- Verify path is in allowed directories- allowed <- isPathAllowed (allowedDirs cap) path- if allowed - then readFile path- else throwIO $ PermissionError $ "Cannot read: " ++ path- --- Restricted capability creation-rootCap :: IO FileReadCap-rootCap = do- -- Check if user has admin rights- isAdmin <- checkAdminRights- if isAdmin- then return $ FileReadCap ["/"] -- Full access- else return $ FileReadCap ["/home/user"] -- Limited access-```--### Resource Pools for Performance--Use resource pooling for expensive resources like database connections.--```haskell-import Data.Pool---- Create a connection pool-initConnectionPool :: Config -> IO (Pool Connection)-initConnectionPool config = createPool- (connect (dbHost config) (dbUser config)) -- create resource- close -- destroy resource- 1 -- stripes (for concurrency)- 60 -- unused resource timeout (seconds)- 10 -- maximum resources per stripe---- Use a resource from the pool-withConnection :: Pool Connection -> (Connection -> IO a) -> IO a-withConnection pool action = withResource pool action-```--## Testing Patterns for Robust Code--### Property-Based Testing--Use property-based testing to identify edge cases that unit tests might miss.--```haskell-import Test.QuickCheck-import Data.List (sort)---- Define properties that should hold for any input-prop_reverseInvolutive :: [Int] -> Bool-prop_reverseInvolutive xs = reverse (reverse xs) == xs--prop_sortIdempotent :: [Int] -> Bool-prop_sortIdempotent xs = sort (sort xs) == sort xs---- Test invariants that your functions should maintain-prop_parseRenderRoundtrip :: Config -> Property-prop_parseRenderRoundtrip config = - parseConfig (renderConfig config) === Just config---- Run the tests-main :: IO ()-main = do- quickCheck prop_reverseInvolutive- quickCheck prop_sortIdempotent- quickCheck prop_parseRenderRoundtrip-```--### Isolated Test Environments--Create isolated, reproducible test environments for reliable testing.--```haskell-import System.IO.Temp (withSystemTempDirectory)-import System.FilePath ((</>))---- Helper for setting up a test environment-withTestEnvironment :: (FilePath -> IO ()) -> IO ()-withTestEnvironment runTest = - withSystemTempDirectory "test-dir" $ \tmpDir -> do- -- Create test files and directories- createDirectoryIfMissing True (tmpDir </> "src")- createDirectoryIfMissing True (tmpDir </> "config")- writeFile (tmpDir </> "src" </> "test.file") "test content"- writeFile (tmpDir </> "config" </> "settings.json") "{\"mode\":\"test\"}"- - -- Run the test with the prepared environment- runTest tmpDir---- Use it in hspec tests-it "processes files correctly" $- withTestEnvironment $ \tmpDir -> do- -- Configure app to use the temp directory- let config = defaultConfig { rootDir = tmpDir }- - -- Run the application- result <- runApp config- - -- Make assertions- result `shouldBe` Success-```--### Golden Tests for Output Verification--Use golden testing to verify your outputs match expected templates.--```haskell-import Test.Tasty.Golden (goldenVsString)-import qualified Data.ByteString.Lazy as BL---- Test that generated output matches a "golden" file-goldenOutputTest :: TestTree-goldenOutputTest = goldenVsString- "report generation" -- test name- "test/golden/expected_report.json" -- golden file path- (BL.fromStrict <$> generateReport testData) -- actual output---- For complex outputs like HTML, use a difference tool-htmlGoldenTest :: TestTree-htmlGoldenTest = goldenVsFileDiff- "page rendering" -- test name- diffCommand -- diff command to use- "test/golden/expected.html" -- golden file- "test/output/actual.html" -- actual output file- (renderPage testData) -- action to generate actual output- where- diffCommand ref new = ["diff", "-u", ref, new]-```--### Table-Driven Testing--Use table-driven testing for testing multiple related cases concisely.--```haskell-import Test.Hspec---- Define test cases as a list of input-output pairs-testCases :: [(String, Int)]-testCases = - [ ("123", 123)- , ("0", 0)- , ("00123", 123)- , ("+123", 123)- , ("-123", -123)- ]---- Test all cases using the same pattern-spec :: Spec-spec = describe "parseNumber" $ do- forM_ testCases $ \(input, expected) ->- it ("parses " ++ show input ++ " correctly") $ do- parseNumber input `shouldBe` Right expected---- For more complex test cases, use records-data ValidationTestCase = ValidationTestCase- { testName :: String- , testInput :: UserInput- , expectedResult :: Either ValidationError ValidatedInput- }--validationTests :: [ValidationTestCase]-validationTests = - [ ValidationTestCase - "valid input" - (UserInput "John" (Just 30) "john@example.com")- (Right $ ValidatedInput "John" 30 "john@example.com")- , ValidationTestCase- "missing age"- (UserInput "John" Nothing "john@example.com")- (Left $ MissingField "age")- ]-```--### Test Fixtures and Mocks--Use fixtures and mocks to test code that depends on external systems.--```haskell--- Define a typecalss for database operations-class Monad m => MonadDB m where- queryUsers :: m [User]- saveUser :: User -> m ()- --- Production implementation-instance MonadDB IO where- queryUsers = queryUsersFromDatabase- saveUser = saveUserToDatabase- --- Test implementation -instance MonadDB (State TestDB) where- queryUsers = gets testDBUsers- saveUser user = modify $ \db -> - db { testDBUsers = user : testDBUsers db }---- Example test-it "creates user profile" $ do- -- Set up initial DB state- let initialDB = TestDB { testDBUsers = [] }- - -- Run operation with mock DB- let (result, finalDB) = runState createUserProfile initialDB- - -- Assert the operation worked correctly- length (testDBUsers finalDB) `shouldBe` 1- --- Function being tested uses constraint for testability-createUserProfile :: MonadDB m => m User-createUserProfile = do- -- Implementation-```--## Debugging and Maintainability Patterns--### Function Decomposition for Testability--Break complex functions into smaller, testable parts that can be individually verified.--```haskell--- Original monolithic function (hard to test and debug)-complexProcess :: Config -> [Input] -> IO [Output]-complexProcess config inputs = do- -- 100+ lines of complex logic with multiple responsibilities- -- and many potential failure points...---- Refactored into testable components-validateInputs :: [Input] -> Either ValidationError [ValidInput]-validateInputs = traverse validateSingleInput--processValidInputs :: Config -> [ValidInput] -> IO [ProcessedData]-processValidInputs config = traverse (processOne config) --generateOutputs :: [ProcessedData] -> [Output]-generateOutputs = map convertToOutput---- Compose them back together with clear error handling-complexProcess :: Config -> [Input] -> IO (Either Error [Output])-complexProcess config inputs = do- case validateInputs inputs of- Left validationError -> - pure $ Left $ ValidationFailed validationError- - Right validInputs -> do- processResult <- try $ processValidInputs config validInputs- case processResult of- Left ex -> - pure $ Left $ ProcessingFailed ex- - Right processed ->- pure $ Right $ generateOutputs processed-```--### Layered Debugging Techniques--Use a combination of tracing approaches for effective debugging.--```haskell-import Debug.Trace (trace, traceShowId, traceM)-import qualified System.IO as IO---- 1. Simple trace for basic logging (but doesn't clutter production code)-withTracing :: Bool -> a -> String -> a-withTracing True x msg = trace msg x-withTracing False x _ = x---- 2. Effectful tracing for debugging monadic code-processItems :: [Item] -> IO [Result]-processItems = mapM $ \item -> do- when debugMode $ traceM $ "Processing: " ++ show item- result <- processItem item- when debugMode $ traceM $ "Result: " ++ show result- return result---- 3. Conditional file logging when trace output is too large-logToFile :: String -> IO ()-logToFile msg = when debugMode $- IO.appendFile "debug.log" (msg ++ "\n")- --- 4. TraceShowId for quick inspection of values in a pipeline-calculateResults :: [Input] -> [Output]-calculateResults = filter isValid - >>> map preprocess - >>> traceShowId -- See the values mid-pipeline- >>> map calculate- >>> filter isSignificant---- 5. Temporary function modification for deeper inspection--- Original function-process :: Item -> Result-process = step1 >>> step2 >>> step3---- Modified during debugging-process :: Item -> Result-process item = - let s1 = step1 item- _ = trace ("After step1: " ++ show s1) ()- s2 = step2 s1- _ = trace ("After step2: " ++ show s2) ()- in step3 s2-```--### Typed Holes for Guided Development--Use typed holes to let the compiler guide your implementation.--```haskell--- Start with the function type signature-processTransaction :: UserId -> Transaction -> Either Error Receipt-processTransaction userId transaction = _implementThis---- The compiler will tell you the expected type of _implementThis---- Gradually fill in implementation guided by holes-processTransaction userId transaction = do- user <- _getUser userId- validated <- _validateTransaction user transaction- _processPayment validated---- Each hole tells you what you need to implement next-_getUser :: UserId -> Either Error User-_getUser = ...--_validateTransaction :: User -> Transaction -> Either Error ValidatedTransaction-_validateTransaction = ...-```--### Equational Reasoning and Step-by-Step Refactoring--Use equational reasoning to verify code transformations.--```haskell--- Original code-sum (map square xs)---- Step 1: Rewrite using function composition-sum . map square $ xs---- Step 2: Introduce a specialized function-sumOfSquares = sum . map square---- Verification:--- sum (map square xs)--- = sum . map square $ xs -- By function composition--- = sumOfSquares xs -- By definition---- More complex example:-processList xs = filter p1 (map f (filter p2 xs))---- Transform step by step:-processList xs = (filter p1 . map f . filter p2) xs---- Extract function:-processList = filter p1 . map f . filter p2---- Each step preserves behavior but improves readability-```--## Performance Patterns and Optimizations--### Hash Function Selection and Implementation--Choose the appropriate hash function based on your actual requirements, not just defaults:--```haskell--- Non-cryptographic fast hashing (xxHash) when you just need speed-import qualified Data.Digest.XXHash.FFI as XXH-import Data.Hashable (hash)--fastChecksum :: BS.ByteString -> Checksum-fastChecksum content =- let -- Use XXH3 hash function (extremely fast)- hashVal = hash (XXH.XXH3 content)- -- Handle potential negative hash values- absHash = abs hashVal- -- Convert to hex string representation- hexStr = showHex absHash ""- in Checksum hexStr---- Cryptographic hashing when security is required-import qualified Crypto.Hash.SHA256 as SHA256-import qualified Data.ByteString.Base16 as Base16--secureChecksum :: BS.ByteString -> Checksum-secureChecksum content =- let hash = SHA256.hash content- hexHash = Base16.encode hash- in Checksum (show hexHash)-```--Performance considerations:-- XXH3 can be 5-15x faster than cryptographic hashes like SHA-256-- For content identification, non-cryptographic hashes are usually sufficient-- Be careful with hash values: they may be negative and need absolute value conversion-- Abstract hash implementation behind a consistent interface--### Efficient ByteString Usage--Use ByteString for efficient text and binary data handling.--```haskell-import qualified Data.ByteString as BS-import qualified Data.ByteString.Char8 as BS8-import qualified Data.ByteString.Lazy as BL-import Data.Word (Word8)---- Converting between ByteString and String (avoid in performance-critical code)-stringToBS :: String -> BS.ByteString-stringToBS = BS8.pack -- For ASCII-only text---- For general Unicode text, use Text instead of String/ByteString-import qualified Data.Text as T-import qualified Data.Text.Encoding as TE--textToBS :: T.Text -> BS.ByteString-textToBS = TE.encodeUtf8--bsToText :: BS.ByteString -> Either String T.Text-bsToText bs = case TE.decodeUtf8' bs of- Left err -> Left $ "UTF-8 decoding error: " ++ show err- Right text -> Right text---- Efficient file reading-readLargeFile :: FilePath -> IO BL.ByteString-readLargeFile = BL.readFile -- Lazy reading for large files---- Stream processing for large data-processLargeFile :: FilePath -> FilePath -> IO ()-processLargeFile input output = do- contents <- BL.readFile input- BL.writeFile output $ BL.filter (/= 0) contents-```--### Strict Fields for Memory Efficiency--Use strictness annotations to avoid space leaks.--```haskell--- Without strictness, can cause space leaks-data Configuration = Configuration- { configPort :: Int- , configHost :: String- , configTimeout :: Int- }---- With strictness annotations, more memory-efficient-data Configuration' = Configuration'- { configPort' :: !Int -- Strict field- , configHost' :: !String- , configTimeout' :: !Int- }---- Strictness and UNPACK for numeric data-data Point = Point- { x :: {-# UNPACK #-} !Double -- Unpacked strict field- , y :: {-# UNPACK #-} !Double- }---- For record types with many fields-{-# LANGUAGE StrictData #-} -- All fields strict by default-data User = User- { userId :: Int- , userName :: String- , userEmail :: String- }-```--### Fusion and Deforestation--Take advantage of list fusion to eliminate intermediate data structures.--```haskell--- This will create an intermediate list-naiveProcess :: [Int] -> Int-naiveProcess xs = sum (filter even (map (*2) xs))---- GHC can optimize this with list fusion-fusedProcess :: [Int] -> Int-fusedProcess = sum . filter even . map (*2)---- Even better: use foldr to fuse everything into a single pass-singlePassProcess :: [Int] -> Int-singlePassProcess = foldr (\x acc -> if even (x*2) then acc + (x*2) else acc) 0---- For more control, use a specialized streaming library-import qualified Streamly.Prelude as S--streamProcess :: [Int] -> IO Int-streamProcess xs = S.fold S.sum - $ S.filter even - $ S.map (*2) - $ S.fromList xs-```--### Handling Numeric Edge Cases--Be aware of edge cases when working with numeric computations, especially hash functions:--```haskell--- Example: Converting hash values to hex strings-import Data.Hashable (hash)-import Numeric (showHex)---- INCORRECT: May fail on negative hash values-toHexStringUnsafe :: Hashable a => a -> String-toHexStringUnsafe x = showHex (hash x) "" -- Fails if hash x is negative---- CORRECT: Handle negative hash values-toHexString :: Hashable a => a -> String-toHexString x = - let hashVal = hash x- -- Take absolute value to ensure showHex works correctly- absHash = abs hashVal- in showHex absHash ""---- Alternative: Use Data.Bits for bit manipulation-import Data.Bits ((.&.))-import Data.Word (Word64)---- This avoids negative numbers entirely by using Word64-toHexStringBits :: Hashable a => a -> String-toHexStringBits x =- let hashVal = hash x- -- Convert to Word64 by masking with all bits set- -- This preserves the exact bit pattern- wordVal = fromIntegral hashVal .&. (maxBound :: Word64)- in showHex wordVal ""-```--Common numeric pitfalls to handle:-- Integer overflow/underflow-- Division by zero-- Negative values in functions expecting positives (like showHex)-- Floating point precision errors-- Range limitations in conversions between numeric types--### Lazy vs. Strict Evaluation Control--Explicitly control evaluation strategy for better performance.--```haskell-import Control.DeepSeq (NFData, force, ($!!))---- Force full evaluation of a structure when needed-processStrictly :: (NFData a) => [a] -> [a]-processStrictly xs = force (map process xs)---- Manually force evaluation to specific depth-data Tree a = Leaf a | Node (Tree a) (Tree a)--forceTree :: Tree a -> ()-forceTree (Leaf _) = ()-forceTree (Node l r) = forceTree l `seq` forceTree r `seq` ()---- Use bang patterns for strict evaluation in function arguments-sumListStrict :: [Int] -> Int-sumListStrict !xs = sum xs -- Force evaluation of xs---- Use BangPatterns language extension for more control-{-# LANGUAGE BangPatterns #-}--foldlStrict :: (b -> a -> b) -> b -> [a] -> b-foldlStrict f !acc [] = acc-foldlStrict f !acc (x:xs) = foldlStrict f (f acc x) xs-```--## Build and Packaging Best Practices--### Cabal Configuration--Properly configure your Cabal file for reliable builds and distribution.--```haskell--- Example cabal file structure with key sections-name: my-project-version: 0.1.0-synopsis: Short description of your project-description: Longer, multi-line description- of your project's purpose and features.-license: MIT-license-file: LICENSE-author: Your Name-maintainer: your.email@example.com-category: Development-build-type: Custom -- Use Custom for custom Setup.hs-cabal-version: 2.0---- Set up custom build if needed-custom-setup- setup-depends: base >= 4.7 && < 5,- Cabal >= 2.0.0.2 && < 3.12,- directory,- filepath,- process---- Documentation files that should be included in source distributions-extra-source-files: README.md- CHANGELOG.md- examples/*.hs---- Files to be installed with the package-data-files: templates/*.txt- data/*.json---- Create an autogenerated module for accessing data-files-auto-generated-modules: Paths_my_project---- Library component-library- -- Modules exposed to users of the library- exposed-modules: MyProject- MyProject.Core- MyProject.Types- - -- Internal modules not exposed to users- other-modules: MyProject.Internal.Util- Paths_my_project -- Auto-generated module for data files- - -- Enable useful warnings- ghc-options: -Wall - -Wcompat- -Wincomplete-record-updates- -Wincomplete-uni-patterns- -Wredundant-constraints- - -- Dependencies with version constraints- build-depends: base >= 4.7 && < 5,- aeson >= 1.4 && < 2.2,- text >= 1.2 && < 2.1,- containers >= 0.6 && < 0.7-```--### Version Range Best Practices--Specify appropriate version ranges for dependencies to avoid compatibility issues.--```--- For most dependencies, specify both lower and upper bounds-build-depends: base >= 4.14 && < 5,- text >= 1.2.4 && < 2.1,- aeson >= 2.0 && < 2.2---- Version range notation examples:--- == 1.0.0 -- Exactly version 1.0.0--- >= 1.0 && < 1.1 -- Greater than or equal to 1.0 and less than 1.1--- ^>= 1.0.0 -- Compatible with version 1.0.0 (>=1.0.0 && <1.1)--- ~> 1.0.0 -- Similar to ^>= but with more restriction-```--### Documentation Integration--Integrate documentation into your build process for better user experience.--```haskell--- In Setup.hs-main = defaultMainWithHooks $ simpleUserHooks- { postBuild = \args buildFlags pkg lbi -> do- -- Run standard post-build first- postBuild simpleUserHooks args buildFlags pkg lbi- - -- Generate documentation- generateDocs pkg lbi- }---- Generate embedded documentation from Markdown-generateDocs :: PackageDescription -> LocalBuildInfo -> IO ()-generateDocs pkg lbi = do- let docsDir = buildDir lbi </> "docs"- createDirectoryIfMissing True docsDir- - -- Process each documentation file- mapM_ (processDoc docsDir) - ["README.md", "TUTORIAL.md", "API.md"]- - -- Use Haddock to generate API documentation- let ghcProg = programPath (haddockProgram (withPrograms lbi))- pkgDb = packageDBFlags lbi- runProcess ghcProg ["--haddock", ...] Nothing Nothing Nothing Nothing-```--### Making Use of Flags and Conditionals--Use flags and conditional compilation for flexible builds.--```haskell--- Define build flags in the cabal file-flag strict- description: Enable stricter GHC options- default: False- manual: True--flag optimize- description: Build with optimization- default: True- manual: False---- Use flags in the build configuration-library- if flag(strict)- ghc-options: -Wall -Werror- else- ghc-options: -Wall- - if flag(optimize)- ghc-options: -O2- else- ghc-options: -O0- - -- Conditional dependencies- if os(windows)- build-depends: Win32- else- build-depends: unix---- For system-specific code- if os(darwin)- cpp-options: -DMACOS- other-modules: System.MacOS.Specific- elif os(linux)- cpp-options: -DLINUX- other-modules: System.Linux.Specific-```--### Custom Setup Scripts--```haskell--- Setup.hs for custom build steps-import Distribution.Simple-import Distribution.Simple.Setup-import Distribution.Simple.LocalBuildInfo-import Distribution.PackageDescription--main = defaultMainWithHooks simpleUserHooks- { postBuild = \args flags pkg lbi -> do- -- Run standard post-build first- postBuild simpleUserHooks args flags pkg lbi- -- Then run custom actions- customAction args flags pkg lbi- }---- Custom build/install actions-customAction :: Args -> BuildFlags -> PackageDescription -> LocalBuildInfo -> IO ()-customAction _ _ pkg lbi = do- -- Access package configuration- let pkgName = unPackageName $ pkgName $ package pkg- buildDir = buildDir lbi- - -- Execute custom build steps- -- ...-```--## Version Number Management--```haskell--- Access version from cabal file-import qualified Paths_<package> as Meta-import Data.Version (showVersion)---- Display version-version :: String-version = showVersion Meta.version-```--## Best Practices--- Favor pure Haskell implementations over shell commands-- Document system dependencies explicitly-- Load resource files from standardized locations, not hardcoded paths-- Derive version information from the cabal file, not hardcoded-- Use type applications for parametric types-- Normalize paths for cross-platform compatibility-- Add explicit type annotations for complex expressions-- Integrate with standard system conventions (man pages, config directories)-- Use Cabal's installation system rather than custom scripts for deployable artifacts--## Documentation Integration--```haskell--- In cabal file-data-files:- doc/*.md, -- Source files- templates/*.txt -- Templates---- In Setup.hs-import Distribution.PackageDescription-import Distribution.Simple.LocalBuildInfo-import Distribution.Simple.BuildPaths (autogenModulesDir)-import Distribution.Simple.Utils (installOrdinaryFiles)---- Generate documentation during build-postBuild _ _ pkg lbi = do- let dataDirName = dataDir lbi- docSrcDir = dataDirName </> "doc"- docDestDir = buildDir lbi </> "doc"- - -- Generate docs from templates- generateDocs docSrcDir docDestDir---- Install documentation to standard locations-copyHook oldHook pkg_descr lbi hooks flags = do- -- First do the standard copy- oldHook pkg_descr lbi hooks flags- - -- Then copy documentation to proper locations- let docDir = case os of- "darwin" -> "/usr/local/share/doc/" ++ pkgName- "linux" -> "/usr/share/doc/" ++ pkgName- _ -> error "Unsupported OS"- - installOrdinaryFiles verbosity docDir [(buildDir lbi </> "doc", "*.html")]-```--## Patterns for Human-AI Collaboration--These patterns are particularly effective when working with AI assistants on Haskell projects.--### Leveraging Types for Verification--Use type checking to verify collaboratively written code and catch misunderstandings early:--```haskell--- Example 1: Use phantom types to enforce usage patterns-data Operation = Read | Write | ReadWrite---- File access with permission enforcement-newtype File (p :: Operation) = File FilePath--readFile :: File p -> IO String-readFile (File path) = -- Implementation--writeFile :: File 'Write -> String -> IO ()-writeFile (File path) content = -- Implementation---- Example 2: Use GADTs to restrict operations-data DatabaseAction a where- Query :: SQL -> DatabaseAction [Row]- Update :: SQL -> DatabaseAction Int- Transaction :: [DatabaseAction a] -> DatabaseAction [a]---- The return type enforces that queries return rows and updates return count-runAction :: DatabaseAction a -> Connection -> IO a---- Example 3: Use newtypes to prevent confusion-newtype FileReadCap = FileReadCap { allowedDirs :: [FilePath] }-newtype FileWriteCap = FileWriteCap { writeDirs :: [FilePath] }---- Won't compile if permissions are mixed up-readFile' :: FileReadCap -> FilePath -> IO String-writeFile' :: FileWriteCap -> FilePath -> String -> IO ()-```--By using these patterns, the AI can catch type errors during development rather than relying solely on runtime testing. The compiler becomes an active participant in the human-AI collaboration.--### Explicit Type Annotations--Add type annotations to make intentions clear and guide AI inference, even when GHC can infer types.--```haskell--- Without annotation (ambiguous intention)-processData input = map process . filter isValid $ input---- With annotation (clear intention)-processData :: [InputData] -> [OutputData]-processData input = map process . filter isValid $ input---- Intermediate type annotations for complex pipelines-processData :: [InputData] -> [OutputData]-processData input = - let validData = filter isValid input :: [InputData]- processedData = map process validData :: [IntermediateData]- result = finalize <$> processedData :: [OutputData]- in result-```--### Named Function Parameters--Use record syntax for complex parameter sets to make function usage self-documenting.--```haskell--- Hard to understand parameter meanings-createUser :: String -> Int -> String -> Bool -> IO User-createUser name age email verified = ...---- Parameters are self-documenting with records-data CreateUserParams = CreateUserParams- { userName :: String- , userAge :: Int- , userEmail :: String- , isVerified :: Bool- }--createUser :: CreateUserParams -> IO User-createUser params = ...---- Usage is clear and order-independent-newUser <- createUser CreateUserParams- { userName = "John"- , userAge = 30- , userEmail = "john@example.com"- , isVerified = True- }-```--### Consistent Error Handling Patterns--Choose a consistent error handling approach and stick to it across the codebase.--```haskell--- Example with ExceptT pattern-type AppM a = ExceptT AppError IO a---- Clear error hierarchy-data AppError- = ValidationError String- | DatabaseError DBError- | AuthError AuthenticationError- | NotFoundError Resource- deriving (Show, Eq)---- Helper functions for error handling-whenM :: Monad m => m Bool -> m () -> m ()-whenM cond action = do- result <- cond- when result action---- Usage-validateInput :: Input -> AppM ValidatedInput-validateInput input = do- whenM (pure $ null $ inputName input) $- throwError $ ValidationError "Name cannot be empty"- - whenM (pure $ inputAge input < 18) $- throwError $ ValidationError "Must be at least 18 years old"- - -- Create validated input after all checks pass- return ValidatedInput- { validName = inputName input- , validAge = inputAge input- }-```--### Build System Patterns--Use simpler build configurations when possible to ease maintenance.--```haskell--- Prefer Simple build-type over Custom when possible-build-type: Simple---- Use common extensions across the project-default-extensions: - OverloadedStrings- LambdaCase- NamedFieldPuns- RecordWildCards- DeriveGeneric- DeriveDataTypeable---- Only use custom Setup.hs when actually needed--- For example, to generate and install man pages-```--### Library Selection for Maintainability--Choose libraries that align with the project's complexity needs and maintainer expertise:--```haskell--- Prefer libraries with clear type signatures and good documentation--- GOOD: Easy for humans and AI to understand this API--- xxhash-ffi provides a clear API with good type signatures-import qualified Data.Digest.XXHash.FFI as XXH-import Data.Hashable (hash)--calculateChecksum :: BS.ByteString -> String-calculateChecksum content = show $ hash (XXH.XXH3 content)---- AVOID: Complex APIs with many type parameters or advanced features --- unless they're truly needed--- Overly complex for simple checksumming needs:-import qualified Crypto.Hash as CH-import qualified Crypto.Hash.Algorithms as CHA--complexChecksum :: BS.ByteString -> String-complexChecksum content = - show (CH.hashWith CHA.SHA256 content :: CH.Digest CHA.SHA256)-```--Guidelines for library selection:-- Choose libraries that match the project's complexity and maintainer expertise-- Prefer libraries with clear documentation and simple, well-typed APIs-- Consider the maintenance burden and dependency footprint-- Avoid over-engineered solutions for simple problems-- Document reasoning for library choices to help future maintainers--### Module Organization for Discovery--Structure modules to facilitate code discovery by humans and AI assistants.--```haskell--- Organize hierarchically with explicit exports-module MyApp- ( -- * Core types- AppConfig(..)- , AppState(..)- - -- * Running the application- , runApp- , runAppWithConfig- - -- * Error handling- , AppError(..)- , handleError- ) where---- Create index modules-module MyApp.Database- ( -- * Re-exports from all database modules- module MyApp.Database.Connection- , module MyApp.Database.Query- , module MyApp.Database.Migration- ) where--import MyApp.Database.Connection-import MyApp.Database.Query-import MyApp.Database.Migration---- Function purpose is clear from name-validateUserInput :: UserInput -> Either ValidationError ValidatedInput-```--### Type-Level Documentation--Embed information in types to make function behavior self-documenting.--```haskell--- Types convey information about the function's behavior-authRequired :: HasAuth r => RIO r Resource-adminOnly :: HasAdminAccess r => RIO r Resource---- Status-tracking in result type-data VerificationStatus = Pending | Verified | Rejected-data Email (s :: VerificationStatus) = Email Text--sendEmail :: Email 'Verified -> Message -> IO ()---- Directional data flow-data Input-data Processed-data Output--data Pipeline s a where- Input :: a -> Pipeline 'Input a- Process :: Pipeline 'Input a -> Pipeline 'Processed a- Output :: Pipeline 'Processed a -> Pipeline 'Output a---- Function chains are guaranteed correct order by types-pipeline :: Data -> Pipeline 'Output Result-pipeline = Output . Process . Input-```
− INSTALLING.md
@@ -1,128 +0,0 @@-# Installing Clod--## Using Homebrew (macOS)--```bash-# Install from the tap-brew tap fuzz/tap-brew install clod--# Or in one command:-brew install fuzz/tap/clod-```--## Using Cabal (All Platforms)--```bash-cabal install clod-```--## Man Pages--Clod includes comprehensive documentation as man pages:--- `clod(1)`: Command usage and options-- `clod(7)`: Project instructions and safeguards-- `clod(8)`: Complete workflow guide for using clod with Claude AI--### Installation--When installing clod through a package manager or Hackage, the man pages are automatically installed to the system's standard man page location:--- **Homebrew**: Man pages are installed to `/opt/homebrew/share/man/` (or equivalent)-- **Hackage/Cabal**: Man pages are included in the standard package installation--No additional configuration is required to use these man pages - they're automatically accessible through the `man` command.--### Generating Man Pages Manually--If you want to generate the man pages manually:--```bash-# Generate man pages in the current directory-bin/generate-man-pages.sh--# Or generate to a specific directory-bin/generate-man-pages.sh /path/to/output/dir-```--This requires pandoc to be installed on your system. The script will create three directories for man page sections (man1, man7, man8) and place the generated man pages in them.--### Installing Man Pages Manually--If you need to install the man pages manually after generating them:--```bash-# System-wide installation (requires admin privileges)-sudo cp man1/clod.1 /usr/local/share/man/man1/-sudo cp man7/clod.7 /usr/local/share/man/man7/-sudo cp man8/clod.8 /usr/local/share/man/man8/--# Or in your home directory-mkdir -p ~/.local/share/man/man1 ~/.local/share/man/man7 ~/.local/share/man/man8-cp man1/clod.1 ~/.local/share/man/man1/-cp man7/clod.7 ~/.local/share/man/man7/-cp man8/clod.8 ~/.local/share/man/man8/-```--## Opening the Staging Directory (macOS)--On macOS, you can directly open the staging directory using the `open` command with the output from clod:--```bash-# Run clod and open the staging directory in Finder-open `clod`--# For scripts, you can capture the output and open it-STAGING_DIR=$(clod)-open "$STAGING_DIR"-```--For other platforms, you can use similar commands appropriate for your operating system, or create a simple wrapper script if needed.--## Viewing Man Pages--After installation, you can view the man pages with:--```bash-man 1 clod # Command reference-man 7 clod # Project instructions-man 8 clod # Workflow guide-```--Or simply:--```bash-man clod # Shows the main command reference (section 1)-```--## Prerequisites--- GHC (Glasgow Haskell Compiler) 9.0 or newer-- libmagic (required for file type detection)- - On macOS: `brew install libmagic`- - On Linux: `apt-get install libmagic-dev` or equivalent for your distribution- - On Windows: Install from source or use package manager-- pandoc (optional, for generating man pages)- - On macOS: `brew install pandoc`- - On Linux: `apt-get install pandoc` or equivalent for your distribution- - On Windows: Install via package manager or from official website--## Troubleshooting--### Missing Man Pages--If you can't access the man pages after installation:--1. Check if pandoc was available during installation (`brew info clod` to check for Homebrew)-2. Generate and install the man pages manually using the instructions above-3. Verify your `MANPATH` environment variable includes the installation directory-4. On some systems, you may need to run `mandb` to update the man page database--### libmagic Issues--If you encounter problems with libmagic:--1. Ensure libmagic is properly installed-2. Check that the dynamic library is in your library path-3. On macOS, you might need to set `DYLD_LIBRARY_PATH` to include the libmagic location
− MAN_PAGES.md
@@ -1,45 +0,0 @@-# Man Page Integration for Clod--This document explains how the man page generation and installation system works for Clod.--## Overview--Clod provides comprehensive man pages following standard Unix conventions:--- `clod(1)` - Command usage and options-- `clod(7)` - Project instructions and safeguards-- `clod(8)` - Workflow guide--The man pages are generated from Markdown sources in the `man/` directory.--## Source Files--The man page source files are Markdown files:--- `man/clod.1.md` - Command reference (section 1)-- `man/clod.7.md` - Project instructions (section 7)-- `man/clod.8.md` - Workflow guide (section 8)--## Generation Process--Man pages are generated during Homebrew installation using the `bin/generate-man-pages.sh` script:--1. The script takes the source .md files and converts them to man format using pandoc-2. Generated man pages are installed to the proper Homebrew man directories-3. This makes them available system-wide via the standard `man clod` command--## Testing and Development--When working with man pages:--1. **Editing Content**: Update the Markdown files in the `man/` directory-2. **Testing Generation**: Run `bin/generate-man-pages.sh` to generate man pages in the current directory-3. **Viewing Locally**: Use `man -l man1/clod.1` to view the generated man pages--## Homebrew Integration--The Homebrew formula in `homebrew-tap/Formula/clod.rb` handles man page installation:--1. During the install process, it runs the generate-man-pages.sh script-2. The generated man pages are copied to the proper system locations-3. Users can then access the documentation with standard man commands
− PROJECT_ARCHITECTURE.md
@@ -1,298 +0,0 @@-# Clod Project Architecture--This document outlines the architecture, components, and design decisions for the Clod project.--## Project Overview--Clod (Claude Optimization Driver) is a tool designed to prepare text files for optimal processing by Claude AI models. It helps:--- Transform text files to be more Claude-friendly-- Generate manifest files for mapping between original and transformed paths-- Create staging directories with optimized files-- Skip binary files automatically-- Respect ignore patterns (.gitignore, .clodignore)-- Detect file changes efficiently--## Key Components--### Module Structure--- `Clod.Types`: Core data types and type classes-- `Clod.Core`: Main application logic-- `Clod.FileSystem`: File system operations- - `Clod.FileSystem.Checksums`: Checksum-based file tracking- - `Clod.FileSystem.Detection`: File type detection- - `Clod.FileSystem.Operations`: File operations- - `Clod.FileSystem.Processing`: File processing- - `Clod.FileSystem.Transformations`: Text transformations-- `Clod.Output`: Output formatting and display-- `Clod.IgnorePatterns`: Pattern matching for ignored files-- `Clod.Capability`: Capability-based security-- `Clod.Config`: Configuration management--## Checksum-Based File Tracking--Clod uses a checksum-based approach for tracking file changes:--- Checksums detect file content changes regardless of modification time-- Works in any directory structure without requiring a Git repository-- Detects file renames by matching identical content with different paths-- Provides a consistent experience across different environments--```haskell--- Calculate SHA-256 checksum of a ByteString-calculateChecksum :: BS.ByteString -> Checksum-calculateChecksum content =- let hash = SHA256.hash content- hexHash = Base16.encode hash- in Checksum (show hexHash)-```--## Database Structure--The database structure enables efficient queries:--- `dbFiles` maps file paths to their entries for quick lookup by path-- `dbChecksums` maps checksums to paths for detecting renamed files-- `dbLastStagingDir` stores the previous staging directory for the `--last` flag-- Entries contain the full file metadata including content hash, modification time, and optimized name--```haskell-data ClodDatabase = ClodDatabase- { dbFiles :: !(Map.Map FilePath FileEntry)- , dbChecksums :: !(Map.Map String FilePath)- , dbLastStagingDir :: !(Maybe FilePath)- , dbLastRunTime :: !UTCTime- } deriving stock (Show, Eq)-```--## Command-Line Interface--Clod provides several command-line options:--- `--verbose`: Enable verbose output-- `--staging`: Specify a custom staging directory-- `--flush`: Flush stale entries from the database-- `--last`: Use the previous staging directory-- `--all-files`: Process all files, not just new or modified ones-- `--modified`: Process only modified files-- `--quiet`: Suppress non-essential output-- `--version`: Show version information-- `--help`: Show help text--## File Change Detection--Clod identifies the following file statuses:--1. `New`: Files not previously in the database-2. `Modified`: Files with a different checksum than the database entry-3. `Unchanged`: Files with the same checksum as the database entry-4. `Deleted`: Files that were in the database but don't exist anymore-5. `Renamed`: Files with an identical checksum but different path--```haskell-detectFileChanges :: FileReadCap -> ClodDatabase -> [FilePath] -> FilePath -> ClodM ([(FilePath, FileStatus)], [(FilePath, FilePath)])-detectFileChanges readCap db filePaths projectRoot = do- -- For each file, calculate checksum and get modification time- fileInfos <- catMaybes <$> forM filePaths (\path -> do- -- Check if file exists and is text, then calculate checksum- -- ...- )- - -- Detect file statuses- changedFiles <- findChangedFiles db fileInfos- - -- Find renamed files- let renamedFiles = findRenamedFiles db changedFiles- - return (changedFiles, renamedFiles)-```--## Binary File Detection--Clod employs the `magic` library for reliable binary file detection:--1. Uses the same detection mechanism as the Unix `file` command-2. Identifies files by examining their content signatures-3. Provides accurate MIME type classification-4. Works consistently across different platforms--```haskell--- Check if a file is a text file using libmagic-safeIsTextFile :: FileReadCap -> FilePath -> ClodM Bool-safeIsTextFile readCap path = do- -- Verify file is within allowed directories first- allowed <- liftIO $ isPathAllowed (allowedReadDirs readCap) path- if not allowed- then throwError $ CapabilityError $ - "Access denied: Cannot read file outside allowed directories: " ++ path- else do- -- Get the MIME type from the magic library- mimeType <- liftIO $ getMimeType path- -- Check if it's a text MIME type- return $ isMimeTypeText mimeType-```--## Capability-Based Security--Clod implements capability-based security for file operations:--- `FileReadCap`: Grants permission to read from specific directories-- `FileWriteCap`: Grants permission to write to specific directories-- `FileTransformCap`: Grants permission to transform file contents--Each capability checks paths against allowed directories before performing operations.--## Manifest Generation--Clod generates manifest files that map transformed file paths to their original locations:--```json-{- "transformed/file.txt": "/original/path/to/file.txt",- "transformed/another-file.txt": "/original/path/to/another-file.txt"-}-```--The manifest helps reconstruct file origins after processing.--## Staging Directory Management--Clod creates staging directories with the following structure:--```-staging-dir/-├── file1.txt-├── file2.txt-├── _path_manifest.dhall-└── ...-```--The `--last` flag functionality allows using the previous staging directory:--```haskell--- Part of the mainLogic function-when lastMode $ do- case dbLastStagingDir database of- Just prevStaging -> do- when verbose $ liftIO $ hPutStrLn stderr $ - "Using previous staging directory: " ++ prevStaging- -- Output the previous staging directory and exit- liftIO $ hPutStrLn stdout prevStaging- throwError $ ConfigError "Using last staging directory as requested"- - Nothing -> do- when verbose $ liftIO $ hPutStrLn stderr - "No previous staging directory available, proceeding with new staging"-```--## Error Handling--Clod uses a monad transformer stack for comprehensive error handling:--```haskell--- Type for the Clod monad-type ClodM a = ExceptT ClodError IO a---- Error types-data ClodError- = FileSystemError FilePath IOError- | ChecksumError String- | DatabaseError String- | ConfigError String- | CapabilityError String- | IgnorePatternError String- deriving (Show, Eq)---- Running the Clod monad-runClodM :: ClodM a -> IO (Either ClodError a)-runClodM = runExceptT-```--## Ignore Pattern Handling--Clod processes ignore patterns similar to .gitignore:--```haskell--- Parse ignore patterns from file-parseIgnorePatterns :: FilePath -> ClodM [IgnorePattern]-parseIgnorePatterns path = do- exists <- liftIO $ doesFileExist path- if not exists- then return []- else do- content <- liftIO $ readFile path- return $ map IgnorePattern $ parseLines content- where- parseLines = filter (not . ignoreLine) . lines- ignoreLine line = null line || "#" `isPrefixOf` line---- Check if a file matches ignore patterns-matchesIgnorePattern :: [IgnorePattern] -> FilePath -> Bool-matchesIgnorePattern patterns path =- any (\(IgnorePattern pattern) -> pathMatchesPattern pattern path) patterns-```--## File Processing Pipeline--The main file processing pipeline consists of several steps:--1. **File Discovery**: Find all files in the project directory-2. **Change Detection**: Identify new, modified, and unchanged files-3. **Filtering**: Apply ignore patterns to exclude files-4. **Text Detection**: Skip binary files-5. **Transformation**: Apply transformations to text files-6. **Staging**: Copy transformed files to the staging directory-7. **Manifest Generation**: Create a mapping between original and transformed paths-8. **Database Update**: Store file metadata for future runs--```haskell--- Main processing logic-processFiles :: FileReadCap -> FileWriteCap -> FileTransformCap -> [FilePath] -> ClodM ProcessingStats-processFiles readCap writeCap transformCap filePaths = do- stats <- foldM processOneFile initialStats filePaths- return stats- where- processOneFile stats path = do- result <- processSingleFile readCap writeCap transformCap path- return $ updateStats stats result-```--## Path Transformation--Clod transforms file paths to make them more Claude-friendly:--```haskell--- Transform a file path for Claude-transformFilePath :: FilePath -> OptimizedName-transformFilePath path =- let parts = splitDirectories path- transformedParts = map transformPathPart parts- result = intercalate "/" transformedParts- in OptimizedName result---- Transform individual path components-transformPathPart :: String -> String-transformPathPart part- | null part = ""- | head part == '.' = "dot--" ++ tail part -- Handle hidden files/dirs- | otherwise = part-```--## Future Development--Potential areas for future enhancement:--1. Parallelized file processing for improved performance on large codebases-2. More sophisticated file transformations based on file type-3. Integration with cloud storage services-4. Remote file processing capabilities-5. GUI interface for easier configuration--## Development Guidelines--1. Follow capability-based security principles for all file operations-2. Use pure functions whenever possible-3. Handle errors with proper error types and messages-4. Ensure comprehensive test coverage for all components-5. Maintain backward compatibility with existing configurations
README.md view
@@ -3,15 +3,18 @@ [](https://hackage.haskell.org/package/clod) [](https://opensource.org/licenses/MIT) -Clod is a utility for preparing and uploading files to Claude AI's Project Knowledge feature. It tracks file changes using checksums, respects `.gitignore` and `.clodignore` patterns, and optimizes filenames for Claude's UI. By efficiently handling file selection and staging, it can significantly reduce AI development costs by 50% or more.--See [HUMAN.md](HUMAN.md) for a complete workflow guide to using `clod` with Claude AI.--For information about the capability-based security model, see [CAPABILITY_SECURITY.md](CAPABILITY_SECURITY.md).--For details on man page integration, see [MAN_PAGES.md](MAN_PAGES.md) and [INSTALLING.md](INSTALLING.md).+Clod is a utility for preparing and uploading files to Claude AI's Project+Knowledge feature. It tracks file changes using checksums, respects+`.gitignore` and `.clodignore` patterns, optimizes filenames for Claude's UI+and provides a filename manifest so Claude can write the files back to their+original locations. By efficiently handling file selection and staging, it can+significantly reduce AI development costs by 50% or more. Unlike other tools+created to solve this problem `clod` does not require any unauthorized access+to Anthropic products, nor is it affected by changes to Claude's UI.+Contributions of Automator code to handle the drag and drops and Project+Knowledge deletes on macOS are welcome, as is similar code for other platforms. -Developed by [BionicFuzz](https://bionicfuzz.com) - World-class technical leadership and execution.+Developed by [Fuzz, Inc](https://fuzz.ink) - World-class technical leadership and execution ## Features @@ -28,6 +31,16 @@ ## Installation +## Homebrew (recommended, binary is Apple Silicon only)++```bash+# On macOS+brew tap fuzz/tap+brew install clod+# Or in one command:+brew install fuzz/tap/clod+```+ ### From Hackage ```bash@@ -42,19 +55,6 @@ cabal install ``` -For detailed installation instructions, including how to install man pages, see [INSTALLING.md](INSTALLING.md).--Man pages are automatically installed when using package managers like Homebrew:-```bash-# On macOS-brew tap fuzz/tap-brew install clod-# Or in one command:-brew install fuzz/tap/clod-```--When installing from Hackage, the man pages are included in the package.- The `clod` program is installed automatically when using `cabal install`. ### Prerequisites@@ -62,9 +62,11 @@ - GHC (Glasgow Haskell Compiler) 9.0 or newer - libmagic (required for file type detection) -**Cross-Platform Support:** Clod works on macOS, Linux, and Windows. The program outputs the path to the staging directory, making it easy to open with your system's file browser or use with any command that accepts a directory path.+**Cross-Platform Support:** Clod works on macOS, Linux, and Windows. The+program outputs the path to the staging directory, making it easy to open with+your system's file browser or use with any command that accepts a directory+path. -Supported file browsers: * macOS: `open` * Linux: `xdg-open`, `gio`, `gnome-open`, or `kde-open` * Windows: `explorer.exe`@@ -76,15 +78,12 @@ ### Basic Usage ```bash-# Process modified files since last run+# Process all files (first run) or modified files since last run clod -# Process all files (respecting .gitignore and .clodignore)+# Process all files regardless of last run (respecting .gitignore and .clodignore) clod --all -# Run in test mode with an optional test directory-clod --test- # On macOS, process files and open the staging directory in Finder open `clod` ```@@ -102,11 +101,12 @@ ### Opening the Staging Directory -Clod outputs the path to the staging directory, which you can use to open it directly in your file browser:+Clod outputs the path to the staging directory, which you can use to open it+directly in your file browser: ```bash # On macOS, process files and open the directory in Finder-open `clod [options]`+open `clod` # For scripts, you can capture the output and open it with your preferred application STAGING_DIR=$(clod [options])@@ -133,14 +133,21 @@ ### Integration with Claude AI +First time: Paste the contents of `project-instructions.md` into the Project+Instructions section+ After running Clod: -1. Navigate to Project Knowledge in your Claude Project (Pro or Team account required)+1. Navigate to Project Knowledge in your Claude Project (Pro or Team account+ required) 2. Drag files from the opened staging folder to Project Knowledge-3. Include the `_path_manifest.dhall` file which maps optimized names back to original paths-4. Paste the contents of `project-instructions.md` into the Project Instructions section-5. **Important**: You must manually delete previous versions of these files from Project Knowledge before starting a new conversation to ensure Claude uses the most recent files-6. Note that the staging directory is temporary and will be cleaned up on your next run of clod (or system reboot)+3. Include the `_path_manifest.dhall` file which maps optimized names back to+ original paths+4. **Important**: You must manually delete previous versions of these files+ from Project Knowledge before starting a new conversation to ensure Claude+ uses the most recent files+5. Note that the staging directory is temporary and will be cleaned up on your+ next run of clod (or system reboot) ## Configuration @@ -153,7 +160,9 @@ ### .clodignore -A `.clodignore` file in your repository root specifies files or patterns to exclude. If this file doesn't exist, Clod will create a default one for you with common patterns for binary files, build directories, and large files.+A `.clodignore` file in your repository root specifies files or patterns to+exclude. If this file doesn't exist, Clod will create a default one for you+with common patterns for binary files, build directories, and large files. ## Development Utilities
− RELEASING.md
@@ -1,101 +0,0 @@-# Release Process for Clod--This document describes the steps to release a new version of Clod.--## Prerequisites--Before releasing, ensure you have:-- [Cabal](https://www.haskell.org/cabal/) installed-- A [Hackage](https://hackage.haskell.org/) account-- The `cabal-install` tool-- [Stack](https://docs.haskellstack.org/en/stable/README/) (optional, for Stackage releases)--## Release Checklist--1. **Update Version Numbers**- - Update version in `clod.cabal`- - Version is automatically accessed via the Paths_clod module (no need to update in Core.hs)- - Ensure version follows [Semantic Versioning](https://semver.org/)--2. **Update Changelog**- - Add a new section to `CHANGELOG.md` with the new version- - Document all notable changes since the last release- - Categorize changes as Added, Changed, Deprecated, Removed, Fixed, or Security--3. **Run Tests**- ```- cabal test- ```--4. **Build Documentation**- ```- cabal haddock --haddock-for-hackage- ```--5. **Create Distribution Package**- ```- cabal sdist- ```--6. **Check Package**- ```- cabal check- ```--7. **Test Package Installation**- ```- cabal v2-build --disable-documentation- ```--8. **Create Release Commit**- ```- git add .- git commit -m "Release version X.Y.Z"- ```--9. **Create Git Tag**- ```- git tag -a vX.Y.Z -m "Release version X.Y.Z"- git push origin vX.Y.Z- ```--10. **Upload to Hackage**- ```- # Option 1: Use the release script- ./bin/release-to-hackage.sh- - # Option 2: Upload manually- cabal upload --publish dist-newstyle/sdist/clod-X.Y.Z.tar.gz- cabal upload --documentation --publish dist-newstyle/clod-X.Y.Z-docs.tar.gz- ```--11. **Prepare for Next Development Cycle**- - Update version in `clod.cabal` to next development version (X.Y.Z-dev)- - Commit changes- ```- git add clod.cabal- git commit -m "Prepare for next development cycle"- ```--## Stackage Integration--To get the package included in Stackage:--1. Fork the [commercialhaskell/stackage](https://github.com/commercialhaskell/stackage) repository-2. Add the package to `build-constraints.yaml`-3. Create a pull request--## GitHub Release--After releasing to Hackage:--1. Go to [GitHub Releases](https://github.com/fuzz/clod/releases)-2. Create a new release based on the tag-3. Add release notes from the CHANGELOG-4. Attach the binary distributions if available--## Troubleshooting--- If the Hackage upload fails, check the validation errors and fix issues-- If documentation doesn't render correctly, ensure all modules are properly documented-- If the check fails, make sure the package meets all Hackage requirements
− SERIALIZATION.md
@@ -1,343 +0,0 @@-# Dhall Serialization Patterns--This document contains patterns and best practices for using Dhall in the project for configuration and serialization.--## Dhall Overview--Dhall is a programmable configuration language that provides:--- A strongly-typed, total, purely functional language-- Type safety with an expressive type system-- Incremental evaluation of configuration-- Ability to import and reuse configuration components--## Basic Serialization--### Type Definitions and Instances--```haskell--- Basic configuration type with Dhall instances-data Config = Config- { configPath :: !FilePath- , configValue :: !Int- , configEnabled :: !Bool- } deriving stock (Show, Eq, Generic)- deriving anyclass (FromDhall, ToDhall)-```--### Loading Configuration--```haskell-import qualified Dhall-import qualified Data.Text as T---- Load configuration from Dhall file-loadConfig :: FilePath -> IO Config-loadConfig path = do- -- Convert FilePath (String) to Text for Dhall input- Dhall.inputFile Dhall.auto (T.pack path)-```--### Error Handling--```haskell-import Control.Exception (SomeException, catch)---- Gracefully handle parsing errors with defaults-loadConfigSafe :: FilePath -> IO Config-loadConfigSafe path = do- (Dhall.inputFile Dhall.auto (T.pack path) :: IO Config) - `catch` \(_ :: SomeException) -> do- putStrLn "Warning: Could not load config, using defaults"- return defaultConfig---- Default configuration-defaultConfig :: Config-defaultConfig = Config- { configPath = "default/path"- , configValue = 42- , configEnabled = False- }-```--## Complex Serialization--### Wrapper Types for Collections--```haskell--- Main data structure with complex types-data ClodDatabase = ClodDatabase- { dbFiles :: !(Map.Map FilePath FileEntry)- , dbChecksums :: !(Map.Map String FilePath)- , dbLastStagingDir :: !(Maybe FilePath)- , dbLastRunTime :: !UTCTime- } deriving stock (Show, Eq)---- Serialization-friendly version-data SerializableClodDatabase = SerializableClodDatabase- { serializedFiles :: ![(FilePath, FileEntry)]- , serializedChecksums :: ![(String, FilePath)]- , serializedLastStagingDir :: !(Maybe FilePath)- , serializedLastRunTime :: !UTCTime- } deriving stock (Show, Eq, Generic)- deriving anyclass (FromDhall, ToDhall)- --- Convert to serializable form-toSerializable :: ClodDatabase -> SerializableClodDatabase-toSerializable db = SerializableClodDatabase- { serializedFiles = Map.toList (dbFiles db)- , serializedChecksums = Map.toList (dbChecksums db)- , serializedLastStagingDir = dbLastStagingDir db- , serializedLastRunTime = dbLastRunTime db- }---- Convert from serializable form-fromSerializable :: SerializableClodDatabase -> ClodDatabase-fromSerializable sdb = ClodDatabase- { dbFiles = Map.fromList (serializedFiles sdb)- , dbChecksums = Map.fromList (serializedChecksums sdb)- , dbLastStagingDir = serializedLastStagingDir sdb- , dbLastRunTime = serializedLastRunTime sdb- }-```--### Composite Type Handling--```haskell--- Format UTCTime for Dhall (as a record with date, time, timeZone fields)-formatUTCTimeDhall :: UTCTime -> String-formatUTCTimeDhall time =- let timeStr = show time- (dateStr, timeWithZone) = span (/= ' ') timeStr- timeStr' = drop 1 $ takeWhile (/= 'U') (drop 1 timeWithZone)- in "{ date = \"" ++ dateStr ++ - "\", time = \"" ++ timeStr' ++ - "\", timeZone = \"UTC\" }"-```--### Saving to Dhall Format--```haskell--- Save database to Dhall format-saveDatabase :: FilePath -> ClodDatabase -> IO ()-saveDatabase path db = do- let serializable = toSerializable db- - -- For complex types, manual construction can be more reliable- let dhallText = T.pack $ - "{ serializedFiles = " ++- formatEntries (serializedFiles serializable) ++- ", serializedChecksums = " ++ - formatChecksums (serializedChecksums serializable) ++- ", serializedLastStagingDir = " ++- formatMaybe (serializedLastStagingDir serializable) ++- ", serializedLastRunTime = " ++- formatUTCTimeDhall (serializedLastRunTime serializable) ++- "}\n"- - -- Write to temp file first for atomic updates- TextIO.writeFile (path ++ ".tmp") dhallText- renameFile (path ++ ".tmp") path- - -- Helper functions for formatting- where- formatEntries [] = "[] : List { _1 : Text, _2 : FileEntry }"- formatEntries entries = - "[\n " ++ intercalate ",\n " [formatEntry e | e <- entries] ++ "\n]"- - formatEntry (path, entry) = - "{ _1 = \"" ++ escape path ++ "\", _2 = " ++ formatFileEntry entry ++ " }"- - formatChecksums [] = "[] : List { _1 : Text, _2 : Text }"- formatChecksums checksums =- "[\n " ++ intercalate ",\n " [formatChecksum c | c <- checksums] ++ "\n]"- - formatChecksum (hash, path) =- "{ _1 = \"" ++ escape hash ++ "\", _2 = \"" ++ escape path ++ "\" }"- - formatMaybe Nothing = "None Text"- formatMaybe (Just s) = "Some \"" ++ escape s ++ "\""- - escape = concatMap escapeChar- escapeChar '"' = "\\\""- escapeChar '\\' = "\\\\"- escapeChar c = [c]-```--### Type Annotations for Empty Lists--```haskell--- Empty lists need explicit type annotations in Dhall-formatEmptyList :: String -> String-formatEmptyList typeName = "[] : List " ++ typeName---- Examples-emptyFiles = "[] : List { _1 : Text, _2 : FileEntry }"-emptyChecksums = "[] : List { _1 : Text, _2 : Text }"-```--## Dhall Configuration Examples--### Simple Configuration--```dhall--- config.dhall-{ configPath = "./data"-, configValue = 42-, configEnabled = True-}-```--### List Configuration--```dhall--- file_types.dhall-{ - textExtensions = - [ -- Documentation- ".txt", ".text", ".md", ".markdown", ".csv", ".tsv"- -- Markup - , ".html", ".htm", ".xhtml", ".xml", ".svg", ".rss"- ]-, binaryExtensions =- [ -- Images- ".jpg", ".jpeg", ".png", ".gif", ".bmp", ".ico"- -- Archives- , ".zip", ".tar", ".gz", ".7z", ".rar"- ]-}-```--### Custom Types and Records--```dhall--- binary_signatures.dhall--- Define a type for binary signatures-let Signature = { name : Text, bytes : List Natural }--let signatures : List Signature =- [ { name = "JPEG", bytes = [0xFF, 0xD8, 0xFF] }- , { name = "PNG", bytes = [0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A] }- , { name = "GIF", bytes = [0x47, 0x49, 0x46, 0x38] }- ]--in { signatures = signatures }-```--## Best Practices--### Defensive Loading--```haskell--- Try to load config, fall back to defaults if anything fails-loadDatabaseSafe :: FilePath -> IO ClodDatabase-loadDatabaseSafe dbPath = do- fileExists <- doesFileExist dbPath- if not fileExists- then do- -- Create a new database- db <- initializeDatabase- saveDatabase dbPath db- return db- else do- -- Try to load, with error handling- (do- sdb <- Dhall.inputFile Dhall.auto (T.pack dbPath) :: IO SerializableClodDatabase- return $ fromSerializable sdb)- `catch` \(e :: SomeException) -> do- putStrLn $ "Warning: Could not parse database: " ++ show e- -- Create a new database- db <- initializeDatabase- saveDatabase dbPath db- return db-```--### Cache Configuration--```haskell--- This is safer than it looks because configuration loading is idempotent--- and the result is referentially transparent-getConfig :: Config-getConfig = unsafePerformIO $ do- loadConfigSafe defaultConfigPath-{-# NOINLINE getConfig #-}-```--### Dhall in Cabal--```-data-files:- resources/config.dhall,- resources/file_types.dhall,- resources/binary_signatures.dhall-```--### Loading Data Files--```haskell-import Paths_clod (getDataFileName)--loadFileTypes :: IO FileTypes-loadFileTypes = do- -- Use getDataFileName to find resource in installed package- path <- getDataFileName "resources/file_types.dhall"- (Dhall.input Dhall.auto (T.pack path) :: IO FileTypes) `catch` \(_ :: SomeException) -> - pure defaultFileTypes-```--## Common Pitfalls--### Record Field Ordering--Dhall records don't require fields in any specific order, but the field names must match exactly:--```haskell--- Dhall type-data Config = Config- { fieldA :: String- , fieldB :: Int- }---- This Dhall is valid even though fields are in different order--- { fieldB = 42, fieldA = "value" }-```--### Type Annotations for Empty Collections--Always provide type annotations for empty collections:--```dhall--- This will fail-{ emptyList = [] }---- This will work-{ emptyList = [] : List Text }-```--### Handling Complex Types--For types like UTCTime that Dhall doesn't natively support, use record representations:--```dhall--- UTCTime representation-{ date = "2023-10-15"-, time = "14:30:45.789012"-, timeZone = "UTC"-}-```--### Escaping String Values--Remember to escape special characters in strings:--```haskell-escapeString :: String -> String-escapeString = concatMap escapeChar- where- escapeChar '"' = "\\\""- escapeChar '\\' = "\\\\"- escapeChar '\n' = "\\n"- escapeChar '\r' = "\\r"- escapeChar '\t' = "\\t"- escapeChar c = [c]-```
− TEST_SAFETY.md
@@ -1,99 +0,0 @@-# TEST SAFETY PROPOSAL--**IMPORTANT: These are proposed ideas and workflows only. They should be ignored when writing code, tests, or documentation for the current implementation.**--## The Problem--When developing software collaboratively between humans and AI assistants, critical functionality can be unintentionally lost during refactoring or implementation changes. This can happen even with test coverage in place, as demonstrated by the incident where the automatic creation of `.clodignore` files was removed without detection.--## Proposed Workflow--### 1. Separate Repositories for Code and Tests--Maintain implementation code and tests in separate repositories with different access controls:--- **Implementation Repository**: AI has read-write access to implementation code-- **Test Repository**: AI can read and run tests but not modify them without explicit human review--### 2. Test Modification Process--When tests need updating:--1. AI generates a detailed modification plan explaining why each test change is needed-2. Human reviews the plan, checking for:- - Removal of tests for still-needed behaviors- - Subtle changes to expected behaviors- - Compatibility with the project specification-3. After approval, a separate AI instance with limited context receives specific instructions to update only the approved tests-4. Tests are run to verify the updates maintain coverage--### 3. Enhanced Test Documentation--#### Test Categorization--Tag tests to indicate their importance for core behaviors:--```haskell--- @behavior-critical: Ensures .clodignore files are auto-created when missing-it "creates a default .clodignore file when one doesn't exist" $ do- ...-```--#### Test Provenance Comments--Add brief headers explaining the purpose of critical tests:--```haskell--- This test verifies the fundamental behavior of creating default configuration--- files when they don't exist, which is essential for first-time users.--- DO NOT REMOVE without explicit approval.-it "creates a default .clodignore file when one doesn't exist" $ do- ...-```--### 4. Implementation-Test Mapping--Create a document that explicitly links key implementation parts to their corresponding tests:--```-| Functionality | Implementation File | Test File |-|-----------------------------|-----------------------------|------------------------------|-| Default file creation | src/Clod/IgnorePatterns.hs | test/Clod/IgnorePatternsSpec.hs |-| ... | ... | ... |-```--### 5. Change Review Automation--When tests are modified, generate a structured report highlighting:--- Tests that were removed-- Tests that had their assertions changed-- New tests added-- Implementation functions no longer covered by tests--## Alternative: Capability-Based Test Protection--As an alternative to separate repositories, implement capability-based access control within the same repository:--1. AI has read-write access to implementation code-2. AI has read-only access to test code by default-3. AI can request explicit permission to modify specific tests, which requires human approval-4. AI can run tests but not modify the test results--## Benefits--This approach would:--1. Preserve critical tests that document expected behaviors-2. Make test changes explicit and subject to human review-3. Maintain institutional knowledge about why certain tests exist-4. Reduce the risk of functionality regression during refactoring--## Next Steps--To implement this workflow:--1. Define clear capability boundaries for test access-2. Create templates for test modification requests-3. Document critical behaviors in a central specification-4. Develop a testing policy that requires human review for test removals
bin/release view
@@ -414,24 +414,13 @@ exit 1 fi -# Create the new lines with proper markers intact-URL_LINE=$(grep "TARBALL_URL_MARKER" "$FORMULA_PATH")-SHA_LINE=$(grep "TARBALL_SHA256_MARKER" "$FORMULA_PATH")--# Extract the pattern from the current lines, preserving spacing-URL_PREFIX=$(echo "$URL_LINE" | sed -E 's/url ".*"/url/') -URL_SUFFIX=$(echo "$URL_LINE" | sed -E 's/.*(".*")/\1/')--SHA_PREFIX=$(echo "$SHA_LINE" | sed -E 's/sha256 ".*"/sha256/') -SHA_SUFFIX=$(echo "$SHA_LINE" | sed -E 's/.*(".*")/\1/')--# Create new lines with the proper URL and SHA-NEW_URL="${URL_PREFIX} \"https://hackage.haskell.org/package/clod-$VERSION/clod-$VERSION.tar.gz\"${URL_SUFFIX}"-NEW_SHA="${SHA_PREFIX} \"$SHA256\"${SHA_SUFFIX}"+# Use simple templates for the complete lines+URL_TEMPLATE=" url \"https://hackage.haskell.org/package/clod-$VERSION/clod-$VERSION.tar.gz\" # TARBALL_URL_MARKER"+SHA_TEMPLATE=" sha256 \"$SHA256\" # TARBALL_SHA256_MARKER" -# Now update the formula-sed -i '' "s|.*TARBALL_URL_MARKER.*|$NEW_URL|" "$FORMULA_PATH"-sed -i '' "s|.*TARBALL_SHA256_MARKER.*|$NEW_SHA|" "$FORMULA_PATH"+# Replace the entire lines with the markers+sed -i '' "s|.*TARBALL_URL_MARKER.*|$URL_TEMPLATE|" "$FORMULA_PATH"+sed -i '' "s|.*TARBALL_SHA256_MARKER.*|$SHA_TEMPLATE|" "$FORMULA_PATH" # Verify the changes were made correctly if ! grep -q "clod-$VERSION.*\.tar\.gz.*TARBALL_URL_MARKER" "$FORMULA_PATH"; then@@ -494,35 +483,13 @@ # Update the formula with bottle information echo "Updating formula with bottle information..." - # Verify that all required marker comments exist- if ! grep -q "BOTTLE_ROOT_URL_MARKER" Formula/clod.rb; then- echo "ERROR: BOTTLE_ROOT_URL_MARKER not found in formula. Cannot update safely."- exit 1- fi- - if ! grep -q "BOTTLE_SHA256_MARKER" Formula/clod.rb; then- echo "ERROR: BOTTLE_SHA256_MARKER not found in formula. Cannot update safely."- exit 1- fi- - # Create the new lines with proper markers intact- ROOT_URL_LINE=$(grep "BOTTLE_ROOT_URL_MARKER" Formula/clod.rb)- BOTTLE_SHA_LINE=$(grep "BOTTLE_SHA256_MARKER" Formula/clod.rb)- - # Extract the pattern from the current lines, preserving spacing- ROOT_URL_PREFIX=$(echo "$ROOT_URL_LINE" | sed -E 's/root_url ".*"/root_url/') - ROOT_URL_SUFFIX=$(echo "$ROOT_URL_LINE" | sed -E 's/.*(".*")/\1/')- - BOTTLE_SHA_PREFIX=$(echo "$BOTTLE_SHA_LINE" | sed -E 's/sha256.*arm64_.* ".*"/sha256 cellar: :any, arm64_'"$macos_version"'/') - BOTTLE_SHA_SUFFIX=$(echo "$BOTTLE_SHA_LINE" | sed -E 's/.*(".*")/\1/')- - # Create new lines with the proper values- NEW_ROOT_URL="${ROOT_URL_PREFIX} \"https://github.com/fuzz/clod/releases/download/v$VERSION\"${ROOT_URL_SUFFIX}"- NEW_BOTTLE_SHA="${BOTTLE_SHA_PREFIX} \"$bottle_sha\"${BOTTLE_SHA_SUFFIX}"+ # Use simple templates for the bottle lines+ ROOT_URL_TEMPLATE=" root_url \"https://github.com/fuzz/clod/releases/download/v$VERSION\" # BOTTLE_ROOT_URL_MARKER"+ BOTTLE_SHA_TEMPLATE=" sha256 cellar: :any, arm64_$macos_version: \"$bottle_sha\" # BOTTLE_SHA256_MARKER" - # Now update the formula- sed -i '' "s|.*BOTTLE_ROOT_URL_MARKER.*|$NEW_ROOT_URL|" Formula/clod.rb- sed -i '' "s|.*BOTTLE_SHA256_MARKER.*|$NEW_BOTTLE_SHA|" Formula/clod.rb+ # Replace the entire lines with the markers+ sed -i '' "s|.*BOTTLE_ROOT_URL_MARKER.*|$ROOT_URL_TEMPLATE|" Formula/clod.rb+ sed -i '' "s|.*BOTTLE_SHA256_MARKER.*|$BOTTLE_SHA_TEMPLATE|" Formula/clod.rb # Verify the changes were made correctly if ! grep -q "v$VERSION.*BOTTLE_ROOT_URL_MARKER" Formula/clod.rb; then
clod.cabal view
@@ -1,6 +1,6 @@ cabal-version: 2.4 name: clod-version: 0.1.16+version: 0.1.18 synopsis: Project file manager for Claude AI integrations description: Clod (Claude Loader) is a utility for preparing and uploading files to Claude AI's Project Knowledge feature. It tracks file changes, respects .gitignore and .clodignore @@ -44,18 +44,9 @@ extra-source-files: README.md- CONTRIBUTING.md - RELEASING.md HUMAN.md- HASKELL_PATTERNS.md- INSTALLING.md- CAPABILITY_SECURITY.md CRITICAL.md- MAN_PAGES.md- PROJECT_ARCHITECTURE.md- SERIALIZATION.md SPEC.md- TEST_SAFETY.md guardrails.md project-instructions.md LICENSE
man/clod.1.md view
@@ -1,4 +1,4 @@-% CLOD(1) Clod 0.1.16+% CLOD(1) Clod 0.1.18 % Fuzz Leonard & Claude <ink@fuzz.ink> % March 2025
man/clod.7.md view
@@ -1,4 +1,4 @@-% CLOD(7) Clod 0.1.16+% CLOD(7) Clod 0.1.18 % Fuzz Leonard & Claude <ink@fuzz.ink> % March 2025
man/clod.8.md view
@@ -1,4 +1,4 @@-% CLOD(8) Clod 0.1.16+% CLOD(8) Clod 0.1.18 % Fuzz Leonard & Claude <ink@fuzz.ink> % March 2025