diff --git a/README.md b/README.md new file mode 100644 --- /dev/null +++ b/README.md @@ -0,0 +1,59 @@ +# Conferer + +## what, why and a bit of how + +VERY UNSTABLE DON'T USE IN PRODUCTION JUST YET + +Conferer is a library that defines ways to get configuration for your Haskell +application and the libraries it uses, which is at heart string keys which map +to other strings. + +To get this map we use `Providers` which define a way to get a key (eg. +`db.username`) that may or may not exist, we then use a list of providers for +getting the value for that key (position on the list defines priority). This +allows adding new providers easily (for example a dhall file provider, +a git repo or a etcd database) + +The other side of this is that we have the `FromConfig` which gets some value +from a `Config` at a certain key possibly using only keys under some namespacing +key + +## Example (not implemented) + +Let's say I want to configure warp. Let's say we wrote this program. + +```haskell +main = do + -- by default gets cli parameters, envvars and json file + config <- getDefaultConfigFor "awesomeapp" + warpConfig :: Warp.Settings <- getKey "warp" config + + Warp.runSettings warpConfig myApp +``` + +Now I need to chage the port of the app, I can change it by either: + +* Setting cli params like `./myApp -Cwarp.port=5555` +* Setting an environment variable called `AWESOMEAPP_WARP_PORT=5555` +* In a json file you can have `{"warp": {"port": 5555}}` + +And you may also get that value from different configuration providers like +redis or etc or whichever you may need. + +## Utilities + +There are as well some utilities to change providers: + +* `Conferer.Provider.Namespace`: All keys must be namespaced and the namespace + is striped for lookup +* `Conferer.Provider.Mapped`: Using a map key to maybe key you can change the + name of a key or even hiding some key +* `Conferer.Provider.Simple`: Get keys from a hardcoded map key to string + +## Future maybe things + +* Interpolate keys with other keys: `{a: "db", b: "${a}_thing"}`, getting `b` + will give `"db_thing"` (maybe) +* A LOT of providers +* A LOT of `FromConfig` implementations +* Docs diff --git a/conferer.cabal b/conferer.cabal --- a/conferer.cabal +++ b/conferer.cabal @@ -1,14 +1,14 @@ -cabal-version: 1.12 +cabal-version: 1.18 -- This file has been generated from package.yaml by hpack version 0.31.2. -- -- see: https://github.com/sol/hpack -- --- hash: eea931123405e74a86b3e1283519e3369618e126404652c3818e55ad1071dad5 +-- hash: a139753b9c1bc2266c7b90feb57d35995b1b57ffa23dcf6f7d8a2b8dad4a7238 name: conferer -version: 0.1.0.0 -synopsis: Configuration management library +version: 0.1.0.1 +synopsis: Configuration management library description: Library to abstract the parsing of many haskell config values from different config sources category: Configuration @@ -19,6 +19,9 @@ license: BSD3 license-file: LICENSE build-type: Simple +extra-doc-files: + README.md + LICENSE library exposed-modules: diff --git a/src/Conferer.hs b/src/Conferer.hs --- a/src/Conferer.hs +++ b/src/Conferer.hs @@ -1,21 +1,94 @@ +-- | +-- Module: Conferer +-- Copyright: (c) 2019 Lucas David Traverso +-- License: MIT +-- Maintainer: Lucas David Traverso +-- Stability: experimental +-- Portability: portable +-- +-- Types and functions for managing configuration effectively module Conferer - ( module Conferer.Core + ( + + -- * How to use this library + -- | This is the most basic example: which uses the default configuration + -- to get a configuration for warp, which can be overriden via env vars, + -- command line arguments of @.properties@ files + -- + -- @ + -- > import Conferer + -- > import Conferer.FetchFromConfig.Warp () -- from package conferer-warp + -- > + -- > main = do + -- > config <- 'defaultConfig' \"awesomeapp\" + -- > warpSettings <- 'getFromConfig' \"warp\" config + -- > runSettings warpSettings application + -- @ + -- + -- In the above example we see that we are getting a configuration value for + -- warp under the key warp, so for example to override it's default value + -- provided by warp the config keys for warp will always look like + -- @warp.something@, for example to override the port for warp (3000 by + -- default) we could call our program as @./my_program --warp.port=8000@. + -- + -- There are two sides to conferer: Getting configuration for other libraries + -- like warp, hspec, snap, etc. and the way we choose to provide values like + -- json files, properties files, env vars, etc. + + -- ** Getting configuration for existing libraries + -- | There is a typeclass 'FetchFromConfig' that defines how to get a type + -- from a config, they are implemented in different packages since the + -- weight of the dependencies would be too high, the package is usually + -- named as @conferer-DEPENDENCY@ where DEPENDENCY is the name of the dependency ( + -- for example: conferer-snap, conferer-warp, conferer-hspec), if you find + -- a library without a conferer port for its config you can create an issue + -- or maybe even create the library yourself! + + -- ** Providing key value pairs for configuration + -- | There is one important type in conferer: 'Config' from which, given a key + -- (eg: @warp@) you can get anything that implements 'FetchFromConfig' (like + -- 'Warp.Settings') + -- + -- Internally a 'Config' is made of many 'Provider's which have a simpler + -- interface: + -- + -- @ + -- 'getKeyInProvider' :: Provider -> Key -> IO (Maybe Text) + -- @ + -- + -- Most configuration providers can be abstracted away as Map String String, + -- and they can use whatever logic they want to turn conferer keys (a list of + -- strings) into a place to look for a string, (for example the env provider + -- requires a string to namespace the env vars that can affect the + -- configuration) + -- + -- Once you have your 'Provider' you can add it to a 'Config' using the + -- 'addProvider' function. One final note: each provider has a different + -- priority, which depends on when is was added to the config ('Provider's + -- added later have lower priority) so the config searches keys in providers + -- in the same order they were added. + + module Conferer.Types + , module Conferer.Core + , defaultConfig + , Key(..) + + -- * Providers , module Conferer.Provider.Env , module Conferer.Provider.Simple , module Conferer.Provider.Namespaced , module Conferer.Provider.Mapping , module Conferer.Provider.CLIArgs , module Conferer.Provider.Null - , defaultConfig - , Key(..) + -- * Re-Exports , (&) ) where import Data.Text (Text) import Data.Function ((&)) -import Conferer.Core -import Conferer.Types +import Conferer.Core (emptyConfig, addProvider, getFromConfig, getKey, unsafeGetKey, (/.)) +import Conferer.Types (Config, Key(..), ProviderCreator, Provider(..), FetchFromConfig) import Conferer.Provider.Env import Conferer.Provider.Simple import Conferer.Provider.Namespaced @@ -25,6 +98,8 @@ +-- | Default config which reads from command line arguments, env vars and +-- property files defaultConfig :: Text -> IO Config defaultConfig appName = do pure emptyConfig diff --git a/src/Conferer/Core.hs b/src/Conferer/Core.hs --- a/src/Conferer/Core.hs +++ b/src/Conferer/Core.hs @@ -6,14 +6,9 @@ import Conferer.Types -unsafeGetKey :: Key -> Config -> IO Text -unsafeGetKey k config = - either (error . Text.unpack) id <$> getKey k config - -getFromConfig :: FetchFromConfig a => Key -> Config -> IO a -getFromConfig k config = - either (error . Text.unpack) id <$> fetch k config - +-- | Most Basic function to interact directly with a 'Config'. It always returns +-- 'Text' in the case of success and implements the logic to traverse +-- providers inside the 'Config'. getKey :: Key -> Config -> IO (Either Text Text) getKey k config = go $ providers config @@ -25,17 +20,32 @@ Just t -> return $ Right t Nothing -> go providers + +-- | Fetch a value from a config key that's parsed using the FetchFromConfig +-- instance. +-- +-- This function throws an exception if the key is not found. +getFromConfig :: FetchFromConfig a => Key -> Config -> IO a +getFromConfig k config = + either (error . Text.unpack) id <$> fetch k config + +-- | Create a new 'Key' by concatenating two existing keys. (/.) :: Key -> Key -> Key parent /. child = Path (unKey parent ++ unKey child) +-- | The empty configuration, this 'Config' is used as the base for +-- most config creating functions. emptyConfig :: Config emptyConfig = Config [] +-- | Instantiate a 'ProviderCreator' using the 'emptyConfig' mkStandaloneProvider :: ProviderCreator -> IO Provider mkStandaloneProvider mkProvider = mkProvider emptyConfig +-- | Instantiate a 'Provider' using an 'ProviderCretor' and a 'Config' and add +-- to the config addProvider :: ProviderCreator -> Config -> IO Config addProvider mkProvider config = do newProvider <- mkProvider config @@ -43,3 +53,8 @@ Config { providers = providers config ++ [ newProvider ] } + +-- | Same as 'getKey' but it throws if the 'Key' isn't found +unsafeGetKey :: Key -> Config -> IO Text +unsafeGetKey k config = + either (error . Text.unpack) id <$> getKey k config diff --git a/src/Conferer/Provider/CLIArgs.hs b/src/Conferer/Provider/CLIArgs.hs --- a/src/Conferer/Provider/CLIArgs.hs +++ b/src/Conferer/Provider/CLIArgs.hs @@ -1,4 +1,14 @@ -module Conferer.Provider.CLIArgs where +module Conferer.Provider.CLIArgs + ( + -- * Command line arguments Provider + -- | This provider provides keys from the command line arguments passed into + -- the program. It only accepts arguments with @--@ and an equals, for + -- example: @./awesomeapp --warp.port=5000@ + mkCLIArgsProvider + , mkCLIArgsProvider' + , parseArgsIntoKeyValue + ) +where import Data.Text (Text) import qualified Data.Text as Text @@ -10,16 +20,20 @@ import Conferer.Provider.Simple +-- | Create a 'ProviderCreator' for CLIArgs from a argument list mkCLIArgsProvider' :: [String] -> ProviderCreator mkCLIArgsProvider' args = \config -> do let configMap = parseArgsIntoKeyValue args mkMapProvider configMap config +-- | Same as 'mkCLIArgsProvider'' but using 'getArgs' to provide the argument +-- list mkCLIArgsProvider :: ProviderCreator mkCLIArgsProvider = \config -> do args <- getArgs mkCLIArgsProvider' args config +-- | Parse an argument list into a dictionary suitable for a 'Provider' parseArgsIntoKeyValue :: [String] -> [(Key, Text)] parseArgsIntoKeyValue = fmap (\(k, s) -> (fromString $ Text.unpack k, s)) . diff --git a/src/Conferer/Provider/Env.hs b/src/Conferer/Provider/Env.hs --- a/src/Conferer/Provider/Env.hs +++ b/src/Conferer/Provider/Env.hs @@ -1,27 +1,42 @@ -module Conferer.Provider.Env where +module Conferer.Provider.Env + ( +-- * Env Provider +-- | This 'Provider' provides config values from env vars given a prefix that's +-- used to avoid colliding with different system configuration +-- +-- For example if you use the 'Prefix' "awesomeapp" and get the 'Key' +-- "warp.port" this provider will try to lookup the env var called +-- @AWESOMEAPP_WARP_PORT@. +-- * Usage +-- | To use this provider simply choose a prefix and add it using the +-- 'addProvider' function like: +-- +-- @ +-- config & 'addProvider' ('mkEnvProvider' "awesomeapp") +-- @ + mkEnvProvider + , mkEnvProvider' + , Prefix + , LookupEnvFunc + , keyToEnvVar + ) where + + import Data.Text (Text) import qualified Data.Text as Text import qualified System.Environment as System import Conferer.Types - -type LookupEnvFunc = (String -> IO (Maybe String)) - - -keyToEnvVar :: Prefix -> Key -> Text -keyToEnvVar prefix (Path keys) = - Text.toUpper - $ Text.intercalate "_" - $ filter (/= mempty) - $ prefix : keys - - +-- | 'ProviderCreator' for env 'Provider' that uses the real 'System.lookupEnv' +-- function mkEnvProvider :: Prefix -> ProviderCreator mkEnvProvider prefix = mkEnvProvider' System.lookupEnv prefix +-- | 'ProviderCreator' for env 'Provider' that allows parameterizing the +-- function used to lookup for testing mkEnvProvider' :: LookupEnvFunc -> Prefix -> ProviderCreator mkEnvProvider' lookupEnv prefix = \_config -> return $ @@ -30,3 +45,23 @@ let envVarName = Text.unpack $ keyToEnvVar prefix k fmap Text.pack <$> lookupEnv envVarName } + +-- | Type alias for the function to lookup env vars +type LookupEnvFunc = String -> IO (Maybe String) + + +-- | A text to namespace env vars +type Prefix = Text + +-- | Get the env name from a prefix and a key by uppercasing and +-- intercalating underscores +-- +-- >>> keyToEnVar "awesomeapp" "warp.port" +-- "AWESOMEAPP_WARP_PORT" +keyToEnvVar :: Prefix -> Key -> Text +keyToEnvVar prefix (Path keys) = + Text.toUpper + $ Text.intercalate "_" + $ filter (/= mempty) + $ prefix : keys + diff --git a/src/Conferer/Provider/Mapping.hs b/src/Conferer/Provider/Mapping.hs --- a/src/Conferer/Provider/Mapping.hs +++ b/src/Conferer/Provider/Mapping.hs @@ -1,10 +1,21 @@ -module Conferer.Provider.Mapping where +module Conferer.Provider.Mapping + ( + -- * Namespaced higher-order provider + -- | This provider takes a provider and returns a new provider that + -- always transforms the key according to either a function for + -- 'mkMappingProvider'' or a 'Map' for 'mkMappingProvider' + mkMappingProvider + , mkMappingProvider' + ) +where import Data.Map (Map) import qualified Data.Map as Map import Conferer.Types +-- | Create a 'ProviderCreator' using a function to transform the supplied keys +-- and another 'ProviderCreator' mkMappingProvider' :: (Key -> Maybe Key) -> ProviderCreator -> ProviderCreator mkMappingProvider' mapper providerCreator config = do configProvider <- providerCreator config @@ -16,6 +27,8 @@ Nothing -> return Nothing } +-- | Create a 'ProviderCreator' using a 'Map' 'Key' 'Key' to transform the supplied keys +-- and another 'ProviderCreator' mkMappingProvider :: Map Key Key -> ProviderCreator -> ProviderCreator mkMappingProvider configMap configProvider = mkMappingProvider' (`Map.lookup` configMap) configProvider diff --git a/src/Conferer/Provider/Namespaced.hs b/src/Conferer/Provider/Namespaced.hs --- a/src/Conferer/Provider/Namespaced.hs +++ b/src/Conferer/Provider/Namespaced.hs @@ -1,9 +1,17 @@ -module Conferer.Provider.Namespaced where +module Conferer.Provider.Namespaced + ( + -- * Namespaced higher-order provider + -- | This provider takes a provider and returns a new provider that + -- always checks that the 'Key' given always starts with certain 'Key' + -- and then strips that prefix before consulting its inner Provider + mkNamespacedProvider + ) where import Data.List (stripPrefix) import Conferer.Types +-- | Create a 'ProviderCreator' from a prefix and another 'ProviderCreator' mkNamespacedProvider :: Key -> ProviderCreator -> ProviderCreator mkNamespacedProvider (Path key) configCreator = \config -> do configProvider <- configCreator config diff --git a/src/Conferer/Provider/Null.hs b/src/Conferer/Provider/Null.hs --- a/src/Conferer/Provider/Null.hs +++ b/src/Conferer/Provider/Null.hs @@ -1,7 +1,14 @@ -module Conferer.Provider.Null where +module Conferer.Provider.Null + ( + -- * Does Nothing Provider + -- | The stub provider that never has a key + mkNullProvider + ) +where import Conferer.Types +-- | Create a null 'Provider' mkNullProvider :: ProviderCreator mkNullProvider _config = return $ Provider diff --git a/src/Conferer/Provider/Simple.hs b/src/Conferer/Provider/Simple.hs --- a/src/Conferer/Provider/Simple.hs +++ b/src/Conferer/Provider/Simple.hs @@ -1,4 +1,12 @@ -module Conferer.Provider.Simple where +module Conferer.Provider.Simple + ( + -- * Simple Provider + -- | This provider provides values from a hardcoded Map passed at creation + -- time that can not be changed afterwards, it's mostly used as a necessary + -- utility + mkMapProvider + , mkMapProvider' + ) where import Data.Map (Map) import qualified Data.Map as Map @@ -6,6 +14,7 @@ import Conferer.Types +-- | Make a 'Provider' from a 'Map' mkMapProvider' :: Map Key Text -> ProviderCreator mkMapProvider' configMap _config = return $ Provider @@ -14,5 +23,6 @@ return $ Map.lookup k configMap } +-- | Make a 'Provider' from 'List' of 'Key', 'Text' pairs mkMapProvider :: [(Key, Text)] -> ProviderCreator mkMapProvider = mkMapProvider' . Map.fromList diff --git a/src/Conferer/Types.hs b/src/Conferer/Types.hs --- a/src/Conferer/Types.hs +++ b/src/Conferer/Types.hs @@ -5,29 +5,43 @@ import Data.Text (Text) import qualified Data.Text as Text +-- | Core interface for library provided configuration, basically consists of +-- getting a 'Key' and informing returning a maybe signaling the value and +-- if it's present in that specific provider data Provider = Provider { getKeyInProvider :: Key -> IO (Maybe Text) } +-- | The way to index 'Provider's, basically list of names that will be adapted +-- to whatever the provider needs newtype Key = Path { unKey :: [Text] } deriving (Show, Eq, Ord) +-- | Collapse a key into a textual representation keyName :: Key -> Text keyName = Text.intercalate "." . unKey +-- | Core type that the user of this library interact with, in the future it may +-- contain more this besides a list of providers data Config = Config { providers :: [Provider] } +-- | The type for creating a provider given a 'Config', some providers require a +-- certain configuration to be initialized (for example: the redis provider +-- needs connection info to connect to the server) type ProviderCreator = Config -> IO Provider +-- | Main typeclass for defining the way to get values from config, hiding the +-- 'Text' based nature of the 'Provider's +-- +-- Here an error means that the value couldn't be parsed and that a reasonable +-- default was not possible. class FetchFromConfig a where fetch :: Key -> Config -> IO (Either Text a) instance IsString Key where fromString s = Path $ filter (/= mempty) $ Text.split (== '.') $ fromString s - -type Prefix = Text