packages feed

haskoin-store 1.16.4 → 1.16.5

raw patch · 9 files changed

+472/−60 lines, 9 filesdep ~haskoin-store-dataPVP ok

version bump matches the API change (PVP)

Dependency ranges changed: haskoin-store-data

API changes (from Hackage documentation)

+ Haskoin.Store.Startup: StartupError :: Text -> StartupError
+ Haskoin.Store.Startup: checkReadable :: Text -> FilePath -> IO ()
+ Haskoin.Store.Startup: connectRedis :: String -> IO Connection
+ Haskoin.Store.Startup: explainDatabase :: FilePath -> ((r -> IO a) -> IO a) -> (r -> IO a) -> IO a
+ Haskoin.Store.Startup: explainListen :: Text -> (IO () -> IO a) -> IO a
+ Haskoin.Store.Startup: instance GHC.Classes.Eq Haskoin.Store.Startup.StartupError
+ Haskoin.Store.Startup: instance GHC.Internal.Exception.Type.Exception Haskoin.Store.Startup.StartupError
+ Haskoin.Store.Startup: instance GHC.Internal.Show.Show Haskoin.Store.Startup.StartupError
+ Haskoin.Store.Startup: makeDataDirectory :: FilePath -> IO ()
+ Haskoin.Store.Startup: newtype StartupError
+ Haskoin.Store.Startup: redisProblem :: Connection -> IO (Maybe Text)

Files

CHANGELOG.md view
@@ -4,6 +4,33 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html) +## [1.16.5] - 2026-11-19++### Added+- **`hsq`, a command-line client for people** (`tools/hsq/hsq`, Python,+  standard library only): a coloured summary of `/health`, refreshed with+  `-w`, and any API path as laid-out JSON, over a Unix socket, HTTP or+  HTTPS, with a configuration per network. `docs/hsq.md` explains it. It is+  not part of the Haskell package.++### Changed+- **Clear messages when the store cannot start.** A new module,+  `Haskoin.Store.Startup`, turns each usual failure into one line that+  says what is wrong and what to do, printed without a backtrace, with+  exit status 1: no Redis or Valkey listening, an unknown Redis host, a+  Redis that wants a password or another one (it used to start and answer+  503 to everything), a URL that is not Redis's, a data directory that+  cannot be made, a database held by another store, a port in use or below+  1024, a host that is not this machine's, a Unix socket in use, missing+  HTTPS files, and an unknown `--net`. Redis is now connected first, before+  the database opens, and with a timeout of five seconds.+- **The warning when Redis stops answering gives the reason**, and says+  that the store answers 503 until Redis is back (it said, wrongly, that+  the store answered from the database).+- **"Not synchronized with the network"**: the message of the 503 that a+  store answers without the block to answer from, with haskoin-store-data+  3.0.1, which it now requires. The code stays `behind-fleet`.+ ## [1.16.4] - 2026-11-18  ### Fixed
README.md view
@@ -938,6 +938,10 @@  ## Monitoring +`tools/hsq/hsq` is a small client for people: `hsq` prints a summary of a+store's health, and `hsq /block/best` any answer of the API, laid out and+coloured, over a Unix socket, HTTP or HTTPS. See `docs/hsq.md`.+ ### Health  A store is unhealthy when Redis does not answer within two seconds, when@@ -976,10 +980,12 @@   during a sync, and the store answers requests at full speed meanwhile.   If the delay makes it fall behind, the check of the blocks says so. -**In a fleet**, a server that does not have the block that the-fleet answers from can answer any request with 503 and-`{"error": "behind-fleet"}`. Have the load balancer try another server, and-alert if one keeps saying it: its `/health` will say why.+**A server that is not synchronized with the network**, because it does+not have the block to answer from yet or cannot reach its Redis, answers+any request with 503 and `{"error": "behind-fleet"}`, with the message+"This server is not synchronized with the network". Have the load balancer+try another server, and alert if one keeps saying it: its `/health` and+its log will say why.  ### Overload 
app/Main.hs view
@@ -5,7 +5,8 @@ module Main (main) where  import Control.Concurrent (getNumCapabilities)-import Control.Concurrent.Async (withAsync)+import Control.Concurrent.Async (ExceptionInLinkedThread (..), withAsync)+import Control.Exception (SomeException, fromException, handle, throwIO) import Control.Monad (when) import Data.Char (toLower) import Data.Default (Default (..))@@ -40,6 +41,7 @@ import Haskoin.Store.Common (loopForever) import Haskoin.Store.Database.Stalls (defaultBackgroundJobs) import Haskoin.Store.Runtime (runtimeMetrics)+import Haskoin.Store.Startup (StartupError (..), checkReadable, makeDataDirectory) import Haskoin.Store.Web.Listen (parseMode, systemdSocket, withUnixSocket) import Haskoin.Store.XPub.Indexer (Settings (..)) import NQE (receive, withSubscription)@@ -69,10 +71,11 @@     switch,     value,   )-import System.Directory (createDirectoryIfMissing, getAppUserDataDirectory)+import System.Directory (getAppUserDataDirectory) import System.Environment (lookupEnv)-import System.Exit (exitSuccess)+import System.Exit (ExitCode (..), exitSuccess, exitWith) import System.FilePath ((</>))+import System.IO (hPutStrLn, stderr) import System.Metrics.StatsD   ( StatConfig       ( namespace,@@ -790,23 +793,46 @@ networkReader s =   case netByName s of     Just net -> Right net-    Nothing -> Left "Network name invalid"+    Nothing -> Left ("--net " <> s <> ": unknown network; use one of " <> intercalate ", " (map (.name) allNets))  -- | Parse the command line and run the store. If no peers are given and discovery -- is off, discovery is turned on.+--+-- A mistake of configuration, or something that the store needs and does+-- not find (Redis, a free port, a database that no other store holds), is+-- printed as one message that says what to do, and the program exits with+-- status 1 ("Haskoin.Store.Startup"). main :: IO ()-main = do+main = handle startupError $ do   c <- execParser . opts =<< defConfig   when c.version $ do     putStrLn haskoinStoreVersion     exitSuccess   -- the cache of xpubs cannot be turned off: its size is all that is set   when (c.redisMaxXPubs < 1) $-    error "--redis-max-xpubs must be at least 1"+    misconfigured "--redis-max-xpubs (or REDIS_MAX_XPUBS) must be at least 1"+  case (c.tlsCert, c.tlsKey) of+    ("", "") -> return ()+    ("", _) -> misconfigured "--tls-key needs --tls-cert: give both, or neither for plain HTTP"+    (_, "") -> misconfigured "--tls-cert needs --tls-key: give both, or neither for plain HTTP"+    (certificate, key) -> checkReadable "--tls-cert" certificate >> checkReadable "--tls-key" key   if null c.peers && not c.discover     then run $ let Config {..} = c in Config {discover = True, ..}     else run c   where+    misconfigured = throwIO . StartupError+    -- also from a thread linked to this one, which wraps what it throws+    startupError e = case startupOf e of+      Just (StartupError msg) -> do+        hPutStrLn stderr ("haskoin-store: " <> T.unpack msg)+        exitWith (ExitFailure 1)+      Nothing -> throwIO e+    startupOf :: SomeException -> Maybe StartupError+    startupOf e = case fromException e of+      Just s -> Just s+      Nothing -> case fromException e of+        Just (ExceptionInLinkedThread _ inner) -> startupOf inner+        Nothing -> Nothing     opts c =       info (helper <*> config c) $         fullDesc@@ -819,12 +845,12 @@ run :: Config -> IO () run cfg =   withLogger level $ \logger -> do-    net <- either error return $ networkReader cfg.net+    net <- either (throwIO . StartupError . T.pack) return $ networkReader cfg.net     with_stats logger net $ \stats -> do       let dir = cfg.dir </> net.name       logInfo (withSource "haskoin-store" logger) $         "Creating working directory (if not found): " <> cs dir-      createDirectoryIfMissing True dir+      makeDataDirectory dir       run_store logger net stats dir $ \store ->         if cfg.noWeb           then withSubscription store.pub $ \sub ->@@ -908,10 +934,10 @@               healthCheckInterval = cfg.healthCheckInterval,               accessLog = cfg.accessLog,               nodeRpc = if null cfg.nodeRpc then Nothing else Just cfg.nodeRpc,+              -- both or neither, which 'main' checked               tls = case (cfg.tlsCert, cfg.tlsKey) of-                ("", "") -> Nothing-                ("", _) -> error "--tls-key needs --tls-cert"-                (_, "") -> error "--tls-cert needs --tls-key"+                ("", _) -> Nothing+                (_, "") -> Nothing                 (certificate, key) -> Just (certificate, key),               logger = withSource "Web" logger             }
haskoin-store.cabal view
@@ -5,7 +5,7 @@ -- see: https://github.com/sol/hpack  name:           haskoin-store-version:        1.16.4+version:        1.16.5 synopsis:       Storage and index for Bitcoin and Bitcoin Cash description:    An indexer for Bitcoin and Bitcoin Cash. It downloads blocks and mempool                 transactions from the peer-to-peer network, indexes them in RocksDB, and@@ -75,6 +75,7 @@       Haskoin.Store.Manager       Haskoin.Store.Outage       Haskoin.Store.Runtime+      Haskoin.Store.Startup       Haskoin.Store.Web       Haskoin.Store.Web.Admission       Haskoin.Store.Web.Congestion@@ -125,7 +126,7 @@     , hashtables >=1.4.2 && <1.5     , haskoin-core >=2.0.1 && <2.1     , haskoin-node >=2.6.1 && <2.7-    , haskoin-store-data >=3.0.0 && <3.1+    , haskoin-store-data >=3.0.1 && <3.1     , hedis >=0.16.4 && <0.17     , http-types >=0.12.6 && <0.13     , lens >=5.3.6 && <5.4@@ -194,7 +195,7 @@     , haskoin-core >=2.0.1 && <2.1     , haskoin-node >=2.6.1 && <2.7     , haskoin-store-    , haskoin-store-data >=3.0.0 && <3.1+    , haskoin-store-data >=3.0.1 && <3.1     , hedis >=0.16.4 && <0.17     , http-types >=0.12.6 && <0.13     , lens >=5.3.6 && <5.4@@ -261,7 +262,7 @@     , haskoin-core >=2.0.1 && <2.1     , haskoin-node >=2.6.1 && <2.7     , haskoin-store-    , haskoin-store-data >=3.0.0 && <3.1+    , haskoin-store-data >=3.0.1 && <3.1     , hedis >=0.16.4 && <0.17     , http-types >=0.12.6 && <0.13     , lens >=5.3.6 && <5.4@@ -361,7 +362,7 @@     , haskoin-core >=2.0.1 && <2.1     , haskoin-node >=2.6.1 && <2.7     , haskoin-store-    , haskoin-store-data >=3.0.0 && <3.1+    , haskoin-store-data >=3.0.1 && <3.1     , hedis >=0.16.4 && <0.17     , hspec >=2.11.18 && <2.12     , http-types >=0.12.6 && <0.13@@ -434,7 +435,7 @@     , haskoin-core >=2.0.1 && <2.1     , haskoin-node >=2.6.1 && <2.7     , haskoin-store-    , haskoin-store-data >=3.0.0 && <3.1+    , haskoin-store-data >=3.0.1 && <3.1     , hedis >=0.16.4 && <0.17     , http-types >=0.12.6 && <0.13     , integration@@ -501,7 +502,7 @@     , haskoin-core >=2.0.1 && <2.1     , haskoin-node >=2.6.1 && <2.7     , haskoin-store-    , haskoin-store-data >=3.0.0 && <3.1+    , haskoin-store-data >=3.0.1 && <3.1     , hedis >=0.16.4 && <0.17     , http-types >=0.12.6 && <0.13     , integration
src/Haskoin/Store/Manager.hs view
@@ -104,6 +104,7 @@     noteBest,   ) import Haskoin.Store.Fleet.Redis (FleetEvent (..))+import Haskoin.Store.Startup (connectRedis, explainDatabase) import Haskoin.Store.XPub.Service (XPubConfig (..)) import NQE   ( Inbox,@@ -230,21 +231,25 @@ -- when the action returns, and an exception in any component is re-thrown in -- the calling thread. ----- The database is opened first, with a thread that watches it for write--- stalls (@withStalls@), then the node, the block store and the cache (if--- 'redis' is set).+-- Redis is connected first (if 'redis' is set), and a store that cannot+-- reach it stops there with a 'StartupError' that says why. Then the+-- database is opened, with a thread that watches it for write stalls+-- (@withStalls@), then the node, the block store and the cache. withStore ::   StoreConfig ->   (Store -> IO a) ->   IO a-withStore cfg action =-  connectDB cfg $ \db ->+withStore cfg action = do+  -- Redis first: a store cannot run without it, and finding out after the+  -- database opened and the node started only takes longer+  redis <- mapM connectRedis cfg.redis+  explainDatabase cfg.db (connectDB cfg) $ \db ->     withStalls cfg db $ \writes ->       withPublisher $ \pub ->         withPublisher $ \node_pub ->           withSubscription node_pub $ \node_sub ->             withNode (nodeCfg cfg db node_pub) $ \node -> do-              member <- storeMember cfg db+              member <- storeMember cfg redis db               withBlockStore (blockStoreCfg cfg node pub db member) $ \block_store ->                 withAsync (nodeForwarder block_store pub node_sub) $ \fwd -> do                   link fwd@@ -278,17 +283,13 @@ cachePrefix net = "hs2:" <> C.pack net.name <> ":"  -- | The store as a member of a fleet, if the configuration says that it is--- one and has a Redis to share: under a name of its own, with a lease of+-- one and has a Redis to share (connected by 'connectRedis', which checked+-- that it answers): under a name of its own, with a lease of -- ten seconds, views of its last six blocks, and the store's limit on the -- age of unconfirmed transactions.-storeMember :: StoreConfig -> DatabaseReader -> IO (Maybe Member)-storeMember cfg db = case cfg.redis of-  Just url -> do-    info <--      if null url-        then return Redis.defaultConnectInfo-        else either fail return (Redis.parseConnectInfo url)-    conn <- Redis.checkedConnect info+storeMember :: StoreConfig -> Maybe Redis.Connection -> DatabaseReader -> IO (Maybe Member)+storeMember cfg redis db = case redis of+  Just conn -> do     name <- C.pack . show <$> (randomIO :: IO Word64)     Just       <$> newMember
+ src/Haskoin/Store/Startup.hs view
@@ -0,0 +1,329 @@+{-# LANGUAGE LambdaCase #-}++-- |+-- Module      : Haskoin.Store.Startup+-- Copyright   : (c) Haskoin Developers+-- License     : MIT+-- Maintainer  : jprupp@protonmail.ch+-- Stability   : experimental+-- Portability : POSIX+--+-- Errors that stop the store from starting, said in words that tell an+-- operator what is wrong and what to do about it.+--+-- = Why+--+-- A store needs a few things from its machine before it can run: a Redis+-- (or Valkey) to share with its fleet, a data directory that it can write,+-- a database that no other process holds, a port or a Unix socket to listen+-- on, and the certificate and key of HTTPS if it is to speak it. When one+-- is missing, the library that finds out throws an exception about its own+-- part, such as @Network.Socket.connect: \<socket: 123\>: does not exist+-- (Connection refused)@ for a Redis that is not running, or a RocksDB+-- @LOCK@ file for a second store on the same directory. Those say what+-- failed, deep down, but not what it was for: the operator has to guess+-- which of the store's connections it was, and how to fix it.+--+-- This module turns each of those failures into a 'StartupError': one+-- paragraph that names the thing that is missing, the setting that points+-- at it (option and environment variable), and the usual fix. The program+-- prints it and exits with status 1, without the backtrace of an uncaught+-- exception, which is noise for a mistake of configuration.+--+-- = Where the checks are+--+-- Each check sits where the thing is first used, rather than in a list of+-- checks run up front: a check made in advance can pass, and the real use+-- fail a moment later for the same reason, with the old message.+--+-- * Redis: 'connectRedis', the first thing that 'Haskoin.Store.withStore'+--   does, before it opens the database or the network, so that a store+--   without Redis stops at once instead of after the database opened.+-- * The database: 'explainDatabase', around the opening of RocksDB.+-- * The data directory: 'makeDataDirectory'.+-- * The web server: 'explainListen', around the binding of its port, and+--   'checkReadable' for the files of HTTPS.+--+-- Errors while the store runs are not stopped here: a Redis that goes+-- away later is logged by the health check with its reason+-- ('redisProblem'), and the store answers 503 until it is back.+module Haskoin.Store.Startup+  ( -- * The error+    StartupError (..),++    -- * Redis+    connectRedis,+    redisProblem,++    -- * Files and directories+    makeDataDirectory,+    checkReadable,++    -- * Explanations of other failures+    explainDatabase,+    explainListen,+  )+where++import Control.Exception (Exception (..), IOException, SomeException, catch, handle, throwIO, try)+import Data.ByteString.Char8 qualified as C+import Data.IORef (newIORef, readIORef, writeIORef)+import Data.List (isInfixOf)+import Data.Maybe (fromMaybe)+import Data.Text (Text)+import Data.Text qualified as T+import Database.Redis qualified as R+import System.Directory (createDirectoryIfMissing)+import System.IO (IOMode (ReadMode), withFile)+import System.IO.Error+  ( ioeGetErrorString,+    isAlreadyInUseError,+    isDoesNotExistError,+    isPermissionError,+  )+import System.Timeout (timeout)++-- | A reason why the store cannot start, in words for the operator. The+-- program prints the text alone and exits with status 1 ('displayException'+-- is the text).+newtype StartupError = StartupError Text+  deriving (Show, Eq)++instance Exception StartupError where+  displayException (StartupError t) = T.unpack t++-- | Throw a 'StartupError' made of these lines, joined with spaces.+failWith :: [Text] -> IO a+failWith = throwIO . StartupError . T.unwords++-- | What every message about Redis ends with: what it is for, and how to+-- point the store at it.+redisHint :: Text+redisHint =+  T.unwords+    [ "haskoin-store needs a Redis or Valkey server to run: it keeps the tip,",+      "the mempool and the xpub cache that the stores of a fleet share.",+      "Start one (for example `valkey-server` or `redis-server`), or point the store",+      "at a running one with --redis-url or REDIS_URL (redis://[:password@]host:port[/db])."+    ]++-- | Connect to the Redis of the fleet at this URL (empty for the default,+-- @redis:\/\/localhost:6379@), and make sure that it answers: a ping must+-- come back as @PONG@. Throws a 'StartupError' that says what went wrong:+-- the URL is not one of Redis, nothing listens there, the name of the host+-- is unknown, Redis wants a password or another one, or it does not answer+-- within five seconds.+--+-- The check of the answer is the point: @checkedConnect@ of hedis only+-- makes sure that a reply came, and a Redis that wants a password replies+-- to a ping with an error, so a store connected to it without one started+-- and then answered 503 to every request, with nothing in its logs but+-- \"Redis does not answer\".+connectRedis :: String -> IO R.Connection+connectRedis url = do+  info <-+    if null url+      then return R.defaultConnectInfo+      else either badURL return (R.parseConnectInfo url)+  -- hedis waits for ever by default for a host that drops the packets+  let info' = info {R.connectTimeout = Just (fromMaybe 5 info.connectTimeout)}+  conn <- R.connect info' `catch` \(e :: IOException) -> unreachable (show e)+  answer <-+    try (timeout 6000000 (R.runRedis conn R.ping)) >>= \case+      Left e -> failed e+      Right r -> return r+  case answer of+    Nothing ->+      failWith ["Redis at", shown, "did not answer within five seconds.", redisHint]+    Just (Right R.Pong) -> return conn+    Just (Right other) ->+      failWith ["Redis at", shown, "answered a ping with", T.pack (show other) <> ".", redisHint]+    Just (Left reply) -> replied reply+  where+    shown = T.pack (if null url then "redis://localhost:6379" else url)+    badURL :: String -> IO b+    badURL why =+      failWith+        [ "--redis-url (or REDIS_URL)",+          T.pack (show url),+          "is not a Redis URL:",+          T.pack why <> ".",+          "It should look like redis://localhost:6379, or redis://:password@host:port/db."+        ]+    unreachable :: String -> IO b+    unreachable why+      | "refused" `isInfixOf` why =+          failWith ["Cannot connect to Redis at", shown <> ": nothing is listening there.", redisHint]+      | "does not exist" `isInfixOf` why || "Name or service not known" `isInfixOf` why =+          failWith ["Cannot connect to Redis at", shown <> ": the host name is unknown.", redisHint]+      | otherwise =+          failWith ["Cannot connect to Redis at", shown <> ":", T.pack why <> ".", redisHint]+    failed :: SomeException -> IO a+    failed e+      | Just (io :: IOException) <- fromException e = unreachable (show io)+      | Just (R.ConnectTimeout _) <- fromException e =+          failWith ["Redis at", shown, "did not accept a connection within five seconds.", redisHint]+      | Just (R.ConnectAuthError reply) <- fromException e = replied reply+      | Just (R.ConnectSelectError reply) <- fromException e =+          failWith+            [ "Redis at",+              shown,+              "refused the database number in the URL:",+              replyText reply <> ".",+              "Use a database that the server has (0 to 15 by default)."+            ]+      | otherwise = throwIO e+    replied :: R.Reply -> IO b+    replied reply+      | any (`T.isPrefixOf` msg) ["NOAUTH", "WRONGPASS", "ERR AUTH", "ERR invalid password"] =+          failWith+            [ "Redis at",+              shown,+              "refused the store:",+              msg <> ".",+              "It needs a password, or another one: put it in the URL,",+              "as redis://:password@host:port, or redis://user:password@host:port."+            ]+      | otherwise =+          failWith ["Redis at", shown, "answered with an error:", msg <> ".", redisHint]+      where+        msg = replyText reply++-- | The text of a reply of Redis, for a message.+replyText :: R.Reply -> Text+replyText =+  T.dropWhileEnd (== '.') . T.strip . \case+    R.Error e -> T.pack (C.unpack e)+    R.SingleLine s -> T.pack (C.unpack s)+    other -> T.pack (show other)++-- | Why Redis does not answer a ping right now, if it does not: for the+-- health check of a store that runs, which logs it. Nothing if it answers+-- @PONG@ within two seconds.+redisProblem :: R.Connection -> IO (Maybe Text)+redisProblem conn =+  handle (\(e :: SomeException) -> return (Just (described e))) $+    timeout 2000000 (R.runRedis conn R.ping) >>= \case+      Nothing -> return (Just "no answer within two seconds")+      Just (Right R.Pong) -> return Nothing+      Just (Right other) -> return (Just ("it answered " <> T.pack (show other)))+      Just (Left reply) -> return (Just (replyText reply))++-- | A failure to reach Redis in a few words: the reason of a refused+-- connection or an unknown host, rather than the text of the socket's+-- exception.+described :: SomeException -> Text+described e+  | "refused" `isInfixOf` why = "connection refused"+  | "does not exist" `isInfixOf` why || "Name or service not known" `isInfixOf` why = "host name unknown"+  | Just R.ConnectionLost <- fromException e = "connection lost"+  | otherwise = T.pack why+  where+    why = displayException e++-- | Make the data directory of the store, and its parents, if they are not+-- there. Throws a 'StartupError' that names the directory and the option+-- (@--dir@, @DIR@) if it cannot be made.+makeDataDirectory :: FilePath -> IO ()+makeDataDirectory dir =+  createDirectoryIfMissing True dir `catch` \(e :: IOException) ->+    failWith+      [ "Cannot create the data directory",+        T.pack dir <> ":",+        T.pack (reason e) <> ".",+        "Set --dir (or DIR) to a directory that the user running haskoin-store can write."+      ]+  where+    reason e+      | isPermissionError e = "permission denied"+      | isDoesNotExistError e = "a parent directory does not exist and cannot be made"+      | otherwise = ioeGetErrorString e++-- | Make sure that a file named by an option can be read, before a library+-- tries and fails with a message that does not name the option. The first+-- argument is the option, such as @--tls-cert@.+checkReadable :: Text -> FilePath -> IO ()+checkReadable option path =+  withFile path ReadMode (\_ -> return ()) `catch` \(e :: IOException) ->+    failWith+      [ option,+        T.pack path <> ":",+        T.pack (reason e) <> "."+      ]+  where+    reason e+      | isDoesNotExistError e = "no such file"+      | isPermissionError e = "the user running haskoin-store may not read it"+      | otherwise = ioeGetErrorString e++-- | Open the database in this directory with a function in the style of+-- @withX@, and run an action with it, explaining the usual failures of the+-- opening: another process holds the database (most often a store that is+-- still running, with the same @--dir@ and network), or the user may not+-- write it. Only the opening is explained: once the action has started,+-- an error of the action goes through as it is, since it is not about+-- opening the database.+explainDatabase :: FilePath -> ((r -> IO a) -> IO a) -> (r -> IO a) -> IO a+explainDatabase dir open action = do+  started <- newIORef False+  open (\r -> writeIORef started True >> action r) `catch` \(e :: IOException) ->+    readIORef started >>= \case+      True -> throwIO e+      False -> explain e+  where+    explain e+      | "LOCK" `isInfixOf` why && ("unavailable" `isInfixOf` why || "lock" `isInfixOf` why) =+          failWith+            [ "The database in",+              T.pack dir,+              "is in use by another process, most likely another haskoin-store",+              "with the same --dir and --net. Stop that one first,",+              "or give this one a data directory of its own with --dir."+            ]+      | "Permission denied" `isInfixOf` why =+          failWith+            [ "Cannot open the database in",+              T.pack dir <> ": permission denied.",+              "Run haskoin-store as the user that owns the directory, or set --dir to another one."+            ]+      | otherwise =+          failWith ["Cannot open the database in", T.pack dir <> ":", T.pack (ioeGetErrorString e)]+      where+        why = show e++-- | Run an action that binds the web server to this address (as the+-- operator gave it, such as @127.0.0.1:3000@ or a socket's path), and+-- explain why it could not: the address is in use, the port needs root,+-- or the host is not an address of this machine. The action gets an+-- action to run as soon as it listens (Warp's @setBeforeMainLoop@): only+-- errors before then are explained, and those after go through as they+-- are, since they are not about binding.+explainListen :: Text -> (IO () -> IO a) -> IO a+explainListen address action = do+  listening <- newIORef False+  action (writeIORef listening True) `catch` \(e :: IOException) ->+    readIORef listening >>= \case+      True -> throwIO e+      False -> explain e+  where+    explain e+      | isAlreadyInUseError e || "in use" `isInfixOf` show e =+          failWith+            [ "Cannot listen on",+              address <> ": another program is using it, perhaps another haskoin-store.",+              "Stop it, or choose another port with --port (or PORT), or another socket with --socket."+            ]+      | isPermissionError e =+          failWith+            [ "Cannot listen on",+              address <> ": permission denied.",+              "Ports below 1024 need root: use a higher --port, or a proxy in front of the store."+            ]+      | "Cannot assign requested address" `isInfixOf` show e || isDoesNotExistError e =+          failWith+            [ "Cannot listen on",+              address <> ": the host is not an address of this machine.",+              "Set --host (or HOST) to one of its addresses, or to * for all of them."+            ]+      | otherwise =+          failWith ["Cannot listen on", address <> ":", T.pack (ioeGetErrorString e)]
src/Haskoin/Store/Web.hs view
@@ -182,6 +182,7 @@     fromJust,     fromMaybe,     isJust,+    isNothing,     listToMaybe,     mapMaybe,     maybeToList,@@ -256,6 +257,7 @@   ) import Haskoin.Store.Fleet.Redis (FleetEvent (..)) import Haskoin.Store.Manager+import Haskoin.Store.Startup (explainListen, redisProblem) import Haskoin.Store.Web.Admission   ( Admission,     Aside (..),@@ -343,6 +345,7 @@   ( defaultSettings,     runSettings,     runSettingsSocket,+    setBeforeMainLoop,     setHost,     setPort,   )@@ -742,12 +745,22 @@ -- servers; this is for a server by itself, and for links between machines -- that should not be in the clear. The files are read when the server -- starts, so a renewed certificate takes a restart.+--+-- A port that cannot be bound stops the program with a 'StartupError'+-- that says why ('explainListen'). runWeb :: WebConfig -> IO ()-runWeb config = serveWeb config $ case config.tls of-  Nothing -> runSettings settings-  Just (certificate, key) -> runTLS (tlsSettings certificate key) settings+runWeb config = serveWeb config $ \app ->+  explainListen address $ \listening ->+    let settings =+          setBeforeMainLoop listening+            . setPort config.port+            . setHost (fromString config.host)+            $ defaultSettings+     in case config.tls of+          Nothing -> runSettings settings app+          Just (certificate, key) -> runTLS (tlsSettings certificate key) settings app   where-    settings = setPort config.port . setHost (fromString config.host) $ defaultSettings+    address = T.pack config.host <> ":" <> T.pack (show config.port)  -- | Like 'runWeb', on a socket that already listens; the host and port of the -- configuration are ignored. Binding port 0 and passing the socket gives a@@ -935,6 +948,8 @@           config.tickerRefresh     runHealthCheck = healthCheck config     cacheCheck = maybe (return False) memberAlive config.store.fleet+    -- what is wrong with Redis, if anything, for the log+    redisCheck = maybe (return (Just "the store has no Redis")) (redisProblem . (.redis)) config.store.fleet     -- The check is run every second by default, and at once when the     -- store has a new best block, which is when its answer changes most:     -- /health answers from the last result, and a load balancer or a@@ -968,9 +983,9 @@           last' <- readIORef pinged           when (now - last' >= 5) $ do             writeIORef pinged now-            redis <- cacheCheck-            atomically (writeTVar up redis)-            tellRedis saidRedis redis+            problem <- redisCheck+            atomically (writeTVar up (isNothing problem))+            tellRedis saidRedis problem     heard newBlock = \case       StoreBestBlock _ -> writeTVar newBlock True       _ -> return ()@@ -988,14 +1003,17 @@         (Just False, True) -> logInfo health' "Healthy again"         (Nothing, True) -> logInfo health' "Healthy"         _ -> return ()-    -- the same for the Redis that a store with a cache answers through;-    -- /health says 503 without it-    tellRedis said redis = do+    -- the same for the Redis that the store answers through, with the+    -- reason that it does not answer (such as a password that it wants);+    -- without it the store answers 503 and /health says so+    tellRedis said problem = do       before <- readIORef said-      writeIORef said redis-      case (before, redis) of-        (True, False) -> logWarn health' "Redis does not answer: answering from the database"-        (False, True) -> logInfo health' "Redis answers again"+      writeIORef said (isNothing problem)+      case (before, problem) of+        (True, Just why) ->+          logWarn health' $+            "Redis does not answer (" <> why <> "): answering 503 to requests until it does"+        (False, Nothing) -> logInfo health' "Redis answers again"         _ -> return ()     -- what a browser asks before a request from another site that is not     -- of the plainest kind: which methods and headers it may use. Without
src/Haskoin/Store/Web/Listen.hs view
@@ -47,7 +47,9 @@  import Control.Exception (bracket, bracketOnError, throwIO, try) import Control.Monad (unless, when)+import Data.Text qualified as T import Foreign.C.String (peekCAString)+import Haskoin.Store.Startup (StartupError (..), explainListen) import Network.Socket   ( Family (AF_UNIX),     SockAddr (SockAddrUnix),@@ -114,11 +116,10 @@   -- the kernel keeps the path in 108 bytes, ending in a zero; the length is   -- that of the path's bytes in the encoding of the file system   pathBytes <- withFilePath path (fmap length . peekCAString)-  when (pathBytes > 107) . throwIO . userError $-    "The path of the Unix socket is longer than 107 bytes, which the system allows: "-      <> path+  when (pathBytes > 107) . throwIO . StartupError . T.pack $+    "--socket " <> path <> ": the path is longer than the 107 bytes that the system allows; use a shorter one"   removeStale path-  bracket open (\s -> close s >> removeFile path) act+  bracket (explainListen (T.pack path) (\ok -> open <* ok)) (\s -> close s >> removeFile path) act   where     open = bracketOnError (socket AF_UNIX Stream defaultProtocol) close $ \s -> do       bind s (SockAddrUnix path)@@ -135,14 +136,17 @@       | otherwise -> throwIO e     Right st -> do       unless (isSocket st) $-        throwIO . userError $-          path <> " exists and is not a socket"+        throwIO . StartupError . T.pack $+          "--socket " <> path <> ": a file that is not a socket is there; remove it, or choose another path"       live <- bracket (socket AF_UNIX Stream defaultProtocol) close $ \s ->         either (const False) (const True)           <$> (try (connect s (SockAddrUnix path)) :: IO (Either IOError ()))       when live $-        throwIO . userError $-          path <> " is in use by another server"+        throwIO . StartupError . T.pack $+          "--socket "+            <> path+            <> ": another server is listening on it, perhaps another haskoin-store; "+            <> "stop it, or choose another path"       removeFile path  -- | Read a file mode written in octal, such as @0660@ or @660@.
swagger.yaml view
@@ -136,7 +136,7 @@     | 500 | `server-error` | Something went wrong on the server; it is logged there. |     | 503 | `server-error` | Reading a wallet into the cache failed; ask again after a second. |     | 503 | | From `/health`: the server is not ready. |-    | 503 | `behind-fleet` | The server is one of a fleet and does not have the block that the fleet answers from; another server of the fleet does. |+    | 503 | `behind-fleet` | The server is not synchronized with the network: it does not have the block to answer from yet, or cannot reach its Redis. Another server of the fleet may have it. |      A value in the path that does not parse, such as a misspelt address, gives     404 rather than 400. Paths are matched by the shape of what is in them, and