packages feed

haskoin-store 1.16.6 → 1.16.7

raw patch · 8 files changed

+88/−17 lines, 8 filesPVP: major bump suggested

API removals or changes: PVP suggests a major version bump

API changes (from Hackage documentation)

+ Haskoin.Store.BlockStore: [explicitPeers] :: BlockStoreConfig -> ![String]
+ Haskoin.Store.BlockStore: asksMempool :: Network -> [SockAddr] -> SockAddr -> Bool
- Haskoin.Store.BlockStore: BlockStoreConfig :: !PeerMgr -> !Chain -> !Publisher StoreEvent -> !DatabaseReader -> !Network -> !BlockHash -> IO () -> !UnixTime -> Tx -> IO () -> ![TxHash] -> IO (HashSet TxHash) -> !Int -> !NominalDiffTime -> !NominalDiffTime -> !NominalDiffTime -> !Maybe Stats -> !Logger -> BlockStoreConfig
+ Haskoin.Store.BlockStore: BlockStoreConfig :: !PeerMgr -> !Chain -> !Publisher StoreEvent -> !DatabaseReader -> !Network -> ![String] -> !BlockHash -> IO () -> !UnixTime -> Tx -> IO () -> ![TxHash] -> IO (HashSet TxHash) -> !Int -> !NominalDiffTime -> !NominalDiffTime -> !NominalDiffTime -> !Maybe Stats -> !Logger -> BlockStoreConfig

Files

CHANGELOG.md view
@@ -4,6 +4,17 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html) +## [1.16.7] - 2026-11-20++### Fixed+- **A Bitcoin store no longer loses its peers when the pool fills again.**+  A refill asked every connected peer for its mempool, and Bitcoin Core+  disconnects a peer that asks unless it whitelists it: on groot the btc+  store lost its three peers at once and answered 503 for seven seconds.+  On Bitcoin only the peers given with `--peer` are asked now, since an+  operator whitelists those; on Bitcoin Cash every peer still is, since+  Bitcoin Cash Node answers anyone.+ ## [1.16.6] - 2026-11-19  ### Fixed
README.md view
@@ -325,7 +325,7 @@ | `-n`, `--net` | `NET` | `bch` | Network: `btc`, `bch`, or a test network (see `--help`) | | `-d`, `--dir` | `DIR` | `~/.haskoin-store` | Data directory | | `--discover` | `DISCOVER` | off | Discover peers through DNS seeds and relayed addresses; on regtest it finds none, so name a `--peer` |-| `-p`, `--peer` | `PEER` | none | Connect to `HOST[:PORT]`; repeat for more, space-separated in `PEER` |+| `-p`, `--peer` | `PEER` | none | Connect to `HOST[:PORT]`; repeat for more, space-separated in `PEER`. On Bitcoin, only these peers are asked for their mempools when the pool fills again: whitelist the store on them (`whitebind=mempool@…`) | | `--max-peers` | `MAX_PEERS` | `20` | Most peers to connect to | | `--peer-timeout` | `PEER_TIMEOUT` | `30` | Seconds before a peer that is silent, or that owes blocks, is dropped | | `--block-stall` | `BLOCK_STALL` | `10` | Longest wait, in seconds, for a block from the syncing peer before another peer is asked too; a quick peer gets four of its recent gaps between blocks, at least 2 s |@@ -401,9 +401,10 @@   blocks, the stores fill it again too. - **Filling the pool again** works best with `--node-rpc`: each store reads   its node's mempool (`getrawmempool`), 2,000 transactions every two-  seconds. Each also asks its peers for their mempools, once. Bitcoin Cash-  Node answers; Bitcoin Core answers only a store it whitelists, and-  disconnects any other that asks (`docs/fleet-mempool.md`).+  seconds. Each also asks its peers for their mempools, once: every peer on+  Bitcoin Cash, whose nodes answer anyone, and on Bitcoin only the peers+  given with `--peer`, since Bitcoin Core disconnects a store that asks+  unless it whitelists it (`docs/fleet-mempool.md`). - **`/events` is the same stream on every store**, written to Redis with   each change of the pool and each move of the tip. - **Xpubs** are cached for the whole fleet (`docs/xpub-indexing.md`).
app/Main.hs view
@@ -390,7 +390,13 @@           metavar "HOSTNAME[:PORT]"             <> long "peer"             <> short 'p'-            <> helpEnv "PEER" "Network peer (as many as required)"+            <> helpEnv+              "PEER"+              ( unwords+                  [ "Network peer (as many as required); on Bitcoin, only these are asked for their mempools",+                    "when the pool fills again, so whitelist the store on them"+                  ]+              )   version <-     switch $       long "version"
haskoin-store.cabal view
@@ -5,7 +5,7 @@ -- see: https://github.com/sol/hpack  name:           haskoin-store-version:        1.16.6+version:        1.16.7 synopsis:       Storage and index for Bitcoin and Bitcoin Cash description:    An indexer for Bitcoin and Bitcoin Cash. It downloads blocks and mempool                 transactions from the peer-to-peer network, indexes the chain in RocksDB and@@ -298,6 +298,7 @@   other-modules:       Haskoin.Store.BlockStore.PendingSpec       Haskoin.Store.BlockStore.SyncSpec+      Haskoin.Store.BlockStoreSpec       Haskoin.Store.CommonSpec       Haskoin.Store.Database.MergeSpec       Haskoin.Store.Database.MigrateSpec
src/Haskoin/Store/BlockStore.hs view
@@ -92,6 +92,7 @@     blockStorePendingTxsSTM,     needsMoreBlocks,     blockStoreRefill,+    asksMempool,     toFetch,   ) where@@ -109,7 +110,7 @@     stateTVar,     writeTVar,   )-import Control.Exception (Exception, throwIO, try)+import Control.Exception (Exception, SomeException, catch, throwIO, try) import Control.Monad   ( filterM,     forM,@@ -189,6 +190,7 @@     releasePeer,     reservePeer,     sendMessage,+    toSockAddr,   ) import Haskoin.Store.BlockStore.Pending (Pending) import Haskoin.Store.BlockStore.Pending qualified as Pending@@ -228,6 +230,7 @@     send,     sendSTM,   )+import Network.Socket (SockAddr) import System.Metrics.StatsD import System.Random (randomIO, randomRIO) @@ -428,6 +431,11 @@     db :: !DatabaseReader,     -- | network constants     net :: !Network,+    -- | the peers given on the command line or in the environment, as+    -- @host:port@: on Bitcoin, only these are asked for their mempools,+    -- since a node whitelists the store there or disconnects it+    -- ('blockStoreRefill')+    explicitPeers :: ![String],     -- | what to do with the best block right after it changes, and when the     -- store starts, before anything else is written to the database: a     -- member of a fleet takes a view of the database for the block here@@ -606,7 +614,7 @@ syncMempool :: BlockStore -> IO () syncMempool self =   guardSync self $ do-    peers <- getPeers self.config.peerMgr+    peers <- getPeers self.config.peerMgr >>= mempoolPeers self     asking <- atomically $ do       waits <- readTVar self.refill       when (waits && not (null peers)) $ writeTVar self.refill False@@ -615,6 +623,28 @@       logInfo self $ "Asking " <> cs (show (length peers)) <> " peers for their mempools"       forM_ peers $ \op -> MMempool `sendMessage` op.mailbox +-- | Of the connected peers, those that answer a request for their mempool+-- ('asksMempool'), with 'explicitPeers' resolved to their addresses. A name+-- that does not resolve leaves its peer out.+mempoolPeers :: BlockStore -> [OnlinePeer] -> IO [OnlinePeer]+mempoolPeers self peers = do+  explicit <-+    if isJust self.config.net.cashAddrPrefix+      then return []+      else concat <$> mapM resolve self.config.explicitPeers+  return [p | p <- peers, asksMempool self.config.net explicit p.address]+  where+    resolve s = toSockAddr self.config.net s `catch` \(_ :: SomeException) -> return []++-- | Whether to ask a peer at an address for its mempool, given the+-- addresses of the peers given explicitly. On Bitcoin Cash, always: Bitcoin+-- Cash Node serves bloom filters by default, and so answers anyone. On+-- Bitcoin, only a peer given explicitly: Bitcoin Core disconnects any peer+-- that asks unless it whitelists it, which an operator does for the peers+-- that the store is told to use, and not for seeds or discovered peers.+asksMempool :: Network -> [SockAddr] -> SockAddr -> Bool+asksMempool net explicit a = isJust net.cashAddrPrefix || a `elem` explicit+ -- | Ask the peers for their mempools, once, and fetch every transaction -- that they announce and the fleet has not seen: for when the pool was -- emptied, because Redis lost its data@@ -623,14 +653,18 @@ -- went with it, so all of it is fetched. The transactions go to 'onTx', -- which offers them to the pool again. ----- Only the peers connected at that time are asked, and only once. Bitcoin--- Core closes the connection of a peer that asks for its mempool unless it--- serves bloom filters (@-peerbloomfilters@, off by default) or it--- whitelists that peer; Bitcoin Cash Node serves bloom filters by default--- and answers. Asking every peer that connects, as the store once did,--- lost its public Bitcoin Core peers one after the other for as long as--- it ran. A node that the store has over RPC (@--node-rpc@) answers in any--- case ('Haskoin.Store.Web.refillLoop').+-- Only the peers connected at that time are asked, and only once, and of+-- those only the ones that answer ('asksMempool'). Bitcoin Core closes the+-- connection of a peer that asks for its mempool unless it serves bloom+-- filters (@-peerbloomfilters@, off by default) or it whitelists that peer;+-- Bitcoin Cash Node serves bloom filters by default and answers anyone. So+-- on Bitcoin only the peers given with @--peer@ are asked, since an+-- operator whitelists those: asking the others, seeds and discovered peers,+-- only loses them, all at once, and with them the health of the store.+-- Asking every peer that connects, as the store once did, lost its public+-- Bitcoin Core peers one after the other for as long as it ran. A node+-- that the store has over RPC (@--node-rpc@) answers in any case+-- ('Haskoin.Store.Web.refillLoop'). blockStoreRefill :: BlockStore -> IO () blockStoreRefill self = do   atomically $ writeTVar self.refill True
src/Haskoin/Store/Manager.hs view
@@ -396,6 +396,7 @@       pub = pub,       db = db,       net = cfg.net,+      explicitPeers = cfg.initPeers,       onBest = maybe (\_ -> return ()) noteBest member,       onTx = maybe (\_ _ -> return ()) memberReceived member,       seen = maybe (\_ -> return mempty) memberSeen member,
swagger.yaml view
@@ -11,7 +11,7 @@   - url: './'  info:-  version: &version 1.16.6+  version: &version 1.16.7   title: Haskoin   contact:     name: JP Rupp
+ test/Haskoin/Store/BlockStoreSpec.hs view
@@ -0,0 +1,17 @@+-- | Which peers the block store asks for their mempools.+module Haskoin.Store.BlockStoreSpec (spec) where++import Haskoin (bch, btc)+import Haskoin.Store.BlockStore (asksMempool)+import Network.Socket (SockAddr (..), tupleToHostAddress)+import Test.Hspec++spec :: Spec+spec = describe "Asking for a mempool" $ do+  let local = SockAddrInet 8335 (tupleToHostAddress (127, 0, 0, 1))+      seed = SockAddrInet 8333 (tupleToHostAddress (192, 0, 2, 7))+  it "asks every peer on Bitcoin Cash, whose nodes answer anyone" $+    map (asksMempool bch []) [local, seed] `shouldBe` [True, True]+  it "asks only the peers given explicitly on Bitcoin, whose nodes disconnect the others" $ do+    map (asksMempool btc [local]) [local, seed] `shouldBe` [True, False]+    asksMempool btc [] local `shouldBe` False