haskoin-store 1.16.6 → 1.16.7
raw patch · 8 files changed
+88/−17 lines, 8 filesPVP: major bump suggested
API removals or changes: PVP suggests a major version bump
API changes (from Hackage documentation)
+ Haskoin.Store.BlockStore: [explicitPeers] :: BlockStoreConfig -> ![String]
+ Haskoin.Store.BlockStore: asksMempool :: Network -> [SockAddr] -> SockAddr -> Bool
- Haskoin.Store.BlockStore: BlockStoreConfig :: !PeerMgr -> !Chain -> !Publisher StoreEvent -> !DatabaseReader -> !Network -> !BlockHash -> IO () -> !UnixTime -> Tx -> IO () -> ![TxHash] -> IO (HashSet TxHash) -> !Int -> !NominalDiffTime -> !NominalDiffTime -> !NominalDiffTime -> !Maybe Stats -> !Logger -> BlockStoreConfig
+ Haskoin.Store.BlockStore: BlockStoreConfig :: !PeerMgr -> !Chain -> !Publisher StoreEvent -> !DatabaseReader -> !Network -> ![String] -> !BlockHash -> IO () -> !UnixTime -> Tx -> IO () -> ![TxHash] -> IO (HashSet TxHash) -> !Int -> !NominalDiffTime -> !NominalDiffTime -> !NominalDiffTime -> !Maybe Stats -> !Logger -> BlockStoreConfig
Files
- CHANGELOG.md +11/−0
- README.md +5/−4
- app/Main.hs +7/−1
- haskoin-store.cabal +2/−1
- src/Haskoin/Store/BlockStore.hs +44/−10
- src/Haskoin/Store/Manager.hs +1/−0
- swagger.yaml +1/−1
- test/Haskoin/Store/BlockStoreSpec.hs +17/−0
CHANGELOG.md view
@@ -4,6 +4,17 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html) +## [1.16.7] - 2026-11-20++### Fixed+- **A Bitcoin store no longer loses its peers when the pool fills again.**+ A refill asked every connected peer for its mempool, and Bitcoin Core+ disconnects a peer that asks unless it whitelists it: on groot the btc+ store lost its three peers at once and answered 503 for seven seconds.+ On Bitcoin only the peers given with `--peer` are asked now, since an+ operator whitelists those; on Bitcoin Cash every peer still is, since+ Bitcoin Cash Node answers anyone.+ ## [1.16.6] - 2026-11-19 ### Fixed
README.md view
@@ -325,7 +325,7 @@ | `-n`, `--net` | `NET` | `bch` | Network: `btc`, `bch`, or a test network (see `--help`) | | `-d`, `--dir` | `DIR` | `~/.haskoin-store` | Data directory | | `--discover` | `DISCOVER` | off | Discover peers through DNS seeds and relayed addresses; on regtest it finds none, so name a `--peer` |-| `-p`, `--peer` | `PEER` | none | Connect to `HOST[:PORT]`; repeat for more, space-separated in `PEER` |+| `-p`, `--peer` | `PEER` | none | Connect to `HOST[:PORT]`; repeat for more, space-separated in `PEER`. On Bitcoin, only these peers are asked for their mempools when the pool fills again: whitelist the store on them (`whitebind=mempool@…`) | | `--max-peers` | `MAX_PEERS` | `20` | Most peers to connect to | | `--peer-timeout` | `PEER_TIMEOUT` | `30` | Seconds before a peer that is silent, or that owes blocks, is dropped | | `--block-stall` | `BLOCK_STALL` | `10` | Longest wait, in seconds, for a block from the syncing peer before another peer is asked too; a quick peer gets four of its recent gaps between blocks, at least 2 s |@@ -401,9 +401,10 @@ blocks, the stores fill it again too. - **Filling the pool again** works best with `--node-rpc`: each store reads its node's mempool (`getrawmempool`), 2,000 transactions every two- seconds. Each also asks its peers for their mempools, once. Bitcoin Cash- Node answers; Bitcoin Core answers only a store it whitelists, and- disconnects any other that asks (`docs/fleet-mempool.md`).+ seconds. Each also asks its peers for their mempools, once: every peer on+ Bitcoin Cash, whose nodes answer anyone, and on Bitcoin only the peers+ given with `--peer`, since Bitcoin Core disconnects a store that asks+ unless it whitelists it (`docs/fleet-mempool.md`). - **`/events` is the same stream on every store**, written to Redis with each change of the pool and each move of the tip. - **Xpubs** are cached for the whole fleet (`docs/xpub-indexing.md`).
app/Main.hs view
@@ -390,7 +390,13 @@ metavar "HOSTNAME[:PORT]" <> long "peer" <> short 'p'- <> helpEnv "PEER" "Network peer (as many as required)"+ <> helpEnv+ "PEER"+ ( unwords+ [ "Network peer (as many as required); on Bitcoin, only these are asked for their mempools",+ "when the pool fills again, so whitelist the store on them"+ ]+ ) version <- switch $ long "version"
haskoin-store.cabal view
@@ -5,7 +5,7 @@ -- see: https://github.com/sol/hpack name: haskoin-store-version: 1.16.6+version: 1.16.7 synopsis: Storage and index for Bitcoin and Bitcoin Cash description: An indexer for Bitcoin and Bitcoin Cash. It downloads blocks and mempool transactions from the peer-to-peer network, indexes the chain in RocksDB and@@ -298,6 +298,7 @@ other-modules: Haskoin.Store.BlockStore.PendingSpec Haskoin.Store.BlockStore.SyncSpec+ Haskoin.Store.BlockStoreSpec Haskoin.Store.CommonSpec Haskoin.Store.Database.MergeSpec Haskoin.Store.Database.MigrateSpec
src/Haskoin/Store/BlockStore.hs view
@@ -92,6 +92,7 @@ blockStorePendingTxsSTM, needsMoreBlocks, blockStoreRefill,+ asksMempool, toFetch, ) where@@ -109,7 +110,7 @@ stateTVar, writeTVar, )-import Control.Exception (Exception, throwIO, try)+import Control.Exception (Exception, SomeException, catch, throwIO, try) import Control.Monad ( filterM, forM,@@ -189,6 +190,7 @@ releasePeer, reservePeer, sendMessage,+ toSockAddr, ) import Haskoin.Store.BlockStore.Pending (Pending) import Haskoin.Store.BlockStore.Pending qualified as Pending@@ -228,6 +230,7 @@ send, sendSTM, )+import Network.Socket (SockAddr) import System.Metrics.StatsD import System.Random (randomIO, randomRIO) @@ -428,6 +431,11 @@ db :: !DatabaseReader, -- | network constants net :: !Network,+ -- | the peers given on the command line or in the environment, as+ -- @host:port@: on Bitcoin, only these are asked for their mempools,+ -- since a node whitelists the store there or disconnects it+ -- ('blockStoreRefill')+ explicitPeers :: ![String], -- | what to do with the best block right after it changes, and when the -- store starts, before anything else is written to the database: a -- member of a fleet takes a view of the database for the block here@@ -606,7 +614,7 @@ syncMempool :: BlockStore -> IO () syncMempool self = guardSync self $ do- peers <- getPeers self.config.peerMgr+ peers <- getPeers self.config.peerMgr >>= mempoolPeers self asking <- atomically $ do waits <- readTVar self.refill when (waits && not (null peers)) $ writeTVar self.refill False@@ -615,6 +623,28 @@ logInfo self $ "Asking " <> cs (show (length peers)) <> " peers for their mempools" forM_ peers $ \op -> MMempool `sendMessage` op.mailbox +-- | Of the connected peers, those that answer a request for their mempool+-- ('asksMempool'), with 'explicitPeers' resolved to their addresses. A name+-- that does not resolve leaves its peer out.+mempoolPeers :: BlockStore -> [OnlinePeer] -> IO [OnlinePeer]+mempoolPeers self peers = do+ explicit <-+ if isJust self.config.net.cashAddrPrefix+ then return []+ else concat <$> mapM resolve self.config.explicitPeers+ return [p | p <- peers, asksMempool self.config.net explicit p.address]+ where+ resolve s = toSockAddr self.config.net s `catch` \(_ :: SomeException) -> return []++-- | Whether to ask a peer at an address for its mempool, given the+-- addresses of the peers given explicitly. On Bitcoin Cash, always: Bitcoin+-- Cash Node serves bloom filters by default, and so answers anyone. On+-- Bitcoin, only a peer given explicitly: Bitcoin Core disconnects any peer+-- that asks unless it whitelists it, which an operator does for the peers+-- that the store is told to use, and not for seeds or discovered peers.+asksMempool :: Network -> [SockAddr] -> SockAddr -> Bool+asksMempool net explicit a = isJust net.cashAddrPrefix || a `elem` explicit+ -- | Ask the peers for their mempools, once, and fetch every transaction -- that they announce and the fleet has not seen: for when the pool was -- emptied, because Redis lost its data@@ -623,14 +653,18 @@ -- went with it, so all of it is fetched. The transactions go to 'onTx', -- which offers them to the pool again. ----- Only the peers connected at that time are asked, and only once. Bitcoin--- Core closes the connection of a peer that asks for its mempool unless it--- serves bloom filters (@-peerbloomfilters@, off by default) or it--- whitelists that peer; Bitcoin Cash Node serves bloom filters by default--- and answers. Asking every peer that connects, as the store once did,--- lost its public Bitcoin Core peers one after the other for as long as--- it ran. A node that the store has over RPC (@--node-rpc@) answers in any--- case ('Haskoin.Store.Web.refillLoop').+-- Only the peers connected at that time are asked, and only once, and of+-- those only the ones that answer ('asksMempool'). Bitcoin Core closes the+-- connection of a peer that asks for its mempool unless it serves bloom+-- filters (@-peerbloomfilters@, off by default) or it whitelists that peer;+-- Bitcoin Cash Node serves bloom filters by default and answers anyone. So+-- on Bitcoin only the peers given with @--peer@ are asked, since an+-- operator whitelists those: asking the others, seeds and discovered peers,+-- only loses them, all at once, and with them the health of the store.+-- Asking every peer that connects, as the store once did, lost its public+-- Bitcoin Core peers one after the other for as long as it ran. A node+-- that the store has over RPC (@--node-rpc@) answers in any case+-- ('Haskoin.Store.Web.refillLoop'). blockStoreRefill :: BlockStore -> IO () blockStoreRefill self = do atomically $ writeTVar self.refill True
src/Haskoin/Store/Manager.hs view
@@ -396,6 +396,7 @@ pub = pub, db = db, net = cfg.net,+ explicitPeers = cfg.initPeers, onBest = maybe (\_ -> return ()) noteBest member, onTx = maybe (\_ _ -> return ()) memberReceived member, seen = maybe (\_ -> return mempty) memberSeen member,
swagger.yaml view
@@ -11,7 +11,7 @@ - url: './' info:- version: &version 1.16.6+ version: &version 1.16.7 title: Haskoin contact: name: JP Rupp
+ test/Haskoin/Store/BlockStoreSpec.hs view
@@ -0,0 +1,17 @@+-- | Which peers the block store asks for their mempools.+module Haskoin.Store.BlockStoreSpec (spec) where++import Haskoin (bch, btc)+import Haskoin.Store.BlockStore (asksMempool)+import Network.Socket (SockAddr (..), tupleToHostAddress)+import Test.Hspec++spec :: Spec+spec = describe "Asking for a mempool" $ do+ let local = SockAddrInet 8335 (tupleToHostAddress (127, 0, 0, 1))+ seed = SockAddrInet 8333 (tupleToHostAddress (192, 0, 2, 7))+ it "asks every peer on Bitcoin Cash, whose nodes answer anyone" $+ map (asksMempool bch []) [local, seed] `shouldBe` [True, True]+ it "asks only the peers given explicitly on Bitcoin, whose nodes disconnect the others" $ do+ map (asksMempool btc [local]) [local, seed] `shouldBe` [True, False]+ asksMempool btc [] local `shouldBe` False