packages feed

haskoin-store-1.16.0: src/Haskoin/Store/Database/Merge.hs

-- |
-- Module      : Haskoin.Store.Database.Merge
-- Copyright   : (c) Haskoin Developers
-- License     : MIT
-- Maintainer  : jprupp@protonmail.ch
-- Stability   : experimental
-- Portability : POSIX
--
-- The merge operators of the database: how RocksDB works out an address
-- balance and a group of spenders from the merge records that the import
-- of a block writes.
--
-- = The problem
--
-- A block changes the balance of every address that it pays or spends
-- from, and adds a spender to the group of every output that it spends.
-- Each was read, changed and written back. On a store of Bitcoin mainnet
-- those two kinds of read were most of the reads of a block: 300 to 850 ms
-- for the groups of spenders and 70 to 300 for the balances, of a read
-- ahead of 640 to 1,440 ms. And they were read only to be changed.
--
-- = Merge records
--
-- RocksDB has a third kind of write besides a put and a delete: a merge
-- record, which says how a value changes and is written without reading
-- anything. The /merge operator/ of the column family works out the value
-- later, from the value that the key had and the records written since:
-- when the key is read, and when a compaction brings them together, after
-- which the key holds a plain value again.
--
-- So the import of a block writes, for each address, the differences to
-- its balance, and for each group of outputs, the spenders to add
-- ('Haskoin.Store.Database.Writer.mergeBalance',
-- 'Haskoin.Store.Database.Writer.mergeSpender'), and reads neither.
--
-- = The two operators
--
-- They are C functions, in @cbits\/haskoin_store_merge.c@, because RocksDB
-- calls them from its own threads. A merge record has the format of a
-- value in both, so that applying records to a value and combining records
-- with each other are one operation:
--
-- * __Balances__ ('balanceOperator'): a record is five differences in the
--   forty bytes of a 'Haskoin.Store.Database.Types.BalVal', and merging
--   adds each of the five numbers modulo 2^64. A key without a value
--   counts from zero.
--
-- * __Groups of spenders__ ('spenderOperator'): a record is a
--   t'Haskoin.Store.Database.Types.SpenderVal' with the entries to add,
--   and merging is the union, the newer entry winning for an output that
--   two have.
--
-- = What it asks
--
-- * __Every program that opens the database sets the operators__: they are
--   in the configuration of the two column families
--   ('Haskoin.Store.Database.Reader.columnFamilyConfig'), so nothing in
--   this package can open it without. RocksDB itself does not refuse to:
--   it then drops the merge records that are still in its write-ahead log,
--   and cannot read the others. That is why a database that this version
--   has opened has a column family that older versions do not know of, and
--   why they can no longer open it
--   ('Haskoin.Store.Database.Reader.layeredFamilyConfig').
--
-- * __Readers pay for the merge.__ A read of a key with merge records
--   pending looks into every table file that has one of them, down to the
--   value, and merges. @haskoin-store-bench merge@ measures it, and
--   @docs\/one-fleet.md@ has the figures.
--
-- * __Reverts read.__ Only the import of a block writes merge records.
--   Taking a block out of the chain, and everything else that changes a
--   balance or a group of spenders, reads the value and writes it whole,
--   or deletes it: a put or a delete hides the merge records under it.
module Haskoin.Store.Database.Merge
  ( balanceOperator,
    spenderOperator,
  )
where

import Database.RocksDB (FullMerge, MergeOperator (..), PartialMerge)
import Foreign.Ptr (FunPtr)

foreign import ccall "&haskoin_store_balance_full_merge" balanceFullMerge :: FunPtr FullMerge

foreign import ccall "&haskoin_store_balance_partial_merge" balancePartialMerge :: FunPtr PartialMerge

foreign import ccall "&haskoin_store_spender_full_merge" spenderFullMerge :: FunPtr FullMerge

foreign import ccall "&haskoin_store_spender_partial_merge" spenderPartialMerge :: FunPtr PartialMerge

-- | The merge operator of the column family of address balances. A merge
-- record is a 'Haskoin.Store.Database.Types.BalVal' of five differences,
-- each added to its number modulo 2^64, so that a number goes down by
-- adding its two's complement. A balance that is not there counts from
-- zero.
balanceOperator :: MergeOperator
balanceOperator =
  MergeFunctions
    { mergeName = "haskoin-store-balance-1",
      fullMerge = balanceFullMerge,
      partialMerge = balancePartialMerge
    }

-- | The merge operator of the column family of the spenders of outputs. A
-- merge record is a t'Haskoin.Store.Database.Types.SpenderVal' with the
-- spenders to add to a group, and merging is the union of the group and
-- the records, the newest entry winning for an output that several have.
spenderOperator :: MergeOperator
spenderOperator =
  MergeFunctions
    { mergeName = "haskoin-store-spender-1",
      fullMerge = spenderFullMerge,
      partialMerge = spenderPartialMerge
    }