diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,34 @@ +## 2.6.2 + + * Build the Bloom filters that `bloomFilter` asks for. The bits per key were + passed to `rocksdb_filterpolicy_create_bloom_full` as an integer, which is + what RocksDB took until 6.23; since 6.24 the function takes a `double`. An + integer and a `double` travel in different registers, so RocksDB read + something else as the bits per key and built no filters: table files had + `filter block size=0`, and a lookup of an absent key read a data block in + every file that could hold it. Nothing failed, and no test noticed. In a + measurement with 2,600,000 keys, such a lookup took about 5.5 microseconds + without the filters and 1.5 with them. + + The function is now imported with the `capi` calling convention, through a + stub compiled against `rocksdb/c.h`, so the C compiler passes the argument + as the installed RocksDB declares it, before or after 6.24. The headers + were already needed to install RocksDB for development; they are now + needed to build this package too. + + Databases opened with `bloomFilter` start building filters when they are + opened with this version: for the table files written from then on, at + about 1.25 bytes per key. Files written before have none until compaction + rewrites them. + * Pass the error pointer that `rocksdb_approximate_sizes` takes since RocksDB + 6.16. `approximateSize` called it with one argument too few, leaving RocksDB + to read a pointer that was never passed, and to write an error message + through it if the call failed. It now throws an `IOError` on failure. + * Both were found by comparing every foreign import with the `rocksdb/c.h` of + RocksDB 11.1: the other 70 match. + * Tests that table files have filters when asked for and not otherwise, and + of `approximateSize`, which had none. + ## 2.6.1 * Documentation, lots of it, with detailed explanations. diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -1,8 +1,13 @@ # rocksdb-haskell-jprupp -[![Hackage](https://img.shields.io/hackage/v/rocksdb-haskell-jprupp.svg)](https://hackage.haskell.org/package/rocksdb-haskell-jprupp) -[![License: BSD-3-Clause](https://img.shields.io/badge/license-BSD--3--Clause-blue.svg)](https://github.com/jprupp/rocksdb-haskell/blob/master/LICENSE) +[![Hackage][hackage-badge]][hackage] +[![License: BSD-3-Clause][license-badge]][license] +[hackage]: https://hackage.haskell.org/package/rocksdb-haskell-jprupp +[hackage-badge]: https://img.shields.io/hackage/v/rocksdb-haskell-jprupp.svg +[license]: https://github.com/jprupp/rocksdb-haskell/blob/master/LICENSE +[license-badge]: https://img.shields.io/badge/license-BSD--3--Clause-blue.svg + Haskell bindings to [RocksDB](https://rocksdb.org), the embedded persistent key-value store. The package on Hackage is `rocksdb-haskell-jprupp`; the module to import is `Database.RocksDB`. @@ -84,7 +89,8 @@ | Column families| Open a database with several independent key spaces; add new ones on open | | Snapshots | Consistent point-in-time reads across many queries and iterators | | Iterators | Ordered scans, forwards and backwards, optionally pinned to a snapshot | -| Tuning | Bloom filters, fixed-length prefix extractor, shared LRU block cache, memtable and WAL limits, open file limit | +| Tuning | Bloom filters, fixed-length prefix extractor, shared LRU block cache | +| Limits | Memtable and WAL size limits, open file limit | | Administration | Properties (memory use, statistics, SST files), approximate sizes, destroy, repair | ### What is not included @@ -120,6 +126,10 @@ non-standard place, pass its directories with `--extra-include-dirs` and `--extra-lib-dirs` (Cabal or Stack). +The headers are needed to build, not only the library: the binding of the +Bloom filter policy is compiled against `rocksdb/c.h`, because RocksDB changed +the type of its argument in 6.24 and the header says which one is installed. + ### Check the library is found ```sh @@ -171,7 +181,7 @@ ```yaml extra-deps: - - rocksdb-haskell-jprupp-2.6.0 + - rocksdb-haskell-jprupp-2.6.2 ``` With `stack --nix`, add RocksDB to the Nix packages: @@ -437,7 +447,11 @@ hundred megabytes to a few gigabytes, `cacheIndexAndFilterBlocks = True`, `maxFiles` of a few thousand, and `dbWriteBufferSize` of a few hundred megabytes. Enable `bloomFilter` when most reads are point lookups of keys that -may not exist. +may not exist. A filter takes about 1.25 bytes per key, held with the index +blocks: in the block cache with `cacheIndexAndFilterBlocks`, in memory outside +it otherwise. A filter belongs to a table file and is written with it, so +turning `bloomFilter` on for an existing database only covers the files +written from then on; the others get a filter when compaction rewrites them. For a large database, setting `maxFiles`, `blockCacheSize`, `cacheIndexAndFilterBlocks` and `dbWriteBufferSize` puts a ceiling on the memory @@ -454,7 +468,9 @@ | Opening | `withDB`, `withDBCF` | | Reading and writing | `get`, `getCF`, `multiGet`, `multiGetCF`, `put`, `putCF`, `delete`, `deleteCF`, `write` | | Snapshots | `withSnapshot`, `createSnapshot`, `releaseSnapshot` | -| Iteration | `withIter`, `withIterCF`, `withIterSnap`, `withIterSnapCF`, `iterSeek`, `iterFirst`, `iterLast`, `iterNext`, `iterPrev`, `iterKey`, `iterValue`, `iterEntry`, `iterValid`, `iterGetError` | +| Creating iterators | `withIter`, `withIterCF`, `withIterSnap`, `withIterSnapCF` | +| Moving iterators | `iterSeek`, `iterFirst`, `iterLast`, `iterNext`, `iterPrev` | +| Reading from iterators | `iterKey`, `iterValue`, `iterEntry`, `iterValid`, `iterGetError` | | Administration | `getProperty`, `getPropertyCF`, `approximateSize`, `destroy`, `repair` | The FFI bindings (`Database.RocksDB.C`) and helpers (`Database.RocksDB.Internal`) @@ -472,6 +488,16 @@ reached the end. Check `iterGetError`. - Calls into RocksDB are `safe` foreign calls, so they do not block other Haskell threads, but each one costs a little more than an `unsafe` call. +- Releases before 2.6.2 built no Bloom filters on RocksDB 6.24 or later, + whatever `bloomFilter` said: they passed the bits per key as an integer to a + function that had come to take a `double`. A database written by one of them + has no filters in its table files. After upgrading, files get them as they + are written, and the memory the filters need comes with them. The size of + the filters of a column family is in the `rocksdb.aggregated-table-properties` + property, as `filter block size`. +- The compiler does not check a foreign import against RocksDB's header. A + binding with the wrong type builds and runs, with garbage in place of the + argument; the two fixed in 2.6.2 went unnoticed for years. ## Development @@ -512,6 +538,10 @@ - Format new code with fourmolu, which the Nix shell provides. - Add tests for the behaviour you change. +- When you add or change a binding in `Database.RocksDB.C`, read the + prototype in `rocksdb/c.h` argument by argument: nothing else checks the + Haskell type against it. If the type of an argument differs between + versions of RocksDB, import the function with `capi`. - Describe the change in `CHANGELOG.md`. Versions follow the [Haskell Package Versioning Policy](https://pvp.haskell.org/). diff --git a/rocksdb-haskell-jprupp.cabal b/rocksdb-haskell-jprupp.cabal --- a/rocksdb-haskell-jprupp.cabal +++ b/rocksdb-haskell-jprupp.cabal @@ -5,7 +5,7 @@ -- see: https://github.com/sol/hpack name: rocksdb-haskell-jprupp -version: 2.6.1 +version: 2.6.2 synopsis: Haskell bindings for RocksDB description: See README at category: Database, FFI diff --git a/src/Database/RocksDB/Base.hs b/src/Database/RocksDB/Base.hs --- a/src/Database/RocksDB/Base.hs +++ b/src/Database/RocksDB/Base.hs @@ -337,7 +337,7 @@ -- | Approximate number of bytes of file system space used by the keys in a -- 'Range' of the default column family. Data still in memtables may not be --- counted. +-- counted. Throws an 'IOError' on failure. approximateSize :: DB -> Range -> IO Int64 approximateSize DB{rocksDB = db_ptr} (from, to) = BU.unsafeUseAsCStringLen from $ \(from_ptr, flen) -> @@ -346,15 +346,13 @@ withArray [intToCSize flen] $ \flen_ptrs -> withArray [to_ptr] $ \to_ptrs -> withArray [intToCSize tlen] $ \tlen_ptrs -> - allocaArray 1 $ \size_ptrs -> do - c_rocksdb_approximate_sizes db_ptr 1 - from_ptrs flen_ptrs - to_ptrs tlen_ptrs - size_ptrs - fmap head $ peekArray 1 size_ptrs >>= mapM toInt64 - - where - toInt64 = return . fromIntegral + alloca $ \size_ptr -> do + throwIfErr "approximate_sizes" $ + c_rocksdb_approximate_sizes db_ptr 1 + from_ptrs flen_ptrs + to_ptrs tlen_ptrs + size_ptr + fromIntegral <$> peek size_ptr -- | Write a key/value pair to the default column family, replacing any -- previous value. Throws an 'IOError' on failure. diff --git a/src/Database/RocksDB/C.hs b/src/Database/RocksDB/C.hs --- a/src/Database/RocksDB/C.hs +++ b/src/Database/RocksDB/C.hs @@ -1,3 +1,4 @@ +{-# LANGUAGE CApiFFI #-} {-# LANGUAGE CPP #-} {-# LANGUAGE EmptyDataDecls #-} {-# LANGUAGE ForeignFunctionInterface #-} @@ -21,6 +22,21 @@ -- for example when they wait for disk or for a compaction to make room, and a -- @safe@ call lets other Haskell threads keep running meanwhile. The small -- extra cost per call is worth that in a concurrent program. +-- +-- = Keeping the types in step with the header +-- +-- The compiler does not check a foreign import against @rocksdb\/c.h@: it +-- takes the Haskell type on trust. An argument of the wrong kind, or a +-- missing one, compiles, links and runs, and RocksDB then works with garbage +-- in its place. RocksDB has changed the types of functions that already +-- existed, twice among the ones bound here: the error pointer added to +-- @rocksdb_approximate_sizes@ in 6.16, and the bits per key of the Bloom +-- filter policies, an @int@ until 6.24 and a @double@ since. When adding or +-- reviewing a binding, read its prototype in the header, argument by +-- argument. Where a type differs between versions of RocksDB, import the +-- function with @capi@ instead of @ccall@, as +-- 'c_rocksdb_filterpolicy_create_bloom_full' is: the C compiler then reads +-- the header that is installed, and converts the argument. module Database.RocksDB.C ( RocksDB @@ -327,6 +343,11 @@ c_rocksdb_property_value_cf :: RocksDB -> ColumnFamily -> CString -> IO CString -- | Binding to @rocksdb_approximate_sizes@. +-- +-- The last argument receives an error, as in every function that can fail. +-- RocksDB added it in 6.16. A binding without it leaves the C function to +-- read a pointer that was never passed, and to write through it on failure. +-- Older versions do not take the argument, and ignore it. foreign import ccall safe "rocksdb/c.h rocksdb_approximate_sizes" c_rocksdb_approximate_sizes :: RocksDB -> CInt @@ -337,6 +358,7 @@ -- ^ range limit keys (array) -> Ptr Word64 -- ^ array of approx. sizes of ranges + -> ErrPtr -> IO () -- | Binding to @rocksdb_destroy_db@. @@ -532,7 +554,8 @@ -> IO () -- | Binding to @rocksdb_block_based_options_set_cache_index_and_filter_blocks_with_high_priority@. -foreign import ccall safe "rocksdb/c.h rocksdb_block_based_options_set_cache_index_and_filter_blocks_with_high_priority" +foreign import ccall safe + "rocksdb/c.h rocksdb_block_based_options_set_cache_index_and_filter_blocks_with_high_priority" c_rocksdb_block_based_options_set_cache_index_and_filter_blocks_with_high_priority :: BlockBasedOptions -> CUChar -> IO () @@ -582,8 +605,19 @@ c_rocksdb_filterpolicy_destroy :: FilterPolicy -> IO () -- | Binding to @rocksdb_filterpolicy_create_bloom_full@. -foreign import ccall safe "rocksdb/c.h rocksdb_filterpolicy_create_bloom_full" - c_rocksdb_filterpolicy_create_bloom_full :: CInt -> IO FilterPolicy +-- +-- The bits per key are a @double@ in C since RocksDB 6.24, and were an @int@ +-- before. Passed as the wrong one of the two, they travel in the wrong +-- register: the function reads whatever the other register holds, takes it +-- for the bits per key, and with too few of them builds no filter at all. +-- Nothing fails; lookups of absent keys are just as slow as without a filter. +-- +-- So this import alone uses the @capi@ calling convention. GHC then calls the +-- function through a stub that it compiles against @rocksdb\/c.h@, and the C +-- compiler converts the argument to the type that the installed RocksDB +-- declares, whichever it is. +foreign import capi safe "rocksdb/c.h rocksdb_filterpolicy_create_bloom_full" + c_rocksdb_filterpolicy_create_bloom_full :: CDouble -> IO FilterPolicy -- -- Read options diff --git a/src/Database/RocksDB/Internal.hs b/src/Database/RocksDB/Internal.hs --- a/src/Database/RocksDB/Internal.hs +++ b/src/Database/RocksDB/Internal.hs @@ -96,7 +96,8 @@ -- bloom filters and prefix seeks , bloomFilter :: !Bool -- ^ use a full bloom filter with 10 bits per key to - -- skip files that cannot contain a key + -- skip files that cannot contain a key; the filters + -- take about 1.25 bytes of memory per key , writeBufferSize :: !(Maybe Int) -- ^ bytes per memtable (RocksDB default 64 MiB); -- per column family diff --git a/test/Spec.hs b/test/Spec.hs --- a/test/Spec.hs +++ b/test/Spec.hs @@ -56,7 +56,10 @@ `shouldReturn` [Just "one50", Nothing, Just "one1", Just "one100", Nothing, Just "one50"] multiGet db [] `shouldReturn` [] ks <- forM [1 .. 2000 :: Int] $ \i -> return (C.pack (show (i `mod` 150))) - multiGet db ks >>= (`shouldBe` map (\k -> if read (C.unpack k) `elem` [1 .. 100 :: Int] then Just ("v" <> k) else Nothing) ks) + let expected k + | read (C.unpack k) `elem` [1 .. 100 :: Int] = Just ("v" <> k) + | otherwise = Nothing + multiGet db ks `shouldReturn` map expected ks describe "Multithreading" $ do it "stores and retrieve items from multiple threads" $ \db -> do let key i = C.pack $ printf "key_%04d" i @@ -264,6 +267,44 @@ getCF db two (key 20000) `shouldReturn` Just value files <- getProperty db SSTables files `shouldSatisfy` maybe False (not . C.null) + describe "Bloom filters" $ do + it "are built for a family that asks for one, and only for it" $ \_ -> + withSystemTempDirectory "rocksdb-tests-bloom" $ \path -> do + let open = def {createIfMissing = True, createMissingColumnFamilies = True} + families = [("with", def {bloomFilter = True}), ("without", def)] + key :: Int -> C.ByteString + key i = C.pack (printf "k%07d" i) + -- bytes of filter blocks in the table files of a family + filters d cf = + getPropertyCF d cf (PropertyName "rocksdb.aggregated-table-properties") + >>= maybe (fail "no table properties") (return . tableProperty "filter block size") + both d = case columnFamilies d of + [a, b] -> return (a, b) + _ -> fail "expected two column families" + withDBCF path open families $ \db -> do + (with, without) <- both db + forM_ [1 .. 5000] $ \i -> putCF db with (key i) "value" >> putCF db without (key i) "value" + -- opening the database again moves what was written from the + -- write-ahead log to table files, which is where filters are + withDBCF path open families $ \db -> do + (with, without) <- both db + -- ten bits for each of the 5000 keys is 6250 bytes + filters db with >>= (`shouldSatisfy` maybe False (> 5000)) + filters db without `shouldReturn` Just 0 + getCF db with (key 2500) `shouldReturn` Just "value" + getCF db with "absent" `shouldReturn` Nothing + describe "Approximate sizes" $ do + it "estimates the space used by a range of keys" $ \_ -> + withSystemTempDirectory "rocksdb-tests-sizes" $ \path -> do + let open = def {createIfMissing = True} + key :: Int -> C.ByteString + key i = C.pack (printf "k%07d" i) + withDB path open $ \db -> + forM_ [1 .. 5000] $ \i -> put db (key i) (C.pack (printf "%0100d" i)) + -- sizes are those of table files; see the test of Bloom filters + withDB path open $ \db -> do + approximateSize db (key 1, key 5000) >>= (`shouldSatisfy` (> 0)) + approximateSize db ("x", "y") `shouldReturn` 0 describe "Properties" $ do it "reports memory use, per database and per column family" $ \db -> do let one = head (columnFamilies db) @@ -321,3 +362,15 @@ let two = columnFamilies db !! 1 putCF db two "new" "cf" getCF db two "new" `shouldReturn` Just "cf" + +-- | A number in the text of the @rocksdb.aggregated-table-properties@ +-- property, which reads @name=value; name=value; ...@. +tableProperty :: C.ByteString -> C.ByteString -> Maybe Int +tableProperty name text = + listToMaybe + [ n + | field <- C.split ';' text, + let (k, v) = C.break (== '=') (C.dropWhile (== ' ') field), + k == name, + Just (n, _) <- [C.readInt (C.drop 1 v)] + ]