diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,15 @@ +## 2.7.1 + + * Documentation and tests only; the library behaves as in 2.7.0. + * The documentation of `prefixSameAsStart` was wrong on one point and silent + on another. It said that the option does nothing after `iterFirst` or + `iterLast`: it does, the iterator stays within the prefix of the key that + it finds there. And it did not say that every key, sought or stored, has + to be at least as long as the `prefixLength`: RocksDB takes the prefix of + a key without checking its length, and reads past the end of a shorter + one. Both are now in the Haddock and in the README, and the first has + tests. + ## 2.7.0 * Bounded iterators. `withIterConfig` creates an iterator that RocksDB itself diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -185,7 +185,7 @@ ```yaml extra-deps: - - rocksdb-haskell-jprupp-2.7.0 + - rocksdb-haskell-jprupp-2.7.1 ``` With `stack --nix`, add RocksDB to the Nix packages: @@ -441,7 +441,9 @@ after seek, which saves creating an iterator for each: the bound moves with every `iterSeek`. It needs a `prefixLength` in the `Config`, and each range has to be a whole prefix of that length. It also lets RocksDB use its - prefix filters to skip the files that have nothing for the prefix. + prefix filters to skip the files that have nothing for the prefix. Every + key, sought or stored, has to be at least as long as the prefix: see + [Things to watch out for](#things-to-watch-out-for). - **`totalOrderSeek`** to walk a database that has a `prefixLength` from end to end. Without it, an iterator there only promises the keys that share the prefix of the key it was sent to. @@ -561,6 +563,15 @@ prefix it was sent to, or that starts with `iterFirst` or `iterLast`, may skip keys. Ask for `totalOrderSeek`, or stay within the prefix with `prefixSameAsStart`. +- With `prefixSameAsStart`, no key may be shorter than the `prefixLength`: + not the keys you seek, and not the keys in the database. RocksDB takes the + prefix of a key without checking its length and reads the memory that + follows a shorter one, so the iterator stops at an arbitrary place. The + binding cannot check this, because an iterator does not know how it was + made. Where keys can be shorter, use an `upperBound`. +- With `prefixSameAsStart`, `iterFirst` and `iterLast` give the first and + the last prefix of the database, and no more: the iterator stays within + the prefix of the key it finds. - Destroy an iterator from `createIteratorConfig` with the action that came with it, not with `destroyIterator`, which would leave its bounds and read options allocated. diff --git a/rocksdb-haskell-jprupp.cabal b/rocksdb-haskell-jprupp.cabal --- a/rocksdb-haskell-jprupp.cabal +++ b/rocksdb-haskell-jprupp.cabal @@ -5,7 +5,7 @@ -- see: https://github.com/sol/hpack name: rocksdb-haskell-jprupp -version: 2.7.0 +version: 2.7.1 synopsis: Haskell bindings for RocksDB description: See README at category: Database, FFI diff --git a/src/Database/RocksDB/Iterator.hs b/src/Database/RocksDB/Iterator.hs --- a/src/Database/RocksDB/Iterator.hs +++ b/src/Database/RocksDB/Iterator.hs @@ -55,7 +55,8 @@ -- -- * at an 'upperBound' or a 'lowerBound', which are keys; -- * at the end of the prefix that it was sent to, with 'prefixSameAsStart', --- in a database or column family that has a 'prefixLength'. +-- in a database or column family that has a 'prefixLength' and no key +-- shorter than it. -- -- Such an iterator becomes invalid at its bound without looking past it. A -- bound also lets RocksDB leave out the table files that hold nothing inside @@ -211,7 +212,8 @@ -- configuration of the database and works with keys of any shape. -- * 'prefixSameAsStart' when one iterator is sent to many ranges, one seek -- after another: the bound moves with each seek. It needs a 'prefixLength', --- and the ranges have to be whole prefixes of that length. +-- the ranges have to be whole prefixes of that length, and no key may be +-- shorter than it. -- * 'totalOrderSeek' when a database with a 'prefixLength' has to be walked -- from end to end. data IterConfig = IterConfig @@ -226,12 +228,21 @@ -- 'prefixUpperBound' gives the bound that ends a prefix. , prefixSameAsStart :: !Bool -- ^ stay within the prefix of the key that the iterator was last sent - -- to with 'iterSeek': the iterator becomes invalid instead of moving - -- to a key with another prefix. The prefix is the first - -- 'Database.RocksDB.prefixLength' bytes of a key. It has no effect in - -- a database or column family without a prefix length, nor together - -- with 'totalOrderSeek', nor after 'iterFirst' or 'iterLast', which - -- name no prefix. + -- to: it becomes invalid instead of moving to a key with another + -- prefix. The prefix is the first 'Database.RocksDB.prefixLength' + -- bytes of a key. After 'iterSeek' it is the prefix of the key + -- sought; after 'iterFirst' or 'iterLast', that of the key found, so + -- that those two give the first or the last prefix of the database + -- and no more. It has no effect in a database or column family + -- without a prefix length, nor together with 'totalOrderSeek'. + -- + -- __Every key has to be at least as long as the prefix__: the keys + -- sought, and the keys stored. RocksDB takes the prefix of a key + -- without looking at its length, so with a shorter key it reads the + -- bytes that follow it in memory, and where the iterator stops is + -- then anyone's guess. This binding cannot check it for you, because + -- an 'Iterator' does not know how it was made. If keys can be + -- shorter, use an 'upperBound', which has no such condition. , totalOrderSeek :: !Bool -- ^ see every key in order, whatever its prefix, in a database or -- column family that has a 'Database.RocksDB.prefixLength'. There, a diff --git a/test/Spec.hs b/test/Spec.hs --- a/test/Spec.hs +++ b/test/Spec.hs @@ -250,6 +250,16 @@ from "aaa2" itr `shouldReturn` ["aaa2"] from "ccc" itr `shouldReturn` ["ccc1"] from "abc" itr `shouldReturn` [] + it "stays within the prefix of the key it finds at either end" $ \db -> do + fill db + withIterConfig db Nothing def {prefixSameAsStart = True} $ \itr -> do + map fst <$> forwards itr `shouldReturn` ["aaa1", "aaa2"] + map fst <$> backwards itr `shouldReturn` ["ccc1"] + it "stays within the prefix going backwards" $ \db -> do + fill db + withIterConfig db Nothing def {prefixSameAsStart = True} $ \itr -> do + iterSeek itr "bbb2" + map fst <$> scan iterPrev itr `shouldReturn` ["bbb2", "bbb1"] it "stays within the prefix when the data is in table files" $ \db -> do fill db -- enough writes to flush the keys out of memory, where the prefix