memcache-0.5.0.0: Database/Memcache/Client.hs
{-# LANGUAGE OverloadedStrings #-}
{-|
Module : Database.Memcache.Client
Description : Memcached Client
Copyright : (c) David Terei, 2016
License : BSD
Maintainer : code@davidterei.com
Stability : stable
Portability : GHC
A Memcached client using the Meta (text) protocol. Memcached is an in-memory
key-value store typically used as a distributed and shared cache. Clients can
connect to a single Memcached server or a cluster of servers; clusters route
each key to a server by ketama consistent hashing, compatible with
libmemcached (see "Database.Memcache.Cluster").
This requires memcached 1.6.10 or newer. Older servers reject the meta set
command this client stores values with, without reading the value that
follows it, and then run the value's bytes as commands. So each new connection
asks the server for its version before it is used: until that check passes,
the only requests sent are the authentication exchange, when credentials are
set, and @version@. Unless the server reports 1.6.10 or later, the connection
is closed and v'ProtocolError' 'UnknownOp' is thrown, quoting the server's
reply; an error reply, such as @CLIENT_ERROR unauthenticated@, throws what it
throws for any request. A refused version isn't retried, and doesn't mark the
server as failed. Through a proxy, the check sees only the proxy's version, so
every memcached behind it must be 1.6.10 or later too.
Authentication uses Memcached's experimental text-protocol auth-file mechanism
(@-Y authfile@), introduced in memcached 1.5.15. Enabling @-Y@ also disables
binary and UDP protocols on the server.
Expected return values (like misses) are returned as part of the return type,
while unexpected errors are thrown as exceptions. Exceptions are either of type
'MemcacheError' or an 'IO' exception thrown by the network.
We support the following logic for handling failure in operations:
* __Timeouts__: we time out any attempt at an operation that takes longer
than 'optsServerTimeout', including time spent waiting for a
free connection to the server, and consider it failed.
* __Retry__: on operation failure (a timeout, a network error, or a
@SERVER_ERROR@ reply whose text names no status, such as
memcached's proxy sends when a backend fails) we close the
connection, if any, and retry the operation, doing this up to
'optsServerRetries' times. Unless the attempt only ran out of time
waiting for a free connection, we also close the server's idle
connections, as what made it fail, such as a restart of the
server, may have broken them too. A reply to an operation's
request that shows the connection is out of step also closes
it. Only 'getMany', 'setMany' and 'deleteMany' retry then, and
close the idle connections too; other operations throw at once,
as the server may have carried out the request.
A request that mustn't be sent twice is retried only after a
failure before any of it was sent, such as a refused
connection or a timeout while waiting for a free connection.
That is 'increment', 'decrement', 'append', 'prepend', 'add',
'delete', and 'cas' and 'replace' with a version other than
@0@. Once such a request may have reached the server, a
failure is thrown, and the server is marked dead as if every
attempt had failed: the server may have carried out the
request, and a retry could apply it twice, or report failure
because the first one succeeded. Before sending one on a
connection from the pool, we check that the server hasn't
closed or reset it, as memcached closes idle connections when
it restarts or after @-o idle_timeout@, and that no bytes from
the server are already waiting on it. If the check fails, the
request is retried on a new connection if retries are left. With
'optsServerRetries' at @0@, that failure is thrown instead, and
the server is marked dead. Other operations are retried after
any server failure, so the server may carry out their request
twice (see 'optsServerRetries'). Set 'optsServerRetries' to @0@
for no retries.
* __Failover__: when an operation against a server in a cluster fails on every
attempt, we mark that server as dead and use the remaining
servers in the cluster to handle all operations: with the
default router, its keys go to the next live server on the
hash ring, and other keys stay where they are. That doesn't
happen when the last attempt only ran out of time waiting for
a free connection, as it never reached the server. After a
configurable period of time has passed, we consider the server
alive again and try to use it. This can lead to consistency
issues (stale data), but is usually fine for caching purposes
and is the common approach in Memcached clients.
Some of this behavior can be configured through the 'Options' data type. We
also have the following concepts exposed by Memcached:
[@version@] Each value has a 'Version' associated with it. This is simply a
numeric, monotonically increasing value. The version field
allows for a primitive version of 'cas' to be implemented.
Operations that take a 'Version' only change a value whose
version matches it, except that a 'Version' of @0@ skips the
check.
[@expiration@] Each value pair has an 'Expiration' associated with it. Once a
a value expires, it will no longer be returned from the cache
until a new value for that key is set. Expirations come in two
forms, the first form interprets the expiration value as the
number of seconds in the future at which the value should be
considered expired. For example, an expiration of @3600@
expires the value in 1 hour. When the value of the expiration
is greater than 30 days however (@2592000@), the expiration
field is instead interpreted as a UNIX timestamp (the number
of seconds since epoch). The timestamp specifies the date at
which the value should expire.
[@flags@] Each value can have a small amount of fixed metadata associated
with it beyond the value itself, these are the 'Flags'.
Usage is roughly as follows:
> {-# LANGUAGE OverloadedStrings #-}
>
> module Main (main) where
>
> import qualified Database.Memcache.Client as M
>
> main :: IO ()
> main = do
> -- use default values: connects to 127.0.0.1:11211
> mc <- M.newClient [M.def] M.def
>
> -- store and then retrieve a key-value pair
> _ <- M.set mc "key" "value" 0 0
> v' <- M.get mc "key"
> case v' of
> Nothing -> putStrLn "Miss!"
> Just (v, _, _) -> putStrLn $ "Hit: " <> show v
-}
module Database.Memcache.Client (
-- * Client creation
newClient, Client, ServerSpec(..), defaultServerSpec, parseServerSpec,
Options(..), defaultOptions, Authentication(..), Username, Password,
-- | 'def' from "Data.Default" gives the same values as
-- 'defaultServerSpec', 'defaultOptions' and 'defaultServerOptions',
-- picked by the type it is used at.
def,
quit, withClient,
-- ** Configuration types
Retries, ServerOptions(..), defaultServerOptions, Server,
-- * Fields & Values
Key, Value, Initial, Delta, Expiration, Flags, Version,
-- * Operations
-- ** Get operations
get, getMany, gat, touch,
-- ** Set operations
set, setMany, cas, add, replace,
-- ** Modify operations
increment, decrement, append, prepend,
-- ** Delete operations
delete, deleteMany, flush,
-- ** Information operations
StatResults, stats, version,
-- * Errors
MemcacheError(..), Status(..), ClientError(..), ProtocolError(..)
) where
import Database.Memcache.Cluster
import Database.Memcache.Errors
import Database.Memcache.Server
import Database.Memcache.Socket
import Database.Memcache.Types
import Control.Exception (uninterruptibleMask_)
import Control.Monad (foldM, forM_, when)
import Data.ByteString (ByteString)
import qualified Data.ByteString as B (any, empty, length, null, stripSuffix)
import qualified Data.ByteString.Char8 as C (split, unwords)
import Data.Default
import Data.Maybe (fromMaybe, listToMaybe)
import Data.Word
import UnliftIO.Exception (Exception, bracket, handle, throwIO)
-- | A Memcached cluster client.
type Client = Cluster
-- | Key/value pairs returned by 'stats'.
type StatResults = [(ByteString, ByteString)]
-- | Create a client for the supplied servers. An empty list uses
-- 'defaultServerSpec'.
--
-- Create one client and share it for the life of your program. It is safe to
-- use from many threads, and it keeps a pool of connections to each server,
-- opened on first use and reused across requests. A client created for each
-- request instead opens (and, with credentials, authenticates) a new
-- connection to every server it uses, every time. Each pool holds as many
-- connections as 'soNumResources' and 'soNumStripes' allow, one by default
-- (with resource-pool 0.3, one for each capability), so concurrent requests to
-- a server beyond that wait for a free connection. The wait counts towards
-- each attempt's 'optsServerTimeout'. If the last attempt runs out of time
-- while still waiting, the server isn't marked down; running out of time
-- afterwards is a server failure, however long the wait was.
--
-- Nothing is sent until an operation needs a connection to a server. Each new
-- connection then checks the server's version, and refuses a server older than
-- memcached 1.6.10: until the check passes, the only requests sent are the
-- authentication exchange, when credentials are set, and @version@.
newClient :: [ServerSpec] -> Options -> IO Client
newClient scs = newCluster $ if null scs then [defaultServerSpec] else scs
-- | Close the client's connections to every server, including servers marked
-- dead. Idle connections are closed straight away. A connection that another
-- thread is using at the time is closed as soon as the request using it
-- finishes, instead of going back to the pool; 'quit' doesn't wait for those
-- requests or interrupt them. Requests that start after 'quit', including
-- later requests of an operation that was already running, never use those
-- connections: they open new ones, which stay open as usual. So does a request
-- that is still waiting for a connection, or opening one, when 'quit' runs.
--
-- This doesn't send Memcached's @quit@ command: closing the connection has the
-- same effect, while sending the command would mean first connecting to any
-- server the client has no connection to. So 'quit' never connects to a
-- server, returns promptly and doesn't throw, even when a server is down.
-- Like 'closeAll', it can't be cut short by an asynchronous exception: one
-- that arrives while it closes connections is delivered once every server's
-- idle connections are closed.
quit :: Client -> IO ()
quit c = getServers c >>= uninterruptibleMask_ . mapM_ closeAll
-- | Create a client with 'newClient', run an action with it, and close the
-- client's connections with 'quit' when the action returns or throws.
--
-- Use it to scope one long-lived client to your program, a worker or a test,
-- and share that client between threads. Don't wrap individual requests in
-- it: every new client opens (and, with credentials, authenticates) its own
-- connection to each server it uses, so a client per request pays that cost
-- on every request.
--
-- 'newClient' doesn't connect to any server (connections are opened on first
-- use), so if it throws there is nothing to close. 'quit' doesn't throw, so
-- 'withClient' returns the action's result, or rethrows its exception,
-- unchanged.
--
-- Don't use the client after 'withClient' returns, or from threads that
-- outlive the action. A request such a thread is making when the action ends
-- completes. If it already has a connection, that connection is then closed.
-- If it is still waiting for a connection, or opening one, it opens a new one,
-- which, like those of the requests the thread starts afterwards, stays open
-- until it has been idle for 'soKeepAlive'.
withClient :: [ServerSpec] -> Options -> (Client -> IO a) -> IO a
withClient specs opts = bracket (newClient specs opts) quit
-- | Retrieve the value for the given key from Memcached, along with its
-- flags and CAS version. Returns 'Nothing' on a miss.
get :: Client -> Key -> IO (Maybe (Value, Flags, Version))
get c k = do
response <- keyedOpExpecting c "GET" getReplies k getRequest
status <- responseStatus response
case (responseCode response, status) of
("VA", NoError) -> return $ Just (valueResult response)
(_, ErrKeyNotFound) -> return Nothing
_ -> failResponse response "GET"
-- | Fetch many keys. Missing keys are omitted from the result.
getMany :: Client -> [Key] -> IO [(Key, (Value, Flags, Version))]
getMany c keys = do
replies <- keyedBatchOp c [(key, getManyRequest) | key <- keys]
reverse <$> foldM collect [] replies
where
collect values (key, response) = do
status <- responseStatus response
case (responseCode response, status) of
("VA", NoError) ->
return ((key, valueResult response) : values)
(_, ErrKeyNotFound) -> return values
_ -> failResponse response "GETMANY"
-- | Get-and-touch: Retrieve the value for the given key from Memcached, and
-- also update the stored key-value pairs expiration time at the server. Use an
-- expiration value of @0@ to store forever.
gat :: Client -> Key -> Expiration -> IO (Maybe (Value, Flags, Version))
gat c k e = do
response <- keyedOpExpecting c "GAT" getReplies k (`gatRequest` e)
status <- responseStatus response
case (responseCode response, status) of
("VA", NoError) -> return $ Just (valueResult response)
(_, ErrKeyNotFound) -> return Nothing
_ -> failResponse response "GAT"
-- | Update the expiration time of a stored key-value pair, returning its
-- new CAS version or 'Nothing'.
-- Use an expiration value of @0@ to store forever.
touch :: Client -> Key -> Expiration -> IO (Maybe Version)
touch c k e = do
response <- keyedOpExpecting c "TOUCH" touchReplies k (`touchRequest` e)
status <- responseStatus response
case status of
NoError | responseCode response == "HD" -> return $ Just (responseCas response)
ErrKeyNotFound -> return Nothing
_ -> failResponse response "TOUCH"
-- | Store a new (or overwrite existing) key-value pair, returning its @Version@
-- identifier. Use an expiration value of @0@ to store forever.
--
-- A value larger than 1 GiB, the largest item any memcached server can store,
-- throws 'OpError' 'ErrValueTooLarge' before anything is sent, as it does in
-- every operation that stores a value.
set :: Client -> Key -> Value -> Flags -> Expiration -> IO Version
set c k v f e = store c k v f e 0 Set "SET"
-- | Store many key-value pairs, as 'set' does, returning the new @Version@ of
-- each. Like 'getMany', the requests to each server are pipelined, and the
-- results are grouped by server rather than in the order given; within a
-- server they keep the input order. A key given more than once is stored
-- once per occurrence, in order, so its last entry in the result is the
-- version now stored. Use an expiration value of @0@ to store forever.
--
-- Every key, and the length of every value, is checked before anything is
-- sent, so an invalid key or a value larger than 1 GiB stores nothing. Every
-- value is therefore evaluated before the first request is sent. Each
-- server's pairs are sent in batches of up to 256 that close early once their
-- requests reach 1 MiB (a larger request goes alone), and each batch, like a
-- 'set', must finish within 'optsServerTimeout'. When a server fails
-- part-way through, the requests it was handling are sent again, storing the
-- same values; the versions returned are from the retry. If this throws,
-- some of the pairs may have been stored.
setMany :: Client -> [(Key, Value, Flags, Expiration)] -> IO [(Key, Version)]
setMany c items = do
mapM_ (\(_, v, _, _) -> checkValue v) items
replies <- keyedBatchOp c [(k, \wire -> setManyRequest wire v f e) | (k, v, f, e) <- items]
mapM stored replies
where
stored (key, response) = do
status <- responseStatus response
case status of
NoError -> return (key, responseCas response)
_ -> failResponse response "SETMANY"
-- | Store a key-value pair, but only if the version specified by the client
-- matches the @Version@ of the key-value pair at the server. The version
-- identifier of the stored key-value pair is returned, or if the key doesn't
-- exist or the version match fails, @Nothing@ is returned. Use an expiration
-- value of @0@ to store forever.
--
-- A version of @0@ skips the check, so the value is stored unconditionally, as
-- with 'set'.
cas :: Client -> Key -> Value -> Flags -> Expiration -> Version -> IO (Maybe Version)
cas c k v f e ver = do
checkValue v
response <- resendUnlessVersioned ver c "SET" storeReplies k (\wire -> storeRequest Set wire v f e ver)
status <- responseStatus response
case status of
NoError -> return $ Just (responseCas response)
ErrKeyNotFound -> return Nothing
ErrKeyExists -> return Nothing
_ -> failResponse response "SET"
-- | Store a new key-value pair, returning its @Version@ identifier. If the
-- key-value pair already exists, then fail (return 'Nothing'). Use an
-- expiration value of @0@ to store forever.
add :: Client -> Key -> Value -> Flags -> Expiration -> IO (Maybe Version)
add c k v f e = do
checkValue v
response <- keyedOpExpectingOnce c "ADD" storeReplies k (\wire -> storeRequest Add wire v f e 0)
status <- responseStatus response
case status of
NoError -> return $ Just (responseCas response)
ErrItemNotStored -> return Nothing
_ -> failResponse response "ADD"
-- | Update the value of an existing key-value pair, returning its new @Version@
-- identifier. If the key doesn't already exist, then fail and return 'Nothing'.
-- Use an expiration value of @0@ to store forever.
--
-- A non-zero version must match the @Version@ of the key-value pair at the
-- server, otherwise 'Nothing' is returned. A version of @0@ skips the check.
replace :: Client -> Key -> Value -> Flags -> Expiration -> Version -> IO (Maybe Version)
replace c k v f e ver = do
checkValue v
response <- resendUnlessVersioned ver c "REPLACE" storeReplies k (\wire -> storeRequest Replace wire v f e ver)
status <- responseStatus response
case status of
NoError -> return $ Just (responseCas response)
ErrKeyNotFound -> return Nothing
ErrKeyExists -> return Nothing
ErrItemNotStored -> return Nothing
_ -> failResponse response "REPLACE"
store :: Cluster -> Key -> Value -> Flags -> Expiration -> Version -> StoreMode -> String -> IO Version
store c k v f e ver mode operation = do
checkValue v
response <- keyedOpExpecting c operation storeReplies k (\wire -> storeRequest mode wire v f e ver)
status <- responseStatus response
case status of
NoError -> return $ responseCas response
_ -> failResponse response operation
-- | 1 GiB, memcached's @ITEM_SIZE_MAX_UPPER_LIMIT@. No memcached server can
-- store a larger item, so none can store a longer value.
maxValueLength :: Int
maxValueLength = 1073741824
-- | Reject a value longer than @maxValueLength@ before anything is sent.
-- memcached can't store it, and one longer than 2147483645 bytes would make
-- it run the rest of the value's bytes as commands. This evaluates the value
-- before a connection is taken.
checkValue :: Value -> IO ()
checkValue value = when (B.length value > maxValueLength) $ throwStatus ErrValueTooLarge
-- | Increment a numeric value stored against a key, returning the incremented
-- value and the @Version@ identifier of the key-value pair. Throws
-- 'OpError' 'ErrValueNonNumeric' if the stored value isn't a number.
--
-- If the key doesn't exist, it is created with the 'Initial' value, which is
-- returned without the delta applied. The expiration is only used for a key
-- created this way, and doesn't change the expiration of an existing key; use
-- @0@ for a created key that never expires. To leave a missing key missing
-- instead, pass an expiration of 'maxBound': nothing is created and 'Nothing'
-- is returned.
--
-- A non-zero version must match the @Version@ of an existing key-value pair,
-- otherwise 'Nothing' is returned. A version of @0@ skips the check.
increment :: Client -> Key -> Initial -> Delta -> Expiration -> Version -> IO (Maybe (Word64, Version))
increment c k i d e ver = arithmetic c k i d e ver Incr "INCREMENT"
-- | Decrement a numeric value stored against a key, returning the decremented
-- value and the @Version@ identifier of the key-value pair. Memcached doesn't
-- decrement below @0@.
--
-- Otherwise this behaves like 'increment': a missing key is created with the
-- 'Initial' value unless the expiration is 'maxBound', a version of @0@ skips
-- the check, and a non-numeric value throws 'OpError' 'ErrValueNonNumeric'.
decrement :: Client -> Key -> Initial -> Delta -> Expiration -> Version -> IO (Maybe (Word64, Version))
decrement c k i d e ver = arithmetic c k i d e ver Decr "DECREMENT"
arithmetic :: Cluster -> Key -> Initial -> Delta -> Expiration -> Version -> ArithMode -> String -> IO (Maybe (Word64, Version))
arithmetic c k i d e ver mode operation = do
response <- keyedOpExpectingOnce c operation arithmeticReplies k (\wire -> arithmeticRequest mode wire i d e ver)
status <- responseStatus response
case status of
NoError -> return $ Just (responseNumber response, responseCas response)
ErrKeyNotFound -> return Nothing
ErrKeyExists -> return Nothing
ErrItemNotStored -> return Nothing
_ -> failResponse response operation
-- | Append a value to an existing key-value pair, returning the new @Version@
-- identifier of the key-value pair when successful, or 'Nothing' if the key
-- doesn't exist.
--
-- A non-zero version must match the @Version@ of the key-value pair at the
-- server, otherwise 'Nothing' is returned. A version of @0@ skips the check.
--
-- memcached 1.6.10 to 1.6.12 return the version given instead of the new one,
-- so @0@ if the check was skipped. Passing that @0@ back as a version skips
-- the check.
append :: Client -> Key -> Value -> Version -> IO (Maybe Version)
append c k v ver = modify c k v ver Append "APPEND"
-- | Prepend a value to an existing key-value pair, returning the new @Version@
-- identifier of the key-value pair when successful, or 'Nothing' if the key
-- doesn't exist.
--
-- A non-zero version must match the @Version@ of the key-value pair at the
-- server, otherwise 'Nothing' is returned. A version of @0@ skips the check.
--
-- memcached 1.6.10 to 1.6.12 return the version given instead of the new one,
-- so @0@ if the check was skipped. Passing that @0@ back as a version skips
-- the check.
prepend :: Client -> Key -> Value -> Version -> IO (Maybe Version)
prepend c k v ver = modify c k v ver Prepend "PREPEND"
modify :: Cluster -> Key -> Value -> Version -> StoreMode -> String -> IO (Maybe Version)
modify c k v ver mode operation = do
checkValue v
response <- keyedOpExpectingOnce c operation storeReplies k (\wire -> modifyRequest mode wire v ver)
status <- responseStatus response
case status of
NoError -> return $ Just (responseCas response)
ErrKeyNotFound -> return Nothing
ErrKeyExists -> return Nothing
ErrItemNotStored -> return Nothing
_ -> failResponse response operation
-- | Delete a key, returning whether it was deleted.
--
-- A non-zero version must match the @Version@ of the key-value pair at the
-- server, otherwise the key isn't deleted and 'False' is returned. A version
-- of @0@ deletes the key unconditionally.
delete :: Client -> Key -> Version -> IO Bool
delete c k ver = do
response <- keyedOpExpectingOnce c "DELETE" deleteReplies k (`deleteRequest` ver)
status <- responseStatus response
case status of
NoError -> return True
ErrKeyNotFound -> return False
ErrKeyExists -> return False
_ -> failResponse response "DELETE"
-- | Delete many keys, pipelining the requests to each server as 'getMany'
-- does. The deletes are unconditional, like 'delete' with a version of @0@.
--
-- This returns once each key has been deleted or found missing, without
-- saying which: when a server fails part-way through, the requests it was
-- handling are sent again, and the retry finds the keys the first attempt
-- deleted missing, so a per-key answer would be wrong for them. Use 'delete'
-- to learn whether a key was present: it never sends a request again once
-- the server may have carried it out, and throws instead.
--
-- Every key is checked before anything is sent, so an invalid key deletes
-- nothing. If this throws, some of the keys may have been deleted.
deleteMany :: Client -> [Key] -> IO ()
deleteMany c keys = do
replies <- keyedBatchOp c [(key, deleteManyRequest) | key <- keys]
forM_ replies $ \(_, response) -> do
status <- responseStatus response
case status of
NoError -> return ()
ErrKeyNotFound -> return ()
_ -> failResponse response "DELETEMANY"
-- | Remove (delete) all currently stored key-value pairs from the cluster. The
-- expiration value can be used to cause this flush to occur in the future
-- rather than immediately. The servers are flushed in turn, and one that
-- fails, or gives a reply other than @OK@, throws and leaves the servers
-- after it unflushed.
flush :: Client -> Maybe Expiration -> IO ()
flush c e = do
results <- allOpExpecting c "FLUSH" ["OK"] (flushRequest e)
forM_ results $ \(_, response) -> do
status <- responseStatus response
when (status /= NoError) $ failResponse response "FLUSH"
-- | Return statistics on the stored key-value pairs at every available server
-- in the cluster. The optional key can be used to select a different set of
-- statistics from the server than the default. Most Memcached servers support
-- @"items"@, @"slabs"@ or @"settings"@. A server's reply of more than 262,144
-- entries, or of more than 16 MiB counting each entry's line as it was sent,
-- fails with 'BadLength'.
--
-- Some groups, such as @cachedump@, list keys as memcached stores them, and a
-- key can hold any bytes, a line break among them. 'stats' reads the reply a
-- line at a time, so the bytes after a key's line break are read as a line of
-- their own. Those bytes can split into entries the server never sent, end the
-- reply early (so the entries after them are missing), or make the request
-- fail. So that the rest of such a reply is never read as the reply to a later
-- request, 'stats' closes each connection it reads a reply from rather than
-- returning it to the pool.
--
-- memcached sends an error only as a reply's first line, and never an empty
-- line. So after the first line, an empty line, or one whose first word is
-- @ERROR@, @CLIENT_ERROR@ or @SERVER_ERROR@, can only be a listed key's bytes.
-- 'stats' throws the error such a line raises, in place of all the servers'
-- entries, once every server has answered. It doesn't send the request again
-- or mark the server dead, since a resend would read the same key and fail
-- again. If a server fails, its failure is thrown instead. The entries that
-- keys' bytes add still count towards the limit of 262,144 entries, and a
-- reply they take over it fails as any other reply that is too long does: the
-- request is sent again, and the server is marked dead if every attempt fails.
stats :: Client -> Maybe Key -> IO [(Server, StatResults)]
stats c key = validateStatsArgs key >> allOp' c (runStats key) >>= mapM rethrow
where
-- 'withSocket' closes the connection when the function throws, so the
-- outcome of reading the reply leaves it as an exception, caught at once.
-- 'retryOp' then sees a success: the error of an empty or error line from
-- a listed key's bytes neither resends the request nor marks the server
-- dead, and is rethrown below once every server has answered. Any other
-- failure, such as a reply too long or an error on the first line, leaves
-- 'withSocket' as itself, for 'retryOp' to handle as usual.
runStats statsArgs server = handle (\(StatsRead result) -> return result) $
withSocket server $ \socket -> do
sendRaw socket (statsRequest statsArgs)
collect socket 0 0 [] >>= throwIO . StatsRead
rethrow (server, result) = (,) server <$> either throwIO return result
-- Arguments may contain spaces (@stats cachedump 1 100@), but control
-- bytes would let a caller inject a second command line.
validateStatsArgs Nothing = return ()
validateStatsArgs (Just args)
| B.null args ||
B.length args > 250 ||
B.any (\byte -> byte < 0x20 || byte == 0x7f) args =
throwIO $ ClientError (KeyTooLong "invalid stats argument")
| otherwise = return ()
-- Each line counts as sent, line end included. The entries kept are
-- slices of their lines, so the spaces between tokens take memory too.
collect socket count size values = do
when (count > maxStatsEntries || size > maxStatsBytes) $
throwIO $ ProtocolError BadLength { protocolError = "stats reply too long" }
line <- recvLine socket
-- Split the line as 'recvResponse' does: on spaces alone, after removing
-- the CR that 'recvLine' leaves on it.
let tokens = filter (not . B.null) (C.split ' ' (fromMaybe line (B.stripSuffix "\r" line)))
keep entry = collect socket (count + 1) (size + B.length line + 1) (entry : values)
-- memcached sends an error only as a reply's first line, and never
-- an empty line, so after the first line either can only be a
-- listed key's bytes carried past its line break: return its error
-- rather than fail the server over it.
keyBytes err
| count > 0 = return (Left err)
| otherwise = throwIO err
case tokens of
[] -> keyBytes (ProtocolError UnknownPkt { protocolError = "" })
code:rest
| code `elem` ["ERROR", "CLIENT_ERROR", "SERVER_ERROR"] ->
statsError (Response code rest B.empty) >>= keyBytes
| code `elem` ["END", "RESET", "OK"] -> return (Right (reverse values))
| otherwise -> case (code, rest) of
("STAT", field:value) -> keep (field, C.unwords value)
_ -> keep (code, C.unwords rest)
-- The error an error line raises, caught so that a key's bytes can be
-- returned rather than thrown. 'responseStatus' throws for each code that
-- gets here, and 'failResponse' would for any it didn't.
statsError response =
handle return (responseStatus response >> failResponse response "STATS")
-- | The outcome of reading a server's 'stats' reply: its entries, or the
-- error of an empty or error line that a listed key's bytes caused. It is
-- thrown out of 'withSocket' so that the connection it was read from is
-- closed, and unwrapped afterwards.
newtype StatsRead = StatsRead (Either MemcacheError StatResults)
deriving Show
instance Exception StatsRead
-- | Most entries 'stats' accepts in one server's reply. Memcached sends four
-- lines for each open connection in @stats conns@, and three for each
-- listening socket, so this allows it on a server with nearly 65,536
-- connections. Most other groups are far shorter:
-- @stats items@ has about 30 lines for each slab class, and @stats sizes@ one
-- for each 32-byte size bucket in use (at most 32,768 with the default 1 MiB
-- item size limit).
maxStatsEntries :: Int
maxStatsEntries = 256 * 1024
-- | Most bytes of entries 'stats' accepts in one server's reply, counting each
-- line as it was sent. Replies to @stats conns@ and @stats cachedump@ are
-- smaller: 262,144 entries of @stats conns@ come to 15 MiB even with
-- five-digit descriptors, the longest Unix socket path and state name, and
-- ten-digit idle times, and memcached stops @stats cachedump@ at 2 MiB.
maxStatsBytes :: Int
maxStatsBytes = 16 * 1024 * 1024
-- | Return the version string of the Memcached cluster. Only queries
-- one server and assumes all servers in the cluster run the same version.
version :: Client -> IO ByteString
version c = do
response <- anyOpExpecting c "VERSION" ["VERSION"] versionRequest
status <- responseStatus response
if responseCode response == "VERSION" && status == NoError
then return $ fromMaybe B.empty (listToMaybe (responseTokens response))
else failResponse response "VERSION"
-- | The codes of the replies, besides error replies, that memcached 1.6.13
-- and later send to each request; a reply with any other code closes the
-- connection, as 'keyedOpExpecting' describes. Up to 1.6.17, memcached run
-- with @-o meta_response_old@ sends @OK@ in place of @HD@. Stores and deletes
-- take it as success; 'touch' doesn't.
getReplies, touchReplies, storeReplies, arithmeticReplies, deleteReplies :: [ByteString]
getReplies = ["VA", "EN"]
touchReplies = ["HD", "EN"]
storeReplies = ["HD", "OK", "NS", "EX", "NF"]
arithmeticReplies = ["VA", "NF", "NS", "EX"]
deleteReplies = ["HD", "OK", "NF", "EX"]
-- | 'keyedOpExpecting' for a store without a CAS version, which can be sent
-- again: it stores the same value. A store with one is sent once, as
-- 'keyedOpExpectingOnce' describes: its success changes the item's version,
-- so a second one would find it changed and report failure.
resendUnlessVersioned :: Version -> Cluster -> String -> [ByteString] -> Key -> (WireKey -> ByteString) -> IO Response
resendUnlessVersioned 0 = keyedOpExpecting
resendUnlessVersioned _ = keyedOpExpectingOnce
-- | The value, flags and CAS version carried by a @VA@ response.
valueResult :: Response -> (Value, Flags, Version)
valueResult response = (responseValue response, responseFlags response, responseCas response)
failResponse :: Response -> String -> IO a
failResponse response operation = do
status <- responseStatus response
throwIO $ if status /= NoError
then OpError status
else ProtocolError UnexpectedResponse
{ protocolError = "Expected " <> operation <> ", got: " <> show (responseCode response) }