diff --git a/ChangeLog.md b/ChangeLog.md new file mode 100644 --- /dev/null +++ b/ChangeLog.md @@ -0,0 +1,183 @@ +# 1.0.0 -- ? + +## Features: +- Support and CI for GHCs: `9.12.2` +- ~50% optimization of select/insert time and alloc (perf test №1) +- Added function `command` for statements with no result +- Added `Bool`, `DateTime64`, `UInt256`, `Enum8`, `Enum16` types support +- Added `Array(T)` partial support (only for primitive types) +- Added partial support for settings passing (watch `addSetting` fuction) +- Added TLS support (see separate package [ClickHaskell-tls](https://hackage.haskell.org/package/ClickHaskell-tls)) +- Improved ClickHaskell typeclass resolution errors in HLS session + +## Fixes: +- Fixed unexpected behavior when the number of result columns was different from expected. + A `UserError` exception `UnmatchedColumnsCount` is now raised in such cases (+ added test) +- Query serialization for UInt128 bigger than 999999999999999934463 (+ added test) + +## Breaking changes: +- ### select/insert API changes + - `insert` and `select` no longer accept raw queries + - The following functions were removed: + `selectFrom`, `selectFromView`, `generateRandom`, `insertInto`. + - Now you should use statement generators instead: + - `select` with `fromTable`, `fromView`, `fromGenerateRandom`, `unsafeMkSelect` + - `insert` with `intoTable`, `unsafeMkInsert` +- ### Generic API changes + `ReadableFrom` and `WritableInto` was replaced with `ClickHaskell`\ + Now you should declare single instance for every API parts + ```haskell + data MyData = + ... + deriving (Generic) + deriving anyclass (ClickHaskell ExampleColumns) + ``` + ```haskell + data MyData = + ... + deriving (Generic) + deriving anyclass + ( ReadableFrom (Columns ExampleColumns) + , WritableInto (Table "profiler" ExampleColumns) + ) + ``` +- ### `Serializable` type class was unexported +- ### `DeserializableColumn` type class was renamed to `SerializableColumn` +- ### `IsChType` instance changes + `ToChType` type family was deleted +- ### `FromChType` typeclass merged into `ToChType` +- ### `parameter` function now doesn't apply toChType +- ### Connection initialization API changes + 1. `ChCredential` renamed to `ConnectionArgs` + 2. `defaultCredentials` renamed to `defaultConnectionArgs` + 3. `openNativeConnection` renamed to `openConnection` + 4. `ConnectionArgs` constructor now are not exported + + You need to use new modifiers:\ + `setHost`, `setPort`, `setUser`, `setDatabase`, `setPassword` + + Connection initialization example: + ```haskell + initMyConnection :: IO Connection + initMyConnection = do + connection <- + openConnection + . setUser "default" + . setPassword "" + . setDatabase "default" + . setPort "9000" + . setHost "localhost" + $ defaultConnectionArgs + pure connection + ``` +- ### `client_name` format was changed + from `ClickHaskell-x.y.z` to `ClickHaskell` + +# 0.2.0 -- 23-03-2025 + +ClickHaskell documentation got it's own domain name: https://clickhaskell.dev/ + + +## Fixes: +- Improved multithreading connection usage (+ added test) +- Unexpected behavior on expected and result column type mismatches (+ added test) + +## Features: +- Additional GHC versions tests: `9.4.8`, `9.8.4`, `9.10.1` +- Query serialization support for UUID (+ added test) +- Export of client errors for exception handling +- Dropped vector dependency +- Introduced memory consumption test (64M limit) on parallel reading and writing of 1 million rows +- Added new reading wrapper for generateRandom function +- Depricating `Ch*` prefixes on types: + - `ChUInt*` -> `UInt*` (type synonyms to Word*) + - `ChInt*` -> `Int*` (reexport of Data.Int) + - `ChDate` -> `Date` (ClickHaskell type) + - `ChDateTime` -> `DateTime` (ClickHaskell type) + - `ChArray` -> `Array` (ClickHaskell type) + - `ChUUID` -> `UUID` (ClickHaskell type) +- openNativeConnection now passes $HOME and $USERNAME variables to query info + +## Breaking changes: +- ### New UserErrors on types and columns names missmatches + This change helps protect a user from unexpected behavior + `UnmatchedType` error occurs when the expected type doesn't match the resulting one + `UnmatchedColumn` error occurs when the expected column name doesn't match the resulting one + ```haskell + data ExpectedName = MkExpectedName + { expectedName :: ChInt64 + } + deriving (Generic) + deriving anyclass (ReadableFrom (Columns ExpectedColumns)) + + type ExpectedColumns = '[ Column "expectedName" ChInt64] + + -- Will throw UnmatchedColumn + void $ + select + @ExpectedColumns @ExpectedName connection + (toChType "SELECT * FROM generateRandom('unexpectedName Int64', 1, 10, 2) LIMIT 1") + pure + + -- Will throw UnmatchedType + void $ + select + @ExpectedColumns @ExpectedName connection + "SELECT * FROM generateRandom('expectedName String', 1, 10, 2) LIMIT 1" + pure + ``` + +- ### Migration to streaming API + The result of selects now exposes the block by block handling result. So you need to pass the handler and to process the list of results + ```haskell + result <- + sum <$> + select + @ExampleColumns + @ExampleData + connection "\ + \SELECT * \ + \FROM generateRandom('a1 Int64', 1, 10, 2) \ + \LIMIT 1_000_00 \ + \ " + ``` + now looks like + ```haskell + result <- + sum <$> + select + @ExampleColumns + @ExampleData + connection "\ + \SELECT * \ + \FROM generateRandom('a1 Int64', 1, 10, 2) \ + \LIMIT 1_000_00 \ + \ " + (pure . sum) + ``` + +- ### DateTime type now parametrized with timezone + Every DateTime type annotations + ```haskell + type A = ChDateTime -- DateTime + type B = ChDateTime -- DateTime('UTC') + ``` + should be changed with + ```haskell + type A = ChDateTime "" -- DateTime + type B = ChDateTime "UTC" -- DateTime('UTC') + ``` + +- ### Migration to single module distribution + You need to move all imports such as + ```haskell + import ClickHaskell.DbTypes (ChInt8) + ``` + to ClickHaskell module + ```haskell + import ClickHaskell (ChInt8) + ``` + +# 0.1.0 -- 04-12-2024 + +Initial release diff --git a/ClickHaskell.cabal b/ClickHaskell.cabal new file mode 100644 --- /dev/null +++ b/ClickHaskell.cabal @@ -0,0 +1,109 @@ +Cabal-version: 3.0 + + +Name: ClickHaskell +Version: 1.0.0 + +Author: Kovalev Dmitry +Maintainer: Kovalev Dmitry +Category: ClickHouse +Synopsis: ClickHouse driver +Description: Small dependency footprint highlevel ClickHouse driver +Tested-with: GHC == 9.4.8, GHC == 9.6.7, GHC == 9.8.4, GHC == 9.10.3, GHC == 9.12.2 +Homepage: https://clickhaskell.dev/ +Bug-reports: https://git.clickhaskell.dev/ +License: BSD-3-Clause +License-File: LICENSE +Copyright: 2023 Kovalev Dmitry +Build-Type: Simple + +extra-doc-files: + README.md + ChangeLog.md + +Source-repository head + Type: git + Location: https://github.com/KovalevDima/ClickHaskell + +Flag dev + Description: Dumps Core representation + Default: False + Manual: True + +Library + Exposed-Modules: + ClickHaskell + ClickHaskell.Protocol + Other-Modules: + ClickHaskell.Columns + ClickHaskell.Connection + ClickHaskell.Protocol.Client + ClickHaskell.Protocol.Data + ClickHaskell.Protocol.Server + ClickHaskell.Protocol.Settings + ClickHaskell.Primitive + ClickHaskell.Statements + HS-Source-Dirs: + ./ + GHC-Options: + -Wall + -Wunused-packages + + Build-depends: + -- GHC included libraries + base >= 4.7 && <5 + , bytestring < 1 + , binary < 9 + , deepseq < 2 + , time < 2 + + -- External dependencies + , network < 4 + , wide-word < 1 + ghc-options: + -Wno-unticked-promoted-constructors + -Wno-dodgy-imports + if flag(dev) + GHC-Options: + -ddump-to-file + -ddump-timings + -ddump-simpl + -ddump-spec + -ddump-simpl-stats + -ddump-rule-firings + -dsuppress-all + -dsuppress-uniques + if impl(ghc >= 9.4) + ghc-options: + -dlint + -dno-suppress-core-sizes + + default-language: Haskell2010 + default-extensions: + AllowAmbiguousTypes + BangPatterns + ConstraintKinds + DataKinds + DefaultSignatures + DeriveAnyClass + DeriveGeneric + DerivingStrategies + DuplicateRecordFields + FunctionalDependencies + GADTs + GeneralizedNewtypeDeriving + ImportQualifiedPost + FlexibleContexts + FlexibleInstances + MultiParamTypeClasses + NamedFieldPuns + NumericUnderscores + OverloadedStrings + RecordWildCards + TupleSections + TypeApplications + TypeFamilies + TypeOperators + ScopedTypeVariables + StandaloneDeriving + UndecidableInstances diff --git a/ClickHaskell.hs b/ClickHaskell.hs new file mode 100644 --- /dev/null +++ b/ClickHaskell.hs @@ -0,0 +1,520 @@ +{- | + Module: ClickHaskell + Copyright: (c) 2023 Dmitry Kovalev + License: BSD-3-Clause + Maintainer: Dmitry Kovalev + Stability: Experimental + + For full documentation, visit: https://clickhaskell.dev/ +-} + +module ClickHaskell + ( + {- * Connection -} + ConnectionArgs, defaultConnectionArgs + , setHost, setPort, setUser, setDatabase, setPassword + , Connection(..), openConnection + {- ** Hacking -} + , overrideInitConnection, BufferArgs(..) + , overrideHostname + , overrideOsUser + , overrideDefaultPort + , overrideMaxRevision + + {- * Statements and commands -} + + {- ** Exceptions -} + , ClientError(..) + , ConnectionError(..) + , UserError(..) + , InternalError(..) + + {- ** Settings -} + , passSettings + , addSetting + + {- ** SELECT -} + {- *** Runner -}, select + {- *** Statements -} + , Select, unsafeMkSelect + , fromGenerateRandom + , fromTable + {- *** View -} + , fromView + , parameter, Parameter, Parameters, viewParameters + + {- ** INSERT -} + , Insert, unsafeMkInsert + , insert + , intoTable + {- *** Modifiers -} + , ToQueryPart(toQueryPart) + + {- ** Ping -}, ping + {- ** Commands -}, command, Command + + {- ** Deriving -} + , ClickHaskell(..) + , ToChType(toChType, fromChType) + , SerializableColumn + , Column, KnownColumn + + + {- * ClickHouse types -} + , IsChType(chTypeName, defaultValueOfTypeName) + , DateTime, DateTime64 + , Int8, Int16, Int32, Int64, Int128(..) + , UInt8, UInt16, UInt32, UInt64, UInt128, UInt256, Word128(..) + , Nullable + , LowCardinality, IsLowCardinalitySupported + , UUID + , Array + , ChString + , Enum8, Enum16 + ) where + +-- Internal +import ClickHaskell.Columns +import ClickHaskell.Connection +import ClickHaskell.Primitive +import ClickHaskell.Statements +import ClickHaskell.Protocol + +-- GHC included +import Control.Concurrent (newMVar, putMVar, takeMVar) +import Control.Exception (Exception, mask, onException, throw, throwIO) +import Control.Monad (when) +import Data.Binary.Get +import Data.ByteString.Builder +import Data.Int (Int16, Int32, Int64, Int8) +import Data.Kind (Type) +import Data.Type.Equality +import Data.Type.Bool +import GHC.Generics (C1, D1, Generic (..), K1 (K1, unK1), M1 (M1, unM1), Meta (MetaSel), Rec0, S1, type (:*:) (..)) +import GHC.Stack (HasCallStack, callStack, prettyCallStack) +import GHC.TypeLits (ErrorMessage (..), TypeError) +import System.Environment (lookupEnv) + +-- External +import Data.WideWord (Int128 (..), Word128 (..)) + +-- * Connection + +openConnection :: HasCallStack => ConnectionArgs -> IO Connection +openConnection creds@MkConnectionArgs{mHostname, mOsUser} = do + hostname <- maybe (lookupEnv "HOSTNAME") (pure . Just) mHostname + osUser <- maybe (lookupEnv "USER") (pure . Just) mOsUser + connectionState <- + createConnectionState auth + . (maybe id overrideHostname hostname) + . (maybe id overrideOsUser osUser) + $ creds + MkConnection <$> newMVar connectionState + +-- * Statements and commands + +-- ** Exceptions + +{- | + A wrapper for all client-related errors + + e.g. `command`, `select`, `insert` etc +-} +data ClientError where + UnmatchedResult :: HasCallStack => UserError -> ClientError + -- ^ Query result unmatched with declared specialization + DatabaseException :: HasCallStack => ExceptionPacket -> ClientError + -- ^ Database responded with an exception packet + InternalError :: HasCallStack => InternalError -> ClientError + deriving anyclass (Exception) + +instance Show ClientError where + show (UnmatchedResult err) = "UserError " <> show err <> "\n" <> prettyCallStack callStack + show (DatabaseException err) = "DatabaseException " <> show err <> "\n" <> prettyCallStack callStack + show (InternalError err) = "InternalError " <> show err <> "\n" <> prettyCallStack callStack + +{- | + Errors intended to be handled by developers +-} +data UserError + = UnmatchedType String + -- ^ Column type mismatch in data packet + | UnmatchedColumn String + -- ^ Column name mismatch in data packet + | UnmatchedColumnsCount String + -- ^ Occurs when actual columns count less or more than expected + deriving (Show, Exception) + + +-- ** Low level + +-- *** SELECT + +{- | + Takes `Select`, `Connection` and __block processing__ function + + Returns __block processing__ result +-} +select :: + forall columns output result + . + HasCallStack + => + ClickHaskell columns output + => + Select columns output -> Connection -> ([output] -> IO result) -> IO [result] +select (MkSelect mkQuery setts) conn f = do + withConnection conn $ \connState -> do + writeToConnection connState + . flip serialize + . mkQueryPacket + . mkQueryArgs connState setts + . mkQuery + $ expectedColumns @columns @output + writeToConnection connState (\rev -> serialize rev . Data $ mkDataPacket "" 0 0) + loopSelect connState [] + where + loopSelect connState@MkConnectionState{..} acc = + readBuffer buffer (deserialize revision) + >>= \packet -> case packet of + DataResponse MkDataPacket{columns_count = 0, rows_count = 0} -> loopSelect connState acc + DataResponse MkDataPacket{columns_count, rows_count} -> do + let expected = columnsCount @columns @output + when (columns_count /= expected) $ + (throw . UnmatchedResult . UnmatchedColumnsCount) + ("Expected " <> show expected <> " columns but got " <> show columns_count) + !result <- f =<< readBuffer buffer (deserializeColumns @columns True revision rows_count) + loopSelect connState (result : acc) + ProfileEvents _ -> loopSelect connState acc + Progress _ -> loopSelect connState acc + ProfileInfo _ -> loopSelect connState acc + EndOfStream -> pure acc + Exception exception -> throwIO (DatabaseException exception) + otherPacket -> throwIO (InternalError $ UnexpectedPacketType $ serverPacketToNum otherPacket) + + +-- *** INSERT + +insert :: + forall columns record + . + HasCallStack + => + ClickHaskell columns record + => + Insert columns record -> Connection -> [record] -> IO () +insert (MkInsert mkQuery dbSettings) conn columnsData = do + withConnection conn $ \connState -> do + writeToConnection connState + . flip serialize + . mkQueryPacket + . mkQueryArgs connState dbSettings + . mkQuery + $ expectedColumns @columns @record + writeToConnection connState (\rev -> serialize rev . Data $ mkDataPacket "" 0 0) + loopInsert connState + where + loopInsert connState@MkConnectionState{..} = do + firstPacket <- readBuffer buffer (deserialize revision) + case firstPacket of + TableColumns _ -> loopInsert connState + DataResponse MkDataPacket{} -> do + _emptyDataPacket <- readBuffer buffer (deserializeColumns @columns @record False revision 0) + let rows = fromIntegral (Prelude.length columnsData) + cols = columnsCount @columns @record + writeToConnection connState (\rev -> serialize rev . Data $ mkDataPacket "" cols rows) + writeToConnection connState (serializeColumns @columns columnsData) + writeToConnection connState (\rev -> serialize rev . Data $ mkDataPacket "" 0 0) + loopInsert connState + EndOfStream -> pure () + Exception exception -> throwIO (DatabaseException exception) + otherPacket -> throwIO (InternalError $ UnexpectedPacketType $ serverPacketToNum otherPacket) + + +-- *** Ping + +{- | + Sends `Ping` packet and handles `Pong` packet +-} +ping :: HasCallStack => Connection -> IO () +ping conn = do + withConnection conn $ \connState@MkConnectionState{revision, buffer} -> do + writeToConnection connState (\rev -> serialize rev Ping) + responsePacket <- readBuffer buffer (deserialize revision) + case responsePacket of + Pong -> pure () + Exception exception -> throwIO (DatabaseException exception) + otherPacket -> throwIO (InternalError $ UnexpectedPacketType $ serverPacketToNum otherPacket) + + +-- *** Commands + +{- | + Might be used for any command without data responses + + For example: CREATE, TRUNCATE, KILL, SET, GRANT + + __Throws exception if any data was returned__ +-} +command :: HasCallStack => Connection -> Command -> IO () +command conn (MkCommand query settings) = do + withConnection conn $ \connState -> do + writeToConnection connState + . flip serialize + . mkQueryPacket + $ mkQueryArgs connState settings query + writeToConnection connState (\rev -> serialize rev . Data $ mkDataPacket "" 0 0) + handleCreate connState + where + handleCreate :: ConnectionState -> IO () + handleCreate MkConnectionState{..} = + readBuffer buffer (deserialize revision) + >>= \packet -> case packet of + EndOfStream -> pure () + Exception exception -> throwIO (DatabaseException exception) + otherPacket -> throwIO (InternalError $ UnexpectedPacketType $ serverPacketToNum otherPacket) + + +-- ** Deriving + +class GenericClickHaskell record columns => ClickHaskell columns record + where + default deserializeColumns :: Bool -> ProtocolRevision -> UVarInt -> Get [record] + deserializeColumns :: Bool -> ProtocolRevision -> UVarInt -> Get [record] + deserializeColumns doCheck rev size = gDeserializeColumns @columns doCheck rev size to + + default serializeColumns :: [record] -> ProtocolRevision -> Builder + serializeColumns :: [record] -> ProtocolRevision -> Builder + serializeColumns records rev = gSerializeRecords @columns rev records from + + default expectedColumns :: [(Builder, Builder)] + expectedColumns :: [(Builder, Builder)] + expectedColumns = gExpectedColumns @columns @(Rep record) + + default columnsCount :: UVarInt + columnsCount :: UVarInt + columnsCount = gColumnsCount @columns @(Rep record) + +type GenericClickHaskell record hasColumns = + ( Generic record + , GClickHaskell hasColumns (Rep record) + ) + + + + +-- * Internal + +mkQueryArgs :: ConnectionState -> DbSettings -> ChString -> QueryPacketArgs +mkQueryArgs MkConnectionState {..} settings query = MkQueryPacketArgs {..} + +-- ** Connection + +readBuffer :: Buffer -> Get a -> IO a +readBuffer MkBuffer{readBuff, writeBuff} parser = runBufferReader (runGetIncremental parser) + where + runBufferReader :: Decoder packet -> IO packet + runBufferReader dec = case dec of + (Partial decoder) -> readBuff >>= runBufferReader . decoder . Just + (Done leftover _consumed packet) -> packet <$ writeBuff leftover + (Fail _leftover _consumed msg) -> throwIO (InternalError $ DeserializationError msg) + + +withConnection :: HasCallStack => Connection -> (ConnectionState -> IO a) -> IO a +withConnection (MkConnection connStateMVar) f = + mask $ \restore -> do + connState <- takeMVar connStateMVar + b <- onException + (restore (f connState)) + (do + newConnState <- recreateConnectionState auth connState + putMVar connStateMVar newConnState + ) + putMVar connStateMVar connState + return b + +auth :: Buffer -> ConnectionArgs -> IO ConnectionState +auth buffer creds@MkConnectionArgs{db, user, pass, mOsUser, mHostname, maxRevision} = do + writeConn buffer + . serialize maxRevision + $ mkHelloPacket db user pass maxRevision + serverPacketType <- readBuffer buffer (deserialize maxRevision) + case serverPacketType of + HelloResponse MkHelloResponse{server_revision} -> do + let conn = + MkConnectionState + { revision = min server_revision maxRevision + , os_user = maybe "" toChType mOsUser + , hostname = maybe "" toChType mHostname + , initial_user = toChType user + , .. + } + writeToConnection conn (\rev -> serialize rev mkAddendum) + pure conn + Exception exception -> throwIO (DatabaseException exception) + otherPacket -> throwIO (InternalError $ UnexpectedPacketType $ serverPacketToNum otherPacket) + + +-- ** Serialization Generic API + +class GClickHaskell (columns :: [Type]) f + where + + {- + Generic deriving can be a bit tricky + + You can think of it as + 1) Columns serialization logic generator + 2) Columns-to-rows(list of records) transposer + -} + gDeserializeColumns :: Bool -> ProtocolRevision -> UVarInt -> (f p -> res) -> Get [res] + gSerializeRecords :: ProtocolRevision -> [res] -> (res -> f p) -> Builder + {- + and affected columns extractor + -} + gExpectedColumns :: [(Builder, Builder)] + gColumnsCount :: UVarInt + +{- + Unwrapping data type constructor + data Record = MkRecord .. +-} +instance + GClickHaskell columns f + => + GClickHaskell columns (D1 c (C1 c2 f)) + where + {-# INLINE gDeserializeColumns #-} + gDeserializeColumns doCheck rev size f = + gDeserializeColumns @columns doCheck rev size (f . M1 . M1) + + {-# INLINE gSerializeRecords #-} + gSerializeRecords rev xs f = gSerializeRecords @columns rev xs (unM1 . unM1 . f) + + gExpectedColumns = gExpectedColumns @columns @f + gColumnsCount = gColumnsCount @columns @f + +{- + Flattening of generic products + + For example + ( + field_1::T1 :*: field_2::T2) + ) :*: ( + field_3::T3 :*: field_4::T4 + ) + + turns into + field_1::T1 :*: ( + field_2::T2 :*: (field_3::T3 :*: field_4::T4) + ) +-} +instance + GClickHaskell columns (left :*: (right1 :*: right2)) + => + GClickHaskell columns ((left :*: right1) :*: right2) + where + {-# INLINE gDeserializeColumns #-} + gDeserializeColumns doCheck rev size f = + gDeserializeColumns @columns @(left :*: (right1 :*: right2)) doCheck rev size + (\(l :*: (r1:*:r2)) -> f ((l :*: r1):*:r2)) + + {-# INLINE gSerializeRecords #-} + gSerializeRecords rev xs f = + gSerializeRecords @columns @(left :*: (right1 :*: right2)) rev xs + ((\((l:*:r1) :*: r2) -> l :*: (r1 :*: r2)) . f) + + gExpectedColumns = gExpectedColumns @columns @(left :*: (right1 :*: right2)) + gColumnsCount = gColumnsCount @columns @(left :*: (right1 :*: right2)) + +{- + Unwrapping a product starting with a field + + field_n::Tn :*: (..) +-} +instance + ( GClickHaskell columns right + , GClickHaskell columns (S1 (MetaSel (Just name) a b f) (Rec0 inputType)) + ) + => + GClickHaskell columns ((S1 (MetaSel (Just name) a b f)) (Rec0 inputType) :*: right) + where + {-# INLINE gDeserializeColumns #-} + gDeserializeColumns doCheck rev size f = do + lefts <- gDeserializeColumns @columns @(S1 (MetaSel (Just name) a b f) (Rec0 inputType)) doCheck rev size id + rights <- gDeserializeColumns @columns @right doCheck rev size id + deserializeProduct (\l r -> f $ l :*: r) lefts rights + + {-# INLINE gSerializeRecords #-} + gSerializeRecords rev xs f + = gSerializeRecords @columns rev xs ((\(l:*:_) -> l) . f) + <> gSerializeRecords @columns rev xs ((\(_:*:r) -> r) . f) + + gExpectedColumns = gExpectedColumns @columns @(S1 (MetaSel (Just name) a b f) (Rec0 inputType)) ++ gExpectedColumns @columns @right + gColumnsCount = gColumnsCount @columns @(S1 (MetaSel (Just name) a b f) (Rec0 inputType)) + gColumnsCount @columns @right + +deserializeProduct :: (l -> r -> a) -> [l] -> [r] -> Get [a] +deserializeProduct f lefts rights = goDeserialize [] lefts rights + where + goDeserialize !acc (l:ls) (r:rs) = goDeserialize ((:acc) $! f l r) ls rs + goDeserialize !acc [] [] = pure acc + goDeserialize _ _ _ = fail "Mismatched lengths in gDeserializeColumns" + +{- + Unwrapping a single generic field (recursion breaker) + + field::Tn +-} +instance + ( KnownColumn (Column name chType) + , SerializableColumn (Column name chType) + , ToChType chType inputType + , mColumn ~ TakeColumn name columns + , If + (mColumn == Nothing) + (TypeError + ( 'Text "There is no column \"" :<>: 'Text name :<>: 'Text "\" in table" + :$$: 'Text "You can't use this field" + ) + ) + (Just (Column name chType) ~ mColumn) + ) => GClickHaskell columns ((S1 (MetaSel (Just name) a b f)) (Rec0 inputType)) + where + {-# INLINE gDeserializeColumns #-} + gDeserializeColumns doCheck rev size f = do + validateColumnHeader @(Column name chType) doCheck rev =<< deserialize @ColumnHeader rev + deserializeColumn @(Column name chType) rev size (f . M1 . K1 . fromChType) + + {-# INLINE gSerializeRecords #-} + gSerializeRecords rev values f + = serialize rev (mkHeader @(Column name chType)) + <> serializeColumn @(Column name chType) rev (toChType . unK1 . unM1 . f) values + + gExpectedColumns = (renderColumnName @(Column name chType), renderColumnType @(Column name chType)) : [] + gColumnsCount = 1 + +validateColumnHeader :: forall column . KnownColumn column => Bool -> ProtocolRevision -> ColumnHeader -> Get () +validateColumnHeader doCheck rev MkColumnHeader{..} = do + let expectedColumnName = toChType (renderColumnName @column) + resultColumnName = name + when (doCheck && resultColumnName /= expectedColumnName) $ + throw . UnmatchedResult . UnmatchedColumn + $ "Got column \"" <> show resultColumnName <> "\" but expected \"" <> show expectedColumnName <> "\"" + + let expectedType = fallbackTypeName rev $ toChType (renderColumnType @column) + resultType = fallbackTypeName rev type_ + when (doCheck && resultType /= expectedType) $ + throw . UnmatchedResult . UnmatchedType + $ "Column " <> show resultColumnName <> " has type " <> show resultType <> ". But expected type is " <> show expectedType + +type family + TakeColumn name columns :: Maybe Type + where + TakeColumn name columns = GoTakeColumn name columns '[] + +type family + GoTakeColumn name (columns :: [Type]) (acc :: [Type]) :: Maybe Type + where + GoTakeColumn name (Column name chType ': columns) acc = Just (Column name chType) + GoTakeColumn name (Column name1 chType ': columns) acc = (GoTakeColumn name columns (Column name1 chType ': acc)) + GoTakeColumn name '[] acc = Nothing diff --git a/ClickHaskell/Columns.hs b/ClickHaskell/Columns.hs new file mode 100644 --- /dev/null +++ b/ClickHaskell/Columns.hs @@ -0,0 +1,288 @@ +module ClickHaskell.Columns where + +-- Internal +import ClickHaskell.Primitive + +-- GHC included +import Data.Binary.Get +import Data.ByteString.Builder +import Data.ByteString.Char8 as BS8 (pack, isPrefixOf) +import Data.Traversable (forM) +import Data.Int +import Data.Kind +import Data.Coerce +import Data.Typeable (Proxy (..)) +import Data.Bits (Bits ((.&.))) +import GHC.Generics (Generic) +import GHC.TypeLits (ErrorMessage (..), KnownSymbol, Symbol, TypeError, symbolVal) + +-- External +import Data.WideWord (Int128 (..)) + +-- * Column + +data Columns (columns :: [Type]) where + Empty :: Columns '[] + AddColumn + :: KnownColumn (Column name chType) + => Column name chType + -> Columns columns + -> Columns (Column name chType ': columns) + +colLen :: Columns columns -> Int +colLen (AddColumn (col :: col) _) = length $ fromColumn @col col +colLen Empty = 0 + +{- | +Column declaration + +For example: + +@ +type MyColumn = Column "myColumn" ChString +@ +-} +data Column (name :: Symbol) (chType :: Type) where + UInt8Column :: [UInt8] -> Column name UInt8 + Int8Column :: [Int8] -> Column name Int8 + UInt16Column :: [UInt16] -> Column name UInt16 + Int16Column :: [Int16] -> Column name Int16 + UInt32Column :: [UInt32] -> Column name UInt32 + Int32Column :: [Int32] -> Column name Int32 + UInt64Column :: [UInt64] -> Column name UInt64 + Int64Column :: [Int64] -> Column name Int64 + UInt128Column :: [UInt128] -> Column name UInt128 + Int128Column :: [Int128] -> Column name Int128 + UInt256Column :: [UInt256] -> Column name UInt256 + DateTimeColumn :: [DateTime tz] -> Column name (DateTime tz) + DateTime64Column :: [DateTime64 precision tz] -> Column name (DateTime64 precision tz) + BoolColumn :: [Bool] -> Column name Bool + Enum8Column :: [Enum8 enums] -> Column name (Enum8 enums) + Enum16Column :: [Enum16 enums] -> Column name (Enum16 enums) + DateColumn :: [Date] -> Column name Date + UUIDColumn :: [UUID] -> Column name UUID + StringColumn :: [ChString] -> Column name ChString + ArrayColumn :: [Array chType] -> Column name (Array chType) + NullableColumn :: [Nullable chType] -> Column name (Nullable chType) + LowCardinalityColumn :: [chType] -> Column name (LowCardinality chType) + +type family GetColumnName column :: Symbol where GetColumnName (Column name columnType) = name +type family GetColumnType column :: Type where GetColumnType (Column name columnType) = columnType + +data ColumnHeader = MkColumnHeader + { name :: ChString + , type_ :: ChString + , is_custom :: UInt8 `SinceRevision` DBMS_MIN_REVISION_WITH_CUSTOM_SERIALIZATION + } deriving (Generic, Serializable) + +fallbackTypeName :: ProtocolRevision -> ChString -> ChString +fallbackTypeName rev (MkChString typeName) = MkChString $ + if rev < mkRev @DBMS_MIN_REVISION_WITH_TIME_ZONE_PARAMETER_IN_DATETIME_DATA_TYPE + && isPrefixOf "DateTime(" typeName + then "DateTime" + else typeName + +class + ( IsChType (GetColumnType column) + , KnownSymbol (GetColumnName column) + ) => + KnownColumn column where + renderColumnName :: Builder + renderColumnName = (stringUtf8 . symbolVal @(GetColumnName column)) Proxy + + renderColumnType :: Builder + renderColumnType = byteString . BS8.pack $ chTypeName @(GetColumnType column) + + mkHeader :: ColumnHeader + mkHeader = let + name = toChType $ renderColumnName @column + type_ = toChType $ chTypeName @(GetColumnType column) + is_custom = AfterRevision 0 + in MkColumnHeader{..} + + toColumn :: [GetColumnType column] -> Column (GetColumnName column) (GetColumnType column) + fromColumn :: Column (GetColumnName column) (GetColumnType column) -> [GetColumnType column] + +instance KnownSymbol name => KnownColumn (Column name UInt8) where + toColumn = UInt8Column + fromColumn (UInt8Column values) = values + +instance KnownSymbol name => KnownColumn (Column name UInt16) where + toColumn = UInt16Column + fromColumn (UInt16Column values) = values + +instance KnownSymbol name => KnownColumn (Column name UInt32) where + toColumn = UInt32Column + fromColumn (UInt32Column values) = values + +instance KnownSymbol name => KnownColumn (Column name UInt64) where + toColumn = UInt64Column + fromColumn (UInt64Column values) = values + +instance KnownSymbol name => KnownColumn (Column name UInt128) where + toColumn = UInt128Column + fromColumn (UInt128Column values) = values + +instance KnownSymbol name => KnownColumn (Column name UInt256) where + toColumn = UInt256Column + fromColumn (UInt256Column values) = values + +instance KnownSymbol name => KnownColumn (Column name Int8) where + toColumn = Int8Column + fromColumn (Int8Column values) = values + +instance KnownSymbol name => KnownColumn (Column name Int16) where + toColumn = Int16Column + fromColumn (Int16Column values) = values + +instance KnownSymbol name => KnownColumn (Column name Int32) where + toColumn = Int32Column + fromColumn (Int32Column values) = values + +instance KnownSymbol name => KnownColumn (Column name Int64) where + toColumn = Int64Column + fromColumn (Int64Column values) = values + +instance KnownSymbol name => KnownColumn (Column name Int128) where + toColumn = Int128Column + fromColumn (Int128Column values) = values + +instance KnownSymbol name => KnownColumn (Column name Date) where + toColumn = DateColumn + fromColumn (DateColumn values) = values + +instance + ( KnownSymbol name + , IsChType (DateTime tz) + ) => + KnownColumn (Column name (DateTime tz)) + where + toColumn = DateTimeColumn + fromColumn (DateTimeColumn values) = values + +instance + ( KnownSymbol name + , IsChType (DateTime64 precision tz) + ) => + KnownColumn (Column name (DateTime64 precision tz)) + where + toColumn = DateTime64Column + fromColumn (DateTime64Column values) = values + +instance + ( KnownSymbol name + , IsChType Bool + ) => + KnownColumn (Column name Bool) + where + toColumn = BoolColumn + fromColumn (BoolColumn values) = values + +instance (KnownSymbol name, KnownSymbol enums) => KnownColumn (Column name (Enum8 enums)) where + toColumn = Enum8Column + fromColumn (Enum8Column values) = values + +instance (KnownSymbol name, KnownSymbol enums) => KnownColumn (Column name (Enum16 enums)) where + toColumn = Enum16Column + fromColumn (Enum16Column values) = values + +instance KnownSymbol name => KnownColumn (Column name UUID) where + toColumn = UUIDColumn + fromColumn (UUIDColumn values) = values + +instance + ( KnownSymbol name + , IsChType chType + , IsChType (Nullable chType) + ) => + KnownColumn (Column name (Nullable chType)) + where + toColumn = NullableColumn + fromColumn (NullableColumn values) = values + +instance KnownSymbol name => KnownColumn (Column name ChString) where + toColumn = StringColumn + fromColumn (StringColumn values) = values + +instance + ( KnownSymbol name + , IsChType (LowCardinality chType) + , IsLowCardinalitySupported chType + ) => + KnownColumn (Column name (LowCardinality chType)) + where + toColumn = LowCardinalityColumn . map coerce + fromColumn (LowCardinalityColumn values) = map coerce values + +instance (KnownSymbol name, IsChType chType) => KnownColumn (Column name (Array chType)) where + toColumn = ArrayColumn + fromColumn (ArrayColumn values) = values + + +class KnownColumn column => SerializableColumn column where + deserializeColumn :: ProtocolRevision -> UVarInt -> (GetColumnType column -> a) -> Get [a] + serializeColumn :: ProtocolRevision -> (a -> GetColumnType column) -> [a] -> Builder + +instance + ( KnownColumn (Column name chType) + , Serializable chType + , IsChType chType + ) => + SerializableColumn (Column name chType) where + {-# INLINE deserializeColumn #-} + deserializeColumn rev rows f = map f <$> replicateGet rev rows + + {-# INLINE serializeColumn #-} + serializeColumn rev f column = foldMap (serialize @chType rev . f) column + +instance {-# OVERLAPPING #-} + ( KnownColumn (Column name (Nullable chType)) + , Serializable chType + , IsChType chType + ) => + SerializableColumn (Column name (Nullable chType)) where + {-# INLINE deserializeColumn #-} + deserializeColumn rev rows f = do + nulls <- replicateGet @UInt8 rev rows + forM nulls (\nulFlag -> case nulFlag of + 0 -> f . Just <$> deserialize @chType rev + _ -> (f Nothing <$ deserialize @chType rev) + ) + + {-# INLINE serializeColumn #-} + serializeColumn rev f column + = foldMap (serialize @UInt8 rev . maybe 1 (const 0) . f) column + <> foldMap (serialize @chType rev . maybe defaultValueOfTypeName id . f) column + +instance {-# OVERLAPPING #-} + ( KnownColumn (Column name (LowCardinality chType)) + , Serializable chType + , IsLowCardinalitySupported chType + , TypeError ('Text "LowCardinality deserialization still unsupported") + ) => + SerializableColumn (Column name (LowCardinality chType)) where + {-# INLINE deserializeColumn #-} + deserializeColumn rev rows f = do + _serializationType <- (.&. 0xf) <$> deserialize @UInt64 rev + _index_size <- deserialize @Int64 rev + -- error $ "Trace | " <> show _serializationType <> " : " <> show _index_size + map f . coerce + <$> replicateGet @chType rev rows + + {-# INLINE serializeColumn #-} + serializeColumn _rev column = undefined column + +instance {-# OVERLAPPING #-} + ( KnownColumn (Column name (Array chType)) + , Serializable chType + ) + => SerializableColumn (Column name (Array chType)) where + {-# INLINE deserializeColumn #-} + deserializeColumn rev rows f = do + offsets <- replicateGet @UInt64 rev rows + forM offsets (fmap (f . MkChArray) . replicateGet @chType rev . fromIntegral) + + {-# INLINE serializeColumn #-} + serializeColumn rev f column + = foldMap (serialize @UInt64 rev . fromIntegral . length . f) column + <> foldMap (foldMap (serialize @chType rev) . f) column diff --git a/ClickHaskell/Connection.hs b/ClickHaskell/Connection.hs new file mode 100644 --- /dev/null +++ b/ClickHaskell/Connection.hs @@ -0,0 +1,279 @@ +module ClickHaskell.Connection where + +-- Internal +import ClickHaskell.Primitive + +-- GHC included +import Control.Concurrent (MVar) +import Control.Exception (SomeException, bracketOnError, catch, finally, throwIO) +import Data.Binary.Builder (Builder, toLazyByteString) +import Data.ByteString as BS (ByteString, null) +import Data.IORef (atomicWriteIORef, newIORef, readIORef) +import Data.Maybe (fromMaybe) +import GHC.Exception (Exception) + +-- External +import Network.Socket +import Network.Socket.ByteString (recv) +import Network.Socket.ByteString.Lazy (sendAll) +import System.Timeout (timeout) + + +-- * Connection + +{- | + Errors occured on connection operations +-} +data ConnectionError + = NoAdressResolved + -- ^ Occurs when 'getAddrInfo' returns an empty result + | EstablishTimeout + -- ^ Occurs on 'socket' connection timeout + | ServerClosedConnection + -- ^ Occurs on 'readSock' empty @""@ result + deriving (Show, Exception) + +{- | + These exceptions might indicate internal bugs. + + If you encounter one, please report it. +-} +data InternalError + = UnexpectedPacketType UVarInt + | DeserializationError String + deriving (Show, Exception) + +writeToConnection :: ConnectionState -> (ProtocolRevision -> Builder) -> IO () +writeToConnection MkConnectionState{revision, buffer} serializer = + (writeConn buffer) (serializer revision) + +data Connection where MkConnection :: (MVar ConnectionState) -> Connection + +data ConnectionState = MkConnectionState + { buffer :: Buffer + , revision :: ProtocolRevision + , initial_user :: ChString + , os_user :: ChString + , hostname :: ChString + , creds :: ConnectionArgs + } + +createConnectionState + :: (Buffer -> ConnectionArgs -> IO ConnectionState) + -> ConnectionArgs + -> IO ConnectionState +createConnectionState postInitAction creds@MkConnectionArgs{..} = do + let port = fromMaybe defPort mPort + buffer <- mkBuffer =<< mkBufferArgs host =<< initSocket =<< resolveHostName host port + postInitAction buffer creds + +recreateConnectionState + :: (Buffer -> ConnectionArgs -> IO ConnectionState) + -> ConnectionState + -> IO ConnectionState +recreateConnectionState postInitAction MkConnectionState{creds, buffer} = do + destroyBuff buffer + createConnectionState postInitAction creds + +data Buffer = MkBuffer + { readBuff :: IO BS.ByteString + , destroyBuff :: IO () + , writeBuff :: BS.ByteString -> IO () + , writeConn :: Builder -> IO () + } + +mkBuffer :: BufferArgs -> IO Buffer +mkBuffer MkBufferArgs{..} = do + buff <- newIORef "" + let writeBuff bs = atomicWriteIORef buff bs + + pure MkBuffer + { writeConn = writeSock + , writeBuff + , readBuff = do + currentBuffer <- readIORef buff + if (not . BS.null) currentBuffer + then writeBuff "" *> pure currentBuffer + else do + sockBytes <- readSock + if BS.null sockBytes + then throwIO ServerClosedConnection + else pure sockBytes + , destroyBuff = do + closeSock + writeBuff "" + } + +data BufferArgs = MkBufferArgs + { writeSock :: Builder -> IO () + , readSock :: IO ByteString + , closeSock :: IO () + } + + + + +-- * Initialization + +{- | + See `defaultConnectionArgs` for documentation +-} +data ConnectionArgs = MkConnectionArgs + { user :: String + , pass :: String + , db :: String + , host :: HostName + , mPort :: Maybe ServiceName + , defPort :: ServiceName + , maxRevision :: ProtocolRevision + , mOsUser :: Maybe String + , mHostname :: Maybe String + , resolveHostName :: HostName -> ServiceName -> IO AddrInfo + , mkBufferArgs :: HostName -> Socket -> IO BufferArgs + , initSocket :: AddrInfo -> IO Socket + } + +{- | + Default connection settings that follow __clickhouse-client__ defaults + + These defaults can be modified with `setUser`, `setPassword`, `setHost`, `setPort`, `setDatabase` + + For TLS support, see [ClickHaskell-tls](https://hackage.haskell.org/package/ClickHaskell-tls) package +-} +defaultConnectionArgs :: ConnectionArgs +defaultConnectionArgs = MkConnectionArgs + { user = "default" + , pass = "" + , host = "localhost" + , db = "default" + , defPort = "9000" + , mPort = Nothing + , mOsUser = Nothing + , mHostname = Nothing + , maxRevision = mkRev @DBMS_TCP_PROTOCOL_VERSION + , resolveHostName = defaultResolveHostName + , initSocket = defaultInitSocket + , mkBufferArgs = defaultMkBufferArgs + } + +defaultResolveHostName :: HostName -> ServiceName -> IO AddrInfo +defaultResolveHostName host port = do + let hints = defaultHints{addrFlags = [AI_ADDRCONFIG], addrSocketType = Stream} + addrs <- getAddrInfo (Just hints) (Just host) (Just port) + case addrs of + [] -> throwIO NoAdressResolved + x:_ -> pure x + +defaultInitSocket :: AddrInfo -> IO Socket +defaultInitSocket AddrInfo{..} = do + maybe (throwIO EstablishTimeout) pure + =<< timeout 3_000_000 ( + bracketOnError + (socket addrFamily addrSocketType addrProtocol) + (\sock -> + catch @SomeException + (finally (shutdown sock ShutdownBoth) (close sock)) + (const $ pure ()) + ) + (\sock -> do + setSocketOption sock NoDelay 1 + setSocketOption sock KeepAlive 1 + connect sock addrAddress + pure sock + ) + ) + +defaultMkBufferArgs :: HostName -> Socket -> IO BufferArgs +defaultMkBufferArgs _hostname sock = + pure $ + let + writeSock = sendAll sock . toLazyByteString + readSock = recv sock 4096 + closeSock = + catch @SomeException + (finally (shutdown sock ShutdownBoth) (close sock)) + (const $ pure ()) + in MkBufferArgs{..} + +{- | + Overrides default user __"default"__ +-} +setUser :: String -> ConnectionArgs -> ConnectionArgs +setUser new MkConnectionArgs{..} = MkConnectionArgs{user=new, ..} + +{- | + Overrides default password __""__ +-} +setPassword :: String -> ConnectionArgs -> ConnectionArgs +setPassword new MkConnectionArgs{..} = MkConnectionArgs{pass=new, ..} + +{- | + Overrides default hostname __"localhost"__ +-} +setHost :: HostName -> ConnectionArgs -> ConnectionArgs +setHost new MkConnectionArgs{..} = MkConnectionArgs{host=new, ..} + +{- | + Set a custom port instead of the default __9000__ (or __9443__ if TLS is used). + + The default port can only be overridden by 'overrideNetwork'. +-} +setPort :: ServiceName -> ConnectionArgs -> ConnectionArgs +setPort new MkConnectionArgs{..} = MkConnectionArgs{mPort=Just new, ..} + +{- | + Overrides default database __"default"__ +-} +setDatabase :: String -> ConnectionArgs -> ConnectionArgs +setDatabase new MkConnectionArgs{..} = MkConnectionArgs{db=new, ..} + + + + +-- * Overriders + +{- | + Overrides default client hostname value which is: + + 1. __$HOSTNAME__ env variable value (if set) + 2. __""__ otherwise + + Client hostname being displayed in ClickHouse logs +-} +overrideHostname :: String -> ConnectionArgs -> ConnectionArgs +overrideHostname new MkConnectionArgs{..} = MkConnectionArgs{mHostname=Just new, ..} + +{- | + Overrides default os_name value which is: + 1. __$USER__ variable value (if set) + 2. __""__ otherwise +-} +overrideOsUser :: String -> ConnectionArgs -> ConnectionArgs +overrideOsUser new MkConnectionArgs{..} = MkConnectionArgs{mOsUser=Just new, ..} + +{- | + This function should be used when you want to override + the default connection behaviour + + Watch __ClickHaskell-tls__ package for example +-} +overrideInitConnection :: (HostName -> Socket -> IO BufferArgs) -> (ConnectionArgs -> ConnectionArgs) +overrideInitConnection new MkConnectionArgs {..} = + MkConnectionArgs{mkBufferArgs = new, ..} + +{- | +Override the default port used when no port was set explicitly via 'setPort'. + +This does not immediately fix the connection port: +if the user has already called 'setPort', that value takes precedence. +Otherwise, the given port becomes the new default. + +Typical use case: provide a different default for TLS connections, e.g. 9443. +-} +overrideDefaultPort :: ServiceName -> (ConnectionArgs -> ConnectionArgs) +overrideDefaultPort new MkConnectionArgs {..} = + MkConnectionArgs {defPort = new, ..} + +overrideMaxRevision :: ProtocolRevision -> (ConnectionArgs -> ConnectionArgs) +overrideMaxRevision new MkConnectionArgs {..} = + MkConnectionArgs {maxRevision = new, ..} diff --git a/ClickHaskell/Primitive.hs b/ClickHaskell/Primitive.hs new file mode 100644 --- /dev/null +++ b/ClickHaskell/Primitive.hs @@ -0,0 +1,815 @@ +{-# LANGUAGE BangPatterns #-} +module ClickHaskell.Primitive where + +-- GHC included +import Control.Applicative (liftA2) +import Control.DeepSeq (NFData) +import Data.Binary.Get +import Data.Bits (Bits (setBit, unsafeShiftL, unsafeShiftR, (.&.), (.|.))) +import Data.Bool (bool) +import Data.ByteString as BS (ByteString, length) +import Data.ByteString.Builder +import Data.ByteString.Char8 as BS8 (pack, unpack, concatMap, singleton, replicate, length) +import Data.ByteString.Lazy (toStrict) +import Data.Coerce (coerce) +import Data.Int (Int16, Int32, Int64, Int8) +import Data.List (uncons) +import Data.String (IsString (..)) +import Data.Time (UTCTime) +import Data.Time.Clock.POSIX +import Data.Type.Bool (Not) +import Data.Type.Equality (type (==)) +import Data.Typeable (Proxy (..)) +import Data.Word (Word16, Word32, Word64, Word8) +import GHC.Generics (C1, D1, Generic (..), K1 (K1), M1 (M1), Meta (MetaSel), Rec0, S1, type (:*:) (..)) +import GHC.TypeLits (ErrorMessage (..), KnownNat, KnownSymbol, Nat, Symbol, TypeError, natVal, symbolVal) +import Prelude hiding (liftA2) + +-- External +import Data.WideWord (Int128 (..), Word256(..), Word128(..)) + +-- * User types + +-- ** Abstractions + +class Serializable chType + where + default serialize :: (Generic chType, GSerial (Rep chType)) => ProtocolRevision -> chType -> Builder + serialize :: ProtocolRevision -> chType -> Builder + serialize rev = gSerialize rev . from + + {-# INLINE deserialize #-} + default deserialize :: (Generic chType, GSerial (Rep chType)) => ProtocolRevision -> Get chType + deserialize :: ProtocolRevision -> Get chType + deserialize rev = to <$> gDeserialize rev + +{-# INLINE replicateGet #-} +replicateGet :: Serializable chType => ProtocolRevision -> UVarInt -> Get [chType] +replicateGet rev cnt0 = loopGet cnt0 + where + loopGet cnt + | cnt == 0 = pure [] + | otherwise = liftA2 (:) (deserialize rev) (loopGet (cnt - 1)) + +instance Serializable prim => Serializable [prim] where + serialize rev list + = serialize @UVarInt rev (fromIntegral $ Prelude.length list) + <> foldMap (serialize @prim rev) list + deserialize rev = do + len <- deserialize @UVarInt rev + replicateGet @prim rev len + {-# INLINE deserialize #-} + +class ToQueryPart chType where + toQueryPart :: chType -> Builder + +class IsChType chType + where + -- | Shows database original type name + -- + -- @ + -- chTypeName \@ChString = \"String\" + -- chTypeName \@(Nullable UInt32) = \"Nullable(UInt32)\" + -- @ + chTypeName :: String + + defaultValueOfTypeName :: chType + +class ToChType chType userType where + toChType :: userType -> chType + fromChType :: chType -> userType + +instance {-# OVERLAPPABLE #-} (IsChType chType, chType ~ inputType) => ToChType chType inputType where + toChType = id + fromChType = id + + +-- ** Int8 + +instance IsChType Int8 where + chTypeName = "Int8" + defaultValueOfTypeName = 0 + +instance Serializable Int8 where + serialize _ = int8 + deserialize _ = getInt8 + {-# INLINE deserialize #-} + +instance ToQueryPart Int8 where + toQueryPart = byteString . BS8.pack . show + + +-- ** Int16 + +instance IsChType Int16 where + chTypeName = "Int16" + defaultValueOfTypeName = 0 + +instance Serializable Int16 where + serialize _ = int16LE + deserialize _ = getInt16le + {-# INLINE deserialize #-} + +instance ToQueryPart Int16 where + toQueryPart = byteString . BS8.pack . show + + +-- ** Int32 + +instance IsChType Int32 where + chTypeName = "Int32" + defaultValueOfTypeName = 0 + +instance Serializable Int32 where + serialize _ = int32LE + deserialize _ = getInt32le + {-# INLINE deserialize #-} + +instance ToQueryPart Int32 where + toQueryPart = byteString . BS8.pack . show + + +-- ** Int64 + +instance IsChType Int64 where + chTypeName = "Int64" + defaultValueOfTypeName = 0 + +instance Serializable Int64 where + serialize _ = int64LE + deserialize _ = getInt64le + {-# INLINE deserialize #-} + +instance ToQueryPart Int64 where + toQueryPart = byteString . BS8.pack . show + + +-- ** Int128 + +instance IsChType Int128 where + chTypeName = "Int128" + defaultValueOfTypeName = 0 + +instance Serializable Int128 where + serialize _ = (\(Int128 hi lo) -> word64LE lo <> word64LE hi) + deserialize _ = do + low <- getWord64le + high <- getWord64le + pure $ Int128 high low + {-# INLINE deserialize #-} + +instance ToQueryPart Int128 where + toQueryPart = byteString . BS8.pack . show + + +-- ** UInt8 + +{- | ClickHouse UInt8 column type -} +type UInt8 = Word8 +instance IsChType UInt8 where + chTypeName = "UInt8" + defaultValueOfTypeName = 0 + +instance Serializable UInt8 where + serialize _ = word8 + deserialize _ = getWord8 + {-# INLINE deserialize #-} + +instance ToQueryPart UInt8 where + toQueryPart = byteString . BS8.pack . show + + +-- ** UInt16 + +{- | ClickHouse UInt16 column type -} +type UInt16 = Word16 +instance IsChType UInt16 where + chTypeName = "UInt16" + defaultValueOfTypeName = 0 + +instance Serializable UInt16 where + serialize _ = word16LE + deserialize _ = getWord16le + {-# INLINE deserialize #-} + +instance ToQueryPart UInt16 where + toQueryPart = byteString . BS8.pack . show + + +-- ** UInt32 + +{- | ClickHouse UInt32 column type -} +type UInt32 = Word32 +instance IsChType UInt32 where + chTypeName = "UInt32" + defaultValueOfTypeName = 0 + +instance Serializable UInt32 where + serialize _ = word32LE + deserialize _ = getWord32le + {-# INLINE deserialize #-} + +instance ToQueryPart UInt32 where + toQueryPart = byteString . BS8.pack . show + + +-- ** UInt64 + +{- | ClickHouse UInt64 column type -} +type UInt64 = Word64 +instance IsChType UInt64 where + chTypeName = "UInt64" + defaultValueOfTypeName = 0 + +instance Serializable UInt64 where + serialize _ = word64LE + deserialize _ = getWord64le + {-# INLINE deserialize #-} + +instance ToQueryPart UInt64 where + toQueryPart = byteString . BS8.pack . show + + +-- ** UInt128 + +{- | ClickHouse UInt128 column type -} +type UInt128 = Word128 +instance IsChType UInt128 where + chTypeName = "UInt128" + defaultValueOfTypeName = 0 + +instance Serializable UInt128 where + serialize _ = (\(Word128 hi lo) -> word64LE lo <> word64LE hi) + deserialize _ = do + low <- getWord64le + high <- getWord64le + pure $ Word128 high low + {-# INLINE deserialize #-} + +instance ToQueryPart UInt128 where + toQueryPart w128 = "'" <> (byteString . BS8.pack . show) w128 <> "'" + + +-- ** Int256 + +{- | ClickHouse UInt128 column type -} +type UInt256 = Word256 +instance IsChType UInt256 where + chTypeName = "UInt256" + defaultValueOfTypeName = 0 + +instance Serializable UInt256 where + serialize _ = (\(Word256 high mid1 mid0 low) -> word64LE low <> word64LE mid0 <> word64LE mid1 <> word64LE high) + deserialize _ = do + low <- getWord64le + mid0 <- getWord64le + mid1 <- getWord64le + high <- getWord64le + pure $ Word256 high mid1 mid0 low + {-# INLINE deserialize #-} + +instance ToQueryPart UInt256 where + toQueryPart w256 = "'" <> (byteString . BS8.pack . show) w256 <> "'" + + +-- ** Date + +{- | ClickHouse Date column type -} +newtype Date = MkDate Word16 + deriving newtype (Show, Eq, Bits, Bounded, Enum, NFData, Num) + +instance IsChType Date where + chTypeName = "Date" + defaultValueOfTypeName = 0 + +instance Serializable Date where + serialize _ (MkDate w16) = word16LE w16 + deserialize _ = MkDate <$> getWord16le + {-# INLINE deserialize #-} + +instance ToChType Date Word16 where + toChType = MkDate + fromChType (MkDate w16) = w16 + + +-- ** ChString + +{- | ClickHouse String column type -} +newtype ChString = MkChString BS.ByteString + deriving newtype (Show, Eq, IsString, NFData) + +instance IsChType ChString where + chTypeName = "String" + defaultValueOfTypeName = "" + +instance Serializable ChString where + serialize rev (MkChString str) = (serialize @UVarInt rev . fromIntegral . BS.length) str <> byteString str + deserialize rev = do + len <- deserialize @UVarInt rev + MkChString <$> (getByteString . fromIntegral) len + {-# INLINE deserialize #-} + +instance ToChType ChString BS.ByteString where + toChType = MkChString + fromChType (MkChString string) = string + +instance ToChType ChString Builder where + toChType = MkChString . toStrict . toLazyByteString + fromChType (MkChString string) = byteString string + +instance ToChType ChString String where + toChType = MkChString . BS8.pack + fromChType (MkChString bs)= BS8.unpack bs + +instance ToQueryPart ChString where + toQueryPart (MkChString string) = "'" <> escapeQuery string <> "'" + where + escapeQuery :: BS.ByteString -> Builder + escapeQuery = byteString . BS8.concatMap (\sym -> + case sym of + '\'' -> "\\\'" + '\\' -> "\\\\" + _ -> BS8.singleton sym + ) + + + +-- ** UUID + +{- | ClickHouse UUID column type -} +newtype UUID = MkUUID Word128 + deriving newtype (Generic, Show, Eq, NFData, Bounded, Enum, Num) +instance IsChType UUID where + chTypeName = "UUID" + defaultValueOfTypeName = 0 + +instance Serializable UUID where + serialize _ = (\(MkUUID (Word128 hi lo)) -> word64LE lo <> word64LE hi) + deserialize _ = do + low <- getWord64le + high <- getWord64le + pure $ MkUUID (Word128 high low) + {-# INLINE deserialize #-} + +instance ToChType UUID (Word64, Word64) where + toChType = MkUUID . uncurry (flip Word128) + fromChType (MkUUID (Word128 w64hi w64lo)) = (w64hi, w64lo) + +instance ToQueryPart UUID where + toQueryPart (MkUUID (Word128 hi lo)) = mconcat + ["'", p 3 hi, p 2 hi, "-", p 1 hi, "-", p 0 hi, "-", p 3 lo, "-", p 2 lo, p 1 lo, p 0 lo, "'"] + where + p :: Int -> Word64 -> Builder + p shiftN word = word16HexFixed $ fromIntegral (word `unsafeShiftR` (shiftN*16)) + + +-- ** Nullable + +{- | ClickHouse Nullable(T) column type + (type synonym for Maybe) + -} +type Nullable = Maybe +instance IsChType chType => IsChType (Nullable chType) + where + chTypeName = "Nullable(" <> chTypeName @chType <> ")" + defaultValueOfTypeName = Nothing + +instance + ToChType inputType chType + => + ToChType (Nullable inputType) (Nullable chType) + where + toChType = fmap (toChType @inputType @chType) + fromChType = fmap (fromChType @inputType) + +instance ToQueryPart chType => ToQueryPart (Nullable chType) + where + toQueryPart = maybe "null" toQueryPart + + +-- ** Enum8 + +newtype Enum8 (enums :: Symbol) = MkEnum8 Int8 + deriving newtype (Serializable, Show, Eq, Num, Bits, Bounded, Enum) + +instance KnownSymbol enums => IsChType (Enum8 enums) where + chTypeName = "Enum8(" <> symbolVal @enums Proxy <> ")" + defaultValueOfTypeName = 0 + +instance ToChType (Enum8 enums) Int8 where + toChType = MkEnum8 + fromChType (MkEnum8 i8)= i8 + +instance ToQueryPart (Enum8 enums) where + toQueryPart = toQueryPart . fromChType @_ @Int8 + + +-- ** Enum16 + +newtype Enum16 (enums :: Symbol) = MkEnum16 Int16 + deriving newtype (Serializable, Show, Eq, Num, Bits, Bounded, Enum) + +instance KnownSymbol enums => IsChType (Enum16 enums) where + chTypeName = "Enum16(" <> symbolVal @enums Proxy <> ")" + defaultValueOfTypeName = 0 + +instance ToChType (Enum16 enums) Int16 where + toChType = MkEnum16 + fromChType (MkEnum16 i16) = i16 + +instance ToQueryPart (Enum16 enums) where + toQueryPart = toQueryPart . fromChType @_ @Int16 + + +-- ** Bool + +instance IsChType Bool where + chTypeName = "Bool" + defaultValueOfTypeName = False + +instance Serializable Bool where + serialize _ = int8 . bool 0 1 + deserialize _ = (\int -> case int of 0->False; _->True) <$> getInt8 + {-# INLINE deserialize #-} + +instance ToQueryPart Bool where + toQueryPart = bool "false" "true" + + +-- ** DateTime + +{- | +ClickHouse DateTime column type (parametrized with timezone) + +>>> chTypeName @(DateTime "") +"DateTime" +>>> chTypeName @(DateTime "UTC") +"DateTime('UTC')" + +__Note:__ 'DateTime' stores whole seconds only, so converting from 'UTCTime' \ +will drop any sub-second precision. + +>>> let myUtcTime = posixSecondsToUTCTime 0.042_042 +>>> toChType @(DateTime "") @UTCTime myUtcTime +0 +-} +newtype DateTime (tz :: Symbol) = MkDateTime Word32 + deriving newtype (Show, Eq, Num, Bits, Enum, Ord, Real, Integral, Bounded, NFData) + +instance KnownSymbol tz => IsChType (DateTime tz) + where + chTypeName = case symbolVal @tz Proxy of + "" -> "DateTime" + tz -> "DateTime('" <> tz <> "')" + defaultValueOfTypeName = MkDateTime 0 + +instance Serializable (DateTime tz) where + serialize _ (MkDateTime w32) = word32LE w32 + deserialize _ = MkDateTime <$> getWord32le + {-# INLINE deserialize #-} + +instance ToChType (DateTime tz) Word32 where + toChType = MkDateTime + fromChType (MkDateTime w32)= w32 + +instance ToChType (DateTime tz) UTCTime where + toChType = MkDateTime . floor . utcTimeToPOSIXSeconds + fromChType (MkDateTime w32) = posixSecondsToUTCTime (fromIntegral w32) + +instance ToQueryPart (DateTime tz) + where + toQueryPart chDateTime = let time = BS8.pack . show . coerce @(DateTime tz) @Word32 $ chDateTime + in byteString (BS8.replicate (10 - BS8.length time) '0' <> time) + + +-- ** DateTime64 + +{- | +ClickHouse DateTime64 column type (parametrized with timezone) + +>>> chTypeName @(DateTime64 3 "") +"DateTime64(3)" +>>> chTypeName @(DateTime64 3 "UTC") +"DateTime64(3, 'UTC')" + +__Note:__ conversion from 'UTCTime' may lose sub-second precision if \ +the @precision@ parameter is lower than the actual timestamp precision. + +>>> let myUtcTime = posixSecondsToUTCTime 42.000_000_042 +>>> toChType @(DateTime64 6 "") @UTCTime myUtcTime +42000000 +>>> toChType @(DateTime64 9 "") @UTCTime myUtcTime +42000000042 +-} +newtype DateTime64 (precision :: Nat) (tz :: Symbol) = MkDateTime64 Word64 + deriving newtype (Show, Eq, Num, Bits, Enum, Ord, Real, Integral, Bounded, NFData) + +instance + (KnownSymbol tz, KnownNat precision) + => + IsChType (DateTime64 precision tz) + where + chTypeName = + let + prec = show (natVal @precision Proxy) + in + case symbolVal @tz Proxy of + "" -> "DateTime64(" <> prec <> ")" + tz -> "DateTime64(" <> prec <> ", '" <> tz <> "')" + defaultValueOfTypeName = MkDateTime64 0 + +instance Serializable (DateTime64 precision tz) where + serialize _ (MkDateTime64 w64) = word64LE w64 + deserialize _ = MkDateTime64 <$> getWord64le + {-# INLINE deserialize #-} + +instance ToChType (DateTime64 precision tz) Word64 where + toChType = MkDateTime64 + fromChType (MkDateTime64 w64) = w64 + +instance KnownNat precision => ToChType (DateTime64 precision tz) UTCTime where + toChType = MkDateTime64 . floor . (* (10 ^ natVal (Proxy @precision))) + . utcTimeToPOSIXSeconds + fromChType (MkDateTime64 w64) = posixSecondsToUTCTime + $ (/ (10 ^ natVal (Proxy @precision))) $ fromIntegral w64 + +-- ToDo: Need to be fixed +-- instance ToQueryPart (DateTime64 precision tz) +-- where +-- toQueryPart chDateTime = +-- let time = BS8.pack . show . fromChType @_ @Word64 $ chDateTime +-- in byteString (BS8.replicate (12 - BS8.length time) '0' <> time) + + +-- ** Array + +-- | ClickHouse Array column type +newtype Array a = MkChArray [a] + deriving newtype (Show, Eq, NFData, Foldable) +instance IsChType chType => IsChType (Array chType) + where + chTypeName = "Array(" <> chTypeName @chType <> ")" + defaultValueOfTypeName = MkChArray [] + +instance ToChType chType inputType => ToChType (Array chType) [inputType] + where + toChType = MkChArray . map toChType + fromChType (MkChArray values) = map fromChType values + +instance (IsChType chType, ToQueryPart chType) => ToQueryPart (Array chType) + where + toQueryPart + = (\x -> "[" <> x <> "]") + . (maybe "" (uncurry (foldl (\a b -> a <> "," <> b))) . uncons + . map (toQueryPart @chType)) . coerce @(Array chType) @[chType] + + +-- ** LowCardinality + +-- | ClickHouse LowCardinality(T) column type +newtype LowCardinality chType = MkLowCardinality chType +instance IsLowCardinalitySupported chType => IsChType (LowCardinality chType) + where + chTypeName = "LowCardinality(" <> chTypeName @chType <> ")" + defaultValueOfTypeName = MkLowCardinality $ defaultValueOfTypeName @chType + +deriving newtype instance (Eq chType, IsLowCardinalitySupported chType) => Eq (LowCardinality chType) +deriving newtype instance (NFData chType, IsLowCardinalitySupported chType) => NFData (LowCardinality chType) +deriving newtype instance IsString (LowCardinality ChString) + +class IsChType chType => IsLowCardinalitySupported chType +instance IsLowCardinalitySupported ChString +instance + ( IsLowCardinalitySupported chType + , IsChType (Nullable chType) + ) => + IsLowCardinalitySupported (Nullable chType) + +instance {-# OVERLAPPABLE #-} + ( IsChType chType + , TypeError + ( 'Text "LowCardinality(" ':<>: 'ShowType chType ':<>: 'Text ") is unsupported" + ':$$: 'Text "Use one of these types:" + ':$$: 'Text " ChString" + ':$$: 'Text " DateTime" + ':$$: 'Text " Nullable(T)" + ) + ) => IsLowCardinalitySupported chType + +instance + ToChType inputType chType + => + ToChType (LowCardinality inputType) chType where + toChType = MkLowCardinality . toChType + fromChType (MkLowCardinality lc)= fromChType @inputType lc + +instance ToQueryPart chType => ToQueryPart (LowCardinality chType) + where + toQueryPart (MkLowCardinality chType) = toQueryPart chType + + +-- ** Generics + +class GSerial f where + gSerialize :: ProtocolRevision -> f p -> Builder + gDeserialize :: ProtocolRevision -> Get (f p) + +instance GSerial f => GSerial (D1 c (C1 c2 f)) where + gSerialize rev (M1 (M1 re)) = gSerialize rev re + {-# INLINE gSerialize #-} + gDeserialize rev = M1 . M1 <$> gDeserialize rev + {-# INLINE gDeserialize #-} + +instance (GSerial left1, GSerial right) => GSerial (left1 :*: right) where + gSerialize rev (l :*: r) = gSerialize rev l <> gSerialize rev r + {-# INLINE gSerialize #-} + gDeserialize rev = do + liftA2 (:*:) + (gDeserialize rev) + (gDeserialize rev) + {-# INLINE gDeserialize #-} + +instance + (Serializable chType, Not (sel == "server_revision") ~ True) + => + GSerial (S1 ('MetaSel ('Just sel) a b c) (Rec0 chType)) where + gSerialize rev (M1 (K1 re)) = serialize rev re + {-# INLINE gSerialize #-} + gDeserialize rev = M1 . K1 <$> deserialize @chType rev + {-# INLINE gDeserialize #-} + + +instance {-# OVERLAPPING #-} + GSerial right + => + GSerial (S1 ('MetaSel ('Just "server_revision") a b c) (Rec0 ProtocolRevision) :*: right) + where + gSerialize rev (M1 (K1 (MkProtocolRevision server_rev)) :*: right)= do + serialize rev server_rev <> gSerialize rev right + {-# INLINE gSerialize #-} + gDeserialize rev = do + chosenRev <- min rev . MkProtocolRevision <$> deserialize @UVarInt rev + liftA2 (:*:) + (pure . M1 . K1 $ chosenRev) + (gDeserialize @right chosenRev) + {-# INLINE gDeserialize #-} + + + + +-- * Protocol parts + +-- ** UVarInt + +{- | + Unsigned variable-length quantity encoding + + Part of protocol implementation +-} +newtype UVarInt = MkUVarInt Word64 + deriving newtype (Show, Eq, Num, Bits, Enum, Ord, Real, Integral, Bounded, NFData) + +instance Serializable UVarInt where + serialize _ = goUVarIntSer + where + goUVarIntSer i + | i < 0x80 = word8 (fromIntegral i) + | otherwise = word8 (setBit (fromIntegral i) 7) <> goUVarIntSer (i `unsafeShiftR` 7) + deserialize _ = goUVarIntDeser 0 (0 :: UVarInt) + where + goUVarIntDeser i o | i < 10 = do + byte <- getWord8 + let o' = o .|. ((fromIntegral byte .&. 0x7f) `unsafeShiftL` (7 * i)) + if byte .&. 0x80 == 0 then pure $! o' else goUVarIntDeser (i + 1) $! o' + goUVarIntDeser _ _ = fail "input exceeds varuint size" + {-# INLINE deserialize #-} + + +-- ** Versioning + +-- >>> show major <> "." <> show minor <> "." <> show patch +-- "1.0.0" +major, minor, patch :: UVarInt +major = 1 +minor = 0 +patch = 0 + +clientName :: ChString +clientName = "ClickHaskell" + +newtype ProtocolRevision = MkProtocolRevision UVarInt + deriving newtype (Eq, Num, Ord, Serializable) + +mkRev :: forall nat . KnownNat nat => ProtocolRevision +mkRev = (fromIntegral . natVal) (Proxy @nat) + +{- | +Protocol implementation part for backward compatilibity formalization + +NB: + + Be carefull with `BeforeRevision` value. + + If **chosen protocol** revision would be >= **revisionNumber** + then you would get an exception during serialization. + + To avoid this: + + 1. On client side - provide `AfterRevision` with some empty value + 2. On proxy side - provide server-to-server packets mapping with fallbacks on revision upgrade + +-} +data Revisioned (revisionNumber :: Nat) b a = BeforeRevision b | AfterRevision a + +type SinceRevision after rev = Revisioned rev () after + +instance + (KnownNat revision, Serializable before, Serializable after) + => + Serializable (Revisioned revision before after) + where + serialize rev sinceRevVal = + if rev < mkRev @revision + then mempty + else case sinceRevVal of + BeforeRevision _b -> error "Protocol-specific implementation error" -- Watch `Revisioned` note + AfterRevision a -> serialize rev a + deserialize rev = + if rev < mkRev @revision + then BeforeRevision <$> deserialize @before rev + else AfterRevision <$> deserialize @after rev + +instance Serializable () where + serialize _ () = "" + deserialize _ = pure () + + +{- + Slightly modified C++ sources: + https://github.com/ClickHouse/ClickHouse/blob/eb4a74d7412a1fcf52727cd8b00b365d6b9ed86c/src/Core/ProtocolDefines.h#L6 +-} +type DBMS_TCP_PROTOCOL_VERSION = DBMS_MIN_PROTOCOL_VERSION_WITH_PARAMETERS; + +type DBMS_MIN_REVISION_WITH_CLIENT_INFO = 54032; +type DBMS_MIN_REVISION_WITH_SERVER_TIMEZONE = 54058; +type DBMS_MIN_REVISION_WITH_QUOTA_KEY_IN_CLIENT_INFO = 54060; +type DBMS_MIN_REVISION_WITH_TABLES_STATUS = 54226; +type DBMS_MIN_REVISION_WITH_TIME_ZONE_PARAMETER_IN_DATETIME_DATA_TYPE = 54337; +type DBMS_MIN_REVISION_WITH_SERVER_DISPLAY_NAME = 54372; +type DBMS_MIN_REVISION_WITH_VERSION_PATCH = 54401; +type DBMS_MIN_REVISION_WITH_SERVER_LOGS = 54406; +type DBMS_MIN_REVISION_WITH_CURRENT_AGGREGATION_VARIANT_SELECTION_METHOD = 54448; +type DBMS_MIN_MAJOR_VERSION_WITH_CURRENT_AGGREGATION_VARIANT_SELECTION_METHOD = 21; +type DBMS_MIN_MINOR_VERSION_WITH_CURRENT_AGGREGATION_VARIANT_SELECTION_METHOD = 4; +type DBMS_MIN_REVISION_WITH_COLUMN_DEFAULTS_METADATA = 54410; +type DBMS_MIN_REVISION_WITH_LOW_CARDINALITY_TYPE = 54405; +type DBMS_MIN_REVISION_WITH_CLIENT_WRITE_INFO = 54420; +type DBMS_MIN_REVISION_WITH_SETTINGS_SERIALIZED_AS_STRINGS = 54429; +type DBMS_MIN_REVISION_WITH_SCALARS = 54429; +type DBMS_MIN_REVISION_WITH_OPENTELEMETRY = 54442; +type DBMS_MIN_REVISION_WITH_AGGREGATE_FUNCTIONS_VERSIONING = 54452; +type DBMS_CLUSTER_INITIAL_PROCESSING_PROTOCOL_VERSION = 1; +type DBMS_CLUSTER_PROCESSING_PROTOCOL_VERSION_WITH_DATA_LAKE_METADATA = 2; +type DBMS_CLUSTER_PROCESSING_PROTOCOL_VERSION = 2; +type DATA_LAKE_TABLE_STATE_SNAPSHOT_PROTOCOL_VERSION = 1; +type DBMS_MIN_SUPPORTED_PARALLEL_REPLICAS_PROTOCOL_VERSION = 3; +type DBMS_PARALLEL_REPLICAS_MIN_VERSION_WITH_MARK_SEGMENT_SIZE_FIELD = 4; +type DBMS_PARALLEL_REPLICAS_MIN_VERSION_WITH_PROJECTION = 5; +type DBMS_PARALLEL_REPLICAS_PROTOCOL_VERSION = 5; +type DBMS_MIN_REVISION_WITH_PARALLEL_REPLICAS = 54453; +type DBMS_MIN_REVISION_WITH_QUERY_AND_LINE_NUMBERS = 54475; +type DBMS_MERGE_TREE_PART_INFO_VERSION = 1; +type DBMS_QUERY_PLAN_SERIALIZATION_VERSION = 0; +type DBMS_MIN_REVISION_WITH_INTERSERVER_SECRET = 54441; +type DBMS_MIN_REVISION_WITH_X_FORWARDED_FOR_IN_CLIENT_INFO = 54443; +type DBMS_MIN_REVISION_WITH_REFERER_IN_CLIENT_INFO = 54447; +type DBMS_MIN_PROTOCOL_VERSION_WITH_DISTRIBUTED_DEPTH = 54448; +type DBMS_MIN_PROTOCOL_VERSION_WITH_INCREMENTAL_PROFILE_EVENTS = 54451; +type DBMS_MIN_REVISION_WITH_CUSTOM_SERIALIZATION = 54454; +type DBMS_MIN_PROTOCOL_VERSION_WITH_INITIAL_QUERY_START_TIME = 54449; +type DBMS_MIN_PROTOCOL_VERSION_WITH_PROFILE_EVENTS_IN_INSERT = 54456; +type DBMS_MIN_PROTOCOL_VERSION_WITH_VIEW_IF_PERMITTED = 54457; +type DBMS_MIN_PROTOCOL_VERSION_WITH_ADDENDUM = 54458; +type DBMS_MIN_PROTOCOL_VERSION_WITH_QUOTA_KEY = 54458; +type DBMS_MIN_PROTOCOL_VERSION_WITH_PARAMETERS = 54459; +type DBMS_MIN_PROTOCOL_VERSION_WITH_SERVER_QUERY_TIME_IN_PROGRESS = 54460; +type DBMS_MIN_PROTOCOL_VERSION_WITH_PASSWORD_COMPLEXITY_RULES = 54461; +type DBMS_MIN_REVISION_WITH_INTERSERVER_SECRET_V2 = 54462; +type DBMS_MIN_PROTOCOL_VERSION_WITH_TOTAL_BYTES_IN_PROGRESS = 54463; +type DBMS_MIN_PROTOCOL_VERSION_WITH_TIMEZONE_UPDATES = 54464; +type DBMS_MIN_REVISION_WITH_SPARSE_SERIALIZATION = 54465; -- ToDo +type DBMS_MIN_REVISION_WITH_SSH_AUTHENTICATION = 54466; +type DBMS_MIN_REVISION_WITH_TABLE_READ_ONLY_CHECK = 54467; +type DBMS_MIN_REVISION_WITH_SYSTEM_KEYWORDS_TABLE = 54468; +type DBMS_MIN_REVISION_WITH_ROWS_BEFORE_AGGREGATION = 54469; +type DBMS_MIN_PROTOCOL_VERSION_WITH_CHUNKED_PACKETS = 54470; +type DBMS_MIN_REVISION_WITH_VERSIONED_PARALLEL_REPLICAS_PROTOCOL = 54471; +type DBMS_MIN_PROTOCOL_VERSION_WITH_INTERSERVER_EXTERNALLY_GRANTED_ROLES = 54472; +type DBMS_MIN_REVISION_WITH_V2_DYNAMIC_AND_JSON_SERIALIZATION = 54473; +type DBMS_MIN_REVISION_WITH_SERVER_SETTINGS = 54474; +type DBMS_MIN_REVISON_WITH_JWT_IN_INTERSERVER = 54476; +type DBMS_MIN_REVISION_WITH_QUERY_PLAN_SERIALIZATION = 54477; +type DBMS_MIN_REVISON_WITH_PARALLEL_BLOCK_MARSHALLING = 54478; +type DBMS_MIN_REVISION_WITH_VERSIONED_CLUSTER_FUNCTION_PROTOCOL = 54479; +type DBMS_MIN_REVISION_WITH_OUT_OF_ORDER_BUCKETS_IN_AGGREGATION = 54480; +type DBMS_MIN_REVISION_WITH_COMPRESSED_LOGS_PROFILE_EVENTS_COLUMNS = 54481; diff --git a/ClickHaskell/Protocol.hs b/ClickHaskell/Protocol.hs new file mode 100644 --- /dev/null +++ b/ClickHaskell/Protocol.hs @@ -0,0 +1,36 @@ +module ClickHaskell.Protocol + ( + {- * Protocol parts -} + + {- ** Shared -} + UVarInt(..), Revisioned(..), ProtocolRevision + {- *** Data packet -}, DataPacket(..), BlockInfo(..) + + {- ** Client -}, ClientPacket(..) + {- *** Hello -}, HelloPacket(..), Addendum(..) + {- *** Query -} + , QueryPacket(..) + , DbSettings(..), QueryParameters(..), QueryStage(..) + , ClientInfo(..), QueryKind(..), Jwt(..) + + {- ** Server -}, ServerPacket(..) + {- *** Hello -}, HelloResponse(..), PasswordComplexityRules(..) + {- *** Exception -}, ExceptionPacket(..) + {- *** Progress -}, ProgressPacket(..) + {- *** ProfileInfo -}, ProfileInfo(..) + {- *** TableColumns -}, TableColumns(..) + + {- * API -} + , addSetting + , mkDataPacket + , mkQueryPacket, QueryPacketArgs(..) + , mkHelloPacket + , mkAddendum + , serverPacketToNum + ) where + +import ClickHaskell.Primitive +import ClickHaskell.Protocol.Client +import ClickHaskell.Protocol.Data +import ClickHaskell.Protocol.Server +import ClickHaskell.Protocol.Settings diff --git a/ClickHaskell/Protocol/Client.hs b/ClickHaskell/Protocol/Client.hs new file mode 100644 --- /dev/null +++ b/ClickHaskell/Protocol/Client.hs @@ -0,0 +1,247 @@ +module ClickHaskell.Protocol.Client where + +-- Internal +import ClickHaskell.Primitive +import ClickHaskell.Protocol.Data (DataPacket) +import ClickHaskell.Protocol.Settings (DbSettings(..)) + +-- GHC +import Data.Int +import GHC.Generics +import Data.ByteString as BS (null) + + +-- * Client packets + +data ClientPacket where + Hello :: HelloPacket -> ClientPacket + Query :: QueryPacket -> ClientPacket + Data :: DataPacket -> ClientPacket + Cancel :: ClientPacket + Ping :: ClientPacket + TablesStatusRequest :: ClientPacket + KeepAlive :: ClientPacket + Scalar :: ClientPacket + IgnoredPartUUIDs :: ClientPacket + ReadTaskResponse :: ClientPacket + MergeTreeReadTaskResponse :: ClientPacket + SSHChallengeRequest :: ClientPacket + SSHChallengeResponse :: ClientPacket + deriving (Generic) + +instance Serializable ClientPacket where + serialize rev packet = case packet of + (Hello p) -> serialize @UVarInt rev 0 <> serialize rev p + (Query p) -> serialize @UVarInt rev 1 <> serialize rev p + (Data p) -> serialize @UVarInt rev 2 <> serialize rev p + (Cancel) -> serialize @UVarInt rev 3 + (Ping) -> serialize @UVarInt rev 4 + (TablesStatusRequest) -> serialize @UVarInt rev 5 + (KeepAlive) -> serialize @UVarInt rev 6 + (Scalar) -> serialize @UVarInt rev 7 + (IgnoredPartUUIDs) -> serialize @UVarInt rev 8 + (ReadTaskResponse) -> serialize @UVarInt rev 9 + (MergeTreeReadTaskResponse) -> serialize @UVarInt rev 10 + (SSHChallengeRequest) -> serialize @UVarInt rev 11 + (SSHChallengeResponse) -> serialize @UVarInt rev 12 + deserialize rev = do + code <- deserialize @UVarInt rev + case code of + 0 -> Hello <$> deserialize rev + 1 -> Query <$> deserialize rev + 2 -> Data <$> deserialize rev + 3 -> pure Cancel + 4 -> pure Ping + 5 -> pure TablesStatusRequest + 6 -> pure KeepAlive + 7 -> pure Scalar + 8 -> pure IgnoredPartUUIDs + 9 -> pure ReadTaskResponse + 10 -> pure MergeTreeReadTaskResponse + 11 -> pure SSHChallengeRequest + 12 -> pure SSHChallengeResponse + num -> fail ("Unknown client packet " <> show num) + +-- ** Hello + +data HelloPacket = MkHelloPacket + { client_name :: ChString + , client_version_major :: UVarInt + , client_version_minor :: UVarInt + , tcp_protocol_version :: ProtocolRevision + , default_database :: ChString + , user :: ChString + , pass :: ChString + } + deriving (Generic, Serializable) + +mkHelloPacket :: String -> String -> String -> ProtocolRevision -> ClientPacket +mkHelloPacket db user pass rev = + Hello + MkHelloPacket + { client_name = clientName + , client_version_major = major + , client_version_minor = minor + , tcp_protocol_version = rev + , default_database = toChType db + , user = toChType user + , pass = toChType pass + } + + +data Addendum = MkAddendum + { quota_key :: ChString `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_QUOTA_KEY + , proto_send_chunked :: ChString `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_CHUNKED_PACKETS + , proto_recv_chunked :: ChString `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_CHUNKED_PACKETS + , replicas_version :: ProtocolRevision `SinceRevision` DBMS_MIN_REVISION_WITH_VERSIONED_PARALLEL_REPLICAS_PROTOCOL + } + deriving (Generic, Serializable) + +mkAddendum :: Addendum +mkAddendum = MkAddendum + { quota_key = AfterRevision "" + , proto_send_chunked = AfterRevision "notchunked" + , proto_recv_chunked = AfterRevision "notchunked" + , replicas_version = AfterRevision $ mkRev @DBMS_PARALLEL_REPLICAS_PROTOCOL_VERSION + } + +-- ** Query + +data QueryPacket = MkQueryPacket + { query_id :: ChString + , client_info :: ClientInfo `SinceRevision` DBMS_MIN_REVISION_WITH_CLIENT_INFO + , settings :: DbSettings + , external_roles :: UVarInt `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_INTERSERVER_EXTERNALLY_GRANTED_ROLES + , interserver_secret :: ChString `SinceRevision` DBMS_MIN_REVISION_WITH_INTERSERVER_SECRET + , query_stage :: QueryStage + , compression :: UVarInt + , query :: ChString + , parameters :: QueryParameters `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_PARAMETERS + } + deriving (Generic, Serializable) + +data QueryPacketArgs = MkQueryPacketArgs + { initial_user :: ChString + , hostname :: ChString + , os_user :: ChString + , query :: ChString + , settings :: DbSettings + , revision :: ProtocolRevision + } + +mkQueryPacket :: QueryPacketArgs -> ClientPacket +mkQueryPacket MkQueryPacketArgs{initial_user, os_user, hostname, query, settings, revision} = + Query + MkQueryPacket + { query_id = "" + , client_info = AfterRevision MkClientInfo + { query_kind = InitialQuery + , initial_user + , initial_query_id = "" + , initial_adress = "0.0.0.0:0" + , initial_time = AfterRevision 0 + , interface_type = 1 -- [tcp - 1, http - 2] + , os_user + , hostname + , client_name = clientName + , client_version_major = major + , client_version_minor = minor + , client_revision = revision + , quota_key = AfterRevision "" + , distrubuted_depth = AfterRevision 0 + , client_version_patch = AfterRevision patch + , open_telemetry = AfterRevision 0 + , collaborate_with_initiator = AfterRevision 0 + , count_participating_replicas = AfterRevision 0 + , number_of_current_replica = AfterRevision 0 + , script_query_number = AfterRevision 0 + , script_line_number = AfterRevision 0 + , jwt = AfterRevision (MkJwt "") + } + , settings + , interserver_secret = AfterRevision "" + , query_stage = Complete + , compression = 0 + , query + , parameters = AfterRevision MkQueryParameters + , external_roles = AfterRevision 0 + } + +data QueryParameters = MkQueryParameters +instance Serializable QueryParameters where + serialize rev _ = + serialize @ChString rev "" + deserialize _rev = + fail "QueryParameters reading unimplemented" + +data QueryStage + = FetchColumns | WithMergeableState | Complete + | WithMergeableStateAfterAggregation + | WithMergeableStateAfterAggregationAndLimit + deriving (Enum) + +instance Serializable QueryStage where + serialize rev = serialize @UVarInt rev . fromIntegral . fromEnum + deserialize rev = do + queryStageCode <- deserialize @UVarInt rev + case queryStageCode of + 0 -> pure FetchColumns + 1 -> pure WithMergeableState + 2 -> pure Complete + 3 -> pure WithMergeableStateAfterAggregation + 4 -> pure WithMergeableStateAfterAggregationAndLimit + num -> fail ("Unknown QueryStage " <> show num) + + +data ClientInfo = MkClientInfo + { query_kind :: QueryKind + , initial_user :: ChString + , initial_query_id :: ChString + , initial_adress :: ChString + , initial_time :: Int64 `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_INITIAL_QUERY_START_TIME + , interface_type :: UInt8 + , os_user :: ChString + , hostname :: ChString + , client_name :: ChString + , client_version_major :: UVarInt + , client_version_minor :: UVarInt + , client_revision :: ProtocolRevision + , quota_key :: ChString `SinceRevision` DBMS_MIN_REVISION_WITH_QUOTA_KEY_IN_CLIENT_INFO + , distrubuted_depth :: UVarInt `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_DISTRIBUTED_DEPTH + , client_version_patch :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_VERSION_PATCH + , open_telemetry :: UInt8 `SinceRevision` DBMS_MIN_REVISION_WITH_OPENTELEMETRY + , collaborate_with_initiator :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_PARALLEL_REPLICAS + , count_participating_replicas :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_PARALLEL_REPLICAS + , number_of_current_replica :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_PARALLEL_REPLICAS + , script_query_number :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_QUERY_AND_LINE_NUMBERS + , script_line_number :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_QUERY_AND_LINE_NUMBERS + , jwt :: Jwt `SinceRevision` DBMS_MIN_REVISON_WITH_JWT_IN_INTERSERVER + } + deriving (Generic, Serializable) + +data QueryKind = NoQuery | InitialQuery | SecondaryQuery +instance Serializable QueryKind where + serialize rev kind = serialize @UInt8 rev $ + case kind of + NoQuery -> 1 + InitialQuery -> 2 + SecondaryQuery -> 3 + deserialize rev = do + kindCode <- deserialize @UInt8 rev + case kindCode of + 1 -> pure NoQuery + 2 -> pure InitialQuery + 3 -> pure SecondaryQuery + num -> fail ("Unknown QueryKind " <> show num) + +data Jwt = MkJwt ChString +instance Serializable Jwt where + serialize rev (MkJwt str@(MkChString bs)) = + if not (BS.null bs) + then serialize @UInt8 rev 1 <> serialize rev str + else serialize @UInt8 rev 0 + deserialize rev = do + has_jwt <- deserialize @UInt8 rev + if has_jwt == 1 + then MkJwt <$> deserialize rev + else pure $ MkJwt "" diff --git a/ClickHaskell/Protocol/Data.hs b/ClickHaskell/Protocol/Data.hs new file mode 100644 --- /dev/null +++ b/ClickHaskell/Protocol/Data.hs @@ -0,0 +1,38 @@ +module ClickHaskell.Protocol.Data where + +-- Internal +import ClickHaskell.Primitive + +-- GHC +import Data.Int +import GHC.Generics + +-- * Common Data packet + +data DataPacket = MkDataPacket + { table_name :: ChString + , block_info :: BlockInfo + , columns_count :: UVarInt + , rows_count :: UVarInt + } + deriving (Generic, Serializable) + +mkDataPacket :: ChString -> UVarInt -> UVarInt -> DataPacket +mkDataPacket table_name columns_count rows_count = + MkDataPacket + { table_name + , block_info = MkBlockInfo + { field_num1 = 1, is_overflows = 0 + , field_num2 = 2, bucket_num = -1 + , eof = 0 + } + , columns_count + , rows_count + } + +data BlockInfo = MkBlockInfo + { field_num1 :: UVarInt, is_overflows :: UInt8 + , field_num2 :: UVarInt, bucket_num :: Int32 + , eof :: UVarInt + } + deriving (Generic, Serializable) diff --git a/ClickHaskell/Protocol/Server.hs b/ClickHaskell/Protocol/Server.hs new file mode 100644 --- /dev/null +++ b/ClickHaskell/Protocol/Server.hs @@ -0,0 +1,182 @@ +module ClickHaskell.Protocol.Server where + +-- Internal +import ClickHaskell.Primitive +import ClickHaskell.Protocol.Data (DataPacket(..)) +import ClickHaskell.Protocol.Settings (DbSettings) + +-- GHC +import Data.Int +import GHC.Generics +import ClickHaskell.Columns (Column, deserializeColumn, ColumnHeader, serializeColumn, mkHeader) +import Control.Monad (when) + +-- * Server packets + +data ServerPacket where + HelloResponse :: HelloResponse -> ServerPacket + DataResponse :: DataPacket -> ServerPacket + Exception :: ExceptionPacket -> ServerPacket + Progress :: ProgressPacket -> ServerPacket + Pong :: ServerPacket + EndOfStream :: ServerPacket + ProfileInfo :: ProfileInfo -> ServerPacket + Totals :: ServerPacket + Extremes :: ServerPacket + TablesStatusResponse :: ServerPacket + Log :: ServerPacket + TableColumns :: TableColumns -> ServerPacket + UUIDs :: ServerPacket + ReadTaskRequest :: ServerPacket + ProfileEvents :: ProfileEventsPacket -> ServerPacket + UnknownPacket :: UVarInt -> ServerPacket + +instance Serializable ServerPacket where + serialize rev packet = case packet of + HelloResponse p -> serialize @UVarInt rev 0 <> serialize rev p + DataResponse p -> serialize @UVarInt rev 1 <> serialize rev p + Exception p -> serialize @UVarInt rev 2 <> serialize rev p + Progress p -> serialize @UVarInt rev 3 <> serialize rev p + Pong -> serialize @UVarInt rev 4 + EndOfStream -> serialize @UVarInt rev 5 + ProfileInfo p -> serialize @UVarInt rev 6 <> serialize rev p + Totals -> serialize @UVarInt rev 7 + Extremes -> serialize @UVarInt rev 8 + TablesStatusResponse -> serialize @UVarInt rev 9 + Log -> serialize @UVarInt rev 10 + TableColumns p -> serialize @UVarInt rev 11 <> serialize rev p + UUIDs -> serialize @UVarInt rev 12 + ReadTaskRequest -> serialize @UVarInt rev 13 + ProfileEvents p -> serialize @UVarInt rev 14 <> serialize rev p + UnknownPacket num -> serialize @UVarInt rev num + deserialize rev = do + packetNum <- deserialize @UVarInt rev + case packetNum of + 0 -> HelloResponse <$> deserialize rev + 1 -> DataResponse <$> deserialize rev + 2 -> Exception <$> deserialize rev + 3 -> Progress <$> deserialize rev + 4 -> pure Pong + 5 -> pure EndOfStream + 6 -> ProfileInfo <$> deserialize rev + 7 -> pure Totals + 8 -> pure Extremes + 9 -> pure TablesStatusResponse + 10 -> pure Log + 11 -> TableColumns <$> deserialize rev + 12 -> pure UUIDs + 13 -> pure ReadTaskRequest + 14 -> ProfileEvents <$> deserialize rev + _ -> pure $ UnknownPacket packetNum + +serverPacketToNum :: ServerPacket -> UVarInt +serverPacketToNum p = case p of + (HelloResponse _) -> 0; (DataResponse _) -> 1 + (Exception _) -> 2; (Progress _) -> 3; + (Pong) -> 4; (EndOfStream) -> 5 + (ProfileInfo _) -> 6; (Totals) -> 7 + (Extremes) -> 8; (TablesStatusResponse) -> 9 + (Log) -> 10; (TableColumns _) -> 11; + (UUIDs) -> 12; (ReadTaskRequest) -> 13 + (ProfileEvents _) -> 14; (UnknownPacket num) -> num + + +{- + https://github.com/ClickHouse/ClickHouse/blob/eb4a74d7412a1fcf52727cd8b00b365d6b9ed86c/src/Client/Connection.cpp#L520 +-} +data HelloResponse = MkHelloResponse + { server_name :: ChString + , server_version_major :: UVarInt + , server_version_minor :: UVarInt + , server_revision :: ProtocolRevision + , server_parallel_replicas_proto :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_VERSIONED_PARALLEL_REPLICAS_PROTOCOL + , server_timezone :: ChString `SinceRevision` DBMS_MIN_REVISION_WITH_SERVER_TIMEZONE + , server_display_name :: ChString `SinceRevision` DBMS_MIN_REVISION_WITH_SERVER_DISPLAY_NAME + , server_version_patch :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_VERSION_PATCH + , proto_send_chunked_srv :: ChString `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_CHUNKED_PACKETS + , proto_recv_chunked_srv :: ChString `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_CHUNKED_PACKETS + , password_complexity_rules :: [PasswordComplexityRules] `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_PASSWORD_COMPLEXITY_RULES + , read_nonce :: UInt64 `SinceRevision` DBMS_MIN_REVISION_WITH_INTERSERVER_SECRET_V2 + , settings :: DbSettings `SinceRevision` DBMS_MIN_REVISION_WITH_SERVER_SETTINGS + , server_query_plan_serialization_version :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_QUERY_PLAN_SERIALIZATION + } + deriving (Generic, Serializable) + +data PasswordComplexityRules = MkPasswordComplexityRules + { original_pattern :: ChString + , exception_message :: ChString + } + deriving (Generic, Serializable) + + +data ExceptionPacket = MkExceptionPacket + { code :: Int32 + , name :: ChString + , message :: ChString + , stack_trace :: ChString + , nested :: UInt8 + } + deriving (Generic, Show, Serializable) + +data ProgressPacket = MkProgressPacket + { rows :: UVarInt + , bytes :: UVarInt + , total_rows :: UVarInt + , total_bytes :: UVarInt `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_TOTAL_BYTES_IN_PROGRESS + , wrote_rows :: UVarInt `SinceRevision` DBMS_MIN_PROTOCOL_VERSION_WITH_TOTAL_BYTES_IN_PROGRESS + , wrote_bytes :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_CLIENT_WRITE_INFO + , elapsed_ns :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_CLIENT_WRITE_INFO + } + deriving (Generic, Serializable) + +data ProfileInfo = MkProfileInfo + { rows :: UVarInt + , blocks :: UVarInt + , bytes :: UVarInt + , applied_limit :: UInt8 + , rows_before_limit :: UVarInt + , calculated_rows_before_limit :: UInt8 + , applied_aggregation :: UInt8 `SinceRevision` DBMS_MIN_REVISION_WITH_ROWS_BEFORE_AGGREGATION + , rows_before_aggregation :: UVarInt `SinceRevision` DBMS_MIN_REVISION_WITH_ROWS_BEFORE_AGGREGATION + } + deriving (Generic, Serializable) + +data TableColumns = MkTableColumns + { table_name :: ChString + , table_columns :: ChString + } + deriving (Generic, Serializable) + +data ProfileEventsPacket = MkProfileEventsPacket + { dataPacket :: DataPacket + , host_name :: [ChString] + , current_time :: [DateTime ""] + , thread_id :: [UInt64] + , type_ :: [Int8] + , name :: [ChString] + , value :: [UInt64] + } deriving (Generic) + +-- ToDo: Simplify +instance Serializable ProfileEventsPacket where + serialize rev MkProfileEventsPacket{..} + = serialize rev dataPacket + <> serialize rev (mkHeader @(Column "host_name" ChString)) <> serializeColumn @(Column "host_name" ChString) rev id host_name + <> serialize rev (mkHeader @(Column "current_time" (DateTime ""))) <> serializeColumn @(Column "current_time" (DateTime "")) rev id current_time + <> serialize rev (mkHeader @(Column "thread_id" UInt64)) <> serializeColumn @(Column "thread_id" UInt64) rev id thread_id + <> serialize rev (mkHeader @(Column "type" Int8)) <> serializeColumn @(Column "type" Int8) rev id type_ + <> serialize rev (mkHeader @(Column "name" ChString)) <> serializeColumn @(Column "name" ChString) rev id name + <> serialize rev (mkHeader @(Column "value" UInt64)) <> serializeColumn @(Column "value" UInt64) rev id value + deserialize rev = do + dataPacket@MkDataPacket{rows_count, columns_count} <- deserialize rev + validateColumnsCount columns_count + !host_name <- deserialize @ColumnHeader rev *> deserializeColumn @(Column "host_name" ChString) rev rows_count id + !current_time <- deserialize @ColumnHeader rev *> deserializeColumn @(Column "current_time" (DateTime "")) rev rows_count id + !thread_id <- deserialize @ColumnHeader rev *> deserializeColumn @(Column "thread_id" UInt64) rev rows_count id + !type_ <- deserialize @ColumnHeader rev *> deserializeColumn @(Column "type" Int8) rev rows_count id + !name <- deserialize @ColumnHeader rev *> deserializeColumn @(Column "name" ChString) rev rows_count id + !value <- deserialize @ColumnHeader rev *> deserializeColumn @(Column "value" UInt64) rev rows_count id + pure $ MkProfileEventsPacket{..} + where + validateColumnsCount count = when (count /= 6) . fail $ + "Unable to parse ProfileEvents packet. Expected 6 columns but got " <> show count diff --git a/ClickHaskell/Protocol/Settings.hs b/ClickHaskell/Protocol/Settings.hs new file mode 100644 --- /dev/null +++ b/ClickHaskell/Protocol/Settings.hs @@ -0,0 +1,1885 @@ +{- + This module is autogenerated. Edit /contribution/genSettings.nix +-} +module ClickHaskell.Protocol.Settings where + +-- Internal +import ClickHaskell.Primitive + +-- GHC +import Data.Binary (Get) +import Data.Binary.Get (lookAhead) +import Data.Bits +import Data.Bool (bool) +import Data.ByteString as BS (null) +import Data.ByteString.Builder (Builder) +import Data.Kind (Type) +import Data.Proxy (Proxy (..)) +import GHC.TypeLits + + +-- * Server settings + +data DbSettings = MkDbSettings [DbSetting] + +addSetting + :: forall name settType + . KnownSetting name settType + => settType + -> DbSettings + -> DbSettings +addSetting val (MkDbSettings xs) = + let setting = toChType (symbolVal @name Proxy) + flags = AfterRevision fIMPORTANT + value = toSettingType val + in MkDbSettings (MkDbSetting{..} : xs) + +data DbSetting = MkDbSetting + { setting :: ChString + , flags :: Flags `SinceRevision` DBMS_MIN_REVISION_WITH_SETTINGS_SERIALIZED_AS_STRINGS + , value :: SettingType + } + +instance Serializable DbSetting where + deserialize rev = do + setting <- deserialize @ChString rev + flags <- deserialize @(Flags `SinceRevision` DBMS_MIN_REVISION_WITH_SETTINGS_SERIALIZED_AS_STRINGS) rev + case lookup setting settingsMap of + Nothing -> fail ("Unsupported setting " <> show setting) + Just MkSettingSerializer{deserializer} -> do + value <- deserializer rev + pure $ MkDbSetting{..} + serialize rev MkDbSetting{setting, flags, value} = + serialize rev setting + <> serialize rev flags + <> case lookup setting settingsMap of + Nothing -> error "Impossible happened. Unknown setting was added to query packet" + Just MkSettingSerializer{serializer} -> serializer rev value + +instance Serializable DbSettings where + serialize rev (MkDbSettings setts) = + foldMap (serialize @DbSetting rev) setts + <> serialize @ChString rev "" + deserialize rev = do + (MkChString setting) <- lookAhead (deserialize @ChString rev) + if BS.null setting + then deserialize @ChString rev *> pure (MkDbSettings []) + else do + sett <- deserialize @DbSetting rev + (\(MkDbSettings setts) -> MkDbSettings (sett : setts)) + <$> deserialize @DbSettings rev + +-- ** Flags + +newtype Flags = MkFlags UVarInt + deriving newtype (Serializable, Num, Eq, Bits) + +-- *** Custom + +fCUSTOM :: Flags +fCUSTOM = 0x02 + +isCustom :: Flags -> Bool +isCustom = (/= 0) . (.&. fCUSTOM) + +setCustom :: Flags -> Flags +setCustom = (.|. fCUSTOM) + +-- *** Important + +fIMPORTANT :: Flags +fIMPORTANT = 0x01 + +isImportant :: Flags -> Bool +isImportant = (/= 0) . (.&. fIMPORTANT) + +setImportant :: Flags -> Flags +setImportant = (.|. fIMPORTANT) + +-- *** Tier + +fTIER :: Flags +fTIER = 0x0c -- 0b1100 == 2 bits + + +class + (Serializable settType, ToQueryPart settType) + => + IsSettingType settType + where + toSettingType :: settType -> SettingType + fromSettingType :: SettingType -> settType + + serializeSettingText :: ProtocolRevision -> SettingType -> Builder + serializeSettingText rev = serialize @ChString rev . toChType . toQueryPart @settType . fromSettingType + + serializeSettingBinary :: ProtocolRevision -> SettingType -> Builder + +data SettingType where + SettingUInt64 :: UInt64 -> SettingType + SettingString :: ChString -> SettingType + SettingBool :: Bool -> SettingType + +instance IsSettingType ChString where + toSettingType str = SettingString str + fromSettingType (SettingString str) = str + fromSettingType _ = error "Impossible" + serializeSettingBinary rev = serialize @ChString rev . fromSettingType + +instance IsSettingType UInt64 where + toSettingType uint64 = SettingUInt64 uint64 + fromSettingType (SettingUInt64 uint64) = uint64 + fromSettingType _ = error "Impossible" + serializeSettingBinary rev = serialize @UVarInt rev . fromIntegral @UInt64 . fromSettingType + +instance IsSettingType Bool where + toSettingType boolean = SettingBool boolean + fromSettingType (SettingBool boolean) = boolean + fromSettingType _ = error "Impossible" + serializeSettingBinary rev = serialize @UVarInt rev . bool 0 1 . fromSettingType + + +data SettingSerializer = + MkSettingSerializer + { deserializer :: ProtocolRevision -> Get SettingType + , serializer :: ProtocolRevision -> SettingType -> Builder + } + +class + ( IsSettingType settType + , KnownSymbol name + ) + => + KnownSetting name settType | name -> settType + where + mkSettingSerializer :: (ChString, SettingSerializer) + mkSettingSerializer = + let name = toChType (symbolVal @name Proxy) + deserializer = \rev -> + if rev >= mkRev @DBMS_MIN_REVISION_WITH_SETTINGS_SERIALIZED_AS_STRINGS + then fail "Deserialization of Settings serializaed as strings is unsuported" + else toSettingType <$> deserialize @settType rev + serializer = \rev -> + if rev >= mkRev @DBMS_MIN_REVISION_WITH_SETTINGS_SERIALIZED_AS_STRINGS + then serializeSettingText @settType rev + else serializeSettingBinary @settType rev + in (name, MkSettingSerializer {deserializer, serializer}) + +data Setting (a :: Symbol) (settType :: Type) + +instance KnownSetting "min_compress_block_size" UInt64 +instance KnownSetting "max_compress_block_size" UInt64 +instance KnownSetting "min_insert_block_size_rows" UInt64 +instance KnownSetting "min_insert_block_size_bytes" UInt64 +instance KnownSetting "min_insert_block_size_rows_for_materialized_views" UInt64 +instance KnownSetting "min_insert_block_size_bytes_for_materialized_views" UInt64 +instance KnownSetting "min_external_table_block_size_rows" UInt64 +instance KnownSetting "min_external_table_block_size_bytes" UInt64 +instance KnownSetting "max_joined_block_size_rows" UInt64 +instance KnownSetting "max_joined_block_size_bytes" UInt64 +instance KnownSetting "min_joined_block_size_rows" UInt64 +instance KnownSetting "min_joined_block_size_bytes" UInt64 +instance KnownSetting "joined_block_split_single_row" Bool +instance KnownSetting "max_insert_threads" UInt64 +instance KnownSetting "max_insert_delayed_streams_for_parallel_write" UInt64 +instance KnownSetting "max_threads_for_indexes" UInt64 +instance KnownSetting "use_concurrency_control" Bool +instance KnownSetting "max_download_buffer_size" UInt64 +instance KnownSetting "max_read_buffer_size_local_fs" UInt64 +instance KnownSetting "max_read_buffer_size_remote_fs" UInt64 +instance KnownSetting "max_distributed_connections" UInt64 +instance KnownSetting "max_query_size" UInt64 +instance KnownSetting "interactive_delay" UInt64 +instance KnownSetting "use_hedged_requests" Bool +instance KnownSetting "allow_changing_replica_until_first_data_packet" Bool +instance KnownSetting "poll_interval" UInt64 +instance KnownSetting "idle_connection_timeout" UInt64 +instance KnownSetting "distributed_connections_pool_size" UInt64 +instance KnownSetting "connections_with_failover_max_tries" UInt64 +instance KnownSetting "s3_strict_upload_part_size" UInt64 +instance KnownSetting "azure_strict_upload_part_size" UInt64 +instance KnownSetting "azure_max_blocks_in_multipart_upload" UInt64 +instance KnownSetting "s3_min_upload_part_size" UInt64 +instance KnownSetting "s3_max_upload_part_size" UInt64 +instance KnownSetting "azure_min_upload_part_size" UInt64 +instance KnownSetting "azure_max_upload_part_size" UInt64 +instance KnownSetting "s3_upload_part_size_multiply_factor" UInt64 +instance KnownSetting "s3_upload_part_size_multiply_parts_count_threshold" UInt64 +instance KnownSetting "s3_max_part_number" UInt64 +instance KnownSetting "s3_allow_multipart_copy" Bool +instance KnownSetting "s3_max_single_operation_copy_size" UInt64 +instance KnownSetting "azure_upload_part_size_multiply_factor" UInt64 +instance KnownSetting "azure_upload_part_size_multiply_parts_count_threshold" UInt64 +instance KnownSetting "s3_max_inflight_parts_for_one_file" UInt64 +instance KnownSetting "azure_max_inflight_parts_for_one_file" UInt64 +instance KnownSetting "s3_max_single_part_upload_size" UInt64 +instance KnownSetting "azure_max_single_part_upload_size" UInt64 +instance KnownSetting "azure_max_single_part_copy_size" UInt64 +instance KnownSetting "s3_max_single_read_retries" UInt64 +instance KnownSetting "azure_max_single_read_retries" UInt64 +instance KnownSetting "azure_max_unexpected_write_error_retries" UInt64 +instance KnownSetting "s3_max_unexpected_write_error_retries" UInt64 +instance KnownSetting "azure_max_redirects" UInt64 +instance KnownSetting "azure_max_get_rps" UInt64 +instance KnownSetting "azure_max_get_burst" UInt64 +instance KnownSetting "azure_max_put_rps" UInt64 +instance KnownSetting "azure_max_put_burst" UInt64 +instance KnownSetting "s3_max_connections" UInt64 +instance KnownSetting "s3_max_get_rps" UInt64 +instance KnownSetting "s3_max_get_burst" UInt64 +instance KnownSetting "s3_max_put_rps" UInt64 +instance KnownSetting "s3_max_put_burst" UInt64 +instance KnownSetting "s3_list_object_keys_size" UInt64 +instance KnownSetting "s3_use_adaptive_timeouts" Bool +instance KnownSetting "azure_use_adaptive_timeouts" Bool +instance KnownSetting "s3_slow_all_threads_after_network_error" Bool +instance KnownSetting "backup_slow_all_threads_after_retryable_s3_error" Bool +instance KnownSetting "azure_list_object_keys_size" UInt64 +instance KnownSetting "s3_truncate_on_insert" Bool +instance KnownSetting "azure_truncate_on_insert" Bool +instance KnownSetting "s3_create_new_file_on_insert" Bool +instance KnownSetting "s3_skip_empty_files" Bool +instance KnownSetting "azure_create_new_file_on_insert" Bool +instance KnownSetting "s3_check_objects_after_upload" Bool +instance KnownSetting "azure_check_objects_after_upload" Bool +instance KnownSetting "s3_allow_parallel_part_upload" Bool +instance KnownSetting "azure_allow_parallel_part_upload" Bool +instance KnownSetting "s3_throw_on_zero_files_match" Bool +instance KnownSetting "hdfs_throw_on_zero_files_match" Bool +instance KnownSetting "azure_throw_on_zero_files_match" Bool +instance KnownSetting "s3_ignore_file_doesnt_exist" Bool +instance KnownSetting "hdfs_ignore_file_doesnt_exist" Bool +instance KnownSetting "azure_ignore_file_doesnt_exist" Bool +instance KnownSetting "azure_sdk_max_retries" UInt64 +instance KnownSetting "azure_sdk_retry_initial_backoff_ms" UInt64 +instance KnownSetting "azure_sdk_retry_max_backoff_ms" UInt64 +instance KnownSetting "azure_request_timeout_ms" UInt64 +instance KnownSetting "azure_connect_timeout_ms" UInt64 +instance KnownSetting "s3_validate_request_settings" Bool +instance KnownSetting "s3_disable_checksum" Bool +instance KnownSetting "s3_request_timeout_ms" UInt64 +instance KnownSetting "s3_connect_timeout_ms" UInt64 +instance KnownSetting "enable_s3_requests_logging" Bool +instance KnownSetting "s3queue_default_zookeeper_path" ChString +instance KnownSetting "s3queue_migrate_old_metadata_to_buckets" Bool +instance KnownSetting "s3queue_enable_logging_to_s3queue_log" Bool +instance KnownSetting "hdfs_replication" UInt64 +instance KnownSetting "hdfs_truncate_on_insert" Bool +instance KnownSetting "hdfs_create_new_file_on_insert" Bool +instance KnownSetting "hdfs_skip_empty_files" Bool +instance KnownSetting "enable_hdfs_pread" Bool +instance KnownSetting "azure_skip_empty_files" Bool +instance KnownSetting "hsts_max_age" UInt64 +instance KnownSetting "extremes" Bool +instance KnownSetting "use_uncompressed_cache" Bool +instance KnownSetting "replace_running_query" Bool +instance KnownSetting "max_remote_read_network_bandwidth" UInt64 +instance KnownSetting "max_remote_write_network_bandwidth" UInt64 +instance KnownSetting "max_local_read_bandwidth" UInt64 +instance KnownSetting "max_local_write_bandwidth" UInt64 +instance KnownSetting "stream_like_engine_allow_direct_select" Bool +instance KnownSetting "stream_like_engine_insert_queue" ChString +instance KnownSetting "dictionary_validate_primary_key_type" Bool +instance KnownSetting "distributed_insert_skip_read_only_replicas" Bool +instance KnownSetting "optimize_move_to_prewhere" Bool +instance KnownSetting "optimize_move_to_prewhere_if_final" Bool +instance KnownSetting "move_all_conditions_to_prewhere" Bool +instance KnownSetting "enable_multiple_prewhere_read_steps" Bool +instance KnownSetting "move_primary_key_columns_to_end_of_prewhere" Bool +instance KnownSetting "allow_reorder_prewhere_conditions" Bool +instance KnownSetting "alter_move_to_space_execute_async" Bool +instance KnownSetting "load_balancing_first_offset" UInt64 +instance KnownSetting "allow_suspicious_low_cardinality_types" Bool +instance KnownSetting "allow_suspicious_fixed_string_types" Bool +instance KnownSetting "allow_suspicious_indices" Bool +instance KnownSetting "allow_suspicious_ttl_expressions" Bool +instance KnownSetting "allow_suspicious_variant_types" Bool +instance KnownSetting "allow_suspicious_primary_key" Bool +instance KnownSetting "allow_suspicious_types_in_group_by" Bool +instance KnownSetting "allow_suspicious_types_in_order_by" Bool +instance KnownSetting "allow_not_comparable_types_in_order_by" Bool +instance KnownSetting "allow_not_comparable_types_in_comparison_functions" Bool +instance KnownSetting "compile_expressions" Bool +instance KnownSetting "min_count_to_compile_expression" UInt64 +instance KnownSetting "compile_aggregate_expressions" Bool +instance KnownSetting "min_count_to_compile_aggregate_expression" UInt64 +instance KnownSetting "compile_sort_description" Bool +instance KnownSetting "min_count_to_compile_sort_description" UInt64 +instance KnownSetting "group_by_two_level_threshold" UInt64 +instance KnownSetting "group_by_two_level_threshold_bytes" UInt64 +instance KnownSetting "distributed_aggregation_memory_efficient" Bool +instance KnownSetting "aggregation_memory_efficient_merge_threads" UInt64 +instance KnownSetting "enable_memory_bound_merging_of_aggregation_results" Bool +instance KnownSetting "enable_positional_arguments" Bool +instance KnownSetting "enable_extended_results_for_datetime_functions" Bool +instance KnownSetting "allow_nonconst_timezone_arguments" Bool +instance KnownSetting "use_legacy_to_time" Bool +instance KnownSetting "function_locate_has_mysql_compatible_argument_order" Bool +instance KnownSetting "group_by_use_nulls" Bool +instance KnownSetting "skip_unavailable_shards" Bool +instance KnownSetting "parallel_distributed_insert_select" UInt64 +instance KnownSetting "distributed_group_by_no_merge" UInt64 +instance KnownSetting "distributed_push_down_limit" UInt64 +instance KnownSetting "optimize_distributed_group_by_sharding_key" Bool +instance KnownSetting "optimize_skip_unused_shards_limit" UInt64 +instance KnownSetting "optimize_skip_unused_shards" Bool +instance KnownSetting "optimize_skip_unused_shards_rewrite_in" Bool +instance KnownSetting "allow_nondeterministic_optimize_skip_unused_shards" Bool +instance KnownSetting "force_optimize_skip_unused_shards" UInt64 +instance KnownSetting "optimize_skip_unused_shards_nesting" UInt64 +instance KnownSetting "force_optimize_skip_unused_shards_nesting" UInt64 +instance KnownSetting "input_format_parallel_parsing" Bool +instance KnownSetting "output_format_parallel_formatting" Bool +instance KnownSetting "output_format_compression_level" UInt64 +instance KnownSetting "output_format_compression_zstd_window_log" UInt64 +instance KnownSetting "allow_special_serialization_kinds_in_output_formats" Bool +instance KnownSetting "enable_parsing_to_custom_serialization" Bool +instance KnownSetting "merge_tree_use_v1_object_and_dynamic_serialization" Bool +instance KnownSetting "merge_tree_min_rows_for_concurrent_read" UInt64 +instance KnownSetting "merge_tree_min_bytes_for_concurrent_read" UInt64 +instance KnownSetting "merge_tree_min_rows_for_seek" UInt64 +instance KnownSetting "merge_tree_min_bytes_for_seek" UInt64 +instance KnownSetting "merge_tree_coarse_index_granularity" UInt64 +instance KnownSetting "merge_tree_max_rows_to_use_cache" UInt64 +instance KnownSetting "merge_tree_max_bytes_to_use_cache" UInt64 +instance KnownSetting "merge_tree_use_deserialization_prefixes_cache" Bool +instance KnownSetting "merge_tree_use_prefixes_deserialization_thread_pool" Bool +instance KnownSetting "do_not_merge_across_partitions_select_final" Bool +instance KnownSetting "split_parts_ranges_into_intersecting_and_non_intersecting_final" Bool +instance KnownSetting "split_intersecting_parts_ranges_into_layers_final" Bool +instance KnownSetting "mysql_max_rows_to_insert" UInt64 +instance KnownSetting "mysql_map_string_to_text_in_show_columns" Bool +instance KnownSetting "mysql_map_fixed_string_to_text_in_show_columns" Bool +instance KnownSetting "optimize_min_equality_disjunction_chain_length" UInt64 +instance KnownSetting "optimize_min_inequality_conjunction_chain_length" UInt64 +instance KnownSetting "min_bytes_to_use_direct_io" UInt64 +instance KnownSetting "min_bytes_to_use_mmap_io" UInt64 +instance KnownSetting "checksum_on_read" Bool +instance KnownSetting "force_index_by_date" Bool +instance KnownSetting "force_primary_key" Bool +instance KnownSetting "use_skip_indexes" Bool +instance KnownSetting "use_skip_indexes_if_final" Bool +instance KnownSetting "use_skip_indexes_if_final_exact_mode" Bool +instance KnownSetting "use_skip_indexes_on_data_read" Bool +instance KnownSetting "materialize_skip_indexes_on_insert" Bool +instance KnownSetting "exclude_materialize_skip_indexes_on_insert" ChString +instance KnownSetting "text_index_use_bloom_filter" Bool +instance KnownSetting "per_part_index_stats" Bool +instance KnownSetting "materialize_statistics_on_insert" Bool +instance KnownSetting "ignore_data_skipping_indices" ChString +instance KnownSetting "force_data_skipping_indices" ChString +instance KnownSetting "secondary_indices_enable_bulk_filtering" Bool +instance KnownSetting "network_compression_method" ChString +instance KnownSetting "priority" UInt64 +instance KnownSetting "log_queries" Bool +instance KnownSetting "log_formatted_queries" Bool +instance KnownSetting "log_queries_cut_to_length" UInt64 +instance KnownSetting "log_processors_profiles" Bool +instance KnownSetting "max_concurrent_queries_for_all_users" UInt64 +instance KnownSetting "max_concurrent_queries_for_user" UInt64 +instance KnownSetting "insert_deduplicate" Bool +instance KnownSetting "async_insert_deduplicate" Bool +instance KnownSetting "insert_quorum_parallel" Bool +instance KnownSetting "select_sequential_consistency" UInt64 +instance KnownSetting "update_sequential_consistency" Bool +instance KnownSetting "table_function_remote_max_addresses" UInt64 +instance KnownSetting "read_backoff_max_throughput" UInt64 +instance KnownSetting "read_backoff_min_events" UInt64 +instance KnownSetting "read_backoff_min_concurrency" UInt64 +instance KnownSetting "enable_http_compression" Bool +instance KnownSetting "http_native_compression_disable_checksumming_on_decompress" Bool +instance KnownSetting "count_distinct_implementation" ChString +instance KnownSetting "add_http_cors_header" Bool +instance KnownSetting "max_http_get_redirects" UInt64 +instance KnownSetting "use_client_time_zone" Bool +instance KnownSetting "send_progress_in_http_headers" Bool +instance KnownSetting "http_headers_progress_interval_ms" UInt64 +instance KnownSetting "http_wait_end_of_query" Bool +instance KnownSetting "http_write_exception_in_output_format" Bool +instance KnownSetting "http_response_buffer_size" UInt64 +instance KnownSetting "fsync_metadata" Bool +instance KnownSetting "join_use_nulls" Bool +instance KnownSetting "join_output_by_rowlist_perkey_rows_threshold" UInt64 +instance KnownSetting "any_join_distinct_right_table_keys" Bool +instance KnownSetting "single_join_prefer_left_table" Bool +instance KnownSetting "query_plan_optimize_join_order_limit" UInt64 +instance KnownSetting "query_plan_join_shard_by_pk_ranges" Bool +instance KnownSetting "query_plan_display_internal_aliases" Bool +instance KnownSetting "query_plan_max_step_description_length" UInt64 +instance KnownSetting "preferred_block_size_bytes" UInt64 +instance KnownSetting "max_replica_delay_for_distributed_queries" UInt64 +instance KnownSetting "fallback_to_stale_replicas_for_distributed_queries" Bool +instance KnownSetting "preferred_max_column_in_block_size_bytes" UInt64 +instance KnownSetting "parts_to_delay_insert" UInt64 +instance KnownSetting "parts_to_throw_insert" UInt64 +instance KnownSetting "number_of_mutations_to_delay" UInt64 +instance KnownSetting "number_of_mutations_to_throw" UInt64 +instance KnownSetting "min_free_disk_bytes_to_perform_insert" UInt64 +instance KnownSetting "final" Bool +instance KnownSetting "partial_result_on_first_cancel" Bool +instance KnownSetting "ignore_on_cluster_for_replicated_udf_queries" Bool +instance KnownSetting "ignore_on_cluster_for_replicated_access_entities_queries" Bool +instance KnownSetting "ignore_on_cluster_for_replicated_named_collections_queries" Bool +instance KnownSetting "unknown_packet_in_send_data" UInt64 +instance KnownSetting "insert_allow_materialized_columns" Bool +instance KnownSetting "http_max_uri_size" UInt64 +instance KnownSetting "http_max_fields" UInt64 +instance KnownSetting "http_max_field_name_size" UInt64 +instance KnownSetting "http_max_field_value_size" UInt64 +instance KnownSetting "http_skip_not_found_url_for_globs" Bool +instance KnownSetting "http_make_head_request" Bool +instance KnownSetting "optimize_throw_if_noop" Bool +instance KnownSetting "use_index_for_in_with_subqueries" Bool +instance KnownSetting "use_index_for_in_with_subqueries_max_values" UInt64 +instance KnownSetting "analyze_index_with_space_filling_curves" Bool +instance KnownSetting "joined_subquery_requires_alias" Bool +instance KnownSetting "empty_result_for_aggregation_by_empty_set" Bool +instance KnownSetting "empty_result_for_aggregation_by_constant_keys_on_empty_set" Bool +instance KnownSetting "allow_distributed_ddl" Bool +instance KnownSetting "allow_suspicious_codecs" Bool +instance KnownSetting "enable_zstd_qat_codec" Bool +instance KnownSetting "enable_deflate_qpl_codec" Bool +instance KnownSetting "query_profiler_real_time_period_ns" UInt64 +instance KnownSetting "query_profiler_cpu_time_period_ns" UInt64 +instance KnownSetting "metrics_perf_events_enabled" Bool +instance KnownSetting "metrics_perf_events_list" ChString +instance KnownSetting "opentelemetry_trace_processors" Bool +instance KnownSetting "opentelemetry_trace_cpu_scheduling" Bool +instance KnownSetting "prefer_column_name_to_alias" Bool +instance KnownSetting "skip_redundant_aliases_in_udf" Bool +instance KnownSetting "prefer_global_in_and_join" Bool +instance KnownSetting "enable_vertical_final" Bool +instance KnownSetting "max_rows_to_read" UInt64 +instance KnownSetting "max_bytes_to_read" UInt64 +instance KnownSetting "max_rows_to_read_leaf" UInt64 +instance KnownSetting "max_bytes_to_read_leaf" UInt64 +instance KnownSetting "max_rows_to_group_by" UInt64 +instance KnownSetting "max_bytes_before_external_group_by" UInt64 +instance KnownSetting "max_rows_to_sort" UInt64 +instance KnownSetting "max_bytes_to_sort" UInt64 +instance KnownSetting "prefer_external_sort_block_bytes" UInt64 +instance KnownSetting "max_bytes_before_external_sort" UInt64 +instance KnownSetting "max_bytes_before_remerge_sort" UInt64 +instance KnownSetting "max_result_rows" UInt64 +instance KnownSetting "max_result_bytes" UInt64 +instance KnownSetting "min_execution_speed" UInt64 +instance KnownSetting "max_execution_speed" UInt64 +instance KnownSetting "min_execution_speed_bytes" UInt64 +instance KnownSetting "max_execution_speed_bytes" UInt64 +instance KnownSetting "max_columns_to_read" UInt64 +instance KnownSetting "max_temporary_columns" UInt64 +instance KnownSetting "max_temporary_non_const_columns" UInt64 +instance KnownSetting "max_sessions_for_user" UInt64 +instance KnownSetting "max_subquery_depth" UInt64 +instance KnownSetting "max_analyze_depth" UInt64 +instance KnownSetting "max_ast_depth" UInt64 +instance KnownSetting "max_ast_elements" UInt64 +instance KnownSetting "max_expanded_ast_elements" UInt64 +instance KnownSetting "readonly" UInt64 +instance KnownSetting "max_rows_in_set" UInt64 +instance KnownSetting "max_bytes_in_set" UInt64 +instance KnownSetting "max_rows_in_join" UInt64 +instance KnownSetting "max_bytes_in_join" UInt64 +instance KnownSetting "join_any_take_last_row" Bool +instance KnownSetting "cross_join_min_rows_to_compress" UInt64 +instance KnownSetting "cross_join_min_bytes_to_compress" UInt64 +instance KnownSetting "default_max_bytes_in_join" UInt64 +instance KnownSetting "partial_merge_join_left_table_buffer_bytes" UInt64 +instance KnownSetting "partial_merge_join_rows_in_right_blocks" UInt64 +instance KnownSetting "join_on_disk_max_files_to_merge" UInt64 +instance KnownSetting "max_rows_in_set_to_optimize_join" UInt64 +instance KnownSetting "compatibility_ignore_collation_in_create_table" Bool +instance KnownSetting "temporary_files_codec" ChString +instance KnownSetting "temporary_files_buffer_size" UInt64 +instance KnownSetting "max_rows_to_transfer" UInt64 +instance KnownSetting "max_bytes_to_transfer" UInt64 +instance KnownSetting "max_rows_in_distinct" UInt64 +instance KnownSetting "max_bytes_in_distinct" UInt64 +instance KnownSetting "max_memory_usage" UInt64 +instance KnownSetting "memory_overcommit_ratio_denominator" UInt64 +instance KnownSetting "max_memory_usage_for_user" UInt64 +instance KnownSetting "memory_overcommit_ratio_denominator_for_user" UInt64 +instance KnownSetting "max_untracked_memory" UInt64 +instance KnownSetting "memory_profiler_step" UInt64 +instance KnownSetting "memory_profiler_sample_min_allocation_size" UInt64 +instance KnownSetting "memory_profiler_sample_max_allocation_size" UInt64 +instance KnownSetting "trace_profile_events" Bool +instance KnownSetting "memory_usage_overcommit_max_wait_microseconds" UInt64 +instance KnownSetting "max_network_bandwidth" UInt64 +instance KnownSetting "max_network_bytes" UInt64 +instance KnownSetting "max_network_bandwidth_for_user" UInt64 +instance KnownSetting "max_network_bandwidth_for_all_users" UInt64 +instance KnownSetting "max_temporary_data_on_disk_size_for_user" UInt64 +instance KnownSetting "max_temporary_data_on_disk_size_for_query" UInt64 +instance KnownSetting "backup_restore_keeper_max_retries" UInt64 +instance KnownSetting "backup_restore_keeper_retry_initial_backoff_ms" UInt64 +instance KnownSetting "backup_restore_keeper_retry_max_backoff_ms" UInt64 +instance KnownSetting "backup_restore_failure_after_host_disconnected_for_seconds" UInt64 +instance KnownSetting "backup_restore_keeper_max_retries_while_initializing" UInt64 +instance KnownSetting "backup_restore_keeper_max_retries_while_handling_error" UInt64 +instance KnownSetting "backup_restore_finish_timeout_after_error_sec" UInt64 +instance KnownSetting "backup_restore_keeper_value_max_size" UInt64 +instance KnownSetting "backup_restore_batch_size_for_keeper_multi" UInt64 +instance KnownSetting "backup_restore_batch_size_for_keeper_multiread" UInt64 +instance KnownSetting "backup_restore_keeper_fault_injection_seed" UInt64 +instance KnownSetting "backup_restore_s3_retry_attempts" UInt64 +instance KnownSetting "backup_restore_s3_retry_initial_backoff_ms" UInt64 +instance KnownSetting "backup_restore_s3_retry_max_backoff_ms" UInt64 +instance KnownSetting "max_backup_bandwidth" UInt64 +instance KnownSetting "restore_replicated_merge_tree_to_shared_merge_tree" Bool +instance KnownSetting "log_profile_events" Bool +instance KnownSetting "log_query_settings" Bool +instance KnownSetting "log_query_threads" Bool +instance KnownSetting "log_query_views" Bool +instance KnownSetting "log_comment" ChString +instance KnownSetting "send_logs_source_regexp" ChString +instance KnownSetting "enable_optimize_predicate_expression" Bool +instance KnownSetting "enable_optimize_predicate_expression_to_final_subquery" Bool +instance KnownSetting "allow_push_predicate_when_subquery_contains_with" Bool +instance KnownSetting "allow_push_predicate_ast_for_distributed_subqueries" Bool +instance KnownSetting "low_cardinality_max_dictionary_size" UInt64 +instance KnownSetting "low_cardinality_use_single_dictionary_for_part" Bool +instance KnownSetting "decimal_check_overflow" Bool +instance KnownSetting "allow_custom_error_code_in_throwif" Bool +instance KnownSetting "prefer_localhost_replica" Bool +instance KnownSetting "max_fetch_partition_retries_count" UInt64 +instance KnownSetting "http_max_multipart_form_data_size" UInt64 +instance KnownSetting "calculate_text_stack_trace" Bool +instance KnownSetting "enable_job_stack_trace" Bool +instance KnownSetting "allow_ddl" Bool +instance KnownSetting "parallel_view_processing" Bool +instance KnownSetting "enable_unaligned_array_join" Bool +instance KnownSetting "optimize_read_in_order" Bool +instance KnownSetting "read_in_order_use_virtual_row" Bool +instance KnownSetting "optimize_read_in_window_order" Bool +instance KnownSetting "optimize_aggregation_in_order" Bool +instance KnownSetting "read_in_order_use_buffering" Bool +instance KnownSetting "aggregation_in_order_max_block_bytes" UInt64 +instance KnownSetting "read_in_order_two_level_merge_threshold" UInt64 +instance KnownSetting "low_cardinality_allow_in_native_format" Bool +instance KnownSetting "cancel_http_readonly_queries_on_client_close" Bool +instance KnownSetting "external_table_functions_use_nulls" Bool +instance KnownSetting "external_table_strict_query" Bool +instance KnownSetting "allow_hyperscan" Bool +instance KnownSetting "max_hyperscan_regexp_length" UInt64 +instance KnownSetting "max_hyperscan_regexp_total_length" UInt64 +instance KnownSetting "reject_expensive_hyperscan_regexps" Bool +instance KnownSetting "allow_simdjson" Bool +instance KnownSetting "allow_introspection_functions" Bool +instance KnownSetting "splitby_max_substrings_includes_remaining_string" Bool +instance KnownSetting "allow_execute_multiif_columnar" Bool +instance KnownSetting "formatdatetime_f_prints_single_zero" Bool +instance KnownSetting "formatdatetime_f_prints_scale_number_of_digits" Bool +instance KnownSetting "formatdatetime_parsedatetime_m_is_month_name" Bool +instance KnownSetting "parsedatetime_parse_without_leading_zeros" Bool +instance KnownSetting "parsedatetime_e_requires_space_padding" Bool +instance KnownSetting "formatdatetime_format_without_leading_zeros" Bool +instance KnownSetting "formatdatetime_e_with_space_padding" Bool +instance KnownSetting "least_greatest_legacy_null_behavior" Bool +instance KnownSetting "h3togeo_lon_lat_result_order" Bool +instance KnownSetting "max_partitions_per_insert_block" UInt64 +instance KnownSetting "throw_on_max_partitions_per_insert_block" Bool +instance KnownSetting "check_query_single_value_result" Bool +instance KnownSetting "allow_drop_detached" Bool +instance KnownSetting "max_parts_to_move" UInt64 +instance KnownSetting "max_table_size_to_drop" UInt64 +instance KnownSetting "max_partition_size_to_drop" UInt64 +instance KnownSetting "postgresql_connection_pool_size" UInt64 +instance KnownSetting "postgresql_connection_attempt_timeout" UInt64 +instance KnownSetting "postgresql_connection_pool_wait_timeout" UInt64 +instance KnownSetting "postgresql_connection_pool_retries" UInt64 +instance KnownSetting "postgresql_connection_pool_auto_close_connection" Bool +instance KnownSetting "glob_expansion_max_elements" UInt64 +instance KnownSetting "odbc_bridge_connection_pool_size" UInt64 +instance KnownSetting "odbc_bridge_use_connection_pooling" Bool +instance KnownSetting "distributed_replica_error_cap" UInt64 +instance KnownSetting "distributed_replica_max_ignored_errors" UInt64 +instance KnownSetting "min_free_disk_space_for_temporary_data" UInt64 +instance KnownSetting "show_table_uuid_in_table_create_query_if_not_nil" Bool +instance KnownSetting "database_atomic_wait_for_drop_and_detach_synchronously" Bool +instance KnownSetting "enable_scalar_subquery_optimization" Bool +instance KnownSetting "optimize_trivial_count_query" Bool +instance KnownSetting "optimize_trivial_approximate_count_query" Bool +instance KnownSetting "optimize_count_from_files" Bool +instance KnownSetting "use_cache_for_count_from_files" Bool +instance KnownSetting "optimize_respect_aliases" Bool +instance KnownSetting "mutations_sync" UInt64 +instance KnownSetting "lightweight_deletes_sync" UInt64 +instance KnownSetting "apply_deleted_mask" Bool +instance KnownSetting "optimize_normalize_count_variants" Bool +instance KnownSetting "optimize_injective_functions_inside_uniq" Bool +instance KnownSetting "count_matches_stop_at_empty_match" Bool +instance KnownSetting "rewrite_count_distinct_if_with_count_distinct_implementation" Bool +instance KnownSetting "convert_query_to_cnf" Bool +instance KnownSetting "optimize_or_like_chain" Bool +instance KnownSetting "optimize_arithmetic_operations_in_aggregate_functions" Bool +instance KnownSetting "optimize_redundant_functions_in_order_by" Bool +instance KnownSetting "optimize_if_chain_to_multiif" Bool +instance KnownSetting "optimize_multiif_to_if" Bool +instance KnownSetting "optimize_if_transform_strings_to_enum" Bool +instance KnownSetting "optimize_functions_to_subcolumns" Bool +instance KnownSetting "optimize_using_constraints" Bool +instance KnownSetting "optimize_substitute_columns" Bool +instance KnownSetting "optimize_append_index" Bool +instance KnownSetting "optimize_time_filter_with_preimage" Bool +instance KnownSetting "normalize_function_names" Bool +instance KnownSetting "enable_early_constant_folding" Bool +instance KnownSetting "deduplicate_blocks_in_dependent_materialized_views" Bool +instance KnownSetting "throw_if_deduplication_in_dependent_materialized_views_enabled_with_async_insert" Bool +instance KnownSetting "materialized_views_ignore_errors" Bool +instance KnownSetting "ignore_materialized_views_with_dropped_target_table" Bool +instance KnownSetting "allow_materialized_view_with_bad_select" Bool +instance KnownSetting "materialized_views_squash_parallel_inserts" Bool +instance KnownSetting "use_compact_format_in_distributed_parts_names" Bool +instance KnownSetting "validate_polygons" Bool +instance KnownSetting "max_parser_depth" UInt64 +instance KnownSetting "max_parser_backtracks" UInt64 +instance KnownSetting "max_recursive_cte_evaluation_depth" UInt64 +instance KnownSetting "allow_settings_after_format_in_insert" Bool +instance KnownSetting "transform_null_in" Bool +instance KnownSetting "allow_nondeterministic_mutations" Bool +instance KnownSetting "validate_mutation_query" Bool +instance KnownSetting "materialize_ttl_after_modify" Bool +instance KnownSetting "function_implementation" ChString +instance KnownSetting "data_type_default_nullable" Bool +instance KnownSetting "cast_keep_nullable" Bool +instance KnownSetting "cast_ipv4_ipv6_default_on_conversion_error" Bool +instance KnownSetting "alter_partition_verbose_result" Bool +instance KnownSetting "system_events_show_zero_values" Bool +instance KnownSetting "optimize_trivial_insert_select" Bool +instance KnownSetting "allow_non_metadata_alters" Bool +instance KnownSetting "enable_global_with_statement" Bool +instance KnownSetting "enable_scopes_for_with_statement" Bool +instance KnownSetting "aggregate_functions_null_for_empty" Bool +instance KnownSetting "optimize_syntax_fuse_functions" Bool +instance KnownSetting "flatten_nested" Bool +instance KnownSetting "asterisk_include_materialized_columns" Bool +instance KnownSetting "asterisk_include_alias_columns" Bool +instance KnownSetting "optimize_skip_merged_partitions" Bool +instance KnownSetting "optimize_on_insert" Bool +instance KnownSetting "optimize_use_implicit_projections" Bool +instance KnownSetting "optimize_use_projection_filtering" Bool +instance KnownSetting "force_optimize_projection" Bool +instance KnownSetting "force_optimize_projection_name" ChString +instance KnownSetting "preferred_optimize_projection_name" ChString +instance KnownSetting "max_projection_rows_to_use_projection_index" UInt64 +instance KnownSetting "min_table_rows_to_use_projection_index" UInt64 +instance KnownSetting "async_socket_for_remote" Bool +instance KnownSetting "async_query_sending_for_remote" Bool +instance KnownSetting "insert_null_as_default" Bool +instance KnownSetting "describe_extend_object_types" Bool +instance KnownSetting "describe_include_subcolumns" Bool +instance KnownSetting "describe_include_virtual_columns" Bool +instance KnownSetting "describe_compact_output" Bool +instance KnownSetting "apply_mutations_on_fly" Bool +instance KnownSetting "apply_patch_parts" Bool +instance KnownSetting "mutations_execute_nondeterministic_on_initiator" Bool +instance KnownSetting "mutations_execute_subqueries_on_initiator" Bool +instance KnownSetting "mutations_max_literal_size_to_replace" UInt64 +instance KnownSetting "delta_lake_log_metadata" Bool +instance KnownSetting "iceberg_delete_data_on_drop" Bool +instance KnownSetting "use_iceberg_metadata_files_cache" Bool +instance KnownSetting "use_query_cache" Bool +instance KnownSetting "enable_writes_to_query_cache" Bool +instance KnownSetting "enable_reads_from_query_cache" Bool +instance KnownSetting "query_cache_max_size_in_bytes" UInt64 +instance KnownSetting "query_cache_max_entries" UInt64 +instance KnownSetting "query_cache_min_query_runs" UInt64 +instance KnownSetting "query_cache_compress_entries" Bool +instance KnownSetting "query_cache_squash_partial_results" Bool +instance KnownSetting "query_cache_share_between_users" Bool +instance KnownSetting "query_cache_tag" ChString +instance KnownSetting "enable_sharing_sets_for_mutations" Bool +instance KnownSetting "use_query_condition_cache" Bool +instance KnownSetting "query_condition_cache_store_conditions_as_plaintext" Bool +instance KnownSetting "enable_shared_storage_snapshot_in_query" Bool +instance KnownSetting "merge_tree_storage_snapshot_sleep_ms" UInt64 +instance KnownSetting "optimize_rewrite_sum_if_to_count_if" Bool +instance KnownSetting "optimize_empty_string_comparisons" Bool +instance KnownSetting "optimize_rewrite_aggregate_function_with_if" Bool +instance KnownSetting "optimize_rewrite_array_exists_to_has" Bool +instance KnownSetting "optimize_rewrite_like_perfect_affix" Bool +instance KnownSetting "execute_exists_as_scalar_subquery" Bool +instance KnownSetting "optimize_rewrite_regexp_functions" Bool +instance KnownSetting "insert_shard_id" UInt64 +instance KnownSetting "collect_hash_table_stats_during_aggregation" Bool +instance KnownSetting "max_size_to_preallocate_for_aggregation" UInt64 +instance KnownSetting "collect_hash_table_stats_during_joins" Bool +instance KnownSetting "max_size_to_preallocate_for_joins" UInt64 +instance KnownSetting "kafka_disable_num_consumers_limit" Bool +instance KnownSetting "allow_experimental_kafka_offsets_storage_in_keeper" Bool +instance KnownSetting "enable_software_prefetch_in_aggregation" Bool +instance KnownSetting "allow_aggregate_partitions_independently" Bool +instance KnownSetting "force_aggregate_partitions_independently" Bool +instance KnownSetting "max_number_of_partitions_for_independent_aggregation" UInt64 +instance KnownSetting "engine_file_empty_if_not_exists" Bool +instance KnownSetting "engine_file_truncate_on_insert" Bool +instance KnownSetting "engine_file_allow_create_multiple_files" Bool +instance KnownSetting "engine_file_skip_empty_files" Bool +instance KnownSetting "engine_url_skip_empty_files" Bool +instance KnownSetting "enable_url_encoding" Bool +instance KnownSetting "database_replicated_initial_query_timeout_sec" UInt64 +instance KnownSetting "database_replicated_enforce_synchronous_settings" Bool +instance KnownSetting "max_distributed_depth" UInt64 +instance KnownSetting "database_replicated_always_detach_permanently" Bool +instance KnownSetting "database_replicated_allow_only_replicated_engine" Bool +instance KnownSetting "database_replicated_allow_replicated_engine_arguments" UInt64 +instance KnownSetting "database_replicated_allow_explicit_uuid" UInt64 +instance KnownSetting "database_replicated_allow_heavy_create" Bool +instance KnownSetting "cloud_mode" Bool +instance KnownSetting "cloud_mode_engine" UInt64 +instance KnownSetting "cloud_mode_database_engine" UInt64 +instance KnownSetting "distributed_ddl_entry_format_version" UInt64 +instance KnownSetting "external_storage_max_read_rows" UInt64 +instance KnownSetting "external_storage_max_read_bytes" UInt64 +instance KnownSetting "external_storage_connect_timeout_sec" UInt64 +instance KnownSetting "external_storage_rw_timeout_sec" UInt64 +instance KnownSetting "allow_experimental_correlated_subqueries" Bool +instance KnownSetting "optimize_aggregators_of_group_by_keys" Bool +instance KnownSetting "optimize_injective_functions_in_group_by" Bool +instance KnownSetting "optimize_group_by_function_keys" Bool +instance KnownSetting "optimize_group_by_constant_keys" Bool +instance KnownSetting "legacy_column_name_of_tuple_literal" Bool +instance KnownSetting "enable_named_columns_in_function_tuple" Bool +instance KnownSetting "query_plan_enable_optimizations" Bool +instance KnownSetting "query_plan_max_optimizations_to_apply" UInt64 +instance KnownSetting "query_plan_lift_up_array_join" Bool +instance KnownSetting "query_plan_push_down_limit" Bool +instance KnownSetting "query_plan_split_filter" Bool +instance KnownSetting "query_plan_merge_expressions" Bool +instance KnownSetting "query_plan_merge_filters" Bool +instance KnownSetting "query_plan_filter_push_down" Bool +instance KnownSetting "query_plan_convert_outer_join_to_inner_join" Bool +instance KnownSetting "query_plan_convert_any_join_to_semi_or_anti_join" Bool +instance KnownSetting "query_plan_merge_filter_into_join_condition" Bool +instance KnownSetting "query_plan_convert_join_to_in" Bool +instance KnownSetting "query_plan_optimize_prewhere" Bool +instance KnownSetting "query_plan_execute_functions_after_sorting" Bool +instance KnownSetting "query_plan_reuse_storage_ordering_for_window_functions" Bool +instance KnownSetting "query_plan_lift_up_union" Bool +instance KnownSetting "query_plan_read_in_order" Bool +instance KnownSetting "query_plan_aggregation_in_order" Bool +instance KnownSetting "query_plan_remove_redundant_sorting" Bool +instance KnownSetting "query_plan_remove_redundant_distinct" Bool +instance KnownSetting "query_plan_try_use_vector_search" Bool +instance KnownSetting "query_plan_enable_multithreading_after_window_functions" Bool +instance KnownSetting "query_plan_optimize_lazy_materialization" Bool +instance KnownSetting "query_plan_max_limit_for_lazy_materialization" UInt64 +instance KnownSetting "enable_lazy_columns_replication" Bool +instance KnownSetting "serialize_query_plan" Bool +instance KnownSetting "correlated_subqueries_substitute_equivalent_expressions" Bool +instance KnownSetting "optimize_qbit_distance_function_reads" Bool +instance KnownSetting "regexp_max_matches_per_row" UInt64 +instance KnownSetting "limit" UInt64 +instance KnownSetting "offset" UInt64 +instance KnownSetting "function_range_max_elements_in_block" UInt64 +instance KnownSetting "function_sleep_max_microseconds_per_block" UInt64 +instance KnownSetting "function_visible_width_behavior" UInt64 +instance KnownSetting "local_filesystem_read_method" ChString +instance KnownSetting "remote_filesystem_read_method" ChString +instance KnownSetting "local_filesystem_read_prefetch" Bool +instance KnownSetting "remote_filesystem_read_prefetch" Bool +instance KnownSetting "merge_tree_min_rows_for_concurrent_read_for_remote_filesystem" UInt64 +instance KnownSetting "merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem" UInt64 +instance KnownSetting "remote_read_min_bytes_for_seek" UInt64 +instance KnownSetting "merge_tree_use_const_size_tasks_for_remote_reading" Bool +instance KnownSetting "merge_tree_determine_task_size_by_prewhere_columns" Bool +instance KnownSetting "merge_tree_compact_parts_min_granules_to_multibuffer_read" UInt64 +instance KnownSetting "async_insert" Bool +instance KnownSetting "wait_for_async_insert" Bool +instance KnownSetting "async_insert_max_data_size" UInt64 +instance KnownSetting "async_insert_max_query_number" UInt64 +instance KnownSetting "async_insert_use_adaptive_busy_timeout" Bool +instance KnownSetting "remote_fs_read_max_backoff_ms" UInt64 +instance KnownSetting "remote_fs_read_backoff_max_tries" UInt64 +instance KnownSetting "cluster_function_process_archive_on_multiple_nodes" Bool +instance KnownSetting "enable_filesystem_cache" Bool +instance KnownSetting "filesystem_cache_name" ChString +instance KnownSetting "enable_filesystem_cache_on_write_operations" Bool +instance KnownSetting "enable_filesystem_cache_log" Bool +instance KnownSetting "read_from_filesystem_cache_if_exists_otherwise_bypass_cache" Bool +instance KnownSetting "filesystem_cache_max_download_size" UInt64 +instance KnownSetting "throw_on_error_from_cache_on_write_operations" Bool +instance KnownSetting "filesystem_cache_segments_batch_size" UInt64 +instance KnownSetting "filesystem_cache_reserve_space_wait_lock_timeout_milliseconds" UInt64 +instance KnownSetting "filesystem_cache_prefer_bigger_buffer_size" Bool +instance KnownSetting "filesystem_cache_boundary_alignment" UInt64 +instance KnownSetting "temporary_data_in_cache_reserve_space_wait_lock_timeout_milliseconds" UInt64 +instance KnownSetting "use_page_cache_for_disks_without_file_cache" Bool +instance KnownSetting "use_page_cache_with_distributed_cache" Bool +instance KnownSetting "read_from_page_cache_if_exists_otherwise_bypass_cache" Bool +instance KnownSetting "page_cache_inject_eviction" Bool +instance KnownSetting "page_cache_block_size" UInt64 +instance KnownSetting "page_cache_lookahead_blocks" UInt64 +instance KnownSetting "load_marks_asynchronously" Bool +instance KnownSetting "enable_filesystem_read_prefetches_log" Bool +instance KnownSetting "allow_prefetched_read_pool_for_remote_filesystem" Bool +instance KnownSetting "allow_prefetched_read_pool_for_local_filesystem" Bool +instance KnownSetting "prefetch_buffer_size" UInt64 +instance KnownSetting "filesystem_prefetch_step_bytes" UInt64 +instance KnownSetting "filesystem_prefetch_step_marks" UInt64 +instance KnownSetting "filesystem_prefetches_limit" UInt64 +instance KnownSetting "use_structure_from_insertion_table_in_table_functions" UInt64 +instance KnownSetting "http_max_tries" UInt64 +instance KnownSetting "http_retry_initial_backoff_ms" UInt64 +instance KnownSetting "http_retry_max_backoff_ms" UInt64 +instance KnownSetting "force_remove_data_recursively_on_drop" Bool +instance KnownSetting "check_table_dependencies" Bool +instance KnownSetting "check_referential_table_dependencies" Bool +instance KnownSetting "allow_unrestricted_reads_from_keeper" Bool +instance KnownSetting "allow_deprecated_database_ordinary" Bool +instance KnownSetting "allow_deprecated_syntax_for_merge_tree" Bool +instance KnownSetting "allow_asynchronous_read_from_io_pool_for_merge_tree" Bool +instance KnownSetting "max_streams_for_merge_tree_reading" UInt64 +instance KnownSetting "force_grouping_standard_compatibility" Bool +instance KnownSetting "schema_inference_use_cache_for_file" Bool +instance KnownSetting "schema_inference_use_cache_for_s3" Bool +instance KnownSetting "schema_inference_use_cache_for_azure" Bool +instance KnownSetting "schema_inference_use_cache_for_hdfs" Bool +instance KnownSetting "schema_inference_use_cache_for_url" Bool +instance KnownSetting "schema_inference_cache_require_modification_time_for_url" Bool +instance KnownSetting "compatibility" ChString +instance KnownSetting "additional_result_filter" ChString +instance KnownSetting "workload" ChString +instance KnownSetting "rename_files_after_processing" ChString +instance KnownSetting "read_through_distributed_cache" Bool +instance KnownSetting "write_through_distributed_cache" Bool +instance KnownSetting "distributed_cache_throw_on_error" Bool +instance KnownSetting "distributed_cache_fetch_metrics_only_from_current_az" Bool +instance KnownSetting "distributed_cache_connect_max_tries" UInt64 +instance KnownSetting "distributed_cache_read_request_max_tries" UInt64 +instance KnownSetting "distributed_cache_receive_response_wait_milliseconds" UInt64 +instance KnownSetting "distributed_cache_receive_timeout_milliseconds" UInt64 +instance KnownSetting "distributed_cache_wait_connection_from_pool_milliseconds" UInt64 +instance KnownSetting "distributed_cache_bypass_connection_pool" Bool +instance KnownSetting "distributed_cache_alignment" UInt64 +instance KnownSetting "distributed_cache_max_unacked_inflight_packets" UInt64 +instance KnownSetting "distributed_cache_data_packet_ack_window" UInt64 +instance KnownSetting "distributed_cache_discard_connection_if_unread_data" Bool +instance KnownSetting "distributed_cache_min_bytes_for_seek" UInt64 +instance KnownSetting "distributed_cache_credentials_refresh_period_seconds" UInt64 +instance KnownSetting "distributed_cache_read_only_from_current_az" Bool +instance KnownSetting "write_through_distributed_cache_buffer_size" UInt64 +instance KnownSetting "table_engine_read_through_distributed_cache" Bool +instance KnownSetting "distributed_cache_connect_backoff_min_ms" UInt64 +instance KnownSetting "distributed_cache_connect_backoff_max_ms" UInt64 +instance KnownSetting "distributed_cache_prefer_bigger_buffer_size" Bool +instance KnownSetting "read_from_distributed_cache_if_exists_otherwise_bypass_cache" Bool +instance KnownSetting "distributed_cache_connect_timeout_ms" UInt64 +instance KnownSetting "distributed_cache_receive_timeout_ms" UInt64 +instance KnownSetting "distributed_cache_send_timeout_ms" UInt64 +instance KnownSetting "distributed_cache_tcp_keep_alive_timeout_ms" UInt64 +instance KnownSetting "filesystem_cache_enable_background_download_for_metadata_files_in_packed_storage" Bool +instance KnownSetting "filesystem_cache_enable_background_download_during_fetch" Bool +instance KnownSetting "parallelize_output_from_storages" Bool +instance KnownSetting "insert_deduplication_token" ChString +instance KnownSetting "count_distinct_optimization" Bool +instance KnownSetting "throw_if_no_data_to_insert" Bool +instance KnownSetting "compatibility_ignore_auto_increment_in_create_table" Bool +instance KnownSetting "multiple_joins_try_to_keep_original_names" Bool +instance KnownSetting "optimize_sorting_by_input_stream_properties" Bool +instance KnownSetting "keeper_max_retries" UInt64 +instance KnownSetting "keeper_retry_initial_backoff_ms" UInt64 +instance KnownSetting "keeper_retry_max_backoff_ms" UInt64 +instance KnownSetting "insert_keeper_max_retries" UInt64 +instance KnownSetting "insert_keeper_retry_initial_backoff_ms" UInt64 +instance KnownSetting "insert_keeper_retry_max_backoff_ms" UInt64 +instance KnownSetting "insert_keeper_fault_injection_seed" UInt64 +instance KnownSetting "force_aggregation_in_order" Bool +instance KnownSetting "http_max_request_param_data_size" UInt64 +instance KnownSetting "function_json_value_return_type_allow_nullable" Bool +instance KnownSetting "function_json_value_return_type_allow_complex" Bool +instance KnownSetting "use_with_fill_by_sorting_prefix" Bool +instance KnownSetting "optimize_uniq_to_count" Bool +instance KnownSetting "use_variant_as_common_type" Bool +instance KnownSetting "enable_order_by_all" Bool +instance KnownSetting "traverse_shadow_remote_data_paths" Bool +instance KnownSetting "geo_distance_returns_float64_on_float64_arguments" Bool +instance KnownSetting "allow_get_client_http_header" Bool +instance KnownSetting "cast_string_to_dynamic_use_inference" Bool +instance KnownSetting "allow_dynamic_type_in_join_keys" Bool +instance KnownSetting "cast_string_to_variant_use_inference" Bool +instance KnownSetting "enable_blob_storage_log" Bool +instance KnownSetting "use_json_alias_for_old_object_type" Bool +instance KnownSetting "allow_create_index_without_type" Bool +instance KnownSetting "create_index_ignore_unique" Bool +instance KnownSetting "print_pretty_type_names" Bool +instance KnownSetting "create_table_empty_primary_key_by_default" Bool +instance KnownSetting "allow_named_collection_override_by_default" Bool +instance KnownSetting "default_view_definer" ChString +instance KnownSetting "cache_warmer_threads" UInt64 +instance KnownSetting "use_async_executor_for_materialized_views" Bool +instance KnownSetting "short_circuit_function_evaluation_for_nulls" Bool +instance KnownSetting "show_data_lake_catalogs_in_system_tables" Bool +instance KnownSetting "delta_lake_enable_expression_visitor_logging" Bool +instance KnownSetting "delta_lake_throw_on_engine_predicate_error" Bool +instance KnownSetting "delta_lake_enable_engine_predicate" Bool +instance KnownSetting "allow_experimental_delta_lake_writes" Bool +instance KnownSetting "allow_deprecated_error_prone_window_functions" Bool +instance KnownSetting "use_iceberg_partition_pruning" Bool +instance KnownSetting "allow_deprecated_snowflake_conversion_functions" Bool +instance KnownSetting "optimize_distinct_in_order" Bool +instance KnownSetting "keeper_map_strict_mode" Bool +instance KnownSetting "restore_replace_external_engines_to_null" Bool +instance KnownSetting "restore_replace_external_table_functions_to_null" Bool +instance KnownSetting "restore_replace_external_dictionary_source_to_null" Bool +instance KnownSetting "parallel_replicas_count" UInt64 +instance KnownSetting "parallel_replica_offset" UInt64 +instance KnownSetting "parallel_replicas_custom_key" ChString +instance KnownSetting "parallel_replicas_custom_key_range_lower" UInt64 +instance KnownSetting "parallel_replicas_custom_key_range_upper" UInt64 +instance KnownSetting "cluster_for_parallel_replicas" ChString +instance KnownSetting "parallel_replicas_allow_in_with_subquery" Bool +instance KnownSetting "parallel_replicas_for_non_replicated_merge_tree" Bool +instance KnownSetting "parallel_replicas_min_number_of_rows_per_replica" UInt64 +instance KnownSetting "parallel_replicas_prefer_local_join" Bool +instance KnownSetting "parallel_replicas_mark_segment_size" UInt64 +instance KnownSetting "parallel_replicas_local_plan" Bool +instance KnownSetting "parallel_replicas_index_analysis_only_on_coordinator" Bool +instance KnownSetting "parallel_replicas_support_projection" Bool +instance KnownSetting "parallel_replicas_only_with_analyzer" Bool +instance KnownSetting "parallel_replicas_insert_select_local_pipeline" Bool +instance KnownSetting "parallel_replicas_for_cluster_engines" Bool +instance KnownSetting "analyzer_compatibility_join_using_top_level_identifier" Bool +instance KnownSetting "analyzer_compatibility_allow_compound_identifiers_in_unflatten_nested" Bool +instance KnownSetting "create_if_not_exists" Bool +instance KnownSetting "enforce_strict_identifier_format" Bool +instance KnownSetting "max_limit_for_vector_search_queries" UInt64 +instance KnownSetting "hnsw_candidate_list_size_for_search" UInt64 +instance KnownSetting "vector_search_with_rescoring" Bool +instance KnownSetting "mongodb_throw_on_unsupported_query" Bool +instance KnownSetting "implicit_select" Bool +instance KnownSetting "optimize_extract_common_expressions" Bool +instance KnownSetting "optimize_and_compare_chain" Bool +instance KnownSetting "push_external_roles_in_interserver_queries" Bool +instance KnownSetting "use_join_disjunctions_push_down" Bool +instance KnownSetting "shared_merge_tree_sync_parts_on_partition_operations" Bool +instance KnownSetting "implicit_table_at_top_level" ChString +instance KnownSetting "allow_general_join_planning" Bool +instance KnownSetting "merge_table_max_tables_to_look_for_schema_inference" UInt64 +instance KnownSetting "validate_enum_literals_in_operators" Bool +instance KnownSetting "max_autoincrement_series" UInt64 +instance KnownSetting "use_hive_partitioning" Bool +instance KnownSetting "parallel_hash_join_threshold" UInt64 +instance KnownSetting "apply_settings_from_server" Bool +instance KnownSetting "allow_archive_path_syntax" Bool +instance KnownSetting "iceberg_insert_max_rows_in_data_file" UInt64 +instance KnownSetting "iceberg_insert_max_bytes_in_data_file" UInt64 +instance KnownSetting "enable_producing_buckets_out_of_order_in_aggregation" Bool +instance KnownSetting "enable_parallel_blocks_marshalling" Bool +instance KnownSetting "min_outstreams_per_resize_after_split" UInt64 +instance KnownSetting "enable_add_distinct_to_in_subqueries" Bool +instance KnownSetting "function_date_trunc_return_type_behavior" UInt64 +instance KnownSetting "jemalloc_enable_profiler" Bool +instance KnownSetting "jemalloc_collect_profile_samples_in_trace_log" Bool +instance KnownSetting "use_roaring_bitmap_iceberg_positional_deletes" Bool +instance KnownSetting "inject_random_order_for_select_without_order_by" Bool +instance KnownSetting "allow_experimental_materialized_postgresql_table" Bool +instance KnownSetting "allow_experimental_funnel_functions" Bool +instance KnownSetting "allow_experimental_nlp_functions" Bool +instance KnownSetting "allow_experimental_hash_functions" Bool +instance KnownSetting "allow_experimental_object_type" Bool +instance KnownSetting "allow_experimental_time_series_table" Bool +instance KnownSetting "allow_experimental_codecs" Bool +instance KnownSetting "throw_on_unsupported_query_inside_transaction" Bool +instance KnownSetting "implicit_transaction" Bool +instance KnownSetting "join_to_sort_minimum_perkey_rows" UInt64 +instance KnownSetting "join_to_sort_maximum_table_rows" UInt64 +instance KnownSetting "allow_experimental_join_right_table_sorting" Bool +instance KnownSetting "use_statistics_cache" Bool +instance KnownSetting "allow_experimental_full_text_index" Bool +instance KnownSetting "query_plan_direct_read_from_text_index" Bool +instance KnownSetting "use_text_index_dictionary_cache" Bool +instance KnownSetting "use_text_index_header_cache" Bool +instance KnownSetting "allow_experimental_window_view" Bool +instance KnownSetting "stop_refreshable_materialized_views_on_startup" Bool +instance KnownSetting "allow_experimental_database_materialized_postgresql" Bool +instance KnownSetting "allow_experimental_qbit_type" Bool +instance KnownSetting "allow_experimental_query_deduplication" Bool +instance KnownSetting "allow_experimental_database_hms_catalog" Bool +instance KnownSetting "allow_experimental_kusto_dialect" Bool +instance KnownSetting "allow_experimental_prql_dialect" Bool +instance KnownSetting "enable_adaptive_memory_spill_scheduler" Bool +instance KnownSetting "allow_experimental_delta_kernel_rs" Bool +instance KnownSetting "allow_experimental_insert_into_iceberg" Bool +instance KnownSetting "allow_experimental_iceberg_compaction" Bool +instance KnownSetting "write_full_path_in_iceberg_metadata" Bool +instance KnownSetting "iceberg_metadata_compression_method" ChString +instance KnownSetting "make_distributed_plan" Bool +instance KnownSetting "distributed_plan_execute_locally" Bool +instance KnownSetting "distributed_plan_default_shuffle_join_bucket_count" UInt64 +instance KnownSetting "distributed_plan_default_reader_bucket_count" UInt64 +instance KnownSetting "distributed_plan_optimize_exchanges" Bool +instance KnownSetting "distributed_plan_force_exchange_kind" ChString +instance KnownSetting "distributed_plan_max_rows_to_broadcast" UInt64 +instance KnownSetting "allow_experimental_ytsaurus_table_engine" Bool +instance KnownSetting "allow_experimental_ytsaurus_table_function" Bool +instance KnownSetting "allow_experimental_ytsaurus_dictionary_source" Bool +instance KnownSetting "distributed_plan_force_shuffle_aggregation" Bool +instance KnownSetting "enable_join_runtime_filters" Bool +instance KnownSetting "join_runtime_filter_exact_values_limit" UInt64 +instance KnownSetting "join_runtime_bloom_filter_bytes" UInt64 +instance KnownSetting "join_runtime_bloom_filter_hash_functions" UInt64 +instance KnownSetting "rewrite_in_to_join" Bool +instance KnownSetting "promql_database" ChString +instance KnownSetting "promql_table" ChString + +settingsMap :: [(ChString, SettingSerializer)] +settingsMap = [ + mkSettingSerializer @"min_compress_block_size", + mkSettingSerializer @"max_compress_block_size", + mkSettingSerializer @"min_insert_block_size_rows", + mkSettingSerializer @"min_insert_block_size_bytes", + mkSettingSerializer @"min_insert_block_size_rows_for_materialized_views", + mkSettingSerializer @"min_insert_block_size_bytes_for_materialized_views", + mkSettingSerializer @"min_external_table_block_size_rows", + mkSettingSerializer @"min_external_table_block_size_bytes", + mkSettingSerializer @"max_joined_block_size_rows", + mkSettingSerializer @"max_joined_block_size_bytes", + mkSettingSerializer @"min_joined_block_size_rows", + mkSettingSerializer @"min_joined_block_size_bytes", + mkSettingSerializer @"joined_block_split_single_row", + mkSettingSerializer @"max_insert_threads", + mkSettingSerializer @"max_insert_delayed_streams_for_parallel_write", + mkSettingSerializer @"max_threads_for_indexes", + mkSettingSerializer @"use_concurrency_control", + mkSettingSerializer @"max_download_buffer_size", + mkSettingSerializer @"max_read_buffer_size_local_fs", + mkSettingSerializer @"max_read_buffer_size_remote_fs", + mkSettingSerializer @"max_distributed_connections", + mkSettingSerializer @"max_query_size", + mkSettingSerializer @"interactive_delay", + mkSettingSerializer @"use_hedged_requests", + mkSettingSerializer @"allow_changing_replica_until_first_data_packet", + mkSettingSerializer @"poll_interval", + mkSettingSerializer @"idle_connection_timeout", + mkSettingSerializer @"distributed_connections_pool_size", + mkSettingSerializer @"connections_with_failover_max_tries", + mkSettingSerializer @"s3_strict_upload_part_size", + mkSettingSerializer @"azure_strict_upload_part_size", + mkSettingSerializer @"azure_max_blocks_in_multipart_upload", + mkSettingSerializer @"s3_min_upload_part_size", + mkSettingSerializer @"s3_max_upload_part_size", + mkSettingSerializer @"azure_min_upload_part_size", + mkSettingSerializer @"azure_max_upload_part_size", + mkSettingSerializer @"s3_upload_part_size_multiply_factor", + mkSettingSerializer @"s3_upload_part_size_multiply_parts_count_threshold", + mkSettingSerializer @"s3_max_part_number", + mkSettingSerializer @"s3_allow_multipart_copy", + mkSettingSerializer @"s3_max_single_operation_copy_size", + mkSettingSerializer @"azure_upload_part_size_multiply_factor", + mkSettingSerializer @"azure_upload_part_size_multiply_parts_count_threshold", + mkSettingSerializer @"s3_max_inflight_parts_for_one_file", + mkSettingSerializer @"azure_max_inflight_parts_for_one_file", + mkSettingSerializer @"s3_max_single_part_upload_size", + mkSettingSerializer @"azure_max_single_part_upload_size", + mkSettingSerializer @"azure_max_single_part_copy_size", + mkSettingSerializer @"s3_max_single_read_retries", + mkSettingSerializer @"azure_max_single_read_retries", + mkSettingSerializer @"azure_max_unexpected_write_error_retries", + mkSettingSerializer @"s3_max_unexpected_write_error_retries", + mkSettingSerializer @"azure_max_redirects", + mkSettingSerializer @"azure_max_get_rps", + mkSettingSerializer @"azure_max_get_burst", + mkSettingSerializer @"azure_max_put_rps", + mkSettingSerializer @"azure_max_put_burst", + mkSettingSerializer @"s3_max_connections", + mkSettingSerializer @"s3_max_get_rps", + mkSettingSerializer @"s3_max_get_burst", + mkSettingSerializer @"s3_max_put_rps", + mkSettingSerializer @"s3_max_put_burst", + mkSettingSerializer @"s3_list_object_keys_size", + mkSettingSerializer @"s3_use_adaptive_timeouts", + mkSettingSerializer @"azure_use_adaptive_timeouts", + mkSettingSerializer @"s3_slow_all_threads_after_network_error", + mkSettingSerializer @"backup_slow_all_threads_after_retryable_s3_error", + mkSettingSerializer @"azure_list_object_keys_size", + mkSettingSerializer @"s3_truncate_on_insert", + mkSettingSerializer @"azure_truncate_on_insert", + mkSettingSerializer @"s3_create_new_file_on_insert", + mkSettingSerializer @"s3_skip_empty_files", + mkSettingSerializer @"azure_create_new_file_on_insert", + mkSettingSerializer @"s3_check_objects_after_upload", + mkSettingSerializer @"azure_check_objects_after_upload", + mkSettingSerializer @"s3_allow_parallel_part_upload", + mkSettingSerializer @"azure_allow_parallel_part_upload", + mkSettingSerializer @"s3_throw_on_zero_files_match", + mkSettingSerializer @"hdfs_throw_on_zero_files_match", + mkSettingSerializer @"azure_throw_on_zero_files_match", + mkSettingSerializer @"s3_ignore_file_doesnt_exist", + mkSettingSerializer @"hdfs_ignore_file_doesnt_exist", + mkSettingSerializer @"azure_ignore_file_doesnt_exist", + mkSettingSerializer @"azure_sdk_max_retries", + mkSettingSerializer @"azure_sdk_retry_initial_backoff_ms", + mkSettingSerializer @"azure_sdk_retry_max_backoff_ms", + mkSettingSerializer @"azure_request_timeout_ms", + mkSettingSerializer @"azure_connect_timeout_ms", + mkSettingSerializer @"s3_validate_request_settings", + mkSettingSerializer @"s3_disable_checksum", + mkSettingSerializer @"s3_request_timeout_ms", + mkSettingSerializer @"s3_connect_timeout_ms", + mkSettingSerializer @"enable_s3_requests_logging", + mkSettingSerializer @"s3queue_default_zookeeper_path", + mkSettingSerializer @"s3queue_migrate_old_metadata_to_buckets", + mkSettingSerializer @"s3queue_enable_logging_to_s3queue_log", + mkSettingSerializer @"hdfs_replication", + mkSettingSerializer @"hdfs_truncate_on_insert", + mkSettingSerializer @"hdfs_create_new_file_on_insert", + mkSettingSerializer @"hdfs_skip_empty_files", + mkSettingSerializer @"enable_hdfs_pread", + mkSettingSerializer @"azure_skip_empty_files", + mkSettingSerializer @"hsts_max_age", + mkSettingSerializer @"extremes", + mkSettingSerializer @"use_uncompressed_cache", + mkSettingSerializer @"replace_running_query", + mkSettingSerializer @"max_remote_read_network_bandwidth", + mkSettingSerializer @"max_remote_write_network_bandwidth", + mkSettingSerializer @"max_local_read_bandwidth", + mkSettingSerializer @"max_local_write_bandwidth", + mkSettingSerializer @"stream_like_engine_allow_direct_select", + mkSettingSerializer @"stream_like_engine_insert_queue", + mkSettingSerializer @"dictionary_validate_primary_key_type", + mkSettingSerializer @"distributed_insert_skip_read_only_replicas", + mkSettingSerializer @"optimize_move_to_prewhere", + mkSettingSerializer @"optimize_move_to_prewhere_if_final", + mkSettingSerializer @"move_all_conditions_to_prewhere", + mkSettingSerializer @"enable_multiple_prewhere_read_steps", + mkSettingSerializer @"move_primary_key_columns_to_end_of_prewhere", + mkSettingSerializer @"allow_reorder_prewhere_conditions", + mkSettingSerializer @"alter_move_to_space_execute_async", + mkSettingSerializer @"load_balancing_first_offset", + mkSettingSerializer @"allow_suspicious_low_cardinality_types", + mkSettingSerializer @"allow_suspicious_fixed_string_types", + mkSettingSerializer @"allow_suspicious_indices", + mkSettingSerializer @"allow_suspicious_ttl_expressions", + mkSettingSerializer @"allow_suspicious_variant_types", + mkSettingSerializer @"allow_suspicious_primary_key", + mkSettingSerializer @"allow_suspicious_types_in_group_by", + mkSettingSerializer @"allow_suspicious_types_in_order_by", + mkSettingSerializer @"allow_not_comparable_types_in_order_by", + mkSettingSerializer @"allow_not_comparable_types_in_comparison_functions", + mkSettingSerializer @"compile_expressions", + mkSettingSerializer @"min_count_to_compile_expression", + mkSettingSerializer @"compile_aggregate_expressions", + mkSettingSerializer @"min_count_to_compile_aggregate_expression", + mkSettingSerializer @"compile_sort_description", + mkSettingSerializer @"min_count_to_compile_sort_description", + mkSettingSerializer @"group_by_two_level_threshold", + mkSettingSerializer @"group_by_two_level_threshold_bytes", + mkSettingSerializer @"distributed_aggregation_memory_efficient", + mkSettingSerializer @"aggregation_memory_efficient_merge_threads", + mkSettingSerializer @"enable_memory_bound_merging_of_aggregation_results", + mkSettingSerializer @"enable_positional_arguments", + mkSettingSerializer @"enable_extended_results_for_datetime_functions", + mkSettingSerializer @"allow_nonconst_timezone_arguments", + mkSettingSerializer @"use_legacy_to_time", + mkSettingSerializer @"function_locate_has_mysql_compatible_argument_order", + mkSettingSerializer @"group_by_use_nulls", + mkSettingSerializer @"skip_unavailable_shards", + mkSettingSerializer @"parallel_distributed_insert_select", + mkSettingSerializer @"distributed_group_by_no_merge", + mkSettingSerializer @"distributed_push_down_limit", + mkSettingSerializer @"optimize_distributed_group_by_sharding_key", + mkSettingSerializer @"optimize_skip_unused_shards_limit", + mkSettingSerializer @"optimize_skip_unused_shards", + mkSettingSerializer @"optimize_skip_unused_shards_rewrite_in", + mkSettingSerializer @"allow_nondeterministic_optimize_skip_unused_shards", + mkSettingSerializer @"force_optimize_skip_unused_shards", + mkSettingSerializer @"optimize_skip_unused_shards_nesting", + mkSettingSerializer @"force_optimize_skip_unused_shards_nesting", + mkSettingSerializer @"input_format_parallel_parsing", + mkSettingSerializer @"output_format_parallel_formatting", + mkSettingSerializer @"output_format_compression_level", + mkSettingSerializer @"output_format_compression_zstd_window_log", + mkSettingSerializer @"allow_special_serialization_kinds_in_output_formats", + mkSettingSerializer @"enable_parsing_to_custom_serialization", + mkSettingSerializer @"merge_tree_use_v1_object_and_dynamic_serialization", + mkSettingSerializer @"merge_tree_min_rows_for_concurrent_read", + mkSettingSerializer @"merge_tree_min_bytes_for_concurrent_read", + mkSettingSerializer @"merge_tree_min_rows_for_seek", + mkSettingSerializer @"merge_tree_min_bytes_for_seek", + mkSettingSerializer @"merge_tree_coarse_index_granularity", + mkSettingSerializer @"merge_tree_max_rows_to_use_cache", + mkSettingSerializer @"merge_tree_max_bytes_to_use_cache", + mkSettingSerializer @"merge_tree_use_deserialization_prefixes_cache", + mkSettingSerializer @"merge_tree_use_prefixes_deserialization_thread_pool", + mkSettingSerializer @"do_not_merge_across_partitions_select_final", + mkSettingSerializer @"split_parts_ranges_into_intersecting_and_non_intersecting_final", + mkSettingSerializer @"split_intersecting_parts_ranges_into_layers_final", + mkSettingSerializer @"mysql_max_rows_to_insert", + mkSettingSerializer @"mysql_map_string_to_text_in_show_columns", + mkSettingSerializer @"mysql_map_fixed_string_to_text_in_show_columns", + mkSettingSerializer @"optimize_min_equality_disjunction_chain_length", + mkSettingSerializer @"optimize_min_inequality_conjunction_chain_length", + mkSettingSerializer @"min_bytes_to_use_direct_io", + mkSettingSerializer @"min_bytes_to_use_mmap_io", + mkSettingSerializer @"checksum_on_read", + mkSettingSerializer @"force_index_by_date", + mkSettingSerializer @"force_primary_key", + mkSettingSerializer @"use_skip_indexes", + mkSettingSerializer @"use_skip_indexes_if_final", + mkSettingSerializer @"use_skip_indexes_if_final_exact_mode", + mkSettingSerializer @"use_skip_indexes_on_data_read", + mkSettingSerializer @"materialize_skip_indexes_on_insert", + mkSettingSerializer @"exclude_materialize_skip_indexes_on_insert", + mkSettingSerializer @"text_index_use_bloom_filter", + mkSettingSerializer @"per_part_index_stats", + mkSettingSerializer @"materialize_statistics_on_insert", + mkSettingSerializer @"ignore_data_skipping_indices", + mkSettingSerializer @"force_data_skipping_indices", + mkSettingSerializer @"secondary_indices_enable_bulk_filtering", + mkSettingSerializer @"network_compression_method", + mkSettingSerializer @"priority", + mkSettingSerializer @"log_queries", + mkSettingSerializer @"log_formatted_queries", + mkSettingSerializer @"log_queries_cut_to_length", + mkSettingSerializer @"log_processors_profiles", + mkSettingSerializer @"max_concurrent_queries_for_all_users", + mkSettingSerializer @"max_concurrent_queries_for_user", + mkSettingSerializer @"insert_deduplicate", + mkSettingSerializer @"async_insert_deduplicate", + mkSettingSerializer @"insert_quorum_parallel", + mkSettingSerializer @"select_sequential_consistency", + mkSettingSerializer @"update_sequential_consistency", + mkSettingSerializer @"table_function_remote_max_addresses", + mkSettingSerializer @"read_backoff_max_throughput", + mkSettingSerializer @"read_backoff_min_events", + mkSettingSerializer @"read_backoff_min_concurrency", + mkSettingSerializer @"enable_http_compression", + mkSettingSerializer @"http_native_compression_disable_checksumming_on_decompress", + mkSettingSerializer @"count_distinct_implementation", + mkSettingSerializer @"add_http_cors_header", + mkSettingSerializer @"max_http_get_redirects", + mkSettingSerializer @"use_client_time_zone", + mkSettingSerializer @"send_progress_in_http_headers", + mkSettingSerializer @"http_headers_progress_interval_ms", + mkSettingSerializer @"http_wait_end_of_query", + mkSettingSerializer @"http_write_exception_in_output_format", + mkSettingSerializer @"http_response_buffer_size", + mkSettingSerializer @"fsync_metadata", + mkSettingSerializer @"join_use_nulls", + mkSettingSerializer @"join_output_by_rowlist_perkey_rows_threshold", + mkSettingSerializer @"any_join_distinct_right_table_keys", + mkSettingSerializer @"single_join_prefer_left_table", + mkSettingSerializer @"query_plan_optimize_join_order_limit", + mkSettingSerializer @"query_plan_join_shard_by_pk_ranges", + mkSettingSerializer @"query_plan_display_internal_aliases", + mkSettingSerializer @"query_plan_max_step_description_length", + mkSettingSerializer @"preferred_block_size_bytes", + mkSettingSerializer @"max_replica_delay_for_distributed_queries", + mkSettingSerializer @"fallback_to_stale_replicas_for_distributed_queries", + mkSettingSerializer @"preferred_max_column_in_block_size_bytes", + mkSettingSerializer @"parts_to_delay_insert", + mkSettingSerializer @"parts_to_throw_insert", + mkSettingSerializer @"number_of_mutations_to_delay", + mkSettingSerializer @"number_of_mutations_to_throw", + mkSettingSerializer @"min_free_disk_bytes_to_perform_insert", + mkSettingSerializer @"final", + mkSettingSerializer @"partial_result_on_first_cancel", + mkSettingSerializer @"ignore_on_cluster_for_replicated_udf_queries", + mkSettingSerializer @"ignore_on_cluster_for_replicated_access_entities_queries", + mkSettingSerializer @"ignore_on_cluster_for_replicated_named_collections_queries", + mkSettingSerializer @"unknown_packet_in_send_data", + mkSettingSerializer @"insert_allow_materialized_columns", + mkSettingSerializer @"http_max_uri_size", + mkSettingSerializer @"http_max_fields", + mkSettingSerializer @"http_max_field_name_size", + mkSettingSerializer @"http_max_field_value_size", + mkSettingSerializer @"http_skip_not_found_url_for_globs", + mkSettingSerializer @"http_make_head_request", + mkSettingSerializer @"optimize_throw_if_noop", + mkSettingSerializer @"use_index_for_in_with_subqueries", + mkSettingSerializer @"use_index_for_in_with_subqueries_max_values", + mkSettingSerializer @"analyze_index_with_space_filling_curves", + mkSettingSerializer @"joined_subquery_requires_alias", + mkSettingSerializer @"empty_result_for_aggregation_by_empty_set", + mkSettingSerializer @"empty_result_for_aggregation_by_constant_keys_on_empty_set", + mkSettingSerializer @"allow_distributed_ddl", + mkSettingSerializer @"allow_suspicious_codecs", + mkSettingSerializer @"enable_zstd_qat_codec", + mkSettingSerializer @"enable_deflate_qpl_codec", + mkSettingSerializer @"query_profiler_real_time_period_ns", + mkSettingSerializer @"query_profiler_cpu_time_period_ns", + mkSettingSerializer @"metrics_perf_events_enabled", + mkSettingSerializer @"metrics_perf_events_list", + mkSettingSerializer @"opentelemetry_trace_processors", + mkSettingSerializer @"opentelemetry_trace_cpu_scheduling", + mkSettingSerializer @"prefer_column_name_to_alias", + mkSettingSerializer @"skip_redundant_aliases_in_udf", + mkSettingSerializer @"prefer_global_in_and_join", + mkSettingSerializer @"enable_vertical_final", + mkSettingSerializer @"max_rows_to_read", + mkSettingSerializer @"max_bytes_to_read", + mkSettingSerializer @"max_rows_to_read_leaf", + mkSettingSerializer @"max_bytes_to_read_leaf", + mkSettingSerializer @"max_rows_to_group_by", + mkSettingSerializer @"max_bytes_before_external_group_by", + mkSettingSerializer @"max_rows_to_sort", + mkSettingSerializer @"max_bytes_to_sort", + mkSettingSerializer @"prefer_external_sort_block_bytes", + mkSettingSerializer @"max_bytes_before_external_sort", + mkSettingSerializer @"max_bytes_before_remerge_sort", + mkSettingSerializer @"max_result_rows", + mkSettingSerializer @"max_result_bytes", + mkSettingSerializer @"min_execution_speed", + mkSettingSerializer @"max_execution_speed", + mkSettingSerializer @"min_execution_speed_bytes", + mkSettingSerializer @"max_execution_speed_bytes", + mkSettingSerializer @"max_columns_to_read", + mkSettingSerializer @"max_temporary_columns", + mkSettingSerializer @"max_temporary_non_const_columns", + mkSettingSerializer @"max_sessions_for_user", + mkSettingSerializer @"max_subquery_depth", + mkSettingSerializer @"max_analyze_depth", + mkSettingSerializer @"max_ast_depth", + mkSettingSerializer @"max_ast_elements", + mkSettingSerializer @"max_expanded_ast_elements", + mkSettingSerializer @"readonly", + mkSettingSerializer @"max_rows_in_set", + mkSettingSerializer @"max_bytes_in_set", + mkSettingSerializer @"max_rows_in_join", + mkSettingSerializer @"max_bytes_in_join", + mkSettingSerializer @"join_any_take_last_row", + mkSettingSerializer @"cross_join_min_rows_to_compress", + mkSettingSerializer @"cross_join_min_bytes_to_compress", + mkSettingSerializer @"default_max_bytes_in_join", + mkSettingSerializer @"partial_merge_join_left_table_buffer_bytes", + mkSettingSerializer @"partial_merge_join_rows_in_right_blocks", + mkSettingSerializer @"join_on_disk_max_files_to_merge", + mkSettingSerializer @"max_rows_in_set_to_optimize_join", + mkSettingSerializer @"compatibility_ignore_collation_in_create_table", + mkSettingSerializer @"temporary_files_codec", + mkSettingSerializer @"temporary_files_buffer_size", + mkSettingSerializer @"max_rows_to_transfer", + mkSettingSerializer @"max_bytes_to_transfer", + mkSettingSerializer @"max_rows_in_distinct", + mkSettingSerializer @"max_bytes_in_distinct", + mkSettingSerializer @"max_memory_usage", + mkSettingSerializer @"memory_overcommit_ratio_denominator", + mkSettingSerializer @"max_memory_usage_for_user", + mkSettingSerializer @"memory_overcommit_ratio_denominator_for_user", + mkSettingSerializer @"max_untracked_memory", + mkSettingSerializer @"memory_profiler_step", + mkSettingSerializer @"memory_profiler_sample_min_allocation_size", + mkSettingSerializer @"memory_profiler_sample_max_allocation_size", + mkSettingSerializer @"trace_profile_events", + mkSettingSerializer @"memory_usage_overcommit_max_wait_microseconds", + mkSettingSerializer @"max_network_bandwidth", + mkSettingSerializer @"max_network_bytes", + mkSettingSerializer @"max_network_bandwidth_for_user", + mkSettingSerializer @"max_network_bandwidth_for_all_users", + mkSettingSerializer @"max_temporary_data_on_disk_size_for_user", + mkSettingSerializer @"max_temporary_data_on_disk_size_for_query", + mkSettingSerializer @"backup_restore_keeper_max_retries", + mkSettingSerializer @"backup_restore_keeper_retry_initial_backoff_ms", + mkSettingSerializer @"backup_restore_keeper_retry_max_backoff_ms", + mkSettingSerializer @"backup_restore_failure_after_host_disconnected_for_seconds", + mkSettingSerializer @"backup_restore_keeper_max_retries_while_initializing", + mkSettingSerializer @"backup_restore_keeper_max_retries_while_handling_error", + mkSettingSerializer @"backup_restore_finish_timeout_after_error_sec", + mkSettingSerializer @"backup_restore_keeper_value_max_size", + mkSettingSerializer @"backup_restore_batch_size_for_keeper_multi", + mkSettingSerializer @"backup_restore_batch_size_for_keeper_multiread", + mkSettingSerializer @"backup_restore_keeper_fault_injection_seed", + mkSettingSerializer @"backup_restore_s3_retry_attempts", + mkSettingSerializer @"backup_restore_s3_retry_initial_backoff_ms", + mkSettingSerializer @"backup_restore_s3_retry_max_backoff_ms", + mkSettingSerializer @"max_backup_bandwidth", + mkSettingSerializer @"restore_replicated_merge_tree_to_shared_merge_tree", + mkSettingSerializer @"log_profile_events", + mkSettingSerializer @"log_query_settings", + mkSettingSerializer @"log_query_threads", + mkSettingSerializer @"log_query_views", + mkSettingSerializer @"log_comment", + mkSettingSerializer @"send_logs_source_regexp", + mkSettingSerializer @"enable_optimize_predicate_expression", + mkSettingSerializer @"enable_optimize_predicate_expression_to_final_subquery", + mkSettingSerializer @"allow_push_predicate_when_subquery_contains_with", + mkSettingSerializer @"allow_push_predicate_ast_for_distributed_subqueries", + mkSettingSerializer @"low_cardinality_max_dictionary_size", + mkSettingSerializer @"low_cardinality_use_single_dictionary_for_part", + mkSettingSerializer @"decimal_check_overflow", + mkSettingSerializer @"allow_custom_error_code_in_throwif", + mkSettingSerializer @"prefer_localhost_replica", + mkSettingSerializer @"max_fetch_partition_retries_count", + mkSettingSerializer @"http_max_multipart_form_data_size", + mkSettingSerializer @"calculate_text_stack_trace", + mkSettingSerializer @"enable_job_stack_trace", + mkSettingSerializer @"allow_ddl", + mkSettingSerializer @"parallel_view_processing", + mkSettingSerializer @"enable_unaligned_array_join", + mkSettingSerializer @"optimize_read_in_order", + mkSettingSerializer @"read_in_order_use_virtual_row", + mkSettingSerializer @"optimize_read_in_window_order", + mkSettingSerializer @"optimize_aggregation_in_order", + mkSettingSerializer @"read_in_order_use_buffering", + mkSettingSerializer @"aggregation_in_order_max_block_bytes", + mkSettingSerializer @"read_in_order_two_level_merge_threshold", + mkSettingSerializer @"low_cardinality_allow_in_native_format", + mkSettingSerializer @"cancel_http_readonly_queries_on_client_close", + mkSettingSerializer @"external_table_functions_use_nulls", + mkSettingSerializer @"external_table_strict_query", + mkSettingSerializer @"allow_hyperscan", + mkSettingSerializer @"max_hyperscan_regexp_length", + mkSettingSerializer @"max_hyperscan_regexp_total_length", + mkSettingSerializer @"reject_expensive_hyperscan_regexps", + mkSettingSerializer @"allow_simdjson", + mkSettingSerializer @"allow_introspection_functions", + mkSettingSerializer @"splitby_max_substrings_includes_remaining_string", + mkSettingSerializer @"allow_execute_multiif_columnar", + mkSettingSerializer @"formatdatetime_f_prints_single_zero", + mkSettingSerializer @"formatdatetime_f_prints_scale_number_of_digits", + mkSettingSerializer @"formatdatetime_parsedatetime_m_is_month_name", + mkSettingSerializer @"parsedatetime_parse_without_leading_zeros", + mkSettingSerializer @"parsedatetime_e_requires_space_padding", + mkSettingSerializer @"formatdatetime_format_without_leading_zeros", + mkSettingSerializer @"formatdatetime_e_with_space_padding", + mkSettingSerializer @"least_greatest_legacy_null_behavior", + mkSettingSerializer @"h3togeo_lon_lat_result_order", + mkSettingSerializer @"max_partitions_per_insert_block", + mkSettingSerializer @"throw_on_max_partitions_per_insert_block", + mkSettingSerializer @"check_query_single_value_result", + mkSettingSerializer @"allow_drop_detached", + mkSettingSerializer @"max_parts_to_move", + mkSettingSerializer @"max_table_size_to_drop", + mkSettingSerializer @"max_partition_size_to_drop", + mkSettingSerializer @"postgresql_connection_pool_size", + mkSettingSerializer @"postgresql_connection_attempt_timeout", + mkSettingSerializer @"postgresql_connection_pool_wait_timeout", + mkSettingSerializer @"postgresql_connection_pool_retries", + mkSettingSerializer @"postgresql_connection_pool_auto_close_connection", + mkSettingSerializer @"glob_expansion_max_elements", + mkSettingSerializer @"odbc_bridge_connection_pool_size", + mkSettingSerializer @"odbc_bridge_use_connection_pooling", + mkSettingSerializer @"distributed_replica_error_cap", + mkSettingSerializer @"distributed_replica_max_ignored_errors", + mkSettingSerializer @"min_free_disk_space_for_temporary_data", + mkSettingSerializer @"show_table_uuid_in_table_create_query_if_not_nil", + mkSettingSerializer @"database_atomic_wait_for_drop_and_detach_synchronously", + mkSettingSerializer @"enable_scalar_subquery_optimization", + mkSettingSerializer @"optimize_trivial_count_query", + mkSettingSerializer @"optimize_trivial_approximate_count_query", + mkSettingSerializer @"optimize_count_from_files", + mkSettingSerializer @"use_cache_for_count_from_files", + mkSettingSerializer @"optimize_respect_aliases", + mkSettingSerializer @"mutations_sync", + mkSettingSerializer @"lightweight_deletes_sync", + mkSettingSerializer @"apply_deleted_mask", + mkSettingSerializer @"optimize_normalize_count_variants", + mkSettingSerializer @"optimize_injective_functions_inside_uniq", + mkSettingSerializer @"count_matches_stop_at_empty_match", + mkSettingSerializer @"rewrite_count_distinct_if_with_count_distinct_implementation", + mkSettingSerializer @"convert_query_to_cnf", + mkSettingSerializer @"optimize_or_like_chain", + mkSettingSerializer @"optimize_arithmetic_operations_in_aggregate_functions", + mkSettingSerializer @"optimize_redundant_functions_in_order_by", + mkSettingSerializer @"optimize_if_chain_to_multiif", + mkSettingSerializer @"optimize_multiif_to_if", + mkSettingSerializer @"optimize_if_transform_strings_to_enum", + mkSettingSerializer @"optimize_functions_to_subcolumns", + mkSettingSerializer @"optimize_using_constraints", + mkSettingSerializer @"optimize_substitute_columns", + mkSettingSerializer @"optimize_append_index", + mkSettingSerializer @"optimize_time_filter_with_preimage", + mkSettingSerializer @"normalize_function_names", + mkSettingSerializer @"enable_early_constant_folding", + mkSettingSerializer @"deduplicate_blocks_in_dependent_materialized_views", + mkSettingSerializer @"throw_if_deduplication_in_dependent_materialized_views_enabled_with_async_insert", + mkSettingSerializer @"materialized_views_ignore_errors", + mkSettingSerializer @"ignore_materialized_views_with_dropped_target_table", + mkSettingSerializer @"allow_materialized_view_with_bad_select", + mkSettingSerializer @"materialized_views_squash_parallel_inserts", + mkSettingSerializer @"use_compact_format_in_distributed_parts_names", + mkSettingSerializer @"validate_polygons", + mkSettingSerializer @"max_parser_depth", + mkSettingSerializer @"max_parser_backtracks", + mkSettingSerializer @"max_recursive_cte_evaluation_depth", + mkSettingSerializer @"allow_settings_after_format_in_insert", + mkSettingSerializer @"transform_null_in", + mkSettingSerializer @"allow_nondeterministic_mutations", + mkSettingSerializer @"validate_mutation_query", + mkSettingSerializer @"materialize_ttl_after_modify", + mkSettingSerializer @"function_implementation", + mkSettingSerializer @"data_type_default_nullable", + mkSettingSerializer @"cast_keep_nullable", + mkSettingSerializer @"cast_ipv4_ipv6_default_on_conversion_error", + mkSettingSerializer @"alter_partition_verbose_result", + mkSettingSerializer @"system_events_show_zero_values", + mkSettingSerializer @"optimize_trivial_insert_select", + mkSettingSerializer @"allow_non_metadata_alters", + mkSettingSerializer @"enable_global_with_statement", + mkSettingSerializer @"enable_scopes_for_with_statement", + mkSettingSerializer @"aggregate_functions_null_for_empty", + mkSettingSerializer @"optimize_syntax_fuse_functions", + mkSettingSerializer @"flatten_nested", + mkSettingSerializer @"asterisk_include_materialized_columns", + mkSettingSerializer @"asterisk_include_alias_columns", + mkSettingSerializer @"optimize_skip_merged_partitions", + mkSettingSerializer @"optimize_on_insert", + mkSettingSerializer @"optimize_use_implicit_projections", + mkSettingSerializer @"optimize_use_projection_filtering", + mkSettingSerializer @"force_optimize_projection", + mkSettingSerializer @"force_optimize_projection_name", + mkSettingSerializer @"preferred_optimize_projection_name", + mkSettingSerializer @"max_projection_rows_to_use_projection_index", + mkSettingSerializer @"min_table_rows_to_use_projection_index", + mkSettingSerializer @"async_socket_for_remote", + mkSettingSerializer @"async_query_sending_for_remote", + mkSettingSerializer @"insert_null_as_default", + mkSettingSerializer @"describe_extend_object_types", + mkSettingSerializer @"describe_include_subcolumns", + mkSettingSerializer @"describe_include_virtual_columns", + mkSettingSerializer @"describe_compact_output", + mkSettingSerializer @"apply_mutations_on_fly", + mkSettingSerializer @"apply_patch_parts", + mkSettingSerializer @"mutations_execute_nondeterministic_on_initiator", + mkSettingSerializer @"mutations_execute_subqueries_on_initiator", + mkSettingSerializer @"mutations_max_literal_size_to_replace", + mkSettingSerializer @"delta_lake_log_metadata", + mkSettingSerializer @"iceberg_delete_data_on_drop", + mkSettingSerializer @"use_iceberg_metadata_files_cache", + mkSettingSerializer @"use_query_cache", + mkSettingSerializer @"enable_writes_to_query_cache", + mkSettingSerializer @"enable_reads_from_query_cache", + mkSettingSerializer @"query_cache_max_size_in_bytes", + mkSettingSerializer @"query_cache_max_entries", + mkSettingSerializer @"query_cache_min_query_runs", + mkSettingSerializer @"query_cache_compress_entries", + mkSettingSerializer @"query_cache_squash_partial_results", + mkSettingSerializer @"query_cache_share_between_users", + mkSettingSerializer @"query_cache_tag", + mkSettingSerializer @"enable_sharing_sets_for_mutations", + mkSettingSerializer @"use_query_condition_cache", + mkSettingSerializer @"query_condition_cache_store_conditions_as_plaintext", + mkSettingSerializer @"enable_shared_storage_snapshot_in_query", + mkSettingSerializer @"merge_tree_storage_snapshot_sleep_ms", + mkSettingSerializer @"optimize_rewrite_sum_if_to_count_if", + mkSettingSerializer @"optimize_empty_string_comparisons", + mkSettingSerializer @"optimize_rewrite_aggregate_function_with_if", + mkSettingSerializer @"optimize_rewrite_array_exists_to_has", + mkSettingSerializer @"optimize_rewrite_like_perfect_affix", + mkSettingSerializer @"execute_exists_as_scalar_subquery", + mkSettingSerializer @"optimize_rewrite_regexp_functions", + mkSettingSerializer @"insert_shard_id", + mkSettingSerializer @"collect_hash_table_stats_during_aggregation", + mkSettingSerializer @"max_size_to_preallocate_for_aggregation", + mkSettingSerializer @"collect_hash_table_stats_during_joins", + mkSettingSerializer @"max_size_to_preallocate_for_joins", + mkSettingSerializer @"kafka_disable_num_consumers_limit", + mkSettingSerializer @"allow_experimental_kafka_offsets_storage_in_keeper", + mkSettingSerializer @"enable_software_prefetch_in_aggregation", + mkSettingSerializer @"allow_aggregate_partitions_independently", + mkSettingSerializer @"force_aggregate_partitions_independently", + mkSettingSerializer @"max_number_of_partitions_for_independent_aggregation", + mkSettingSerializer @"engine_file_empty_if_not_exists", + mkSettingSerializer @"engine_file_truncate_on_insert", + mkSettingSerializer @"engine_file_allow_create_multiple_files", + mkSettingSerializer @"engine_file_skip_empty_files", + mkSettingSerializer @"engine_url_skip_empty_files", + mkSettingSerializer @"enable_url_encoding", + mkSettingSerializer @"database_replicated_initial_query_timeout_sec", + mkSettingSerializer @"database_replicated_enforce_synchronous_settings", + mkSettingSerializer @"max_distributed_depth", + mkSettingSerializer @"database_replicated_always_detach_permanently", + mkSettingSerializer @"database_replicated_allow_only_replicated_engine", + mkSettingSerializer @"database_replicated_allow_replicated_engine_arguments", + mkSettingSerializer @"database_replicated_allow_explicit_uuid", + mkSettingSerializer @"database_replicated_allow_heavy_create", + mkSettingSerializer @"cloud_mode", + mkSettingSerializer @"cloud_mode_engine", + mkSettingSerializer @"cloud_mode_database_engine", + mkSettingSerializer @"distributed_ddl_entry_format_version", + mkSettingSerializer @"external_storage_max_read_rows", + mkSettingSerializer @"external_storage_max_read_bytes", + mkSettingSerializer @"external_storage_connect_timeout_sec", + mkSettingSerializer @"external_storage_rw_timeout_sec", + mkSettingSerializer @"allow_experimental_correlated_subqueries", + mkSettingSerializer @"optimize_aggregators_of_group_by_keys", + mkSettingSerializer @"optimize_injective_functions_in_group_by", + mkSettingSerializer @"optimize_group_by_function_keys", + mkSettingSerializer @"optimize_group_by_constant_keys", + mkSettingSerializer @"legacy_column_name_of_tuple_literal", + mkSettingSerializer @"enable_named_columns_in_function_tuple", + mkSettingSerializer @"query_plan_enable_optimizations", + mkSettingSerializer @"query_plan_max_optimizations_to_apply", + mkSettingSerializer @"query_plan_lift_up_array_join", + mkSettingSerializer @"query_plan_push_down_limit", + mkSettingSerializer @"query_plan_split_filter", + mkSettingSerializer @"query_plan_merge_expressions", + mkSettingSerializer @"query_plan_merge_filters", + mkSettingSerializer @"query_plan_filter_push_down", + mkSettingSerializer @"query_plan_convert_outer_join_to_inner_join", + mkSettingSerializer @"query_plan_convert_any_join_to_semi_or_anti_join", + mkSettingSerializer @"query_plan_merge_filter_into_join_condition", + mkSettingSerializer @"query_plan_convert_join_to_in", + mkSettingSerializer @"query_plan_optimize_prewhere", + mkSettingSerializer @"query_plan_execute_functions_after_sorting", + mkSettingSerializer @"query_plan_reuse_storage_ordering_for_window_functions", + mkSettingSerializer @"query_plan_lift_up_union", + mkSettingSerializer @"query_plan_read_in_order", + mkSettingSerializer @"query_plan_aggregation_in_order", + mkSettingSerializer @"query_plan_remove_redundant_sorting", + mkSettingSerializer @"query_plan_remove_redundant_distinct", + mkSettingSerializer @"query_plan_try_use_vector_search", + mkSettingSerializer @"query_plan_enable_multithreading_after_window_functions", + mkSettingSerializer @"query_plan_optimize_lazy_materialization", + mkSettingSerializer @"query_plan_max_limit_for_lazy_materialization", + mkSettingSerializer @"enable_lazy_columns_replication", + mkSettingSerializer @"serialize_query_plan", + mkSettingSerializer @"correlated_subqueries_substitute_equivalent_expressions", + mkSettingSerializer @"optimize_qbit_distance_function_reads", + mkSettingSerializer @"regexp_max_matches_per_row", + mkSettingSerializer @"limit", + mkSettingSerializer @"offset", + mkSettingSerializer @"function_range_max_elements_in_block", + mkSettingSerializer @"function_sleep_max_microseconds_per_block", + mkSettingSerializer @"function_visible_width_behavior", + mkSettingSerializer @"local_filesystem_read_method", + mkSettingSerializer @"remote_filesystem_read_method", + mkSettingSerializer @"local_filesystem_read_prefetch", + mkSettingSerializer @"remote_filesystem_read_prefetch", + mkSettingSerializer @"merge_tree_min_rows_for_concurrent_read_for_remote_filesystem", + mkSettingSerializer @"merge_tree_min_bytes_for_concurrent_read_for_remote_filesystem", + mkSettingSerializer @"remote_read_min_bytes_for_seek", + mkSettingSerializer @"merge_tree_use_const_size_tasks_for_remote_reading", + mkSettingSerializer @"merge_tree_determine_task_size_by_prewhere_columns", + mkSettingSerializer @"merge_tree_compact_parts_min_granules_to_multibuffer_read", + mkSettingSerializer @"async_insert", + mkSettingSerializer @"wait_for_async_insert", + mkSettingSerializer @"async_insert_max_data_size", + mkSettingSerializer @"async_insert_max_query_number", + mkSettingSerializer @"async_insert_use_adaptive_busy_timeout", + mkSettingSerializer @"remote_fs_read_max_backoff_ms", + mkSettingSerializer @"remote_fs_read_backoff_max_tries", + mkSettingSerializer @"cluster_function_process_archive_on_multiple_nodes", + mkSettingSerializer @"enable_filesystem_cache", + mkSettingSerializer @"filesystem_cache_name", + mkSettingSerializer @"enable_filesystem_cache_on_write_operations", + mkSettingSerializer @"enable_filesystem_cache_log", + mkSettingSerializer @"read_from_filesystem_cache_if_exists_otherwise_bypass_cache", + mkSettingSerializer @"filesystem_cache_max_download_size", + mkSettingSerializer @"throw_on_error_from_cache_on_write_operations", + mkSettingSerializer @"filesystem_cache_segments_batch_size", + mkSettingSerializer @"filesystem_cache_reserve_space_wait_lock_timeout_milliseconds", + mkSettingSerializer @"filesystem_cache_prefer_bigger_buffer_size", + mkSettingSerializer @"filesystem_cache_boundary_alignment", + mkSettingSerializer @"temporary_data_in_cache_reserve_space_wait_lock_timeout_milliseconds", + mkSettingSerializer @"use_page_cache_for_disks_without_file_cache", + mkSettingSerializer @"use_page_cache_with_distributed_cache", + mkSettingSerializer @"read_from_page_cache_if_exists_otherwise_bypass_cache", + mkSettingSerializer @"page_cache_inject_eviction", + mkSettingSerializer @"page_cache_block_size", + mkSettingSerializer @"page_cache_lookahead_blocks", + mkSettingSerializer @"load_marks_asynchronously", + mkSettingSerializer @"enable_filesystem_read_prefetches_log", + mkSettingSerializer @"allow_prefetched_read_pool_for_remote_filesystem", + mkSettingSerializer @"allow_prefetched_read_pool_for_local_filesystem", + mkSettingSerializer @"prefetch_buffer_size", + mkSettingSerializer @"filesystem_prefetch_step_bytes", + mkSettingSerializer @"filesystem_prefetch_step_marks", + mkSettingSerializer @"filesystem_prefetches_limit", + mkSettingSerializer @"use_structure_from_insertion_table_in_table_functions", + mkSettingSerializer @"http_max_tries", + mkSettingSerializer @"http_retry_initial_backoff_ms", + mkSettingSerializer @"http_retry_max_backoff_ms", + mkSettingSerializer @"force_remove_data_recursively_on_drop", + mkSettingSerializer @"check_table_dependencies", + mkSettingSerializer @"check_referential_table_dependencies", + mkSettingSerializer @"allow_unrestricted_reads_from_keeper", + mkSettingSerializer @"allow_deprecated_database_ordinary", + mkSettingSerializer @"allow_deprecated_syntax_for_merge_tree", + mkSettingSerializer @"allow_asynchronous_read_from_io_pool_for_merge_tree", + mkSettingSerializer @"max_streams_for_merge_tree_reading", + mkSettingSerializer @"force_grouping_standard_compatibility", + mkSettingSerializer @"schema_inference_use_cache_for_file", + mkSettingSerializer @"schema_inference_use_cache_for_s3", + mkSettingSerializer @"schema_inference_use_cache_for_azure", + mkSettingSerializer @"schema_inference_use_cache_for_hdfs", + mkSettingSerializer @"schema_inference_use_cache_for_url", + mkSettingSerializer @"schema_inference_cache_require_modification_time_for_url", + mkSettingSerializer @"compatibility", + mkSettingSerializer @"additional_result_filter", + mkSettingSerializer @"workload", + mkSettingSerializer @"rename_files_after_processing", + mkSettingSerializer @"read_through_distributed_cache", + mkSettingSerializer @"write_through_distributed_cache", + mkSettingSerializer @"distributed_cache_throw_on_error", + mkSettingSerializer @"distributed_cache_fetch_metrics_only_from_current_az", + mkSettingSerializer @"distributed_cache_connect_max_tries", + mkSettingSerializer @"distributed_cache_read_request_max_tries", + mkSettingSerializer @"distributed_cache_receive_response_wait_milliseconds", + mkSettingSerializer @"distributed_cache_receive_timeout_milliseconds", + mkSettingSerializer @"distributed_cache_wait_connection_from_pool_milliseconds", + mkSettingSerializer @"distributed_cache_bypass_connection_pool", + mkSettingSerializer @"distributed_cache_alignment", + mkSettingSerializer @"distributed_cache_max_unacked_inflight_packets", + mkSettingSerializer @"distributed_cache_data_packet_ack_window", + mkSettingSerializer @"distributed_cache_discard_connection_if_unread_data", + mkSettingSerializer @"distributed_cache_min_bytes_for_seek", + mkSettingSerializer @"distributed_cache_credentials_refresh_period_seconds", + mkSettingSerializer @"distributed_cache_read_only_from_current_az", + mkSettingSerializer @"write_through_distributed_cache_buffer_size", + mkSettingSerializer @"table_engine_read_through_distributed_cache", + mkSettingSerializer @"distributed_cache_connect_backoff_min_ms", + mkSettingSerializer @"distributed_cache_connect_backoff_max_ms", + mkSettingSerializer @"distributed_cache_prefer_bigger_buffer_size", + mkSettingSerializer @"read_from_distributed_cache_if_exists_otherwise_bypass_cache", + mkSettingSerializer @"distributed_cache_connect_timeout_ms", + mkSettingSerializer @"distributed_cache_receive_timeout_ms", + mkSettingSerializer @"distributed_cache_send_timeout_ms", + mkSettingSerializer @"distributed_cache_tcp_keep_alive_timeout_ms", + mkSettingSerializer @"filesystem_cache_enable_background_download_for_metadata_files_in_packed_storage", + mkSettingSerializer @"filesystem_cache_enable_background_download_during_fetch", + mkSettingSerializer @"parallelize_output_from_storages", + mkSettingSerializer @"insert_deduplication_token", + mkSettingSerializer @"count_distinct_optimization", + mkSettingSerializer @"throw_if_no_data_to_insert", + mkSettingSerializer @"compatibility_ignore_auto_increment_in_create_table", + mkSettingSerializer @"multiple_joins_try_to_keep_original_names", + mkSettingSerializer @"optimize_sorting_by_input_stream_properties", + mkSettingSerializer @"keeper_max_retries", + mkSettingSerializer @"keeper_retry_initial_backoff_ms", + mkSettingSerializer @"keeper_retry_max_backoff_ms", + mkSettingSerializer @"insert_keeper_max_retries", + mkSettingSerializer @"insert_keeper_retry_initial_backoff_ms", + mkSettingSerializer @"insert_keeper_retry_max_backoff_ms", + mkSettingSerializer @"insert_keeper_fault_injection_seed", + mkSettingSerializer @"force_aggregation_in_order", + mkSettingSerializer @"http_max_request_param_data_size", + mkSettingSerializer @"function_json_value_return_type_allow_nullable", + mkSettingSerializer @"function_json_value_return_type_allow_complex", + mkSettingSerializer @"use_with_fill_by_sorting_prefix", + mkSettingSerializer @"optimize_uniq_to_count", + mkSettingSerializer @"use_variant_as_common_type", + mkSettingSerializer @"enable_order_by_all", + mkSettingSerializer @"traverse_shadow_remote_data_paths", + mkSettingSerializer @"geo_distance_returns_float64_on_float64_arguments", + mkSettingSerializer @"allow_get_client_http_header", + mkSettingSerializer @"cast_string_to_dynamic_use_inference", + mkSettingSerializer @"allow_dynamic_type_in_join_keys", + mkSettingSerializer @"cast_string_to_variant_use_inference", + mkSettingSerializer @"enable_blob_storage_log", + mkSettingSerializer @"use_json_alias_for_old_object_type", + mkSettingSerializer @"allow_create_index_without_type", + mkSettingSerializer @"create_index_ignore_unique", + mkSettingSerializer @"print_pretty_type_names", + mkSettingSerializer @"create_table_empty_primary_key_by_default", + mkSettingSerializer @"allow_named_collection_override_by_default", + mkSettingSerializer @"default_view_definer", + mkSettingSerializer @"cache_warmer_threads", + mkSettingSerializer @"use_async_executor_for_materialized_views", + mkSettingSerializer @"short_circuit_function_evaluation_for_nulls", + mkSettingSerializer @"show_data_lake_catalogs_in_system_tables", + mkSettingSerializer @"delta_lake_enable_expression_visitor_logging", + mkSettingSerializer @"delta_lake_throw_on_engine_predicate_error", + mkSettingSerializer @"delta_lake_enable_engine_predicate", + mkSettingSerializer @"allow_experimental_delta_lake_writes", + mkSettingSerializer @"allow_deprecated_error_prone_window_functions", + mkSettingSerializer @"use_iceberg_partition_pruning", + mkSettingSerializer @"allow_deprecated_snowflake_conversion_functions", + mkSettingSerializer @"optimize_distinct_in_order", + mkSettingSerializer @"keeper_map_strict_mode", + mkSettingSerializer @"restore_replace_external_engines_to_null", + mkSettingSerializer @"restore_replace_external_table_functions_to_null", + mkSettingSerializer @"restore_replace_external_dictionary_source_to_null", + mkSettingSerializer @"parallel_replicas_count", + mkSettingSerializer @"parallel_replica_offset", + mkSettingSerializer @"parallel_replicas_custom_key", + mkSettingSerializer @"parallel_replicas_custom_key_range_lower", + mkSettingSerializer @"parallel_replicas_custom_key_range_upper", + mkSettingSerializer @"cluster_for_parallel_replicas", + mkSettingSerializer @"parallel_replicas_allow_in_with_subquery", + mkSettingSerializer @"parallel_replicas_for_non_replicated_merge_tree", + mkSettingSerializer @"parallel_replicas_min_number_of_rows_per_replica", + mkSettingSerializer @"parallel_replicas_prefer_local_join", + mkSettingSerializer @"parallel_replicas_mark_segment_size", + mkSettingSerializer @"parallel_replicas_local_plan", + mkSettingSerializer @"parallel_replicas_index_analysis_only_on_coordinator", + mkSettingSerializer @"parallel_replicas_support_projection", + mkSettingSerializer @"parallel_replicas_only_with_analyzer", + mkSettingSerializer @"parallel_replicas_insert_select_local_pipeline", + mkSettingSerializer @"parallel_replicas_for_cluster_engines", + mkSettingSerializer @"analyzer_compatibility_join_using_top_level_identifier", + mkSettingSerializer @"analyzer_compatibility_allow_compound_identifiers_in_unflatten_nested", + mkSettingSerializer @"create_if_not_exists", + mkSettingSerializer @"enforce_strict_identifier_format", + mkSettingSerializer @"max_limit_for_vector_search_queries", + mkSettingSerializer @"hnsw_candidate_list_size_for_search", + mkSettingSerializer @"vector_search_with_rescoring", + mkSettingSerializer @"mongodb_throw_on_unsupported_query", + mkSettingSerializer @"implicit_select", + mkSettingSerializer @"optimize_extract_common_expressions", + mkSettingSerializer @"optimize_and_compare_chain", + mkSettingSerializer @"push_external_roles_in_interserver_queries", + mkSettingSerializer @"use_join_disjunctions_push_down", + mkSettingSerializer @"shared_merge_tree_sync_parts_on_partition_operations", + mkSettingSerializer @"implicit_table_at_top_level", + mkSettingSerializer @"allow_general_join_planning", + mkSettingSerializer @"merge_table_max_tables_to_look_for_schema_inference", + mkSettingSerializer @"validate_enum_literals_in_operators", + mkSettingSerializer @"max_autoincrement_series", + mkSettingSerializer @"use_hive_partitioning", + mkSettingSerializer @"parallel_hash_join_threshold", + mkSettingSerializer @"apply_settings_from_server", + mkSettingSerializer @"allow_archive_path_syntax", + mkSettingSerializer @"iceberg_insert_max_rows_in_data_file", + mkSettingSerializer @"iceberg_insert_max_bytes_in_data_file", + mkSettingSerializer @"enable_producing_buckets_out_of_order_in_aggregation", + mkSettingSerializer @"enable_parallel_blocks_marshalling", + mkSettingSerializer @"min_outstreams_per_resize_after_split", + mkSettingSerializer @"enable_add_distinct_to_in_subqueries", + mkSettingSerializer @"function_date_trunc_return_type_behavior", + mkSettingSerializer @"jemalloc_enable_profiler", + mkSettingSerializer @"jemalloc_collect_profile_samples_in_trace_log", + mkSettingSerializer @"use_roaring_bitmap_iceberg_positional_deletes", + mkSettingSerializer @"inject_random_order_for_select_without_order_by", + mkSettingSerializer @"allow_experimental_materialized_postgresql_table", + mkSettingSerializer @"allow_experimental_funnel_functions", + mkSettingSerializer @"allow_experimental_nlp_functions", + mkSettingSerializer @"allow_experimental_hash_functions", + mkSettingSerializer @"allow_experimental_object_type", + mkSettingSerializer @"allow_experimental_time_series_table", + mkSettingSerializer @"allow_experimental_codecs", + mkSettingSerializer @"throw_on_unsupported_query_inside_transaction", + mkSettingSerializer @"implicit_transaction", + mkSettingSerializer @"join_to_sort_minimum_perkey_rows", + mkSettingSerializer @"join_to_sort_maximum_table_rows", + mkSettingSerializer @"allow_experimental_join_right_table_sorting", + mkSettingSerializer @"use_statistics_cache", + mkSettingSerializer @"allow_experimental_full_text_index", + mkSettingSerializer @"query_plan_direct_read_from_text_index", + mkSettingSerializer @"use_text_index_dictionary_cache", + mkSettingSerializer @"use_text_index_header_cache", + mkSettingSerializer @"allow_experimental_window_view", + mkSettingSerializer @"stop_refreshable_materialized_views_on_startup", + mkSettingSerializer @"allow_experimental_database_materialized_postgresql", + mkSettingSerializer @"allow_experimental_qbit_type", + mkSettingSerializer @"allow_experimental_query_deduplication", + mkSettingSerializer @"allow_experimental_database_hms_catalog", + mkSettingSerializer @"allow_experimental_kusto_dialect", + mkSettingSerializer @"allow_experimental_prql_dialect", + mkSettingSerializer @"enable_adaptive_memory_spill_scheduler", + mkSettingSerializer @"allow_experimental_delta_kernel_rs", + mkSettingSerializer @"allow_experimental_insert_into_iceberg", + mkSettingSerializer @"allow_experimental_iceberg_compaction", + mkSettingSerializer @"write_full_path_in_iceberg_metadata", + mkSettingSerializer @"iceberg_metadata_compression_method", + mkSettingSerializer @"make_distributed_plan", + mkSettingSerializer @"distributed_plan_execute_locally", + mkSettingSerializer @"distributed_plan_default_shuffle_join_bucket_count", + mkSettingSerializer @"distributed_plan_default_reader_bucket_count", + mkSettingSerializer @"distributed_plan_optimize_exchanges", + mkSettingSerializer @"distributed_plan_force_exchange_kind", + mkSettingSerializer @"distributed_plan_max_rows_to_broadcast", + mkSettingSerializer @"allow_experimental_ytsaurus_table_engine", + mkSettingSerializer @"allow_experimental_ytsaurus_table_function", + mkSettingSerializer @"allow_experimental_ytsaurus_dictionary_source", + mkSettingSerializer @"distributed_plan_force_shuffle_aggregation", + mkSettingSerializer @"enable_join_runtime_filters", + mkSettingSerializer @"join_runtime_filter_exact_values_limit", + mkSettingSerializer @"join_runtime_bloom_filter_bytes", + mkSettingSerializer @"join_runtime_bloom_filter_hash_functions", + mkSettingSerializer @"rewrite_in_to_join", + mkSettingSerializer @"promql_database", + mkSettingSerializer @"promql_table" + ] diff --git a/ClickHaskell/Statements.hs b/ClickHaskell/Statements.hs new file mode 100644 --- /dev/null +++ b/ClickHaskell/Statements.hs @@ -0,0 +1,174 @@ +module ClickHaskell.Statements where + +-- Internal +import ClickHaskell.Primitive +import ClickHaskell.Protocol.Settings (DbSettings (..)) + +-- GHC included +import Data.ByteString.Builder (Builder, byteString) +import Data.ByteString.Char8 as BS8 (pack) +import Data.Kind (Type) +import Data.List (intersperse) +import Data.Proxy (Proxy (..)) +import Data.String (IsString (..)) +import GHC.TypeLits + + +-- * Statements + +tableName :: forall name . KnownSymbol name => Builder +tableName = (byteString . BS8.pack) (symbolVal $ Proxy @name) + +class Statement statement where + {- | + Wrapper for settings passing + -} + passSettings :: (DbSettings -> DbSettings) -> statement -> statement + +instance Statement (Select cols output) where + passSettings pass (MkSelect mkQuery dbSettings) = MkSelect mkQuery (pass dbSettings) + +instance Statement (Insert cols input) where + passSettings pass (MkInsert mkQuery dbSettings) = MkInsert mkQuery (pass dbSettings) + +instance Statement (Command) where + passSettings pass (MkCommand query dbSettings) = MkCommand query (pass dbSettings) + + +-- ** Command + +data Command + where + MkCommand :: ChString -> DbSettings -> Command + +instance IsString Command where + fromString str = MkCommand (toChType str) (MkDbSettings []) + + +-- ** SELECT + +{-| + SELECT statement abstraction +-} +data Select (columns :: [Type]) output + where + MkSelect :: ([(Builder, Builder)] -> ChString) -> DbSettings -> Select columns output + +unsafeMkSelect :: ([(Builder, Builder)] -> Builder) -> Select columns output +unsafeMkSelect s = MkSelect (toChType . s) (MkDbSettings []) + +{-| + Type-safe wrapper for statements like + + @SELECT ${columns} FROM ${table}@ +-} +fromTable :: + forall name columns output + . + KnownSymbol name + => + Select columns output +fromTable = unsafeMkSelect $ \cols -> + "SELECT " <> selectedColumns cols <> + " FROM " <> tableName @name + where + selectedColumns = + mconcat . intersperse ", " . map (\(name, _) -> name) + +fromView :: + forall name columns output params + . + KnownSymbol name + => + (Parameters '[] -> Parameters params) -> Select columns output +fromView interpreter = unsafeMkSelect $ \cols -> + "SELECT " <> selectedColumns cols <> + " FROM " <> tableName @name <> viewParameters interpreter + where + selectedColumns = + mconcat . intersperse ", " . map (\(name, _) -> name) + +fromGenerateRandom :: + forall columns output + . + (UInt64, UInt64, UInt64) -> UInt64 -> Select columns output +fromGenerateRandom (randomSeed, maxStrLen, maxArrayLen) limit = query + where + query = unsafeMkSelect $ \cols -> + "SELECT * FROM generateRandom(" <> + "'" <> columnsAndTypes cols <> "'" <> "," <> + toQueryPart randomSeed <> "," <> + toQueryPart maxStrLen <> "," <> + toQueryPart maxArrayLen <> + ")" <> + " LIMIT " <> toQueryPart limit <> ";" + + columnsAndTypes = + mconcat . intersperse ", " . map (\(name, tyype) -> name <> " " <> tyype) + + +-- ** INSERT + +{-| + INSERT statement generation abstraction +-} +data Insert (columns :: [Type]) output + where + MkInsert :: ([(Builder, Builder)] -> ChString) -> DbSettings -> Insert columns output + +unsafeMkInsert :: ([(Builder, Builder)] -> Builder) -> Insert columns output +unsafeMkInsert s = MkInsert (toChType . s) (MkDbSettings []) + +intoTable :: + forall name columns output + . + KnownSymbol name + => + Insert columns output +intoTable = unsafeMkInsert mkQuery + where + mkQuery cols = + "INSERT INTO " <> tableName @name <> + " (" <> mkInsertColumns cols <> ") VALUES" + mkInsertColumns cols = + (mconcat . intersperse ", " . map (\(name, _) -> name)) cols + + + + +-- * Parameters + +type family KnownParameter param + where + KnownParameter (Parameter name parType) = (KnownSymbol name, IsChType parType, ToQueryPart parType) + +data Parameter (name :: Symbol) (chType :: Type) = MkParamater chType + +data Parameters parameters where + NoParameters :: Parameters '[] + AddParameter + :: KnownParameter (Parameter name chType) + => Parameter name chType + -> Parameters parameters + -> Parameters (Parameter name chType ': parameters) + +{- | +>>> viewParameters (parameter @"a3" ("a3Val" :: ChString) . parameter @"a2" ("a2Val" :: ChString)) +"(a3='a3Val', a2='a2Val')" +-} +viewParameters :: (Parameters '[] -> Parameters passedParameters) -> Builder +viewParameters interpreter = "(" <> renderParameters (interpreter NoParameters) <> ")" + +renderParameters :: Parameters params -> Builder +renderParameters NoParameters = "" +renderParameters (AddParameter param NoParameters) = renderParameter param +renderParameters (AddParameter param moreParams) = renderParameter param <> ", " <> renderParameters moreParams + + +parameter + :: KnownParameter (Parameter name t) + => t -> Parameters params -> Parameters (Parameter name t ': params) +parameter val = AddParameter (MkParamater val) + +renderParameter :: forall name chType . KnownParameter (Parameter name chType) => Parameter name chType -> Builder +renderParameter (MkParamater chType) = (byteString . BS8.pack . symbolVal @name) Proxy <> "=" <> toQueryPart chType diff --git a/LICENSE b/LICENSE new file mode 100644 --- /dev/null +++ b/LICENSE @@ -0,0 +1,28 @@ +BSD 3-Clause License + +Copyright (c) 2025, Dmitry Kovalev + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/README.md b/README.md new file mode 100644 --- /dev/null +++ b/README.md @@ -0,0 +1,5 @@ +# ClickHaskell + +Haskell implementation of [ClickHouse](https://clickhouse.com/) DBMS Native protocol and client + +Haddock documentation are incomplete. Visit [Home page](https://clickhaskell.dev/) to learn more diff --git a/Setup.hs b/Setup.hs new file mode 100644 --- /dev/null +++ b/Setup.hs @@ -0,0 +1,2 @@ +import Distribution.Simple +main = defaultMain