splice 0.2 → 0.3

raw patch · 5 files changed

+276/−274 lines, 5 filesPVP ok

version bump matches the API change (PVP)

API changes (from Hackage documentation)

Files

• splice.cabal +22/−22
• src/Network/Socket/Splice.hs +29/−0
• src/Network/Socket/Splice.hsc +0/−252
• src/Network/Socket/Splice/Internal.hsc +149/−0
• src/System/IO/Splice/Linux.hsc +76/−0

@@ -1,27 +1,25 @@ name:          splice-version:       0.2+version:       0.3 stability:     stable on Linux, experimental on other operating systems-synopsis:      Socket to Socket Data Splicing-description:   A library that implements efficient socket to socket-               data transfer loops for proxy servers.-               .-               On Linux, it uses the zero-copy splice() system call:-               <http://kerneltrap.org/node/6505>.+synopsis:      Socket to Socket Data Splicing (supports all operating systems)+description:   A library that implements most efficient socket to socket+               data transfer loops for proxy servers on each operating system.                .-               On all other operating systems, it currently falls back-               to a portable Haskell implementation that allocates a-               constant-sized memory buffer before it enters an inner-               loop which then uses hGetBufSome and hPutBuf; this avoids-               lots of tiny allocations as would otherwise be caused by-               recv and sendAll functions from Network.Socket.ByteString.+               On Linux, it uses and exposes the zero-copy @splice()@ system+               call: <http://kerneltrap.org/node/6505>.                .-               This work is funded by Corsis Research and used in:+               On other operating systems, it currently falls back to a portable+               Haskell implementation.                .-               PortFusion ]-[ayabusa – German-               research project for building the simplest-               high-performance distributed reverse / forward proxy.+               [Corsis Research]+               This work is funded by Corsis Research+               (<http://corsis.eu>) for the development of                .-               <https://sourceforge.net/p/portfusion/wiki/RoadMap/>+               [PortFusion \]-\[ayabusa (はやぶさ) (Hayabusa)]+               – German-Japanese joint research project for building the+               simplest and most concise high-performance distributed reverse /+               forward proxy possible+               (<https://sourceforge.net/p/portfusion/wiki/RoadMap/>). license:       BSD3 license-file:  LICENSE author:        Cetin Sert <fusion@corsis.eu>@@ -31,12 +29,12 @@ build-type:    Simple cabal-version: >=1.2 copyright:     Copyright © 2012 Cetin Sert-tested-with:   GHC >= 7.4.3+tested-with:   GHC >= 7.0   flag portable     description: explicitly enable portable splice implemented in Haskell-    default    : False+    default    : True   library@@ -44,15 +42,17 @@     hs-source-dirs:  src      exposed-modules: Network.Socket.Splice+    other-modules:   Network.Socket.Splice.Internal      if os(linux) && !flag(portable)+      exposed-modules: System.IO.Splice.Linux       build-depends: base    >= 4 && <= 6,                      network >= 2 && <= 4,                      unix    >= 2 && <= 4       cpp-options: -DLINUX_SPLICE-      ghc-options: -O2 -O3 -fllvm -optlo-O3+      ghc-options: -W -O2 -O3 -fllvm -optlo-O3     else       build-depends: base    >= 4 && <= 6,                      network >= 2 && <= 4       cpp-options: -ULINUX_SPLICE-      ghc-options: -O2 -O3+      ghc-options: -W -O2 -O3

@@ -0,0 +1,29 @@+{- |+  This library implements most efficient socket to socket data transfer loops+  for proxy servers on each operating system.+               +  On Linux, it uses and exposes the zero-copy @splice()@ system call:+  <http://kerneltrap.org/node/6505>.++  On other operating systems, it currently falls back to a portable Haskell+  implementation – which first allocates a constant-sized memory buffer in user+  address, then enters an inner loop which uses 'System.IO.hGetBufSome'+  and 'System.IO.hPutBuf' on this user-space buffer. This avoids tony of tiny+  allocations as might otherwise be caused by 'Network.Socket.ByteString.recv'+  and 'Network.Socket.ByteString.sendAll' from the @bytestring@ package.+-}+--  +-- Module      : Network.Socket.Splice+-- Copyright   : (c) Cetin Sert 2012+-- License     : BSD3+-- Maintainer  : fusion@corsis.eu+-- Stability   : stable+-- Portability : GHC-only, works on all OSes++module Network.Socket.Splice (++  module Network.Socket.Splice.Internal++  ) where++import Network.Socket.Splice.Internal

@@ -1,252 +0,0 @@-{- |-  This library implements efficient socket to socket-  data transfer loops for proxy servers.-               -   On Linux, it uses the zero-copy splice() system call:-   <http://kerneltrap.org/node/6505>.--   On all other operating systems, it currently falls back-   to a portable Haskell implementation that allocates a-   constant-sized memory buffer before it enters an inner-   loop which then uses hGetBufSome and hPutBuf; this avoids-   lots of tiny allocations as would otherwise be caused by-   recv and sendAll functions from Network.Socket.ByteString.--}---  --- Module      : Network.Socket.Splice--- Copyright   : (c) Cetin Sert 2012--- License     : BSD-style--- Maintainer  : fusion@corsis.eu--- Stability   : stable--- Portability : GHC-only, works on all OSes--#ifdef LINUX_SPLICE-#include <fcntl.h>-{-# LANGUAGE CPP #-}-{-# LANGUAGE ForeignFunctionInterface #-}-#endif--module Network.Socket.Splice (--  -- * Cross-platform API for Socket to Socket Data Transfer Loops-  {- | 'splice' is the cross-platform API for continous, uni-directional-       data transfer between two network sockets.--       It is an /infinite loop/ that is intended to be used with-       'Control.Concurrent.forkIO':--       > void . forkIO . try_ $ splice 1024 sourceSocket targetSocket-       > void . forkIO . try_ $ splice 1024 targetSocket sourceSocket-  -}--    splice-  , ChunkSize-  , zeroCopy--  -- * Combinators for Exception Handling-  , try_---  -- * Linux splice() Components-  {- | These are available only on Linux and-       will be moved to a different namespace-       in later releases. Their names will stay-       the same.-  -}--#ifdef LINUX_SPLICE-  , c_splice-  , sPLICE_F_MOVE-  , sPLICE_F_MORE-  , sPLICE_F_NONBLOCK-#endif--  ) where--import Data.Word-import Foreign.Ptr--import Network.Socket-import Control.Monad-import Control.Exception-import System.IO-import System.Posix.Types-import System.Posix.Internals-import GHC.IO.Handle.FD--#ifdef LINUX_SPLICE-import Data.Int-import Data.Bits-import Unsafe.Coerce-import Foreign.C.Types-import Foreign.C.Error-import System.Posix.IO-#else-import Foreign.Marshal.Alloc-#endif----- | Indicates whether 'splice' uses zero-copy system calls---   or the portable user mode Haskell substitue implementation.-zeroCopy :: Bool -- ^ True: uses zero-copy system calls; otherwise: portable.-zeroCopy =-#ifdef LINUX_SPLICE-  True-#else-  False-#endif--------------------------------------------------------------------------------------- | The numeric type used to recommend chunk sizes for moving---   data between sockets used by both the Linux 'splice' and---   the portable implementation of 'splice'.-type ChunkSize =-#ifdef LINUX_SPLICE-  (#type size_t)-#else-  Int-#endif----- | Pipes data from one socket to another in an---   **infinite loop**.------   On Linux this happens in kernel space with---   zero copying between kernel and user spaces.------   On other operating systems, a portable---   implementation utilizes a user space buffer---   and works on handles instead of file descriptors.-splice-  :: ChunkSize -- ^ Chunk size.-  -> Socket    -- ^ Source socket.-  -> Socket    -- ^ Target socket.-  -> IO ()     -- ^ Infinite loop.-splice len sIn sOut = do--  let throwRecv0 = error "Network.Socket.Splice.splice ended"--  let fdIn  = fdSocket sIn-  let fdOut = fdSocket sOut --#ifdef LINUX_SPLICE--  print "LINUX-SPLICE"--  (r,w) <- createPipe     -- r: read  end of pipe-  print ('+',r,w)         -- w: write end of pipe-  let s = Fd fdIn         -- s: source socket-  let t = Fd fdOut        -- t: target socket-  let n = nullPtr  -  let u = unsafeCoerce :: (#type ssize_t) -> (#type size_t)-  let check = throwErrnoIfMinus1 "Network.Socket.Splice.splice"-  let flags = sPLICE_F_MOVE .|. sPLICE_F_MORE-  let setNonBlockingMode v = do setNonBlockingFD fdIn  v-                                setNonBlockingFD fdOut v--  setNonBlockingMode False-  finally-    (forever $ do -       bytes <- check $ c_splice s n w n    len    flags-       if bytes > 0-         then           c_splice r n t n (u bytes) flags-         else           throwRecv0)-    (do closeFd r-        closeFd w-        try_ $ setNonBlockingMode True-        print ('-',r,w))--#else--  s <- fdToHandle fdIn-  t <- fdToHandle fdOut-  hSetBuffering s NoBuffering-  hSetBuffering t NoBuffering-  a <- mallocBytes len :: IO (Ptr Word8)--  print "PORTABLE-SPLICE"--  finally-    (forever $ do-       bytes <-   hGetBufSome s a len-       if bytes > 0-         then     hPutBuf     t a bytes-         else     throwRecv0)-    (do free a-        try_ $ hClose s-        try_ $ hClose t)--#endif----- | Similar to 'Control.Exception.Base.try' but used when an---   obvious exception is expected whose type can be safely---   ignored.-try_-  :: IO () -- ^ The action to run which can throw any exception.-  -> IO () -- ^ The new action where exceptions are silenced.-try_ a = (try a :: IO (Either SomeException ())) >> return ()--------------------------------------------------------------------------------------#ifdef LINUX_SPLICE--- SPLICE---- fcntl.h--- ssize_t splice(---   int          fd_in,---   loff_t*      off_in,---   int          fd_out,---   loff_t*      off_out,---   size_t       len,---   unsigned int flags--- );---- | Moves data between two file descriptors without---   copying between kernel address space and user ---   address space. It transfers up to 'len' bytes of---   data from the file descriptor 'fd_in' to the file---   file descriptor 'fd_out', where one of the---   descriptors must refer to a pipe.------   'c_splice' is NOT a loop and needs to called repeatedly.---   For an example, see the source code of 'splice'.-foreign import ccall "splice"-  c_splice-  :: Fd                  -- ^ fd_in-  -> Ptr (#type loff_t)  -- ^ off_in-  -> Fd                  -- ^ fd_out-  -> Ptr (#type loff_t)  -- ^ off_out-  -> (#type size_t)      -- ^ len-  -> Word                -- ^ flags-  -> IO (#type ssize_t)  -- ^ number of bytes moved or -1 on error----- | Attempt to move pages instead of copying. This is---   only a hint to the kernel: pages may stil be copied---   if the kernel cannot move the pages from the pipe,---   or if the pipe buffers don't refer to full pages.-sPLICE_F_MOVE :: Word-sPLICE_F_MOVE = (#const "SPLICE_F_MOVE")----- | More data will be coming in a subsequent splice.---   This is a helpful hint when 'fd_out' refers to a---   socket.-sPLICE_F_MORE :: Word-sPLICE_F_MORE = (#const "SPLICE_F_MORE")----- | Do not block on I/O. This makes the splice pipe---   operations nonblocking, but splice() may nevertheless---   block because the file descriptors that are spliced---   to/from may block (unless they have the O_NONBLOCK flag---   set).-sPLICE_F_NONBLOCK :: Word-sPLICE_F_NONBLOCK = (#const "SPLICE_F_NONBLOCK")--#endif

@@ -0,0 +1,149 @@+-- | Implementation.++#ifdef LINUX_SPLICE+#include <fcntl.h>+#endif+{-# LANGUAGE CPP #-}+{-# LANGUAGE ForeignFunctionInterface #-}+++module Network.Socket.Splice.Internal (++  -- * Cross-platform API for Socket to Socket Data Transfer Loops+  {- | 'splice' is the cross-platform API for continous, uni-directional+       data transfer between two network sockets.++       It is an /infinite loop/ that is intended to be used with+       'Control.Concurrent.forkIO':++       > void . forkIO . try_ $ splice 1024 sourceSocket targetSocket+       > void . forkIO . try_ $ splice 1024 targetSocket sourceSocket+  -}++    splice+  , ChunkSize+  , zeroCopy++  -- * Combinators for Exception Handling+  , try_++  ) where+++import Data.Word+import Foreign.Ptr++import Network.Socket+import Control.Monad+import Control.Exception++#ifdef LINUX_SPLICE+import Data.Int+import Data.Bits+import System.Posix.IO+import Unsafe.Coerce+import Foreign.C.Error+import System.Posix.Types+import System.Posix.Internals+import qualified System.IO.Splice.Linux as L+#else+import System.IO+import GHC.IO.Handle.FD+import Foreign.Marshal.Alloc+#endif+++--------------------------------------------------------------------------------+++-- | Indicates whether 'splice' uses zero-copy system calls or the portable user +--   space Haskell implementation.+zeroCopy :: Bool -- ^ @True@ if 'splice' uses zero-copy system calls;+                 --   otherwise, false.+zeroCopy =+#ifdef LINUX_SPLICE+  True+#else+  False+#endif+++-- | The numeric type to recommend chunk sizes for moving data between sockets+--   used by both zero-copy and portable implementations of 'splice'.+type ChunkSize =+#ifdef LINUX_SPLICE+  L.ChunkSize+#else+  Int+#endif+++-- | Pipes data from one socket to another in an /infinite loop/.+--+--   On Linux this uses the @splice()@ system call and eliminates copying+--   between kernel and user address spaces.+--+--   On other operating systems, a portable Haskell implementation utilizes a+--   user space buffer.+splice+  :: ChunkSize -- ^ chunk size.+  -> Socket    -- ^ source socket.+  -> Socket    -- ^ target socket.+  -> IO ()     -- ^ infinite loop.+splice len sIn sOut = do++  let throwRecv0 = error "Network.Socket.Splice.splice ended"++  let fdIn  = fdSocket sIn+  let fdOut = fdSocket sOut ++#ifdef LINUX_SPLICE++  (r,w) <- createPipe     -- r,w: read / write ends of pipe+  let s = Fd fdIn         -- s  : source socket+  let t = Fd fdOut        -- t  : target socket+  let n = nullPtr  +  let u = unsafeCoerce :: (#type ssize_t) -> (#type size_t)+  let check = throwErrnoIfMinus1 "Network.Socket.Splice.splice"+  let flags = L.sPLICE_F_MOVE .|. L.sPLICE_F_MORE+  let setNonBlockingMode v = do setNonBlockingFD fdIn  v+                                setNonBlockingFD fdOut v++  setNonBlockingMode False+  finally+    (forever $ do +       bytes <- check $ L.c_splice s n w n    len    flags+       if bytes > 0+         then           L.c_splice r n t n (u bytes) flags+         else           throwRecv0)+    (do closeFd r+        closeFd w+        try_ $ setNonBlockingMode True)++#else++  s <- fdToHandle fdIn+  t <- fdToHandle fdOut+  hSetBuffering s NoBuffering+  hSetBuffering t NoBuffering+  a <- mallocBytes len :: IO (Ptr Word8)++  finally+    (forever $ do+       bytes <-   hGetBufSome s a len+       if bytes > 0+         then     hPutBuf     t a bytes+         else     throwRecv0)+    (do free a+        try_ $ hClose s+        try_ $ hClose t)++#endif+++-- | Similar to 'Control.Exception.Base.try' but used when an obvious exception+--   is expected which can be safely ignored.+try_+  :: IO () -- ^ action to run which can throw /any/ exception.+  -> IO () -- ^ new action where exceptions are silenced.+try_ a = (try a :: IO (Either SomeException ())) >> return ()

@@ -0,0 +1,76 @@+{- | Exposes the GNU\/Linux system call @splice()@.++     /This module is only available (compiled & exposed) on Linux./+-}+--  +-- Module      : Network.Socket.Splice+-- Copyright   : (c) Cetin Sert 2012+-- License     : BSD3+-- Maintainer  : fusion@corsis.eu+-- Stability   : stable+-- Portability : GNU\/Linux-only++#include <fcntl.h>+{-# LANGUAGE CPP #-}+{-# LANGUAGE ForeignFunctionInterface #-}++module System.IO.Splice.Linux (++    c_splice+  , ChunkSize+  , sPLICE_F_MOVE+  , sPLICE_F_MORE+  , sPLICE_F_NONBLOCK++  ) where+++import Data.Int+import Data.Word+import Foreign.Ptr+import Foreign.C.Types+import System.Posix.Types+++-- | The numeric type used by 'c_splice' for chunk size recommendations when+--   moving data.+type ChunkSize = (#type size_t)++-- | Moves data between two file descriptors without copying between kernel+--   address space and user address space. It transfers up to 'len' bytes of+--   data from the file descriptor 'fd_in' to the file descriptor 'fd_out',+--   where one of the descriptors must refer to a pipe.+--+--   'c_splice' is /NOT/ a loop and needs to be called repeatedly.+--+--   For an example, see 'Network.Socket.Splice.Internal.splice'.+foreign import ccall "splice"+  c_splice+  :: Fd                  -- ^ @fd_in@.+  -> Ptr (#type loff_t)  -- ^ @off_in@.+  -> Fd                  -- ^ @fd_out@.+  -> Ptr (#type loff_t)  -- ^ @off_out@.+  -> ChunkSize           -- ^ @len@.+  -> Word                -- ^ @flags@.+  -> IO (#type ssize_t)  -- ^ number of bytes moved if successful; otherwise -1.+++-- | Attempt to move pages instead of copying. This is only a hint to the+--   kernel: pages may stil be copied (/in kernel address space/) if the kernel+--   cannot move the pages from the pipe, or if the pipe buffers don't refer to+--   full pages.+sPLICE_F_MOVE :: Word+sPLICE_F_MOVE = (#const "SPLICE_F_MOVE")+++-- | More data will be coming in a subsequent splice. This is a helpful hint+--   when 'fd_out' refers to a socket.+sPLICE_F_MORE :: Word+sPLICE_F_MORE = (#const "SPLICE_F_MORE")+++-- | Do not block on I\/O. This makes the splice pipe operations nonblocking,+--   but splice() may nevertheless block because the file descriptors that are+--   spliced to\/from may block (unless they have the O_NONBLOCK flag set).+sPLICE_F_NONBLOCK :: Word+sPLICE_F_NONBLOCK = (#const "SPLICE_F_NONBLOCK")