diff --git a/LICENSE b/LICENSE new file mode 100644 --- /dev/null +++ b/LICENSE @@ -0,0 +1,165 @@ + GNU LESSER GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + + This version of the GNU Lesser General Public License incorporates +the terms and conditions of version 3 of the GNU General Public +License, supplemented by the additional permissions listed below. + + 0. Additional Definitions. + + As used herein, "this License" refers to version 3 of the GNU Lesser +General Public License, and the "GNU GPL" refers to version 3 of the GNU +General Public License. + + "The Library" refers to a covered work governed by this License, +other than an Application or a Combined Work as defined below. + + An "Application" is any work that makes use of an interface provided +by the Library, but which is not otherwise based on the Library. +Defining a subclass of a class defined by the Library is deemed a mode +of using an interface provided by the Library. + + A "Combined Work" is a work produced by combining or linking an +Application with the Library. The particular version of the Library +with which the Combined Work was made is also called the "Linked +Version". + + The "Minimal Corresponding Source" for a Combined Work means the +Corresponding Source for the Combined Work, excluding any source code +for portions of the Combined Work that, considered in isolation, are +based on the Application, and not on the Linked Version. + + The "Corresponding Application Code" for a Combined Work means the +object code and/or source code for the Application, including any data +and utility programs needed for reproducing the Combined Work from the +Application, but excluding the System Libraries of the Combined Work. + + 1. Exception to Section 3 of the GNU GPL. + + You may convey a covered work under sections 3 and 4 of this License +without being bound by section 3 of the GNU GPL. + + 2. Conveying Modified Versions. + + If you modify a copy of the Library, and, in your modifications, a +facility refers to a function or data to be supplied by an Application +that uses the facility (other than as an argument passed when the +facility is invoked), then you may convey a copy of the modified +version: + + a) under this License, provided that you make a good faith effort to + ensure that, in the event an Application does not supply the + function or data, the facility still operates, and performs + whatever part of its purpose remains meaningful, or + + b) under the GNU GPL, with none of the additional permissions of + this License applicable to that copy. + + 3. Object Code Incorporating Material from Library Header Files. + + The object code form of an Application may incorporate material from +a header file that is part of the Library. You may convey such object +code under terms of your choice, provided that, if the incorporated +material is not limited to numerical parameters, data structure +layouts and accessors, or small macros, inline functions and templates +(ten or fewer lines in length), you do both of the following: + + a) Give prominent notice with each copy of the object code that the + Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the object code with a copy of the GNU GPL and this license + document. + + 4. Combined Works. + + You may convey a Combined Work under terms of your choice that, +taken together, effectively do not restrict modification of the +portions of the Library contained in the Combined Work and reverse +engineering for debugging such modifications, if you also do each of +the following: + + a) Give prominent notice with each copy of the Combined Work that + the Library is used in it and that the Library and its use are + covered by this License. + + b) Accompany the Combined Work with a copy of the GNU GPL and this license + document. + + c) For a Combined Work that displays copyright notices during + execution, include the copyright notice for the Library among + these notices, as well as a reference directing the user to the + copies of the GNU GPL and this license document. + + d) Do one of the following: + + 0) Convey the Minimal Corresponding Source under the terms of this + License, and the Corresponding Application Code in a form + suitable for, and under terms that permit, the user to + recombine or relink the Application with a modified version of + the Linked Version to produce a modified Combined Work, in the + manner specified by section 6 of the GNU GPL for conveying + Corresponding Source. + + 1) Use a suitable shared library mechanism for linking with the + Library. A suitable mechanism is one that (a) uses at run time + a copy of the Library already present on the user's computer + system, and (b) will operate properly with a modified version + of the Library that is interface-compatible with the Linked + Version. + + e) Provide Installation Information, but only if you would otherwise + be required to provide such information under section 6 of the + GNU GPL, and only to the extent that such information is + necessary to install and execute a modified version of the + Combined Work produced by recombining or relinking the + Application with a modified version of the Linked Version. (If + you use option 4d0, the Installation Information must accompany + the Minimal Corresponding Source and Corresponding Application + Code. If you use option 4d1, you must provide the Installation + Information in the manner specified by section 6 of the GNU GPL + for conveying Corresponding Source.) + + 5. Combined Libraries. + + You may place library facilities that are a work based on the +Library side by side in a single library together with other library +facilities that are not Applications and are not covered by this +License, and convey such a combined library under terms of your +choice, if you do both of the following: + + a) Accompany the combined library with a copy of the same work based + on the Library, uncombined with any other library facilities, + conveyed under the terms of this License. + + b) Give prominent notice with the combined library that part of it + is a work based on the Library, and explaining where to find the + accompanying uncombined form of the same work. + + 6. Revised Versions of the GNU Lesser General Public License. + + The Free Software Foundation may publish revised and/or new versions +of the GNU Lesser General Public License from time to time. Such new +versions will be similar in spirit to the present version, but may +differ in detail to address new problems or concerns. + + Each version is given a distinguishing version number. If the +Library as you received it specifies that a certain numbered version +of the GNU Lesser General Public License "or any later version" +applies to it, you have the option of following the terms and +conditions either of that published version or of any later version +published by the Free Software Foundation. If the Library as you +received it does not specify a version number of the GNU Lesser +General Public License, you may choose any version of the GNU Lesser +General Public License ever published by the Free Software Foundation. + + If the Library as you received it specifies that a proxy can decide +whether future versions of the GNU Lesser General Public License shall +apply, that proxy's public statement of acceptance of any version is +permanent authorization for you to choose that version for the +Library. diff --git a/Setup.hs b/Setup.hs new file mode 100644 --- /dev/null +++ b/Setup.hs @@ -0,0 +1,2 @@ +import Distribution.Simple +main = defaultMain diff --git a/chunky.cabal b/chunky.cabal new file mode 100644 --- /dev/null +++ b/chunky.cabal @@ -0,0 +1,54 @@ +-- Initial chunky.cabal generated by cabal init. For further +-- documentation, see http://haskell.org/cabal/users-guide/ + +name: chunky +version: 0.1.0.0 +category: Codec +synopsis: Human-readable storage of text/binary objects. +description: Allows serializing data into binary files, preserving human readability + as far as possible. It allows mixing binary and textual data in the same file, and + is a light-weight alternative to parsec & co. The generated files are not intended + to be human-editable, only readable. + If the stored binary data is utf8-encoded text, the whole file can be read in any + text-editor. +license: LGPL-3 +license-file: LICENSE +author: Philipp Hausmann +maintainer: hackage@314.ch +-- copyright: +build-type: Simple +cabal-version: >=1.8 + +source-repository head + type: git + location: git://github.com/phile314/chunky.git + +library + exposed-modules: Codec.Chunky + , Codec.Chunky.Internal + -- other-modules: + build-depends: base ==4.6.* + , binary >= 0.5.1.1 && < 0.8 + , bytestring >= 0.10.0.0 && < 0.11 + , text >= 0.11.3.1 && < 1.3 + hs-source-dirs: src/lib + +executable chunky-test + hs-source-dirs: src/chunky-test + main-is: Main.hs + build-depends: base ==4.6.* + , binary >= 0.5.1.1 && < 0.8 + , bytestring >= 0.10.0.0 && < 0.11 + , chunky >= 0.1.0.0 + , text >= 0.11.3.1 && < 1.3 + +test-suite test-roundtrip + type: exitcode-stdio-1.0 + main-is: RoundtripTest.hs + hs-source-dirs: test/ + build-depends: base ==4.6.* + , binary >= 0.5.1.1 && < 0.8 + , bytestring >= 0.10.0.0 && < 0.11 + , chunky >= 0.1.0.0 + , HUnit >= 1.2.5.2 && < 1.3 + , text >= 0.11.3.1 && < 1.3 diff --git a/src/chunky-test/Main.hs b/src/chunky-test/Main.hs new file mode 100644 --- /dev/null +++ b/src/chunky-test/Main.hs @@ -0,0 +1,26 @@ +{-# LANGUAGE OverloadedStrings, DoAndIfThenElse #-} + +module Main where + +import qualified Codec.Chunky as C +import qualified Data.Text as T +import Data.Text.Encoding +import qualified Data.ByteString.Lazy as B +import Control.Applicative +import Data.Binary.Get +import Data.Binary.Put + +chunks :: [C.Chunk] +chunks = [ C.Textual "Test Textual" "Blabsdfsdf\n\nafsdfsadfs" + , C.Binary "Test Binary" (B.fromStrict $ encodeUtf8 "bladsfdsg\n\ndfg") + , C.Binary "Test Binary2" (B.pack [0xFF, 0xFF, 0xFF, 0xEF, 0xBB, 0xBF, 0x00]) + ] + +main :: IO () +main = do + B.writeFile "test.chnk" $ runPut $ C.putChunks chunks + rr <- runGet (C.readChunks (length chunks)) <$> B.readFile "test.chnk" + if rr == chunks then + putStrLn "Got back correct result!" + else + putStrLn "Wrong result !?" diff --git a/src/lib/Codec/Chunky.hs b/src/lib/Codec/Chunky.hs new file mode 100644 --- /dev/null +++ b/src/lib/Codec/Chunky.hs @@ -0,0 +1,54 @@ +-- | Example output of this library looks like this, opened in a simple text editor (gedit): +-- +-- @ +-- ==================== +-- Title: Test Textual +-- Type: TXT Length: 00000000000000000022 +-- ==================== +-- Blabsdfsdf +-- +-- afsdfsadfs +-- ==================== +-- ==================== +-- Title: Test Binary +-- Type: BIN Length: 00000000000000000014 +-- ==================== +-- bladsfdsg +-- +-- dfg +-- ==================== +-- ==================== +-- Title: Test Binary2 +-- Type: BIN Length: 00000000000000000007 +-- ==================== +-- \FF\FF\FF\00 +-- ==================== +-- @ +-- +-- You should make sure that the file is written in binary mode, e.g. use +-- the readFile and writeFile methods from Data.ByteString.Lazy. +-- +module Codec.Chunky + ( -- * Support for writing/reading chunks + Chunk (..) + , readChunks + , putChunks + + -- * Some useful helper functions when parsing/writing simple binary files. + , expectStr + , expectStrLn + , putStr + , putStrLn + , readStr + , readStrLn + , putStrPadded + , putStrLnPadded + , readStrPadded + , readStrLnPadded + ) +where + +import Prelude () + +import Codec.Chunky.Internal + diff --git a/src/lib/Codec/Chunky/Internal.hs b/src/lib/Codec/Chunky/Internal.hs new file mode 100644 --- /dev/null +++ b/src/lib/Codec/Chunky/Internal.hs @@ -0,0 +1,155 @@ +{-# LANGUAGE OverloadedStrings, DoAndIfThenElse #-} + +module Codec.Chunky.Internal + ( Chunk (..) + , readChunks + , putChunks + + , expectStr + , expectStrLn + , putStr + , putStrLn + , readStr + , readStrLn + , putStrPadded + , putStrLnPadded + , readStrPadded + , readStrLnPadded + ) +where + +import Prelude hiding (putStr, putStrLn) + +import Data.Binary.Get +import Data.Binary.Put +import qualified Data.ByteString.Lazy as B +import Data.Text as T +import Data.Text.Encoding +import Control.Applicative +import Data.Int +import Control.Monad + + +-- Title can be up to 30 characters. +data Chunk + = Binary + { title :: T.Text + , bContent :: B.ByteString + } + | Textual + { title :: T.Text + , tContent :: T.Text + } + deriving (Show, Read, Eq, Ord) + + +readChunks :: Int -> Get [Chunk] +readChunks n = replicateM n parseChunk + +parseChunk :: Get Chunk +parseChunk = do + expectStrLn sep + expectStr "Title: " + title <- readStrLnPadded 30 + + expectStr "Type: " + ty' <- readStr 3 + + expectStr " Length: " + len <- fromInteger . read . T.unpack <$> readStrLn 20 + + expectStrLn sep + + bd <- getLazyByteString len + + expectStr "\n" + expectStrLn sep + + return $ case ty' of + "TXT" -> Textual title (decodeUtf8 $ B.toStrict bd) + "BIN" -> Binary title bd + +putChunks :: [Chunk] -> Put +putChunks cs = mapM printChunk cs >> return () + +printChunk :: Chunk -> Put +printChunk chnk = do + putStrLn sep + putStr $ "Title: " + putStrLnPadded 30 (title chnk) + putStr "Type: " + body <- case chnk of + (Textual _ bd) -> do + putStr "TXT " + return $ B.fromStrict $ encodeUtf8 bd + (Binary _ bd) -> do + putStr "BIN " + return bd + let len = B.length body + putStrLn $ "Length: " `T.append` (padLeft "0" 20 $ T.pack $ show len) + putStrLn sep + putLazyByteString body + putStr "\n" + putStrLn sep + +-- TODO actually call this... +validate :: Chunk -> () +validate chnk = + -- title must not start/end with whitespace + case True of + _ | T.strip (title chnk) /= title chnk -> + error "Title must not start/end with whitespaces." + _ | T.length (title chnk) > 20 -> + error "Title must not be longer than 20 characters." + _ -> () + +expectStrLn :: T.Text -> Get T.Text +expectStrLn t = T.init <$> expectStr (t `T.append` "\n") + +expectStr :: T.Text -> Get T.Text +expectStr t = do + let tAsBytes = B.fromStrict $ encodeUtf8 t + act <- getLazyByteString (B.length tAsBytes) + if act == tAsBytes then + return t + else + error $ "Malformed input!" + +readStrLn :: Int64 -> Get T.Text +readStrLn n = readStr n <* expectStr "\n" + +readStr :: Int64 -> Get T.Text +readStr n = decodeUtf8 . B.toStrict <$> getLazyByteString n + +putStrLn :: T.Text -> Put +putStrLn s = putStr (s `T.append` "\n") + +putStr :: T.Text -> Put +putStr = putLazyByteString . B.fromStrict . encodeUtf8 + +putStrPadded :: Int -> T.Text -> Put +putStrPadded n t = case (T.stripEnd t == t, T.length t <= n) of + (True, True) -> putStr (padRight " " n t) + (False, _) -> error "Padded strings must not end with whitespaces." + (_, False) -> error "Padded strings must not be longer than the allocated space." + +putStrLnPadded :: Int -> T.Text -> Put +putStrLnPadded n t = putStrPadded n t >> putStr "\n" + +readStrPadded :: Int64 -> Get T.Text +readStrPadded n = T.stripEnd <$> readStr n + +readStrLnPadded :: Int64 -> Get T.Text +readStrLnPadded n = do + r <- readStrPadded n + expectStr "\n" + return r + +padRight :: T.Text -> Int -> T.Text -> T.Text +padRight s n t = t `T.append` (T.replicate (n - T.length t) s) + +padLeft :: T.Text -> Int -> T.Text -> T.Text +padLeft s n t = (T.replicate (n - T.length t) s) `T.append` t + +sep :: T.Text +sep = T.replicate 20 "=" diff --git a/test/RoundtripTest.hs b/test/RoundtripTest.hs new file mode 100644 --- /dev/null +++ b/test/RoundtripTest.hs @@ -0,0 +1,35 @@ +{-# LANGUAGE OverloadedStrings, DoAndIfThenElse #-} + +module Main where + +import Codec.Chunky +import qualified Data.Text as T +import Data.Text.Encoding +import qualified Data.ByteString.Lazy as B +import Test.HUnit +import System.Exit +import Data.Binary.Get +import Data.Binary.Put + +chunks :: [Chunk] +chunks = [ Textual "Test Textual" "Blabsdfsdf\n\nafsdfsadfs" + , Binary "Test Binary" (B.fromStrict $ encodeUtf8 "bladsfdsg\n\ndfg") + , Binary "Test Binary2" (B.pack [0xFF, 0xFF, 0xFF, 0xEF, 0xBB, 0xBF, 0x00]) + ] + +tests = TestList [TestLabel "Roundtrip Test" rndtripTest] + +rndtripTest = TestCase (do + let bs = runPut $ putChunks chunks + let act = runGet (readChunks (length chunks)) bs + + assertEqual "" chunks act + ) + +main :: IO () +main = do + cnts <- runTestTT $ tests + if (errors cnts > 0 || failures cnts > 0) then + exitWith (ExitFailure 1) + else + exitSuccess