epub-metadata 2.2.0.0 → 2.2.0.1
raw patch · 10 files changed
+217/−428 lines, 10 filesdep +HUnitsetup-changedPVP ok
version bump matches the API change (PVP)
Dependencies added: HUnit
API changes (from Hackage documentation)
Files
- README +0/−91
- Setup.hs +1/−10
- TODO +0/−3
- epub-metadata.cabal +17/−4
- src/Codec/Epub/Opf/Format/Metadata.hs +5/−5
- testsuite/opf-parse.hs +194/−0
- testsuite/runtests.hs +0/−182
- util/gentags.sh +0/−3
- util/prefs/boring +0/−117
- util/show-opf.hs +0/−13
− README
@@ -1,91 +0,0 @@-------Building:-- Easy with cabal-install, of course:-- $ cabal install epub-metadata-- Or the conventional way:-- $ runhaskell Setup.hs configure- $ runhaskell Setup.hs build- $ runhaskell Setup.hs test- $ runhaskell Setup.hs haddock- $ runhaskell Setup.hs install---------Why was this done?-- The motivation for this project grew out of my desire to take charge- of missing or incorrect ePub metadata in books I have purchased. I- started out using the Calibre open source tools for examining this- info. Limitations and incomplete implementation of those tools led- me here to build a more complete implementation in the programming- language that I love beyond all others.---------Why didn't I just use existing solutions?-- - Calibre ebook-meta utility-- I experienced various problems using this software, such as:-- Incomplete and in some cases incorrect handling of tags that can- exist more than once, particularly when they are differentiated- using attributes according to the spec.-- Unable to display many fields in the OPF Package Document metadata- specification. Unable to manipulate data that is represented as- attributes of tags in the OPF spec.-- Astonishingly slow performance. The command-line tool in this- new Haskell project is more than 45 times faster at parsing- and displaying ePub metadata. I'm going to blame Python here for- Calibre's performance. This has had a big impact on projects where- I've been processing hundreds of ePubs in batch operations.-- To be fair, an effort is being made in Calibre to work with both- ePub and Sony LRF book documents. That is going to naturally require- a lowest-common-denominator approach. My focus here was to work- with ePub only, and thoroughly support the OPF specification.--- - epub on Hackage, EPUB E-Book construction support library-- The focus of this project seems to be with building new documents,- not parsing existing files. And there is a specific attempt to- do more than the metadata, to gather up the content and other- metafiles that make up an ePub for creation.-- Examining Codec.Ebook.OPF.Types, most of the metadata fields- from the OPF Package Document spec are missing or aren't modeled- thoroughly. I felt to contribute to this project, I would have- had to significantly rip up the types and redesign them.-- At this time I felt it was a better solution for me to start fresh- with modelling these types and code to manipulate them. That said,- I would be very interested in combining the epub and epub-metadata- projects at some point in some way that makes sense.---------A word about the version numbering scheme:-- 4-part: major.minor.status.build- 3-part: major.status.build-- status:- 0 alpha- 1 beta- 2 release candidate- 3 release-- examples:- 1.3.0.2 v1.3 alpha build 2- 1.2.1.0 v1.2 beta build 0- 4.2.24 v4 release candidate build 24- 2.10.3.5 v2.10 release build 5 (say they were bug fixes)- 1.5.2.20090818 Can even use a date for build- v1.5 release candidate 2009-08-18 build
Setup.hs view
@@ -5,15 +5,6 @@ -- Author: Dino Morelli <dino@ui3.info> import Distribution.Simple-import System.Cmd ( system ) -main = defaultMainWithHooks (simpleUserHooks - { runTests = testRunner- } )-- where- -- Target for running all unit tests- testRunner _ _ _ _ = do- system $ "runhaskell -isrc testsuite/runtests.hs"- return ()+main = defaultMain
− TODO
@@ -1,3 +0,0 @@-- Better errors on parse failure, this will emerge along with more unit testing--- More unit testing
epub-metadata.cabal view
@@ -1,6 +1,6 @@ name: epub-metadata-version: 2.2.0.0-cabal-version: >= 1.2+version: 2.2.0.1+cabal-version: >= 1.8 build-type: Simple license: BSD3 license-file: LICENSE@@ -12,8 +12,13 @@ synopsis: Library for parsing and manipulating ePub files and OPF package data description: Library for parsing and manipulating ePub files and OPF package data. An attempt has been made here to very thoroughly implement the OPF Package Document specification. category: Codec, Text-tested-with: GHC >= 6.12.3+tested-with: GHC >= 7.0.3+extra-source-files: testsuite/*.hs, testsuite/*.opf +source-repository head+ type: darcs+ location: http://ui3.info/darcs/epub-metadata/+ library exposed-modules: Codec.Epub.Archive Codec.Epub.IO,@@ -30,8 +35,16 @@ Codec.Epub.Opf.Package.Metadata, Codec.Epub.Opf.Package.Spine, Codec.Epub.Opf.Parse- hs-source-dirs: src build-depends: base >= 3 && < 5, bytestring, containers, directory, filepath, hxt >= 9, mtl, regex-compat, zip-archive+ hs-source-dirs: src+ ghc-options: -Wall++test-suite opf-parse+ type: exitcode-stdio-1.0+ main-is: opf-parse.hs+ build-depends: base >= 3 && < 5, bytestring, directory, filepath,+ HUnit, hxt >= 9, mtl, regex-compat, zip-archive+ hs-source-dirs: src testsuite ghc-options: -Wall
src/Codec/Epub/Opf/Format/Metadata.hs view
@@ -19,7 +19,7 @@ tellTitle (MetaTitle Nothing title) = tellSeq $ printf "title: %s\n" title tellTitle (MetaTitle lang title) = tellSeq $ printf "title\n%s%s" (formatSubline "lang" lang)- (formatSubline "title" (Just title))+ (formatSubline "text" (Just title)) tellCreator :: MonadWriter (Seq Char) m => MetaCreator -> m ()@@ -29,7 +29,7 @@ tellSeq $ printf "creator\n%s%s%s" (formatSubline "role" role) (formatSubline "file-as" fileAs)- (formatSubline "creator" (Just creator))+ (formatSubline "text" (Just creator)) tellContributor :: MonadWriter (Seq Char) m => MetaCreator -> m ()@@ -39,7 +39,7 @@ tellSeq $ printf "contributor\n%s%s%s" (formatSubline "role" role) (formatSubline "file-as" fileAs)- (formatSubline "creator" (Just contributor))+ (formatSubline "text" (Just contributor)) tellDate :: MonadWriter (Seq Char) m => MetaDate -> m ()@@ -48,7 +48,7 @@ tellDate (MetaDate event date) = tellSeq $ printf "date\n%s%s" (formatSubline "event" event)- (formatSubline "date" (Just date))+ (formatSubline "text" (Just date)) tellType :: MonadWriter (Seq Char) m => Maybe String -> m ()@@ -64,7 +64,7 @@ tellSeq $ printf "identifier\n%s%s%s" (formatSubline "id" (Just idVal)) (formatSubline "scheme" scheme)- (formatSubline "identifier" (Just content))+ (formatSubline "text" (Just content)) tellSource :: MonadWriter (Seq Char) m => Maybe String -> m ()
+ testsuite/opf-parse.hs view
@@ -0,0 +1,194 @@+-- Copyright: 2010, 2011 Dino Morelli+-- License: BSD3 (see LICENSE)+-- Author: Dino Morelli <dino@ui3.info>++import Control.Monad.Error+import System.Exit+import System.FilePath+import Test.HUnit ( Counts (..), Test (..), assertEqual, runTestTT )++import Codec.Epub.Opf.Package+import Codec.Epub.Opf.Parse+++main :: IO ()+main = do+ counts <- runTestTT tests+ exit $ testsPassed counts+++exit :: Bool -> IO ()+exit True = exitWith ExitSuccess+exit False = exitWith $ ExitFailure 1+++testsPassed :: Counts -> Bool+testsPassed (Counts _ _ e f) = (e == 0) && (f == 0)+++tests :: Test+tests = TestList+ [ testFull+ , testMinimal+ , testMissingAll+ ]+++{- A fairly comprehensive test containing all possible things+ Not complete at this time because the library can't parse it all yet!+-}+testFull :: Test+testFull = TestCase $ do+ xmlString <- readFile $ "testsuite" </> "testFull.opf"+ actual <- runErrorT $ parseXmlToOpf xmlString+ let expected =+ Right Package+ { opVersion = "2.0"+ , opUniqueId = "isbn"+ , opMeta = Metadata+ { metaTitles =+ [ MetaTitle Nothing "Title Of This Book"+ , MetaTitle (Just "fr") "Titre De Ce Livre"+ ]+ , metaCreators = + [ MetaCreator+ (Just "aut")+ (Just "Wiggins, Josephine B.")+ "Josephine B. Wiggins"+ , MetaCreator+ (Just "aut")+ Nothing+ "Horatio Cromwell"+ , MetaCreator+ Nothing+ Nothing+ "Natalia Jenkins"+ ]+ , metaContributors = + [ MetaCreator + (Just "ill") + (Just "Knickerbocker, Reginald Q.") + "Reginald Q. Knickerbocker"+ , MetaCreator + (Just "edt") + Nothing "Beverly Abercrombie"+ ]+ , metaSubjects = ["Fiction","Science Fiction"]+ , metaDescription = Just "This document is a stub used for unit testing. It is missing the rest of the tags that normally occur after metadata."+ , metaPublisher = Just "Fictional Books Ltd."+ , metaDates = + [ MetaDate (Just "published") "2010"+ , MetaDate (Just "created") "2010-05-07"+ , MetaDate (Just "modified") "2010-05-08T10:20:57"+ , MetaDate Nothing "2009"+ ]+ , metaType = Just "test OPF Package Document"+ , metaFormat = Just "ePub publication"+ , metaIds = + [ MetaId "isbn" (Just "ISBN") "1-82057-821-9"+ , MetaId "other" Nothing "1386506873266"+ ]+ , metaSource = Just "document source"+ , metaLangs = ["en-us"]+ , metaRelation = Just "document relation"+ , metaCoverage = Just "coverage information"+ , metaRights = Just "Copyright: 2010 Dino Morelli, License: BSD3"+ }+ , opManifest = + [ ManifestItem + { mfiId = "ncx"+ , mfiHref = "toc.ncx"+ , mfiMediaType = "application/x-dtbncx+xml"+ }+ , ManifestItem + { mfiId = "titlePage"+ , mfiHref = "content/titlePage.html"+ , mfiMediaType = "application/xhtml+xml"+ }+ , ManifestItem + { mfiId = "someContent"+ , mfiHref = "content/someContent.html"+ , mfiMediaType = "application/xhtml+xml"+ }+ ]+ , opSpine = Spine+ { spineToc = "ncx"+ , spineItemrefs = + [ SpineItemref {siIdRef = "titlePage", siLinear = Nothing}+ , SpineItemref {siIdRef = "someContent", siLinear = Nothing}+ ]+ }+ , opGuide = + [ GuideRef + { grType = "title-page"+ , grTitle = Just "Title page"+ , grHref = "content/titlePage.html"+ }+ , GuideRef + { grType = "text"+ , grTitle = Just "Title Of This Book"+ , grHref = "content/someContent.html"+ }+ ]+ }+ assertEqual "very full" expected actual+++{- Test the absolute minimum set of fields allowed while remaining + compliant with the spec+-}+testMinimal :: Test+testMinimal = TestCase $ do+ xmlString <- liftIO $ readFile $ "testsuite" </> "testMinimal.opf"+ actual <- runErrorT $ parseXmlToOpf xmlString+ let expected = + Right Package + { opVersion = "2.0"+ , opUniqueId = "isbn"+ , opMeta = Metadata + { metaTitles = [MetaTitle Nothing "Title Of This Book"]+ , metaCreators = []+ , metaContributors = []+ , metaSubjects = []+ , metaDescription = Nothing+ , metaPublisher = Nothing+ , metaDates = []+ , metaType = Nothing+ , metaFormat = Nothing+ , metaIds = [MetaId "isbn" (Just "ISBN") "1-82057-821-9"]+ , metaSource = Nothing+ , metaLangs = ["en-us"]+ , metaRelation = Nothing+ , metaCoverage = Nothing+ , metaRights = Nothing+ }+ , opManifest = + [ ManifestItem + { mfiId = "ncx"+ , mfiHref = "toc.ncx"+ , mfiMediaType = "application/x-dtbncx+xml"+ }+ ]+ , opSpine = Spine {spineToc = "ncx", spineItemrefs = []}+ , opGuide = []+ }+ assertEqual "minimal" expected actual+++{- Test data missing everything important: package version and + unique-identifier attributes, title, identifier and language tags+-}+testMissingAll :: Test+testMissingAll = TestCase $ do+ xmlString <- readFile $ "testsuite" </> "testMissingAll.opf"+ actual <- runErrorT $ parseXmlToOpf xmlString+ let expected =+ Right Package + { opVersion = ""+ , opUniqueId = ""+ , opMeta = emptyMetadata+ , opManifest = []+ , opSpine = Spine {spineToc = "", spineItemrefs = []}+ , opGuide = []+ }+ assertEqual "missing all" expected actual
− testsuite/runtests.hs
@@ -1,182 +0,0 @@--- Copyright: 2010, 2011 Dino Morelli--- License: BSD3 (see LICENSE)--- Author: Dino Morelli <dino@ui3.info>--import Control.Monad.Error-import System.FilePath-import Test.HUnit ( Counts, Test (..), assertEqual, runTestTT )-import Test.HUnit.Base ( Assertion )--import Codec.Epub.Opf.Package-import Codec.Epub.Opf.Parse---main = runTestTT tests >> return ()---tests :: Test-tests = TestList- [ testFull- , testMinimal- , testMissingAll- ]---{- A fairly comprehensive test containing all possible things- Not complete at this time because the library can't parse it all yet!--}-testFull :: Test-testFull = TestCase $ do- xmlString <- readFile $ "testsuite" </> "testFull.opf"- actual <- runErrorT $ parseXmlToOpf xmlString- let expected =- Right Package- { opVersion = "2.0"- , opUniqueId = "isbn"- , opMeta = Metadata- { metaTitles =- [ MetaTitle Nothing "Title Of This Book"- , MetaTitle (Just "fr") "Titre De Ce Livre"- ]- , metaCreators = - [ MetaCreator- (Just "aut")- (Just "Wiggins, Josephine B.")- "Josephine B. Wiggins"- , MetaCreator- (Just "aut")- Nothing- "Horatio Cromwell"- , MetaCreator- Nothing- Nothing- "Natalia Jenkins"- ]- , metaContributors = - [ MetaCreator - (Just "ill") - (Just "Knickerbocker, Reginald Q.") - "Reginald Q. Knickerbocker"- , MetaCreator - (Just "edt") - Nothing "Beverly Abercrombie"- ]- , metaSubjects = ["Fiction","Science Fiction"]- , metaDescription = Just "This document is a stub used for unit testing. It is missing the rest of the tags that normally occur after metadata."- , metaPublisher = Just "Fictional Books Ltd."- , metaDates = - [ MetaDate (Just "published") "2010"- , MetaDate (Just "created") "2010-05-07"- , MetaDate (Just "modified") "2010-05-08T10:20:57"- , MetaDate Nothing "2009"- ]- , metaType = Just "test OPF Package Document"- , metaFormat = Just "ePub publication"- , metaIds = - [ MetaId "isbn" (Just "ISBN") "1-82057-821-9"- , MetaId "other" Nothing "1386506873266"- ]- , metaSource = Just "document source"- , metaLangs = ["en-us"]- , metaRelation = Just "document relation"- , metaCoverage = Just "coverage information"- , metaRights = Just "Copyright: 2010 Dino Morelli, License: BSD3"- }- , opManifest = - [ ManifestItem - { mfiId = "ncx"- , mfiHref = "toc.ncx"- , mfiMediaType = "application/x-dtbncx+xml"- }- , ManifestItem - { mfiId = "titlePage"- , mfiHref = "content/titlePage.html"- , mfiMediaType = "application/xhtml+xml"- }- , ManifestItem - { mfiId = "someContent"- , mfiHref = "content/someContent.html"- , mfiMediaType = "application/xhtml+xml"- }- ]- , opSpine = Spine- { spineToc = "ncx"- , spineItemrefs = - [ SpineItemref {siIdRef = "titlePage", siLinear = Nothing}- , SpineItemref {siIdRef = "someContent", siLinear = Nothing}- ]- }- , opGuide = - [ GuideRef - { grType = "title-page"- , grTitle = Just "Title page"- , grHref = "content/titlePage.html"- }- , GuideRef - { grType = "text"- , grTitle = Just "Title Of This Book"- , grHref = "content/someContent.html"- }- ]- }- assertEqual "very full" expected actual---{- Test the absolute minimum set of fields allowed while remaining - compliant with the spec--}-testMinimal :: Test-testMinimal = TestCase $ do- xmlString <- liftIO $ readFile $ "testsuite" </> "testMinimal.opf"- actual <- runErrorT $ parseXmlToOpf xmlString- let expected = - Right Package - { opVersion = "2.0"- , opUniqueId = "isbn"- , opMeta = Metadata - { metaTitles = [MetaTitle Nothing "Title Of This Book"]- , metaCreators = []- , metaContributors = []- , metaSubjects = []- , metaDescription = Nothing- , metaPublisher = Nothing- , metaDates = []- , metaType = Nothing- , metaFormat = Nothing- , metaIds = [MetaId "isbn" (Just "ISBN") "1-82057-821-9"]- , metaSource = Nothing- , metaLangs = ["en-us"]- , metaRelation = Nothing- , metaCoverage = Nothing- , metaRights = Nothing- }- , opManifest = - [ ManifestItem - { mfiId = "ncx"- , mfiHref = "toc.ncx"- , mfiMediaType = "application/x-dtbncx+xml"- }- ]- , opSpine = Spine {spineToc = "ncx", spineItemrefs = []}- , opGuide = []- }- assertEqual "minimal" expected actual---{- Test data missing everything important: package version and - unique-identifier attributes, title, identifier and language tags--}-testMissingAll :: Test-testMissingAll = TestCase $ do- xmlString <- readFile $ "testsuite" </> "testMissingAll.opf"- actual <- runErrorT $ parseXmlToOpf xmlString- let expected =- Right Package - { opVersion = ""- , opUniqueId = ""- , opMeta = emptyMetadata- , opManifest = []- , opSpine = Spine {spineToc = "", spineItemrefs = []}- , opGuide = []- }- assertEqual "missing all" expected actual
− util/gentags.sh
@@ -1,3 +0,0 @@-#! /bin/sh--find src -regex '.*\..?hs' | xargs hasktags -c
− util/prefs/boring
@@ -1,117 +0,0 @@-# Boring file regexps:--### compiler and interpreter intermediate files-# haskell (ghc) interfaces-\.hi$-\.hi-boot$-\.o-boot$-# object files-\.o$-\.o\.cmd$-# profiling haskell-\.p_hi$-\.p_o$-# haskell program coverage resp. profiling info-\.tix$-\.prof$-# fortran module files-\.mod$-# linux kernel-\.ko\.cmd$-\.mod\.c$-(^|/)\.tmp_versions($|/)-# *.ko files aren't boring by default because they might-# be Korean translations rather than kernel modules-# \.ko$-# python, emacs, java byte code-\.py[co]$-\.elc$-\.class$-# objects and libraries; lo and la are libtool things-\.(obj|a|exe|so|lo|la)$-# compiled zsh configuration files-\.zwc$-# Common LISP output files for CLISP and CMUCL-\.(fas|fasl|sparcf|x86f)$--### build and packaging systems-# cabal intermediates-\.installed-pkg-config-\.setup-config-# standard cabal build dir, might not be boring for everybody-# ^dist(/|$)-# autotools-(^|/)autom4te\.cache($|/)-(^|/)config\.(log|status)$-# microsoft web expression, visual studio metadata directories-\_vti_cnf$-\_vti_pvt$-# gentoo tools-\.revdep-rebuild.*-# generated dependencies-^\.depend$--### version control systems-# cvs-(^|/)CVS($|/)-\.cvsignore$-# cvs, emacs locks-^\.#-# rcs-(^|/)RCS($|/)-,v$-# subversion-(^|/)\.svn($|/)-# mercurial-(^|/)\.hg($|/)-# git-(^|/)\.git($|/)-# bzr-\.bzr$-# sccs-(^|/)SCCS($|/)-# darcs-(^|/)_darcs($|/)-(^|/)\.darcsrepo($|/)-^\.darcs-temp-mail$--darcs-backup[[:digit:]]+$-# gnu arch-(^|/)(\+|,)-(^|/)vssver\.scc$-\.swp$-(^|/)MT($|/)-(^|/)\{arch\}($|/)-(^|/).arch-ids($|/)-# bitkeeper-(^|/)BitKeeper($|/)-(^|/)ChangeSet($|/)--### miscellaneous-# backup files-~$-\.bak$-\.BAK$-# patch originals and rejects-\.orig$-\.rej$-# X server-\..serverauth.*-# image spam-\#-(^|/)Thumbs\.db$-# vi, emacs tags-(^|/)(tags|TAGS)$-#(^|/)\.[^/]-# core dumps-(^|/|\.)core$-# partial broken files (KIO copy operations)-\.part$-# waf files, see http://code.google.com/p/waf/-(^|/)\.waf-[[:digit:].]+-[[:digit:]]+($|/)-(^|/)\.lock-wscript$-# mac os finder-(^|/)\.DS_Store$-# cabal-(^|/)dist($|/)--(^|/)bin($|/)
− util/show-opf.hs
@@ -1,13 +0,0 @@-#! /usr/bin/runhaskell -isrc---- Copyright: 2010, 2011 Dino Morelli--- License: BSD3 (see LICENSE)--- Author: Dino Morelli <dino@ui3.info>--import System.Environment ( getArgs )--import Codec.Epub.Opf.Parse---main :: IO ()-main = (fmap head) getArgs >>= readFile >>= parseXmlToOpf >>= print