razom-text-util (empty) → 0.1.0.0

raw patch · 15 files changed

+634/−0 lines, 15 filesdep +QuickCheckdep +basedep +razom-text-utilsetup-changed

Dependencies added: QuickCheck, base, razom-text-util, regex-applicative, smaoin, text-position

Files

• AUTHORS +1/−0
• COPYING +121/−0
• ChangeLog +17/−0
• INSTALL +13/−0
• NEWS +18/−0
• README +21/−0
• Setup.hs +2/−0
• razom-text-util.cabal +50/−0
• src/Text/Razom.hs +29/−0
• src/Text/Razom/Char.hs +26/−0
• src/Text/Razom/Number.hs +75/−0
• src/Text/Razom/Types.hs +37/−0
• src/Text/Razom/Uid.hs +107/−0
• src/Text/Razom/Value.hs +55/−0
• test/test.hs +62/−0

@@ -0,0 +1,1 @@+fr33domlover <fr33domlover@riseup.net>

@@ -0,0 +1,121 @@+Creative Commons Legal Code++CC0 1.0 Universal++    CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE+    LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN+    ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS+    INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES+    REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS+    PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM+    THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED+    HEREUNDER.++Statement of Purpose++The laws of most jurisdictions throughout the world automatically confer+exclusive Copyright and Related Rights (defined below) upon the creator+and subsequent owner(s) (each and all, an "owner") of an original work of+authorship and/or a database (each, a "Work").++Certain owners wish to permanently relinquish those rights to a Work for+the purpose of contributing to a commons of creative, cultural and+scientific works ("Commons") that the public can reliably and without fear+of later claims of infringement build upon, modify, incorporate in other+works, reuse and redistribute as freely as possible in any form whatsoever+and for any purposes, including without limitation commercial purposes.+These owners may contribute to the Commons to promote the ideal of a free+culture and the further production of creative, cultural and scientific+works, or to gain reputation or greater distribution for their Work in+part through the use and efforts of others.++For these and/or other purposes and motivations, and without any+expectation of additional consideration or compensation, the person+associating CC0 with a Work (the "Affirmer"), to the extent that he or she+is an owner of Copyright and Related Rights in the Work, voluntarily+elects to apply CC0 to the Work and publicly distribute the Work under its+terms, with knowledge of his or her Copyright and Related Rights in the+Work and the meaning and intended legal effect of CC0 on those rights.++1. Copyright and Related Rights. A Work made available under CC0 may be+protected by copyright and related or neighboring rights ("Copyright and+Related Rights"). Copyright and Related Rights include, but are not+limited to, the following:++  i. the right to reproduce, adapt, distribute, perform, display,+     communicate, and translate a Work;+ ii. moral rights retained by the original author(s) and/or performer(s);+iii. publicity and privacy rights pertaining to a person's image or+     likeness depicted in a Work;+ iv. rights protecting against unfair competition in regards to a Work,+     subject to the limitations in paragraph 4(a), below;+  v. rights protecting the extraction, dissemination, use and reuse of data+     in a Work;+ vi. database rights (such as those arising under Directive 96/9/EC of the+     European Parliament and of the Council of 11 March 1996 on the legal+     protection of databases, and under any national implementation+     thereof, including any amended or successor version of such+     directive); and+vii. other similar, equivalent or corresponding rights throughout the+     world based on applicable law or treaty, and any national+     implementations thereof.++2. Waiver. To the greatest extent permitted by, but not in contravention+of, applicable law, Affirmer hereby overtly, fully, permanently,+irrevocably and unconditionally waives, abandons, and surrenders all of+Affirmer's Copyright and Related Rights and associated claims and causes+of action, whether now known or unknown (including existing as well as+future claims and causes of action), in the Work (i) in all territories+worldwide, (ii) for the maximum duration provided by applicable law or+treaty (including future time extensions), (iii) in any current or future+medium and for any number of copies, and (iv) for any purpose whatsoever,+including without limitation commercial, advertising or promotional+purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each+member of the public at large and to the detriment of Affirmer's heirs and+successors, fully intending that such Waiver shall not be subject to+revocation, rescission, cancellation, termination, or any other legal or+equitable action to disrupt the quiet enjoyment of the Work by the public+as contemplated by Affirmer's express Statement of Purpose.++3. Public License Fallback. Should any part of the Waiver for any reason+be judged legally invalid or ineffective under applicable law, then the+Waiver shall be preserved to the maximum extent permitted taking into+account Affirmer's express Statement of Purpose. In addition, to the+extent the Waiver is so judged Affirmer hereby grants to each affected+person a royalty-free, non transferable, non sublicensable, non exclusive,+irrevocable and unconditional license to exercise Affirmer's Copyright and+Related Rights in the Work (i) in all territories worldwide, (ii) for the+maximum duration provided by applicable law or treaty (including future+time extensions), (iii) in any current or future medium and for any number+of copies, and (iv) for any purpose whatsoever, including without+limitation commercial, advertising or promotional purposes (the+"License"). The License shall be deemed effective as of the date CC0 was+applied by Affirmer to the Work. Should any part of the License for any+reason be judged legally invalid or ineffective under applicable law, such+partial invalidity or ineffectiveness shall not invalidate the remainder+of the License, and in such case Affirmer hereby affirms that he or she+will not (i) exercise any of his or her remaining Copyright and Related+Rights in the Work or (ii) assert any associated claims and causes of+action with respect to the Work, in either case contrary to Affirmer's+express Statement of Purpose.++4. Limitations and Disclaimers.++ a. No trademark or patent rights held by Affirmer are waived, abandoned,+    surrendered, licensed or otherwise affected by this document.+ b. Affirmer offers the Work as-is and makes no representations or+    warranties of any kind concerning the Work, express, implied,+    statutory or otherwise, including without limitation warranties of+    title, merchantability, fitness for a particular purpose, non+    infringement, or the absence of latent or other defects, accuracy, or+    the present or absence of errors, whether or not discoverable, all to+    the greatest extent permissible under applicable law.+ c. Affirmer disclaims responsibility for clearing rights of other persons+    that may apply to the Work or any use thereof, including without+    limitation any person's Copyright and Related Rights in the Work.+    Further, Affirmer disclaims responsibility for obtaining any necessary+    consents, permissions or other rights required for any use of the+    Work.+ d. Affirmer understands and acknowledges that Creative Commons is not a+    party to this document and has no duty or obligation with respect to+    this CC0 or use of the Work.

@@ -0,0 +1,17 @@+The changes are recorded by the version control system, Darcs. To see a log+quickly from the terminal, run:++  $ darcs changes --repo http://darcs.rel4tion.org/repos/razom-text-util++There is also a web interface at <http://darcs.rel4tion.org> which, among other+things, can display the history log.++To see the log in a local clone, first get a copy of the repository if you+haven't yet:++  $ darcs get http://darcs.rel4tion.org/repos/razom-text-util++Then move into the newly created directory and run darcs:++  $ cd razom-text-util+  $ darcs changes

@@ -0,0 +1,13 @@+Install from Hackage:++    $ cabal install razom-text-util++Install from unpacked release tarball or source repo:++    $ cd razom-text-util+    $ cabal install++Just play with it without installing:++    $ cabal build+    $ cabal repl

@@ -0,0 +1,18 @@+razom-text-util 0.1.0.0 -- 2015-05-05+=====================================++General, build and documentation changes:++* (This is the first release, so everything is new)++New APIs, features and enhancements:++* (This is the first release, so everything is a new feature)++Bug fixes:++* (This is the first release and there's no bug tracking yet)++Dependency changes:++* (No dependencies at this point)

@@ -0,0 +1,21 @@+This is a Haskell library of utilities for writing code that handles semantic+information documents. It is mainly intended for collecting common tools used+by the Razom information language libraries, such as language-idan and+language-kort.++See the .cabal file for more info and link to project website the version+control.++The official download locations are:++- The project website+- Torrents+- Hackage++This library is free software, and is committed to software freedom. It is+released to the public domain using the CC0 Public Domain Dedication. For the+boring "legal" details see the file 'COPYING'.++See the file 'INSTALL' for hints on installation. The file 'ChangeLog' explains+how to see the history log of the changes done in the code. 'NEWS' provides a+friendly overview of the changes for each release.

@@ -0,0 +1,2 @@+import Distribution.Simple+main = defaultMain

@@ -0,0 +1,50 @@+name:                razom-text-util+version:             0.1.0.0+synopsis:            Common text/parsing tools for Razom language packages.+description:         This is a library of utilities for writing code that+                     handles semantic information documents (text files). It is+                     mainly intended for collecting common tools used by the+                     Razom information language libraries, but can be used by+                     any text related package which needs these tools.+homepage:            http://rel4tion.org/projects/razom-text-util/+bug-reports:         http://rel4tion.org/projects/razom-text-util/tickets/+license:             PublicDomain+license-file:        COPYING+author:              fr33domlover+maintainer:          fr33domlover@riseup.net+copyright:           ♡ Copying is an act of love. Please copy, reuse and share.+category:            Text+build-type:          Simple+extra-source-files:  AUTHORS ChangeLog COPYING INSTALL NEWS README+cabal-version:       >=1.10++source-repository head+  type:                darcs+  location:            http://darcs.rel4tion.org/repos/razom-text-util/++library+  exposed-modules:     Text.Razom+                     , Text.Razom.Char+                     , Text.Razom.Number+                     , Text.Razom.Types+                     , Text.Razom.Uid+                     , Text.Razom.Value+  -- other-modules:       +  -- other-extensions:    +  build-depends:       base              >=4.7 && <5+                     , regex-applicative >=0.3.1+                     , smaoin            >=0.1.0.0+                     , text-position     >=0.1.0.0+  hs-source-dirs:      src+  default-language:    Haskell2010++test-suite test+  default-language:    Haskell2010+  type:                exitcode-stdio-1.0+  hs-source-dirs:      test+  main-is:             test.hs+  build-depends:       base              >=4.7 && <5+                     , QuickCheck        >=2.8+                     , razom-text-util+                     , regex-applicative >=0.3.1+                     , smaoin            >=0.1.1.1

@@ -0,0 +1,29 @@+{- This file is part of razom-text-util.+ -+ - Written in 2015 by fr33domlover <fr33domlover@rel4tion.org>.+ -+ - ♡ Copying is an act of love. Please copy, reuse and share.+ -+ - The author(s) have dedicated all copyright and related and neighboring+ - rights to this software to the public domain worldwide. This software is+ - distributed without any warranty.+ -+ - You should have received a copy of the CC0 Public Domain Dedication along+ - with this software. If not, see+ - <http://creativecommons.org/publicdomain/zero/1.0/>.+ -}++module Text.Razom+    ( module Text.Razom.Char+    , module Text.Razom.Number+    , module Text.Razom.Types+    , module Text.Razom.Uid+    , module Text.Razom.Value+    )+where++import Text.Razom.Char+import Text.Razom.Number+import Text.Razom.Types+import Text.Razom.Uid+import Text.Razom.Value

@@ -0,0 +1,26 @@+{- This file is part of razom-text-util.+ -+ - Written in 2015 by fr33domlover <fr33domlover@rel4tion.org>.+ -+ - ♡ Copying is an act of love. Please copy, reuse and share.+ -+ - The author(s) have dedicated all copyright and related and neighboring+ - rights to this software to the public domain worldwide. This software is+ - distributed without any warranty.+ -+ - You should have received a copy of the CC0 Public Domain Dedication along+ - with this software. If not, see+ - <http://creativecommons.org/publicdomain/zero/1.0/>.+ -}++module Text.Razom.Char+    ( isGraphical+    )+where++import Data.Char++-- | Selects Unicode characters with basic type Graphical.+isGraphical :: Char -> Bool+isGraphical c = isLetter c || isMark c || isNumber c || isPunctuation c+             || isSymbol c || generalCategory c == Space

@@ -0,0 +1,75 @@+{- This file is part of razom-text-util.+ -+ - Written in 2015 by fr33domlover <fr33domlover@rel4tion.org>.+ -+ - ♡ Copying is an act of love. Please copy, reuse and share.+ -+ - The author(s) have dedicated all copyright and related and neighboring+ - rights to this software to the public domain worldwide. This software is+ - distributed without any warranty.+ -+ - You should have received a copy of the CC0 Public Domain Dedication along+ - with this software. If not, see+ - <http://creativecommons.org/publicdomain/zero/1.0/>.+ -}++module Text.Razom.Number+    ( isBinDigit+    , digitsToNum+    , binit+    , octit+    , digit+    , hexit+    , binary+    , octal+    , decimal+    , hexadecimal+    , signed+    , fromDigits+    )+where++import Data.Char (digitToInt, isOctDigit)+import Data.Smaoin (RealNum (..))+import Text.Razom.Types+import Text.Regex.Applicative+import Text.Regex.Applicative.Common++-- | Selects ASCII binary digits, i.e. @\'0\'@ and @\'1@\'.+isBinDigit :: Char -> Bool+isBinDigit c = c == '0' || c == '1'++-- | Given the radix and an integer represented as a list of digits, construct+-- a number of the represented value.+--+-- >>> digitsToNum 2 [1, 0, 1, 1, 0, 1]+-- 45+--+-- >>> digitsToNum 16 [15, 15]+-- 255+digitsToNum :: (Num a) => a -> [a] -> a+digitsToNum radix = foldl (\ n i -> n * radix + i) 0++binit :: Num a => Regex a+binit = fromIntegral . digitToInt <$> psym isBinDigit++octit :: Num a => Regex a+octit = fromIntegral . digitToInt <$> psym isOctDigit++hexit = hexDigit++binary :: Num a => Regex a+binary = digitsToNum 2 <$> some binit++octal :: Num a => Regex a+octal = digitsToNum 8 <$> some octit++-- | Read a decimal number string into a 'RealNum' value. The parameters are+-- the digits before the decimal point (integer part) and after it (fraction+-- part). Example with the number 142.857:+--+-- >>> fromDigits [1, 4, 2] [8, 5, 7]+-- RealNum 142857 -3+fromDigits :: [Int] -> [Int] -> RealNum+fromDigits s e = RealNum (fromIntegral $ digitsToNum 10 $ s ++ e)+                         (fromIntegral $ negate $ length e)

@@ -0,0 +1,37 @@+{- This file is part of razom-text-util.+ -+ - Written in 2015 by fr33domlover <fr33domlover@rel4tion.org>.+ -+ - ♡ Copying is an act of love. Please copy, reuse and share.+ -+ - The author(s) have dedicated all copyright and related and neighboring+ - rights to this software to the public domain worldwide. This software is+ - distributed without any warranty.+ -+ - You should have received a copy of the CC0 Public Domain Dedication along+ - with this software. If not, see+ - <http://creativecommons.org/publicdomain/zero/1.0/>.+ -}++module Text.Razom.Types+    ( Regex+    , PosRegex+    , LexError (..)+    )+where++import Data.Position (Position, PosRE)+import Text.Regex.Applicative (RE)++-- | A regular expression over Unicode characters.+type Regex a = RE Char a++-- | A position-aware regular expression over Unicode characters.+type PosRegex a = PosRE Char a++-- | An error returned when applying a regex fails. Parameters:+--+-- (1) Location of the error+-- (2) Previous tokens parsed successfully+-- (3) The rest of the text+data LexError t = LexError Position [t] String deriving Show

@@ -0,0 +1,107 @@+{- This file is part of razom-text-util.+ -+ - Written in 2015 by fr33domlover <fr33domlover@rel4tion.org>.+ -+ - ♡ Copying is an act of love. Please copy, reuse and share.+ -+ - The author(s) have dedicated all copyright and related and neighboring+ - rights to this software to the public domain worldwide. This software is+ - distributed without any warranty.+ -+ - You should have received a copy of the CC0 Public Domain Dedication along+ - with this software. If not, see+ - <http://creativecommons.org/publicdomain/zero/1.0/>.+ -}++module Text.Razom.Uid+    ( uid+    , generator+    , escapeUid+    , unescapeUid+    )+where++import Data.Char (isSpace)+import Text.Razom.Char+import Text.Razom.Types+import Text.Regex.Applicative++uidcharp c = isGraphical c+          && not (isSpace c)+          && c /= '\\'+          && c /= '>'+uidfstp c  = uidcharp c && c /= '%'++uidopen  = sym '<'+uidclose = sym '>'++-- | Regex which matches and returns a Uid, unescaped and with delimiters+-- removed.+--+-- >>> match uid "<12<34\\>56\\\\7>"+-- Just "12<34>56\\7"+uid :: Regex String+uid = unescapeUid <$> (uidopen *> uidbody <* uidclose)+    where+    uidbody  = snd <$> withMatched (uidfirst <* many (mkpart uidcharp))+    uidfirst = Nothing <$ string "\\%" <|> Nothing <$ mkpart uidfstp+    mkpart p = Nothing <$ psym p+           <|> Nothing <$ string "\\\\"+           <|> Nothing <$ string "\\>"++-- | Regex which matches and returns a Uid generator, without delimiters and+-- the leading prefix character.+--+-- >>> match generator "<%>"+-- Just ""+--+-- >>> match generator "<%mylabel>"+-- Just "mylabel"+generator :: Regex String+generator = (:) <$> uidopen *> sym '%' *> many (psym uidcharp) <* uidclose++escapeUidChar :: Char -> String+escapeUidChar '\\' = "\\\\"+escapeUidChar '>'  = "\\>"+escapeUidChar c    = [c]++unescapeUidChar :: String -> (Char, String)+unescapeUidChar ('\\':c:cs) =+    case c of+        '\\' -> ('\\', cs)+        '>'  -> ('>',  cs)+        x    -> error $ "invalid escape sequence: \\" ++ [x]+unescapeUidChar (c:cs)   = (c, cs)+unescapeUidChar []       = error "empty string"++-- | Convert a Uid string into a form which can be put in a semantic document.+-- It doesn't add delimiters.+--+-- >>> escapeUid "1>2\\3"+-- "1\\>2\\\\3"+--+-- To prevent confusion, the above is the same as:+--+-- > ['1', '\\', '>', '2', '\\', '\\', '3']+escapeUid :: String -> String+escapeUid s =+    case s of+        ('%':cs) -> '\\' : '%' : f cs+        cs       -> f cs+    where+    f = foldr ((++) . escapeUidChar) ""++-- | Convert a Uid as encoded in a semantic document into normal form, i.e.+-- converting escape sequences to their target characters.+--+-- >>> unescapeUid "1\\>2\\\\3"+-- "1>2\\3"+unescapeUid :: String -> String+unescapeUid s =+    case s of+        []            -> []+        ('\\':'%':cs) -> '%' : f cs+        cs            -> f cs+    where+    f [] = []+    f s  = let (c, cs) = unescapeUidChar s in c : f cs

@@ -0,0 +1,55 @@+{- This file is part of razom-text-util.+ -+ - Written in 2015 by fr33domlover <fr33domlover@rel4tion.org>.+ -+ - ♡ Copying is an act of love. Please copy, reuse and share.+ -+ - The author(s) have dedicated all copyright and related and neighboring+ - rights to this software to the public domain worldwide. This software is+ - distributed without any warranty.+ -+ - You should have received a copy of the CC0 Public Domain Dedication along+ - with this software. If not, see+ - <http://creativecommons.org/publicdomain/zero/1.0/>.+ -}++module Text.Razom.Value+    ( literal+    , escapeMinimal+    , unescapeMinimal+    )+where++import Text.Razom.Char+import Text.Razom.Types+import Text.Regex.Applicative++-- | Matches a delimited generic value literal and returns it, without+-- delimiters and still escaped (since unescaping may be type-specific).+--+-- >>> match literal "{{1\\}2\\\\3}}"+-- Just "1\\}2\\\\3"+literal :: Regex String+literal = string "{{" *> lbody <* string "}}"+    where+    lbody = snd <$> withMatched (some lpart)+    lpart = Nothing <$ lchar <|> Nothing <$ sym '\\' <* psym isGraphical+    lchar = psym $ \ c -> isGraphical c && c /= '\\' && c /= '}'++-- | Given a value literal in unescaped form, do minimal default escapes: for+-- the escape character itself (backslash) and for the closing delimiter+-- character (@'{'@).+escapeMinimal :: String -> String+escapeMinimal []        = []+escapeMinimal ('\\':cs) = '\\' : '\\' : escapeMinimal cs+escapeMinimal ('}':cs)  = '\\' : '}'  : escapeMinimal cs+escapeMinimal (c:cs)    = c : escapeMinimal cs++-- | Given an escaped value literal, do minimal default unescapes: for the+-- escape character and for the closing delimiter.+unescapeMinimal :: String -> String+unescapeMinimal [] = []+unescapeMinimal ('\\':'\\':cs) = '\\' : unescapeMinimal cs+unescapeMinimal ('\\':'}':cs)  = '}'  : unescapeMinimal cs+unescapeMinimal ('\\':c:cs)    = '\\' : c : unescapeMinimal cs+unescapeMinimal (c:cs)         = c : unescapeMinimal cs

@@ -0,0 +1,62 @@+{- This file is part of razom-text-util.+ -+ - Written in 2015 by fr33domlover <fr33domlover@rel4tion.org>.+ -+ - ♡ Copying is an act of love. Please copy, reuse and share.+ -+ - The author(s) have dedicated all copyright and related and neighboring+ - rights to this software to the public domain worldwide. This software is+ - distributed without any warranty.+ -+ - You should have received a copy of the CC0 Public Domain Dedication along+ - with this software. If not, see+ - <http://creativecommons.org/publicdomain/zero/1.0/>.+ -}++import Control.Monad (unless)+import Data.Smaoin (RealNum (..))+import System.Exit+import Test.QuickCheck+import Test.QuickCheck.Test+import Text.Printf+import Text.Razom+import Text.Regex.Applicative (match)++main :: IO ()+main = do+    results <- mapM (\ (name ,action) -> printf "%-25s: " name >> action) tests+    unless (all isSuccess results) exitFailure++prop_digitsToNum :: Bool+prop_digitsToNum = digitsToNum 2  [1, 0, 1, 1, 0, 1] == 45+                && digitsToNum 16 [15, 15]           == 255++prop_fromDigits :: Bool+prop_fromDigits = fromDigits [1, 4, 2] [8, 5, 7] == RealNum 142857 (-3)++prop_uid :: Bool+prop_uid = match uid "<12<34\\>46\\\\7>" == Just "12<34>46\\7"++prop_generator :: Bool+prop_generator = match generator "<%>"      == Just ""+              && match generator "<%label>" == Just "label"++prop_escapeUid :: Bool+prop_escapeUid = escapeUid "1>2\\3" == "1\\>2\\\\3"++prop_unescapeUid :: Bool+prop_unescapeUid = unescapeUid "1\\>2\\\\3" == "1>2\\3"++prop_literal :: Bool+prop_literal = match literal "{{1\\}2\\\\3}}" == Just "1\\}2\\\\3"++tests :: [(String, IO Result)]+tests =+    [ ("digitsToNum", quickCheckResult prop_digitsToNum)+    , ("fromDigits",  quickCheckResult prop_fromDigits)+    , ("uid",         quickCheckResult prop_uid)+    , ("generator",   quickCheckResult prop_generator)+    , ("escapeUid",   quickCheckResult prop_escapeUid)+    , ("unescapeUid", quickCheckResult prop_unescapeUid)+    , ("literal",     quickCheckResult prop_literal)+    ]