diff --git a/AUTHORS b/AUTHORS new file mode 100644 --- /dev/null +++ b/AUTHORS @@ -0,0 +1,1 @@ +fr33domlover diff --git a/COPYING b/COPYING new file mode 100644 --- /dev/null +++ b/COPYING @@ -0,0 +1,121 @@ +Creative Commons Legal Code + +CC0 1.0 Universal + + CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE + LEGAL SERVICES. DISTRIBUTION OF THIS DOCUMENT DOES NOT CREATE AN + ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS + INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES + REGARDING THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS + PROVIDED HEREUNDER, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM + THE USE OF THIS DOCUMENT OR THE INFORMATION OR WORKS PROVIDED + HEREUNDER. + +Statement of Purpose + +The laws of most jurisdictions throughout the world automatically confer +exclusive Copyright and Related Rights (defined below) upon the creator +and subsequent owner(s) (each and all, an "owner") of an original work of +authorship and/or a database (each, a "Work"). + +Certain owners wish to permanently relinquish those rights to a Work for +the purpose of contributing to a commons of creative, cultural and +scientific works ("Commons") that the public can reliably and without fear +of later claims of infringement build upon, modify, incorporate in other +works, reuse and redistribute as freely as possible in any form whatsoever +and for any purposes, including without limitation commercial purposes. +These owners may contribute to the Commons to promote the ideal of a free +culture and the further production of creative, cultural and scientific +works, or to gain reputation or greater distribution for their Work in +part through the use and efforts of others. + +For these and/or other purposes and motivations, and without any +expectation of additional consideration or compensation, the person +associating CC0 with a Work (the "Affirmer"), to the extent that he or she +is an owner of Copyright and Related Rights in the Work, voluntarily +elects to apply CC0 to the Work and publicly distribute the Work under its +terms, with knowledge of his or her Copyright and Related Rights in the +Work and the meaning and intended legal effect of CC0 on those rights. + +1. Copyright and Related Rights. A Work made available under CC0 may be +protected by copyright and related or neighboring rights ("Copyright and +Related Rights"). Copyright and Related Rights include, but are not +limited to, the following: + + i. the right to reproduce, adapt, distribute, perform, display, + communicate, and translate a Work; + ii. moral rights retained by the original author(s) and/or performer(s); +iii. publicity and privacy rights pertaining to a person's image or + likeness depicted in a Work; + iv. rights protecting against unfair competition in regards to a Work, + subject to the limitations in paragraph 4(a), below; + v. rights protecting the extraction, dissemination, use and reuse of data + in a Work; + vi. database rights (such as those arising under Directive 96/9/EC of the + European Parliament and of the Council of 11 March 1996 on the legal + protection of databases, and under any national implementation + thereof, including any amended or successor version of such + directive); and +vii. other similar, equivalent or corresponding rights throughout the + world based on applicable law or treaty, and any national + implementations thereof. + +2. Waiver. To the greatest extent permitted by, but not in contravention +of, applicable law, Affirmer hereby overtly, fully, permanently, +irrevocably and unconditionally waives, abandons, and surrenders all of +Affirmer's Copyright and Related Rights and associated claims and causes +of action, whether now known or unknown (including existing as well as +future claims and causes of action), in the Work (i) in all territories +worldwide, (ii) for the maximum duration provided by applicable law or +treaty (including future time extensions), (iii) in any current or future +medium and for any number of copies, and (iv) for any purpose whatsoever, +including without limitation commercial, advertising or promotional +purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each +member of the public at large and to the detriment of Affirmer's heirs and +successors, fully intending that such Waiver shall not be subject to +revocation, rescission, cancellation, termination, or any other legal or +equitable action to disrupt the quiet enjoyment of the Work by the public +as contemplated by Affirmer's express Statement of Purpose. + +3. Public License Fallback. Should any part of the Waiver for any reason +be judged legally invalid or ineffective under applicable law, then the +Waiver shall be preserved to the maximum extent permitted taking into +account Affirmer's express Statement of Purpose. In addition, to the +extent the Waiver is so judged Affirmer hereby grants to each affected +person a royalty-free, non transferable, non sublicensable, non exclusive, +irrevocable and unconditional license to exercise Affirmer's Copyright and +Related Rights in the Work (i) in all territories worldwide, (ii) for the +maximum duration provided by applicable law or treaty (including future +time extensions), (iii) in any current or future medium and for any number +of copies, and (iv) for any purpose whatsoever, including without +limitation commercial, advertising or promotional purposes (the +"License"). The License shall be deemed effective as of the date CC0 was +applied by Affirmer to the Work. Should any part of the License for any +reason be judged legally invalid or ineffective under applicable law, such +partial invalidity or ineffectiveness shall not invalidate the remainder +of the License, and in such case Affirmer hereby affirms that he or she +will not (i) exercise any of his or her remaining Copyright and Related +Rights in the Work or (ii) assert any associated claims and causes of +action with respect to the Work, in either case contrary to Affirmer's +express Statement of Purpose. + +4. Limitations and Disclaimers. + + a. No trademark or patent rights held by Affirmer are waived, abandoned, + surrendered, licensed or otherwise affected by this document. + b. Affirmer offers the Work as-is and makes no representations or + warranties of any kind concerning the Work, express, implied, + statutory or otherwise, including without limitation warranties of + title, merchantability, fitness for a particular purpose, non + infringement, or the absence of latent or other defects, accuracy, or + the present or absence of errors, whether or not discoverable, all to + the greatest extent permissible under applicable law. + c. Affirmer disclaims responsibility for clearing rights of other persons + that may apply to the Work or any use thereof, including without + limitation any person's Copyright and Related Rights in the Work. + Further, Affirmer disclaims responsibility for obtaining any necessary + consents, permissions or other rights required for any use of the + Work. + d. Affirmer understands and acknowledges that Creative Commons is not a + party to this document and has no duty or obligation with respect to + this CC0 or use of the Work. diff --git a/ChangeLog b/ChangeLog new file mode 100644 --- /dev/null +++ b/ChangeLog @@ -0,0 +1,17 @@ +The changes are recorded by the version control system, Darcs. To see a log +quickly from the terminal, run: + + $ darcs changes --repo http://darcs.rel4tion.org/repos/razom-text-util + +There is also a web interface at which, among other +things, can display the history log. + +To see the log in a local clone, first get a copy of the repository if you +haven't yet: + + $ darcs get http://darcs.rel4tion.org/repos/razom-text-util + +Then move into the newly created directory and run darcs: + + $ cd razom-text-util + $ darcs changes diff --git a/INSTALL b/INSTALL new file mode 100644 --- /dev/null +++ b/INSTALL @@ -0,0 +1,13 @@ +Install from Hackage: + + $ cabal install razom-text-util + +Install from unpacked release tarball or source repo: + + $ cd razom-text-util + $ cabal install + +Just play with it without installing: + + $ cabal build + $ cabal repl diff --git a/NEWS b/NEWS new file mode 100644 --- /dev/null +++ b/NEWS @@ -0,0 +1,18 @@ +razom-text-util 0.1.0.0 -- 2015-05-05 +===================================== + +General, build and documentation changes: + +* (This is the first release, so everything is new) + +New APIs, features and enhancements: + +* (This is the first release, so everything is a new feature) + +Bug fixes: + +* (This is the first release and there's no bug tracking yet) + +Dependency changes: + +* (No dependencies at this point) diff --git a/README b/README new file mode 100644 --- /dev/null +++ b/README @@ -0,0 +1,21 @@ +This is a Haskell library of utilities for writing code that handles semantic +information documents. It is mainly intended for collecting common tools used +by the Razom information language libraries, such as language-idan and +language-kort. + +See the .cabal file for more info and link to project website the version +control. + +The official download locations are: + +- The project website +- Torrents +- Hackage + +This library is free software, and is committed to software freedom. It is +released to the public domain using the CC0 Public Domain Dedication. For the +boring "legal" details see the file 'COPYING'. + +See the file 'INSTALL' for hints on installation. The file 'ChangeLog' explains +how to see the history log of the changes done in the code. 'NEWS' provides a +friendly overview of the changes for each release. diff --git a/Setup.hs b/Setup.hs new file mode 100644 --- /dev/null +++ b/Setup.hs @@ -0,0 +1,2 @@ +import Distribution.Simple +main = defaultMain diff --git a/razom-text-util.cabal b/razom-text-util.cabal new file mode 100644 --- /dev/null +++ b/razom-text-util.cabal @@ -0,0 +1,50 @@ +name: razom-text-util +version: 0.1.0.0 +synopsis: Common text/parsing tools for Razom language packages. +description: This is a library of utilities for writing code that + handles semantic information documents (text files). It is + mainly intended for collecting common tools used by the + Razom information language libraries, but can be used by + any text related package which needs these tools. +homepage: http://rel4tion.org/projects/razom-text-util/ +bug-reports: http://rel4tion.org/projects/razom-text-util/tickets/ +license: PublicDomain +license-file: COPYING +author: fr33domlover +maintainer: fr33domlover@riseup.net +copyright: ♡ Copying is an act of love. Please copy, reuse and share. +category: Text +build-type: Simple +extra-source-files: AUTHORS ChangeLog COPYING INSTALL NEWS README +cabal-version: >=1.10 + +source-repository head + type: darcs + location: http://darcs.rel4tion.org/repos/razom-text-util/ + +library + exposed-modules: Text.Razom + , Text.Razom.Char + , Text.Razom.Number + , Text.Razom.Types + , Text.Razom.Uid + , Text.Razom.Value + -- other-modules: + -- other-extensions: + build-depends: base >=4.7 && <5 + , regex-applicative >=0.3.1 + , smaoin >=0.1.0.0 + , text-position >=0.1.0.0 + hs-source-dirs: src + default-language: Haskell2010 + +test-suite test + default-language: Haskell2010 + type: exitcode-stdio-1.0 + hs-source-dirs: test + main-is: test.hs + build-depends: base >=4.7 && <5 + , QuickCheck >=2.8 + , razom-text-util + , regex-applicative >=0.3.1 + , smaoin >=0.1.1.1 diff --git a/src/Text/Razom.hs b/src/Text/Razom.hs new file mode 100644 --- /dev/null +++ b/src/Text/Razom.hs @@ -0,0 +1,29 @@ +{- This file is part of razom-text-util. + - + - Written in 2015 by fr33domlover . + - + - ♡ Copying is an act of love. Please copy, reuse and share. + - + - The author(s) have dedicated all copyright and related and neighboring + - rights to this software to the public domain worldwide. This software is + - distributed without any warranty. + - + - You should have received a copy of the CC0 Public Domain Dedication along + - with this software. If not, see + - . + -} + +module Text.Razom + ( module Text.Razom.Char + , module Text.Razom.Number + , module Text.Razom.Types + , module Text.Razom.Uid + , module Text.Razom.Value + ) +where + +import Text.Razom.Char +import Text.Razom.Number +import Text.Razom.Types +import Text.Razom.Uid +import Text.Razom.Value diff --git a/src/Text/Razom/Char.hs b/src/Text/Razom/Char.hs new file mode 100644 --- /dev/null +++ b/src/Text/Razom/Char.hs @@ -0,0 +1,26 @@ +{- This file is part of razom-text-util. + - + - Written in 2015 by fr33domlover . + - + - ♡ Copying is an act of love. Please copy, reuse and share. + - + - The author(s) have dedicated all copyright and related and neighboring + - rights to this software to the public domain worldwide. This software is + - distributed without any warranty. + - + - You should have received a copy of the CC0 Public Domain Dedication along + - with this software. If not, see + - . + -} + +module Text.Razom.Char + ( isGraphical + ) +where + +import Data.Char + +-- | Selects Unicode characters with basic type Graphical. +isGraphical :: Char -> Bool +isGraphical c = isLetter c || isMark c || isNumber c || isPunctuation c + || isSymbol c || generalCategory c == Space diff --git a/src/Text/Razom/Number.hs b/src/Text/Razom/Number.hs new file mode 100644 --- /dev/null +++ b/src/Text/Razom/Number.hs @@ -0,0 +1,75 @@ +{- This file is part of razom-text-util. + - + - Written in 2015 by fr33domlover . + - + - ♡ Copying is an act of love. Please copy, reuse and share. + - + - The author(s) have dedicated all copyright and related and neighboring + - rights to this software to the public domain worldwide. This software is + - distributed without any warranty. + - + - You should have received a copy of the CC0 Public Domain Dedication along + - with this software. If not, see + - . + -} + +module Text.Razom.Number + ( isBinDigit + , digitsToNum + , binit + , octit + , digit + , hexit + , binary + , octal + , decimal + , hexadecimal + , signed + , fromDigits + ) +where + +import Data.Char (digitToInt, isOctDigit) +import Data.Smaoin (RealNum (..)) +import Text.Razom.Types +import Text.Regex.Applicative +import Text.Regex.Applicative.Common + +-- | Selects ASCII binary digits, i.e. @\'0\'@ and @\'1@\'. +isBinDigit :: Char -> Bool +isBinDigit c = c == '0' || c == '1' + +-- | Given the radix and an integer represented as a list of digits, construct +-- a number of the represented value. +-- +-- >>> digitsToNum 2 [1, 0, 1, 1, 0, 1] +-- 45 +-- +-- >>> digitsToNum 16 [15, 15] +-- 255 +digitsToNum :: (Num a) => a -> [a] -> a +digitsToNum radix = foldl (\ n i -> n * radix + i) 0 + +binit :: Num a => Regex a +binit = fromIntegral . digitToInt <$> psym isBinDigit + +octit :: Num a => Regex a +octit = fromIntegral . digitToInt <$> psym isOctDigit + +hexit = hexDigit + +binary :: Num a => Regex a +binary = digitsToNum 2 <$> some binit + +octal :: Num a => Regex a +octal = digitsToNum 8 <$> some octit + +-- | Read a decimal number string into a 'RealNum' value. The parameters are +-- the digits before the decimal point (integer part) and after it (fraction +-- part). Example with the number 142.857: +-- +-- >>> fromDigits [1, 4, 2] [8, 5, 7] +-- RealNum 142857 -3 +fromDigits :: [Int] -> [Int] -> RealNum +fromDigits s e = RealNum (fromIntegral $ digitsToNum 10 $ s ++ e) + (fromIntegral $ negate $ length e) diff --git a/src/Text/Razom/Types.hs b/src/Text/Razom/Types.hs new file mode 100644 --- /dev/null +++ b/src/Text/Razom/Types.hs @@ -0,0 +1,37 @@ +{- This file is part of razom-text-util. + - + - Written in 2015 by fr33domlover . + - + - ♡ Copying is an act of love. Please copy, reuse and share. + - + - The author(s) have dedicated all copyright and related and neighboring + - rights to this software to the public domain worldwide. This software is + - distributed without any warranty. + - + - You should have received a copy of the CC0 Public Domain Dedication along + - with this software. If not, see + - . + -} + +module Text.Razom.Types + ( Regex + , PosRegex + , LexError (..) + ) +where + +import Data.Position (Position, PosRE) +import Text.Regex.Applicative (RE) + +-- | A regular expression over Unicode characters. +type Regex a = RE Char a + +-- | A position-aware regular expression over Unicode characters. +type PosRegex a = PosRE Char a + +-- | An error returned when applying a regex fails. Parameters: +-- +-- (1) Location of the error +-- (2) Previous tokens parsed successfully +-- (3) The rest of the text +data LexError t = LexError Position [t] String deriving Show diff --git a/src/Text/Razom/Uid.hs b/src/Text/Razom/Uid.hs new file mode 100644 --- /dev/null +++ b/src/Text/Razom/Uid.hs @@ -0,0 +1,107 @@ +{- This file is part of razom-text-util. + - + - Written in 2015 by fr33domlover . + - + - ♡ Copying is an act of love. Please copy, reuse and share. + - + - The author(s) have dedicated all copyright and related and neighboring + - rights to this software to the public domain worldwide. This software is + - distributed without any warranty. + - + - You should have received a copy of the CC0 Public Domain Dedication along + - with this software. If not, see + - . + -} + +module Text.Razom.Uid + ( uid + , generator + , escapeUid + , unescapeUid + ) +where + +import Data.Char (isSpace) +import Text.Razom.Char +import Text.Razom.Types +import Text.Regex.Applicative + +uidcharp c = isGraphical c + && not (isSpace c) + && c /= '\\' + && c /= '>' +uidfstp c = uidcharp c && c /= '%' + +uidopen = sym '<' +uidclose = sym '>' + +-- | Regex which matches and returns a Uid, unescaped and with delimiters +-- removed. +-- +-- >>> match uid "<12<34\\>56\\\\7>" +-- Just "12<34>56\\7" +uid :: Regex String +uid = unescapeUid <$> (uidopen *> uidbody <* uidclose) + where + uidbody = snd <$> withMatched (uidfirst <* many (mkpart uidcharp)) + uidfirst = Nothing <$ string "\\%" <|> Nothing <$ mkpart uidfstp + mkpart p = Nothing <$ psym p + <|> Nothing <$ string "\\\\" + <|> Nothing <$ string "\\>" + +-- | Regex which matches and returns a Uid generator, without delimiters and +-- the leading prefix character. +-- +-- >>> match generator "<%>" +-- Just "" +-- +-- >>> match generator "<%mylabel>" +-- Just "mylabel" +generator :: Regex String +generator = (:) <$> uidopen *> sym '%' *> many (psym uidcharp) <* uidclose + +escapeUidChar :: Char -> String +escapeUidChar '\\' = "\\\\" +escapeUidChar '>' = "\\>" +escapeUidChar c = [c] + +unescapeUidChar :: String -> (Char, String) +unescapeUidChar ('\\':c:cs) = + case c of + '\\' -> ('\\', cs) + '>' -> ('>', cs) + x -> error $ "invalid escape sequence: \\" ++ [x] +unescapeUidChar (c:cs) = (c, cs) +unescapeUidChar [] = error "empty string" + +-- | Convert a Uid string into a form which can be put in a semantic document. +-- It doesn't add delimiters. +-- +-- >>> escapeUid "1>2\\3" +-- "1\\>2\\\\3" +-- +-- To prevent confusion, the above is the same as: +-- +-- > ['1', '\\', '>', '2', '\\', '\\', '3'] +escapeUid :: String -> String +escapeUid s = + case s of + ('%':cs) -> '\\' : '%' : f cs + cs -> f cs + where + f = foldr ((++) . escapeUidChar) "" + +-- | Convert a Uid as encoded in a semantic document into normal form, i.e. +-- converting escape sequences to their target characters. +-- +-- >>> unescapeUid "1\\>2\\\\3" +-- "1>2\\3" +unescapeUid :: String -> String +unescapeUid s = + case s of + [] -> [] + ('\\':'%':cs) -> '%' : f cs + cs -> f cs + where + f [] = [] + f s = let (c, cs) = unescapeUidChar s in c : f cs diff --git a/src/Text/Razom/Value.hs b/src/Text/Razom/Value.hs new file mode 100644 --- /dev/null +++ b/src/Text/Razom/Value.hs @@ -0,0 +1,55 @@ +{- This file is part of razom-text-util. + - + - Written in 2015 by fr33domlover . + - + - ♡ Copying is an act of love. Please copy, reuse and share. + - + - The author(s) have dedicated all copyright and related and neighboring + - rights to this software to the public domain worldwide. This software is + - distributed without any warranty. + - + - You should have received a copy of the CC0 Public Domain Dedication along + - with this software. If not, see + - . + -} + +module Text.Razom.Value + ( literal + , escapeMinimal + , unescapeMinimal + ) +where + +import Text.Razom.Char +import Text.Razom.Types +import Text.Regex.Applicative + +-- | Matches a delimited generic value literal and returns it, without +-- delimiters and still escaped (since unescaping may be type-specific). +-- +-- >>> match literal "{{1\\}2\\\\3}}" +-- Just "1\\}2\\\\3" +literal :: Regex String +literal = string "{{" *> lbody <* string "}}" + where + lbody = snd <$> withMatched (some lpart) + lpart = Nothing <$ lchar <|> Nothing <$ sym '\\' <* psym isGraphical + lchar = psym $ \ c -> isGraphical c && c /= '\\' && c /= '}' + +-- | Given a value literal in unescaped form, do minimal default escapes: for +-- the escape character itself (backslash) and for the closing delimiter +-- character (@'{'@). +escapeMinimal :: String -> String +escapeMinimal [] = [] +escapeMinimal ('\\':cs) = '\\' : '\\' : escapeMinimal cs +escapeMinimal ('}':cs) = '\\' : '}' : escapeMinimal cs +escapeMinimal (c:cs) = c : escapeMinimal cs + +-- | Given an escaped value literal, do minimal default unescapes: for the +-- escape character and for the closing delimiter. +unescapeMinimal :: String -> String +unescapeMinimal [] = [] +unescapeMinimal ('\\':'\\':cs) = '\\' : unescapeMinimal cs +unescapeMinimal ('\\':'}':cs) = '}' : unescapeMinimal cs +unescapeMinimal ('\\':c:cs) = '\\' : c : unescapeMinimal cs +unescapeMinimal (c:cs) = c : unescapeMinimal cs diff --git a/test/test.hs b/test/test.hs new file mode 100644 --- /dev/null +++ b/test/test.hs @@ -0,0 +1,62 @@ +{- This file is part of razom-text-util. + - + - Written in 2015 by fr33domlover . + - + - ♡ Copying is an act of love. Please copy, reuse and share. + - + - The author(s) have dedicated all copyright and related and neighboring + - rights to this software to the public domain worldwide. This software is + - distributed without any warranty. + - + - You should have received a copy of the CC0 Public Domain Dedication along + - with this software. If not, see + - . + -} + +import Control.Monad (unless) +import Data.Smaoin (RealNum (..)) +import System.Exit +import Test.QuickCheck +import Test.QuickCheck.Test +import Text.Printf +import Text.Razom +import Text.Regex.Applicative (match) + +main :: IO () +main = do + results <- mapM (\ (name ,action) -> printf "%-25s: " name >> action) tests + unless (all isSuccess results) exitFailure + +prop_digitsToNum :: Bool +prop_digitsToNum = digitsToNum 2 [1, 0, 1, 1, 0, 1] == 45 + && digitsToNum 16 [15, 15] == 255 + +prop_fromDigits :: Bool +prop_fromDigits = fromDigits [1, 4, 2] [8, 5, 7] == RealNum 142857 (-3) + +prop_uid :: Bool +prop_uid = match uid "<12<34\\>46\\\\7>" == Just "12<34>46\\7" + +prop_generator :: Bool +prop_generator = match generator "<%>" == Just "" + && match generator "<%label>" == Just "label" + +prop_escapeUid :: Bool +prop_escapeUid = escapeUid "1>2\\3" == "1\\>2\\\\3" + +prop_unescapeUid :: Bool +prop_unescapeUid = unescapeUid "1\\>2\\\\3" == "1>2\\3" + +prop_literal :: Bool +prop_literal = match literal "{{1\\}2\\\\3}}" == Just "1\\}2\\\\3" + +tests :: [(String, IO Result)] +tests = + [ ("digitsToNum", quickCheckResult prop_digitsToNum) + , ("fromDigits", quickCheckResult prop_fromDigits) + , ("uid", quickCheckResult prop_uid) + , ("generator", quickCheckResult prop_generator) + , ("escapeUid", quickCheckResult prop_escapeUid) + , ("unescapeUid", quickCheckResult prop_unescapeUid) + , ("literal", quickCheckResult prop_literal) + ]