unicode-data-names (empty) → 0.1.0
raw patch · 11 files changed
+1138/−0 lines, 11 filesdep +basedep +deepseqdep +hspec
Dependencies added: base, deepseq, hspec, tasty, tasty-bench, unicode-data, unicode-data-names
Files
- Changelog.md +5/−0
- LICENSE +249/−0
- README.md +35/−0
- bench/Main.hs +60/−0
- lib/Unicode/Char/General/Names.hs +90/−0
- lib/Unicode/Internal/Char/UnicodeData/DerivedName.hs too large to diff
- lib/Unicode/Internal/Char/UnicodeData/NameAliases.hs +443/−0
- test/Main.hs +1/−0
- test/Unicode/Char/General/NamesSpec.hs +83/−0
- test/export_all_chars.hs +42/−0
- unicode-data-names.cabal +130/−0
+ Changelog.md view
@@ -0,0 +1,5 @@+# Changelog++## 0.1.0 (June 2022)++- Initial release
+ LICENSE view
@@ -0,0 +1,249 @@++ Apache License+ Version 2.0, January 2004+ http://www.apache.org/licenses/++ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION++ 1. Definitions.++ "License" shall mean the terms and conditions for use, reproduction,+ and distribution as defined by Sections 1 through 9 of this document.++ "Licensor" shall mean the copyright owner or entity authorized by+ the copyright owner that is granting the License.++ "Legal Entity" shall mean the union of the acting entity and all+ other entities that control, are controlled by, or are under common+ control with that entity. For the purposes of this definition,+ "control" means (i) the power, direct or indirect, to cause the+ direction or management of such entity, whether by contract or+ otherwise, or (ii) ownership of fifty percent (50%) or more of the+ outstanding shares, or (iii) beneficial ownership of such entity.++ "You" (or "Your") shall mean an individual or Legal Entity+ exercising permissions granted by this License.++ "Source" form shall mean the preferred form for making modifications,+ including but not limited to software source code, documentation+ source, and configuration files.++ "Object" form shall mean any form resulting from mechanical+ transformation or translation of a Source form, including but+ not limited to compiled object code, generated documentation,+ and conversions to other media types.++ "Work" shall mean the work of authorship, whether in Source or+ Object form, made available under the License, as indicated by a+ copyright notice that is included in or attached to the work.++ "Derivative Works" shall mean any work, whether in Source or Object+ form, that is based on (or derived from) the Work and for which the+ editorial revisions, annotations, elaborations, or other modifications+ represent, as a whole, an original work of authorship. For the purposes+ of this License, Derivative Works shall not include works that remain+ separable from, or merely link (or bind by name) to the interfaces of,+ the Work and Derivative Works thereof.++ "Contribution" shall mean any work of authorship, including+ the original version of the Work and any modifications or additions+ to that Work or Derivative Works thereof, that is intentionally+ submitted to Licensor for inclusion in the Work by the copyright owner+ or by an individual or Legal Entity authorized to submit on behalf of+ the copyright owner. For the purposes of this definition, "submitted"+ means any form of electronic, verbal, or written communication sent+ to the Licensor or its representatives, including but not limited to+ communication on electronic mailing lists, source code control systems,+ and issue tracking systems that are managed by, or on behalf of, the+ Licensor for the purpose of discussing and improving the Work, but+ excluding communication that is conspicuously marked or otherwise+ designated in writing by the copyright owner as "Not a Contribution."++ "Contributor" shall mean Licensor and any individual or Legal Entity+ on behalf of whom a Contribution has been received by Licensor and+ subsequently incorporated within the Work.++ 2. Grant of Copyright License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ copyright license to reproduce, prepare Derivative Works of,+ publicly display, publicly perform, sublicense, and distribute the+ Work and such Derivative Works in Source or Object form.++ 3. Grant of Patent License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ (except as stated in this section) patent license to make, have made,+ use, offer to sell, sell, import, and otherwise transfer the Work,+ where such license applies only to those patent claims licensable+ by such Contributor that are necessarily infringed by their+ Contribution(s) alone or by combination of their Contribution(s)+ with the Work to which such Contribution(s) was submitted. If You+ institute patent litigation against any entity (including a+ cross-claim or counterclaim in a lawsuit) alleging that the Work+ or a Contribution incorporated within the Work constitutes direct+ or contributory patent infringement, then any patent licenses+ granted to You under this License for that Work shall terminate+ as of the date such litigation is filed.++ 4. Redistribution. You may reproduce and distribute copies of the+ Work or Derivative Works thereof in any medium, with or without+ modifications, and in Source or Object form, provided that You+ meet the following conditions:++ (a) You must give any other recipients of the Work or+ Derivative Works a copy of this License; and++ (b) You must cause any modified files to carry prominent notices+ stating that You changed the files; and++ (c) You must retain, in the Source form of any Derivative Works+ that You distribute, all copyright, patent, trademark, and+ attribution notices from the Source form of the Work,+ excluding those notices that do not pertain to any part of+ the Derivative Works; and++ (d) If the Work includes a "NOTICE" text file as part of its+ distribution, then any Derivative Works that You distribute must+ include a readable copy of the attribution notices contained+ within such NOTICE file, excluding those notices that do not+ pertain to any part of the Derivative Works, in at least one+ of the following places: within a NOTICE text file distributed+ as part of the Derivative Works; within the Source form or+ documentation, if provided along with the Derivative Works; or,+ within a display generated by the Derivative Works, if and+ wherever such third-party notices normally appear. The contents+ of the NOTICE file are for informational purposes only and+ do not modify the License. You may add Your own attribution+ notices within Derivative Works that You distribute, alongside+ or as an addendum to the NOTICE text from the Work, provided+ that such additional attribution notices cannot be construed+ as modifying the License.++ You may add Your own copyright statement to Your modifications and+ may provide additional or different license terms and conditions+ for use, reproduction, or distribution of Your modifications, or+ for any such Derivative Works as a whole, provided Your use,+ reproduction, and distribution of the Work otherwise complies with+ the conditions stated in this License.++ 5. Submission of Contributions. Unless You explicitly state otherwise,+ any Contribution intentionally submitted for inclusion in the Work+ by You to the Licensor shall be under the terms and conditions of+ this License, without any additional terms or conditions.+ Notwithstanding the above, nothing herein shall supersede or modify+ the terms of any separate license agreement you may have executed+ with Licensor regarding such Contributions.++ 6. Trademarks. This License does not grant permission to use the trade+ names, trademarks, service marks, or product names of the Licensor,+ except as required for reasonable and customary use in describing the+ origin of the Work and reproducing the content of the NOTICE file.++ 7. Disclaimer of Warranty. Unless required by applicable law or+ agreed to in writing, Licensor provides the Work (and each+ Contributor provides its Contributions) on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or+ implied, including, without limitation, any warranties or conditions+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A+ PARTICULAR PURPOSE. You are solely responsible for determining the+ appropriateness of using or redistributing the Work and assume any+ risks associated with Your exercise of permissions under this License.++ 8. Limitation of Liability. In no event and under no legal theory,+ whether in tort (including negligence), contract, or otherwise,+ unless required by applicable law (such as deliberate and grossly+ negligent acts) or agreed to in writing, shall any Contributor be+ liable to You for damages, including any direct, indirect, special,+ incidental, or consequential damages of any character arising as a+ result of this License or out of the use or inability to use the+ Work (including but not limited to damages for loss of goodwill,+ work stoppage, computer failure or malfunction, or any and all+ other commercial damages or losses), even if such Contributor+ has been advised of the possibility of such damages.++ 9. Accepting Warranty or Additional Liability. While redistributing+ the Work or Derivative Works thereof, You may choose to offer,+ and charge a fee for, acceptance of support, warranty, indemnity,+ or other liability obligations and/or rights consistent with this+ License. However, in accepting such obligations, You may act only+ on Your own behalf and on Your sole responsibility, not on behalf+ of any other Contributor, and only if You agree to indemnify,+ defend, and hold each Contributor harmless for any liability+ incurred by, or claims asserted against, such Contributor by reason+ of your accepting any such warranty or additional liability.++ END OF TERMS AND CONDITIONS++-------------------------------------------------------------------------------+This distribution includes portions of code from the "unicode-transforms"+package (https://github.com/composewell/unicode-transforms/) which is+available under BSD-3-Clause license as described below.+-------------------------------------------------------------------------------++Copyright (c) 2016, Harendra Kumar+All rights reserved.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++1. Redistributions of source code must retain the above copyright notice, this+list of conditions and the following disclaimer.++2. Redistributions in binary form must reproduce the above copyright notice,+this list of conditions and the following disclaimer in the documentation+and/or other materials provided with the distribution.++3. Neither the name of the copyright holder nor the names of its contributors+may be used to endorse or promote products derived from this software without+specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.++-------------------------------------------------------------------------------+This distribution includes portions of code from the "unicode-transforms"+package (https://github.com/composewell/unicode-transforms/)+which included portions of code from the "prose"+(https://github.com/llelf/prose) package available under BSD-3-Clause+license as described below.+-------------------------------------------------------------------------------++Copyright (c) 2014–2015, Antonio Nikishaev++All rights reserved.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++ * Redistributions of source code must retain the above copyright+ notice, this list of conditions and the following disclaimer.++ * Redistributions in binary form must reproduce the above+ copyright notice, this list of conditions and the following+ disclaimer in the documentation and/or other materials provided+ with the distribution.++ * Neither the name of Antonio Nikishaev nor the names of other+ contributors may be used to endorse or promote products derived+ from this software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ README.md view
@@ -0,0 +1,35 @@+# README++`unicode-data-names` provides Haskell APIs to efficiently access the Unicode+character names and aliases from the+[Unicode character database](https://www.unicode.org/ucd/).++The Haskell data structures are generated programmatically from the+Unicode character database (UCD) files. The latest Unicode version+supported by this library is+[`14.0.0`](https://www.unicode.org/versions/Unicode14.0.0/).++Please see the+[Haddock documentation](https://hackage.haskell.org/package/unicode-data-names)+for reference documentation.++## Comparing with Python++In order to check Unicode implementation in Haskell, we compare the results obtained+with Python.++__Warning:__ A Python version with the _exact same Unicode version_ is required.++```bash+cabal run -f "export-all-chars" -v0 export-all-chars > ./test/all_chars.csv+python3 ./test/check.py -v ./test/all_chars.csv+```++## Licensing++`unicode-data-names` is an [open source](https://github.com/composewell/unicode-data)+project available under a liberal [Apache-2.0 license](LICENSE).++## Contributing++As an open project we welcome contributions.
+ bench/Main.hs view
@@ -0,0 +1,60 @@+{-# LANGUAGE ExistentialQuantification #-}++import Control.DeepSeq (NFData, deepseq)+import Data.Ix (Ix(..))+import Test.Tasty.Bench (Benchmark, bgroup, bcompare, bench, nf, defaultMain)++import qualified Unicode.Char.General.Names as Names+import qualified Unicode.Internal.Char.UnicodeData.DerivedName as DerivedName+import qualified Unicode.Internal.Char.UnicodeData.NameAliases as NameAliases++-- | A unit benchmark+data Bench = forall a. (NFData a) => Bench+ { _title :: !String -- ^ Name+ , _func :: Char -> a -- ^ Function to benchmark+ }++main :: IO ()+main = defaultMain+ [ bgroup "Unicode.Char.General.Names"+ -- Character classification+ [ bgroup' "name"+ [ Bench "CString" DerivedName.name+ , Bench "String" Names.name+ ]+ , bgroup' "correctedName"+ [ Bench "String" Names.correctedName+ ]+ , bgroup' "nameOrAlias"+ [ Bench "String" Names.name+ ]+ , bgroup' "nameAliasesByType"+ [ Bench "CString"+ (\c -> (`NameAliases.nameAliasesByType` c) <$> [minBound..maxBound])+ , Bench "String"+ (\c -> (`Names.nameAliasesByType` c) <$> [minBound..maxBound])+ ]+ , bgroup' "nameAliasesWithTypes"+ [ Bench "CString" (show . NameAliases.nameAliasesWithTypes)+ , Bench "String" (show . Names.nameAliasesWithTypes)+ ]+ ]+ ]+ where+ bgroup' groupTitle bs = bgroup groupTitle+ [ benchNF' groupTitle title f+ | Bench title f <- bs+ ]++ -- [NOTE] Works if groupTitle uniquely identifies the benchmark group.+ benchNF' groupTitle title = case title of+ "CString" -> benchNF title+ _ ->+ bcompare ("$NF == \"CString\" && $(NF-1) == \"" ++ groupTitle ++ "\"")+ . benchNF title++ benchNF :: forall a. (NFData a) => String -> (Char -> a) -> Benchmark+ benchNF t f = bench t $ nf (fold_ f) (minBound, maxBound)++ fold_ :: forall a. (NFData a) => (Char -> a) -> (Char, Char) -> ()+ fold_ f = foldr (deepseq . f) () . range
+ lib/Unicode/Char/General/Names.hs view
@@ -0,0 +1,90 @@+-- |+-- Module : Unicode.Char.General.Names+-- Copyright : (c) 2022 Composewell Technologies and Contributors+-- License : Apache-2.0+-- Maintainer : streamly@composewell.com+-- Stability : experimental+--+-- Unicode character names and name aliases.+-- See Unicode standard 14.0.0, section 4.8.+--+-- @since 0.1.0++module Unicode.Char.General.Names+ ( -- * Name+ name+ , nameOrAlias+ , correctedName+ -- * Name Aliases+ , NameAliases.NameAliasType(..)+ , nameAliases+ , nameAliasesByType+ , nameAliasesWithTypes+ ) where++import Control.Applicative ((<|>))+import Data.Maybe (listToMaybe)+import Foreign.C.String (CString, peekCAString)+import System.IO.Unsafe (unsafePerformIO)++import qualified Unicode.Internal.Char.UnicodeData.DerivedName as DerivedName+import qualified Unicode.Internal.Char.UnicodeData.NameAliases as NameAliases++-- | Name of a character, if defined.+--+-- @since 0.1.0+{-# INLINE name #-}+name :: Char -> Maybe String+name = fmap unpack . DerivedName.name++-- | Returns /corrected/ name of a character (see 'NameAliases.Correction'),+-- if defined, otherwise returns its original 'name' if defined.+--+-- @since 0.1.0+{-# INLINE correctedName #-}+correctedName :: Char -> Maybe String+correctedName c =+ listToMaybe (nameAliasesByType NameAliases.Correction c) <|> name c++-- | Returns a character’s 'name' if defined,+-- otherwise returns its first name alias if defined.+--+-- @since 0.1.0+nameOrAlias :: Char -> Maybe String+nameOrAlias c = name c <|> case NameAliases.nameAliasesWithTypes c of+ (_, n:_):_ -> Just (unpack n)+ _ -> Nothing++-- | All name aliases of a character, if defined.+-- The names are listed in the original order of the UCD.+--+-- See 'nameAliasesWithTypes' for the detailed list by alias type.+--+-- @since 0.1.0+{-# INLINE nameAliases #-}+nameAliases :: Char -> [String]+nameAliases = fmap unpack . NameAliases.nameAliases++-- | Name aliases of a character for a specific name alias type.+--+-- @since 0.1.0+{-# INLINE nameAliasesByType #-}+nameAliasesByType :: NameAliases.NameAliasType -> Char -> [String]+nameAliasesByType t = fmap unpack . NameAliases.nameAliasesByType t++-- | Detailed character names aliases.+-- The names are listed in the original order of the UCD.+--+-- See 'nameAliases' if the alias type is not required.+--+-- @since 0.1.0+{-# INLINE nameAliasesWithTypes #-}+nameAliasesWithTypes :: Char -> [(NameAliases.NameAliasType, [String])]+nameAliasesWithTypes+ = fmap (fmap (fmap unpack))+ . NameAliases.nameAliasesWithTypes++-- Note: names are ASCII. See Unicode Standard 14.0.0, section 4.8.+{-# INLINE unpack #-}+unpack :: CString -> String+unpack = unsafePerformIO . peekCAString
+ lib/Unicode/Internal/Char/UnicodeData/DerivedName.hs view
file too large to diff
+ lib/Unicode/Internal/Char/UnicodeData/NameAliases.hs view
@@ -0,0 +1,443 @@+-- autogenerated from https://www.unicode.org/Public/14.0.0/ucd/NameAliases.txt+-- |+-- Module : Unicode.Internal.Char.UnicodeData.NameAliases+-- Copyright : (c) 2022 Composewell Technologies and Contributors+-- License : Apache-2.0+-- Maintainer : streamly@composewell.com+-- Stability : experimental++{-# OPTIONS_HADDOCK hide #-}++module Unicode.Internal.Char.UnicodeData.NameAliases+(NameAliasType(..), nameAliases, nameAliasesByType, nameAliasesWithTypes)+where++import Data.Ix (Ix)+import Data.Maybe (fromMaybe)+import Foreign.C.String (CString)+import GHC.Exts (Ptr(..))++-- | Type of name alias. See Unicode Standard 14.0.0, section 4.8.+--+-- @since 0.1.0+data NameAliasType+ = Correction+ -- ^ Corrections for serious problems in the character names.+ | Control+ -- ^ ISO 6429 names for @C0@ and @C1@ control functions, and other+ -- commonly occurring names for control codes.+ | Alternate+ -- ^ A few widely used alternate names for format characters.+ | Figment+ -- ^ Several documented labels for @C1@ control code points which+ -- were never actually approved in any standard.+ | Abbreviation+ -- ^ Commonly occurring abbreviations (or acronyms) for control codes,+ -- format characters, spaces, and variation selectors.+ deriving (Enum, Bounded, Eq, Ord, Ix, Show)++-- | All name aliases of a character.+-- The names are listed in the original order of the UCD.+--+-- See 'nameAliasesWithTypes' for the detailed list by alias type.+--+-- @since 0.1.0+{-# INLINE nameAliases #-}+nameAliases :: Char -> [CString]+nameAliases = mconcat . fmap snd . nameAliasesWithTypes++-- | Name aliases of a character for a specific name alias type.+--+-- @since 0.1.0+{-# INLINE nameAliasesByType #-}+nameAliasesByType :: NameAliasType -> Char -> [CString]+nameAliasesByType t = fromMaybe mempty . lookup t . nameAliasesWithTypes++-- | Detailed character names aliases.+-- The names are listed in the original order of the UCD.+--+-- See 'nameAliases' if the alias type is not required.+--+-- @since 0.1.0+nameAliasesWithTypes :: Char -> [(NameAliasType, [CString])]+nameAliasesWithTypes = \case+ '\x0000' -> [(Control,[Ptr "NULL\0"#]),(Abbreviation,[Ptr "NUL\0"#])]+ '\x0001' -> [(Control,[Ptr "START OF HEADING\0"#]),(Abbreviation,[Ptr "SOH\0"#])]+ '\x0002' -> [(Control,[Ptr "START OF TEXT\0"#]),(Abbreviation,[Ptr "STX\0"#])]+ '\x0003' -> [(Control,[Ptr "END OF TEXT\0"#]),(Abbreviation,[Ptr "ETX\0"#])]+ '\x0004' -> [(Control,[Ptr "END OF TRANSMISSION\0"#]),(Abbreviation,[Ptr "EOT\0"#])]+ '\x0005' -> [(Control,[Ptr "ENQUIRY\0"#]),(Abbreviation,[Ptr "ENQ\0"#])]+ '\x0006' -> [(Control,[Ptr "ACKNOWLEDGE\0"#]),(Abbreviation,[Ptr "ACK\0"#])]+ '\x0007' -> [(Control,[Ptr "ALERT\0"#]),(Abbreviation,[Ptr "BEL\0"#])]+ '\x0008' -> [(Control,[Ptr "BACKSPACE\0"#]),(Abbreviation,[Ptr "BS\0"#])]+ '\x0009' -> [(Control,[Ptr "CHARACTER TABULATION\0"#,Ptr "HORIZONTAL TABULATION\0"#]),(Abbreviation,[Ptr "HT\0"#,Ptr "TAB\0"#])]+ '\x000a' -> [(Control,[Ptr "LINE FEED\0"#,Ptr "NEW LINE\0"#,Ptr "END OF LINE\0"#]),(Abbreviation,[Ptr "LF\0"#,Ptr "NL\0"#,Ptr "EOL\0"#])]+ '\x000b' -> [(Control,[Ptr "LINE TABULATION\0"#,Ptr "VERTICAL TABULATION\0"#]),(Abbreviation,[Ptr "VT\0"#])]+ '\x000c' -> [(Control,[Ptr "FORM FEED\0"#]),(Abbreviation,[Ptr "FF\0"#])]+ '\x000d' -> [(Control,[Ptr "CARRIAGE RETURN\0"#]),(Abbreviation,[Ptr "CR\0"#])]+ '\x000e' -> [(Control,[Ptr "SHIFT OUT\0"#,Ptr "LOCKING-SHIFT ONE\0"#]),(Abbreviation,[Ptr "SO\0"#])]+ '\x000f' -> [(Control,[Ptr "SHIFT IN\0"#,Ptr "LOCKING-SHIFT ZERO\0"#]),(Abbreviation,[Ptr "SI\0"#])]+ '\x0010' -> [(Control,[Ptr "DATA LINK ESCAPE\0"#]),(Abbreviation,[Ptr "DLE\0"#])]+ '\x0011' -> [(Control,[Ptr "DEVICE CONTROL ONE\0"#]),(Abbreviation,[Ptr "DC1\0"#])]+ '\x0012' -> [(Control,[Ptr "DEVICE CONTROL TWO\0"#]),(Abbreviation,[Ptr "DC2\0"#])]+ '\x0013' -> [(Control,[Ptr "DEVICE CONTROL THREE\0"#]),(Abbreviation,[Ptr "DC3\0"#])]+ '\x0014' -> [(Control,[Ptr "DEVICE CONTROL FOUR\0"#]),(Abbreviation,[Ptr "DC4\0"#])]+ '\x0015' -> [(Control,[Ptr "NEGATIVE ACKNOWLEDGE\0"#]),(Abbreviation,[Ptr "NAK\0"#])]+ '\x0016' -> [(Control,[Ptr "SYNCHRONOUS IDLE\0"#]),(Abbreviation,[Ptr "SYN\0"#])]+ '\x0017' -> [(Control,[Ptr "END OF TRANSMISSION BLOCK\0"#]),(Abbreviation,[Ptr "ETB\0"#])]+ '\x0018' -> [(Control,[Ptr "CANCEL\0"#]),(Abbreviation,[Ptr "CAN\0"#])]+ '\x0019' -> [(Control,[Ptr "END OF MEDIUM\0"#]),(Abbreviation,[Ptr "EOM\0"#])]+ '\x001a' -> [(Control,[Ptr "SUBSTITUTE\0"#]),(Abbreviation,[Ptr "SUB\0"#])]+ '\x001b' -> [(Control,[Ptr "ESCAPE\0"#]),(Abbreviation,[Ptr "ESC\0"#])]+ '\x001c' -> [(Control,[Ptr "INFORMATION SEPARATOR FOUR\0"#,Ptr "FILE SEPARATOR\0"#]),(Abbreviation,[Ptr "FS\0"#])]+ '\x001d' -> [(Control,[Ptr "INFORMATION SEPARATOR THREE\0"#,Ptr "GROUP SEPARATOR\0"#]),(Abbreviation,[Ptr "GS\0"#])]+ '\x001e' -> [(Control,[Ptr "INFORMATION SEPARATOR TWO\0"#,Ptr "RECORD SEPARATOR\0"#]),(Abbreviation,[Ptr "RS\0"#])]+ '\x001f' -> [(Control,[Ptr "INFORMATION SEPARATOR ONE\0"#,Ptr "UNIT SEPARATOR\0"#]),(Abbreviation,[Ptr "US\0"#])]+ '\x0020' -> [(Abbreviation,[Ptr "SP\0"#])]+ '\x007f' -> [(Control,[Ptr "DELETE\0"#]),(Abbreviation,[Ptr "DEL\0"#])]+ '\x0080' -> [(Figment,[Ptr "PADDING CHARACTER\0"#]),(Abbreviation,[Ptr "PAD\0"#])]+ '\x0081' -> [(Figment,[Ptr "HIGH OCTET PRESET\0"#]),(Abbreviation,[Ptr "HOP\0"#])]+ '\x0082' -> [(Control,[Ptr "BREAK PERMITTED HERE\0"#]),(Abbreviation,[Ptr "BPH\0"#])]+ '\x0083' -> [(Control,[Ptr "NO BREAK HERE\0"#]),(Abbreviation,[Ptr "NBH\0"#])]+ '\x0084' -> [(Control,[Ptr "INDEX\0"#]),(Abbreviation,[Ptr "IND\0"#])]+ '\x0085' -> [(Control,[Ptr "NEXT LINE\0"#]),(Abbreviation,[Ptr "NEL\0"#])]+ '\x0086' -> [(Control,[Ptr "START OF SELECTED AREA\0"#]),(Abbreviation,[Ptr "SSA\0"#])]+ '\x0087' -> [(Control,[Ptr "END OF SELECTED AREA\0"#]),(Abbreviation,[Ptr "ESA\0"#])]+ '\x0088' -> [(Control,[Ptr "CHARACTER TABULATION SET\0"#,Ptr "HORIZONTAL TABULATION SET\0"#]),(Abbreviation,[Ptr "HTS\0"#])]+ '\x0089' -> [(Control,[Ptr "CHARACTER TABULATION WITH JUSTIFICATION\0"#,Ptr "HORIZONTAL TABULATION WITH JUSTIFICATION\0"#]),(Abbreviation,[Ptr "HTJ\0"#])]+ '\x008a' -> [(Control,[Ptr "LINE TABULATION SET\0"#,Ptr "VERTICAL TABULATION SET\0"#]),(Abbreviation,[Ptr "VTS\0"#])]+ '\x008b' -> [(Control,[Ptr "PARTIAL LINE FORWARD\0"#,Ptr "PARTIAL LINE DOWN\0"#]),(Abbreviation,[Ptr "PLD\0"#])]+ '\x008c' -> [(Control,[Ptr "PARTIAL LINE BACKWARD\0"#,Ptr "PARTIAL LINE UP\0"#]),(Abbreviation,[Ptr "PLU\0"#])]+ '\x008d' -> [(Control,[Ptr "REVERSE LINE FEED\0"#,Ptr "REVERSE INDEX\0"#]),(Abbreviation,[Ptr "RI\0"#])]+ '\x008e' -> [(Control,[Ptr "SINGLE SHIFT TWO\0"#,Ptr "SINGLE-SHIFT-2\0"#]),(Abbreviation,[Ptr "SS2\0"#])]+ '\x008f' -> [(Control,[Ptr "SINGLE SHIFT THREE\0"#,Ptr "SINGLE-SHIFT-3\0"#]),(Abbreviation,[Ptr "SS3\0"#])]+ '\x0090' -> [(Control,[Ptr "DEVICE CONTROL STRING\0"#]),(Abbreviation,[Ptr "DCS\0"#])]+ '\x0091' -> [(Control,[Ptr "PRIVATE USE ONE\0"#,Ptr "PRIVATE USE-1\0"#]),(Abbreviation,[Ptr "PU1\0"#])]+ '\x0092' -> [(Control,[Ptr "PRIVATE USE TWO\0"#,Ptr "PRIVATE USE-2\0"#]),(Abbreviation,[Ptr "PU2\0"#])]+ '\x0093' -> [(Control,[Ptr "SET TRANSMIT STATE\0"#]),(Abbreviation,[Ptr "STS\0"#])]+ '\x0094' -> [(Control,[Ptr "CANCEL CHARACTER\0"#]),(Abbreviation,[Ptr "CCH\0"#])]+ '\x0095' -> [(Control,[Ptr "MESSAGE WAITING\0"#]),(Abbreviation,[Ptr "MW\0"#])]+ '\x0096' -> [(Control,[Ptr "START OF GUARDED AREA\0"#,Ptr "START OF PROTECTED AREA\0"#]),(Abbreviation,[Ptr "SPA\0"#])]+ '\x0097' -> [(Control,[Ptr "END OF GUARDED AREA\0"#,Ptr "END OF PROTECTED AREA\0"#]),(Abbreviation,[Ptr "EPA\0"#])]+ '\x0098' -> [(Control,[Ptr "START OF STRING\0"#]),(Abbreviation,[Ptr "SOS\0"#])]+ '\x0099' -> [(Figment,[Ptr "SINGLE GRAPHIC CHARACTER INTRODUCER\0"#]),(Abbreviation,[Ptr "SGC\0"#])]+ '\x009a' -> [(Control,[Ptr "SINGLE CHARACTER INTRODUCER\0"#]),(Abbreviation,[Ptr "SCI\0"#])]+ '\x009b' -> [(Control,[Ptr "CONTROL SEQUENCE INTRODUCER\0"#]),(Abbreviation,[Ptr "CSI\0"#])]+ '\x009c' -> [(Control,[Ptr "STRING TERMINATOR\0"#]),(Abbreviation,[Ptr "ST\0"#])]+ '\x009d' -> [(Control,[Ptr "OPERATING SYSTEM COMMAND\0"#]),(Abbreviation,[Ptr "OSC\0"#])]+ '\x009e' -> [(Control,[Ptr "PRIVACY MESSAGE\0"#]),(Abbreviation,[Ptr "PM\0"#])]+ '\x009f' -> [(Control,[Ptr "APPLICATION PROGRAM COMMAND\0"#]),(Abbreviation,[Ptr "APC\0"#])]+ '\x00a0' -> [(Abbreviation,[Ptr "NBSP\0"#])]+ '\x00ad' -> [(Abbreviation,[Ptr "SHY\0"#])]+ '\x01a2' -> [(Correction,[Ptr "LATIN CAPITAL LETTER GHA\0"#])]+ '\x01a3' -> [(Correction,[Ptr "LATIN SMALL LETTER GHA\0"#])]+ '\x034f' -> [(Abbreviation,[Ptr "CGJ\0"#])]+ '\x061c' -> [(Abbreviation,[Ptr "ALM\0"#])]+ '\x0709' -> [(Correction,[Ptr "SYRIAC SUBLINEAR COLON SKEWED LEFT\0"#])]+ '\x0cde' -> [(Correction,[Ptr "KANNADA LETTER LLLA\0"#])]+ '\x0e9d' -> [(Correction,[Ptr "LAO LETTER FO FON\0"#])]+ '\x0e9f' -> [(Correction,[Ptr "LAO LETTER FO FAY\0"#])]+ '\x0ea3' -> [(Correction,[Ptr "LAO LETTER RO\0"#])]+ '\x0ea5' -> [(Correction,[Ptr "LAO LETTER LO\0"#])]+ '\x0fd0' -> [(Correction,[Ptr "TIBETAN MARK BKA- SHOG GI MGO RGYAN\0"#])]+ '\x11ec' -> [(Correction,[Ptr "HANGUL JONGSEONG YESIEUNG-KIYEOK\0"#])]+ '\x11ed' -> [(Correction,[Ptr "HANGUL JONGSEONG YESIEUNG-SSANGKIYEOK\0"#])]+ '\x11ee' -> [(Correction,[Ptr "HANGUL JONGSEONG SSANGYESIEUNG\0"#])]+ '\x11ef' -> [(Correction,[Ptr "HANGUL JONGSEONG YESIEUNG-KHIEUKH\0"#])]+ '\x180b' -> [(Abbreviation,[Ptr "FVS1\0"#])]+ '\x180c' -> [(Abbreviation,[Ptr "FVS2\0"#])]+ '\x180d' -> [(Abbreviation,[Ptr "FVS3\0"#])]+ '\x180e' -> [(Abbreviation,[Ptr "MVS\0"#])]+ '\x180f' -> [(Abbreviation,[Ptr "FVS4\0"#])]+ '\x200b' -> [(Abbreviation,[Ptr "ZWSP\0"#])]+ '\x200c' -> [(Abbreviation,[Ptr "ZWNJ\0"#])]+ '\x200d' -> [(Abbreviation,[Ptr "ZWJ\0"#])]+ '\x200e' -> [(Abbreviation,[Ptr "LRM\0"#])]+ '\x200f' -> [(Abbreviation,[Ptr "RLM\0"#])]+ '\x202a' -> [(Abbreviation,[Ptr "LRE\0"#])]+ '\x202b' -> [(Abbreviation,[Ptr "RLE\0"#])]+ '\x202c' -> [(Abbreviation,[Ptr "PDF\0"#])]+ '\x202d' -> [(Abbreviation,[Ptr "LRO\0"#])]+ '\x202e' -> [(Abbreviation,[Ptr "RLO\0"#])]+ '\x202f' -> [(Abbreviation,[Ptr "NNBSP\0"#])]+ '\x205f' -> [(Abbreviation,[Ptr "MMSP\0"#])]+ '\x2060' -> [(Abbreviation,[Ptr "WJ\0"#])]+ '\x2066' -> [(Abbreviation,[Ptr "LRI\0"#])]+ '\x2067' -> [(Abbreviation,[Ptr "RLI\0"#])]+ '\x2068' -> [(Abbreviation,[Ptr "FSI\0"#])]+ '\x2069' -> [(Abbreviation,[Ptr "PDI\0"#])]+ '\x2118' -> [(Correction,[Ptr "WEIERSTRASS ELLIPTIC FUNCTION\0"#])]+ '\x2448' -> [(Correction,[Ptr "MICR ON US SYMBOL\0"#])]+ '\x2449' -> [(Correction,[Ptr "MICR DASH SYMBOL\0"#])]+ '\x2b7a' -> [(Correction,[Ptr "LEFTWARDS TRIANGLE-HEADED ARROW WITH DOUBLE VERTICAL STROKE\0"#])]+ '\x2b7c' -> [(Correction,[Ptr "RIGHTWARDS TRIANGLE-HEADED ARROW WITH DOUBLE VERTICAL STROKE\0"#])]+ '\xa015' -> [(Correction,[Ptr "YI SYLLABLE ITERATION MARK\0"#])]+ '\xaa6e' -> [(Correction,[Ptr "MYANMAR LETTER KHAMTI LLA\0"#])]+ '\xfe00' -> [(Abbreviation,[Ptr "VS1\0"#])]+ '\xfe01' -> [(Abbreviation,[Ptr "VS2\0"#])]+ '\xfe02' -> [(Abbreviation,[Ptr "VS3\0"#])]+ '\xfe03' -> [(Abbreviation,[Ptr "VS4\0"#])]+ '\xfe04' -> [(Abbreviation,[Ptr "VS5\0"#])]+ '\xfe05' -> [(Abbreviation,[Ptr "VS6\0"#])]+ '\xfe06' -> [(Abbreviation,[Ptr "VS7\0"#])]+ '\xfe07' -> [(Abbreviation,[Ptr "VS8\0"#])]+ '\xfe08' -> [(Abbreviation,[Ptr "VS9\0"#])]+ '\xfe09' -> [(Abbreviation,[Ptr "VS10\0"#])]+ '\xfe0a' -> [(Abbreviation,[Ptr "VS11\0"#])]+ '\xfe0b' -> [(Abbreviation,[Ptr "VS12\0"#])]+ '\xfe0c' -> [(Abbreviation,[Ptr "VS13\0"#])]+ '\xfe0d' -> [(Abbreviation,[Ptr "VS14\0"#])]+ '\xfe0e' -> [(Abbreviation,[Ptr "VS15\0"#])]+ '\xfe0f' -> [(Abbreviation,[Ptr "VS16\0"#])]+ '\xfe18' -> [(Correction,[Ptr "PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRACKET\0"#])]+ '\xfeff' -> [(Alternate,[Ptr "BYTE ORDER MARK\0"#]),(Abbreviation,[Ptr "BOM\0"#,Ptr "ZWNBSP\0"#])]+ '\x122d4' -> [(Correction,[Ptr "CUNEIFORM SIGN NU11 TENU\0"#])]+ '\x122d5' -> [(Correction,[Ptr "CUNEIFORM SIGN NU11 OVER NU11 BUR OVER BUR\0"#])]+ '\x16e56' -> [(Correction,[Ptr "MEDEFAIDRIN CAPITAL LETTER H\0"#])]+ '\x16e57' -> [(Correction,[Ptr "MEDEFAIDRIN CAPITAL LETTER NG\0"#])]+ '\x16e76' -> [(Correction,[Ptr "MEDEFAIDRIN SMALL LETTER H\0"#])]+ '\x16e77' -> [(Correction,[Ptr "MEDEFAIDRIN SMALL LETTER NG\0"#])]+ '\x1b001' -> [(Correction,[Ptr "HENTAIGANA LETTER E-1\0"#])]+ '\x1d0c5' -> [(Correction,[Ptr "BYZANTINE MUSICAL SYMBOL FTHORA SKLIRON CHROMA VASIS\0"#])]+ '\xe0100' -> [(Abbreviation,[Ptr "VS17\0"#])]+ '\xe0101' -> [(Abbreviation,[Ptr "VS18\0"#])]+ '\xe0102' -> [(Abbreviation,[Ptr "VS19\0"#])]+ '\xe0103' -> [(Abbreviation,[Ptr "VS20\0"#])]+ '\xe0104' -> [(Abbreviation,[Ptr "VS21\0"#])]+ '\xe0105' -> [(Abbreviation,[Ptr "VS22\0"#])]+ '\xe0106' -> [(Abbreviation,[Ptr "VS23\0"#])]+ '\xe0107' -> [(Abbreviation,[Ptr "VS24\0"#])]+ '\xe0108' -> [(Abbreviation,[Ptr "VS25\0"#])]+ '\xe0109' -> [(Abbreviation,[Ptr "VS26\0"#])]+ '\xe010a' -> [(Abbreviation,[Ptr "VS27\0"#])]+ '\xe010b' -> [(Abbreviation,[Ptr "VS28\0"#])]+ '\xe010c' -> [(Abbreviation,[Ptr "VS29\0"#])]+ '\xe010d' -> [(Abbreviation,[Ptr "VS30\0"#])]+ '\xe010e' -> [(Abbreviation,[Ptr "VS31\0"#])]+ '\xe010f' -> [(Abbreviation,[Ptr "VS32\0"#])]+ '\xe0110' -> [(Abbreviation,[Ptr "VS33\0"#])]+ '\xe0111' -> [(Abbreviation,[Ptr "VS34\0"#])]+ '\xe0112' -> [(Abbreviation,[Ptr "VS35\0"#])]+ '\xe0113' -> [(Abbreviation,[Ptr "VS36\0"#])]+ '\xe0114' -> [(Abbreviation,[Ptr "VS37\0"#])]+ '\xe0115' -> [(Abbreviation,[Ptr "VS38\0"#])]+ '\xe0116' -> [(Abbreviation,[Ptr "VS39\0"#])]+ '\xe0117' -> [(Abbreviation,[Ptr "VS40\0"#])]+ '\xe0118' -> [(Abbreviation,[Ptr "VS41\0"#])]+ '\xe0119' -> [(Abbreviation,[Ptr "VS42\0"#])]+ '\xe011a' -> [(Abbreviation,[Ptr "VS43\0"#])]+ '\xe011b' -> [(Abbreviation,[Ptr "VS44\0"#])]+ '\xe011c' -> [(Abbreviation,[Ptr "VS45\0"#])]+ '\xe011d' -> [(Abbreviation,[Ptr "VS46\0"#])]+ '\xe011e' -> [(Abbreviation,[Ptr "VS47\0"#])]+ '\xe011f' -> [(Abbreviation,[Ptr "VS48\0"#])]+ '\xe0120' -> [(Abbreviation,[Ptr "VS49\0"#])]+ '\xe0121' -> [(Abbreviation,[Ptr "VS50\0"#])]+ '\xe0122' -> [(Abbreviation,[Ptr "VS51\0"#])]+ '\xe0123' -> [(Abbreviation,[Ptr "VS52\0"#])]+ '\xe0124' -> [(Abbreviation,[Ptr "VS53\0"#])]+ '\xe0125' -> [(Abbreviation,[Ptr "VS54\0"#])]+ '\xe0126' -> [(Abbreviation,[Ptr "VS55\0"#])]+ '\xe0127' -> [(Abbreviation,[Ptr "VS56\0"#])]+ '\xe0128' -> [(Abbreviation,[Ptr "VS57\0"#])]+ '\xe0129' -> [(Abbreviation,[Ptr "VS58\0"#])]+ '\xe012a' -> [(Abbreviation,[Ptr "VS59\0"#])]+ '\xe012b' -> [(Abbreviation,[Ptr "VS60\0"#])]+ '\xe012c' -> [(Abbreviation,[Ptr "VS61\0"#])]+ '\xe012d' -> [(Abbreviation,[Ptr "VS62\0"#])]+ '\xe012e' -> [(Abbreviation,[Ptr "VS63\0"#])]+ '\xe012f' -> [(Abbreviation,[Ptr "VS64\0"#])]+ '\xe0130' -> [(Abbreviation,[Ptr "VS65\0"#])]+ '\xe0131' -> [(Abbreviation,[Ptr "VS66\0"#])]+ '\xe0132' -> [(Abbreviation,[Ptr "VS67\0"#])]+ '\xe0133' -> [(Abbreviation,[Ptr "VS68\0"#])]+ '\xe0134' -> [(Abbreviation,[Ptr "VS69\0"#])]+ '\xe0135' -> [(Abbreviation,[Ptr "VS70\0"#])]+ '\xe0136' -> [(Abbreviation,[Ptr "VS71\0"#])]+ '\xe0137' -> [(Abbreviation,[Ptr "VS72\0"#])]+ '\xe0138' -> [(Abbreviation,[Ptr "VS73\0"#])]+ '\xe0139' -> [(Abbreviation,[Ptr "VS74\0"#])]+ '\xe013a' -> [(Abbreviation,[Ptr "VS75\0"#])]+ '\xe013b' -> [(Abbreviation,[Ptr "VS76\0"#])]+ '\xe013c' -> [(Abbreviation,[Ptr "VS77\0"#])]+ '\xe013d' -> [(Abbreviation,[Ptr "VS78\0"#])]+ '\xe013e' -> [(Abbreviation,[Ptr "VS79\0"#])]+ '\xe013f' -> [(Abbreviation,[Ptr "VS80\0"#])]+ '\xe0140' -> [(Abbreviation,[Ptr "VS81\0"#])]+ '\xe0141' -> [(Abbreviation,[Ptr "VS82\0"#])]+ '\xe0142' -> [(Abbreviation,[Ptr "VS83\0"#])]+ '\xe0143' -> [(Abbreviation,[Ptr "VS84\0"#])]+ '\xe0144' -> [(Abbreviation,[Ptr "VS85\0"#])]+ '\xe0145' -> [(Abbreviation,[Ptr "VS86\0"#])]+ '\xe0146' -> [(Abbreviation,[Ptr "VS87\0"#])]+ '\xe0147' -> [(Abbreviation,[Ptr "VS88\0"#])]+ '\xe0148' -> [(Abbreviation,[Ptr "VS89\0"#])]+ '\xe0149' -> [(Abbreviation,[Ptr "VS90\0"#])]+ '\xe014a' -> [(Abbreviation,[Ptr "VS91\0"#])]+ '\xe014b' -> [(Abbreviation,[Ptr "VS92\0"#])]+ '\xe014c' -> [(Abbreviation,[Ptr "VS93\0"#])]+ '\xe014d' -> [(Abbreviation,[Ptr "VS94\0"#])]+ '\xe014e' -> [(Abbreviation,[Ptr "VS95\0"#])]+ '\xe014f' -> [(Abbreviation,[Ptr "VS96\0"#])]+ '\xe0150' -> [(Abbreviation,[Ptr "VS97\0"#])]+ '\xe0151' -> [(Abbreviation,[Ptr "VS98\0"#])]+ '\xe0152' -> [(Abbreviation,[Ptr "VS99\0"#])]+ '\xe0153' -> [(Abbreviation,[Ptr "VS100\0"#])]+ '\xe0154' -> [(Abbreviation,[Ptr "VS101\0"#])]+ '\xe0155' -> [(Abbreviation,[Ptr "VS102\0"#])]+ '\xe0156' -> [(Abbreviation,[Ptr "VS103\0"#])]+ '\xe0157' -> [(Abbreviation,[Ptr "VS104\0"#])]+ '\xe0158' -> [(Abbreviation,[Ptr "VS105\0"#])]+ '\xe0159' -> [(Abbreviation,[Ptr "VS106\0"#])]+ '\xe015a' -> [(Abbreviation,[Ptr "VS107\0"#])]+ '\xe015b' -> [(Abbreviation,[Ptr "VS108\0"#])]+ '\xe015c' -> [(Abbreviation,[Ptr "VS109\0"#])]+ '\xe015d' -> [(Abbreviation,[Ptr "VS110\0"#])]+ '\xe015e' -> [(Abbreviation,[Ptr "VS111\0"#])]+ '\xe015f' -> [(Abbreviation,[Ptr "VS112\0"#])]+ '\xe0160' -> [(Abbreviation,[Ptr "VS113\0"#])]+ '\xe0161' -> [(Abbreviation,[Ptr "VS114\0"#])]+ '\xe0162' -> [(Abbreviation,[Ptr "VS115\0"#])]+ '\xe0163' -> [(Abbreviation,[Ptr "VS116\0"#])]+ '\xe0164' -> [(Abbreviation,[Ptr "VS117\0"#])]+ '\xe0165' -> [(Abbreviation,[Ptr "VS118\0"#])]+ '\xe0166' -> [(Abbreviation,[Ptr "VS119\0"#])]+ '\xe0167' -> [(Abbreviation,[Ptr "VS120\0"#])]+ '\xe0168' -> [(Abbreviation,[Ptr "VS121\0"#])]+ '\xe0169' -> [(Abbreviation,[Ptr "VS122\0"#])]+ '\xe016a' -> [(Abbreviation,[Ptr "VS123\0"#])]+ '\xe016b' -> [(Abbreviation,[Ptr "VS124\0"#])]+ '\xe016c' -> [(Abbreviation,[Ptr "VS125\0"#])]+ '\xe016d' -> [(Abbreviation,[Ptr "VS126\0"#])]+ '\xe016e' -> [(Abbreviation,[Ptr "VS127\0"#])]+ '\xe016f' -> [(Abbreviation,[Ptr "VS128\0"#])]+ '\xe0170' -> [(Abbreviation,[Ptr "VS129\0"#])]+ '\xe0171' -> [(Abbreviation,[Ptr "VS130\0"#])]+ '\xe0172' -> [(Abbreviation,[Ptr "VS131\0"#])]+ '\xe0173' -> [(Abbreviation,[Ptr "VS132\0"#])]+ '\xe0174' -> [(Abbreviation,[Ptr "VS133\0"#])]+ '\xe0175' -> [(Abbreviation,[Ptr "VS134\0"#])]+ '\xe0176' -> [(Abbreviation,[Ptr "VS135\0"#])]+ '\xe0177' -> [(Abbreviation,[Ptr "VS136\0"#])]+ '\xe0178' -> [(Abbreviation,[Ptr "VS137\0"#])]+ '\xe0179' -> [(Abbreviation,[Ptr "VS138\0"#])]+ '\xe017a' -> [(Abbreviation,[Ptr "VS139\0"#])]+ '\xe017b' -> [(Abbreviation,[Ptr "VS140\0"#])]+ '\xe017c' -> [(Abbreviation,[Ptr "VS141\0"#])]+ '\xe017d' -> [(Abbreviation,[Ptr "VS142\0"#])]+ '\xe017e' -> [(Abbreviation,[Ptr "VS143\0"#])]+ '\xe017f' -> [(Abbreviation,[Ptr "VS144\0"#])]+ '\xe0180' -> [(Abbreviation,[Ptr "VS145\0"#])]+ '\xe0181' -> [(Abbreviation,[Ptr "VS146\0"#])]+ '\xe0182' -> [(Abbreviation,[Ptr "VS147\0"#])]+ '\xe0183' -> [(Abbreviation,[Ptr "VS148\0"#])]+ '\xe0184' -> [(Abbreviation,[Ptr "VS149\0"#])]+ '\xe0185' -> [(Abbreviation,[Ptr "VS150\0"#])]+ '\xe0186' -> [(Abbreviation,[Ptr "VS151\0"#])]+ '\xe0187' -> [(Abbreviation,[Ptr "VS152\0"#])]+ '\xe0188' -> [(Abbreviation,[Ptr "VS153\0"#])]+ '\xe0189' -> [(Abbreviation,[Ptr "VS154\0"#])]+ '\xe018a' -> [(Abbreviation,[Ptr "VS155\0"#])]+ '\xe018b' -> [(Abbreviation,[Ptr "VS156\0"#])]+ '\xe018c' -> [(Abbreviation,[Ptr "VS157\0"#])]+ '\xe018d' -> [(Abbreviation,[Ptr "VS158\0"#])]+ '\xe018e' -> [(Abbreviation,[Ptr "VS159\0"#])]+ '\xe018f' -> [(Abbreviation,[Ptr "VS160\0"#])]+ '\xe0190' -> [(Abbreviation,[Ptr "VS161\0"#])]+ '\xe0191' -> [(Abbreviation,[Ptr "VS162\0"#])]+ '\xe0192' -> [(Abbreviation,[Ptr "VS163\0"#])]+ '\xe0193' -> [(Abbreviation,[Ptr "VS164\0"#])]+ '\xe0194' -> [(Abbreviation,[Ptr "VS165\0"#])]+ '\xe0195' -> [(Abbreviation,[Ptr "VS166\0"#])]+ '\xe0196' -> [(Abbreviation,[Ptr "VS167\0"#])]+ '\xe0197' -> [(Abbreviation,[Ptr "VS168\0"#])]+ '\xe0198' -> [(Abbreviation,[Ptr "VS169\0"#])]+ '\xe0199' -> [(Abbreviation,[Ptr "VS170\0"#])]+ '\xe019a' -> [(Abbreviation,[Ptr "VS171\0"#])]+ '\xe019b' -> [(Abbreviation,[Ptr "VS172\0"#])]+ '\xe019c' -> [(Abbreviation,[Ptr "VS173\0"#])]+ '\xe019d' -> [(Abbreviation,[Ptr "VS174\0"#])]+ '\xe019e' -> [(Abbreviation,[Ptr "VS175\0"#])]+ '\xe019f' -> [(Abbreviation,[Ptr "VS176\0"#])]+ '\xe01a0' -> [(Abbreviation,[Ptr "VS177\0"#])]+ '\xe01a1' -> [(Abbreviation,[Ptr "VS178\0"#])]+ '\xe01a2' -> [(Abbreviation,[Ptr "VS179\0"#])]+ '\xe01a3' -> [(Abbreviation,[Ptr "VS180\0"#])]+ '\xe01a4' -> [(Abbreviation,[Ptr "VS181\0"#])]+ '\xe01a5' -> [(Abbreviation,[Ptr "VS182\0"#])]+ '\xe01a6' -> [(Abbreviation,[Ptr "VS183\0"#])]+ '\xe01a7' -> [(Abbreviation,[Ptr "VS184\0"#])]+ '\xe01a8' -> [(Abbreviation,[Ptr "VS185\0"#])]+ '\xe01a9' -> [(Abbreviation,[Ptr "VS186\0"#])]+ '\xe01aa' -> [(Abbreviation,[Ptr "VS187\0"#])]+ '\xe01ab' -> [(Abbreviation,[Ptr "VS188\0"#])]+ '\xe01ac' -> [(Abbreviation,[Ptr "VS189\0"#])]+ '\xe01ad' -> [(Abbreviation,[Ptr "VS190\0"#])]+ '\xe01ae' -> [(Abbreviation,[Ptr "VS191\0"#])]+ '\xe01af' -> [(Abbreviation,[Ptr "VS192\0"#])]+ '\xe01b0' -> [(Abbreviation,[Ptr "VS193\0"#])]+ '\xe01b1' -> [(Abbreviation,[Ptr "VS194\0"#])]+ '\xe01b2' -> [(Abbreviation,[Ptr "VS195\0"#])]+ '\xe01b3' -> [(Abbreviation,[Ptr "VS196\0"#])]+ '\xe01b4' -> [(Abbreviation,[Ptr "VS197\0"#])]+ '\xe01b5' -> [(Abbreviation,[Ptr "VS198\0"#])]+ '\xe01b6' -> [(Abbreviation,[Ptr "VS199\0"#])]+ '\xe01b7' -> [(Abbreviation,[Ptr "VS200\0"#])]+ '\xe01b8' -> [(Abbreviation,[Ptr "VS201\0"#])]+ '\xe01b9' -> [(Abbreviation,[Ptr "VS202\0"#])]+ '\xe01ba' -> [(Abbreviation,[Ptr "VS203\0"#])]+ '\xe01bb' -> [(Abbreviation,[Ptr "VS204\0"#])]+ '\xe01bc' -> [(Abbreviation,[Ptr "VS205\0"#])]+ '\xe01bd' -> [(Abbreviation,[Ptr "VS206\0"#])]+ '\xe01be' -> [(Abbreviation,[Ptr "VS207\0"#])]+ '\xe01bf' -> [(Abbreviation,[Ptr "VS208\0"#])]+ '\xe01c0' -> [(Abbreviation,[Ptr "VS209\0"#])]+ '\xe01c1' -> [(Abbreviation,[Ptr "VS210\0"#])]+ '\xe01c2' -> [(Abbreviation,[Ptr "VS211\0"#])]+ '\xe01c3' -> [(Abbreviation,[Ptr "VS212\0"#])]+ '\xe01c4' -> [(Abbreviation,[Ptr "VS213\0"#])]+ '\xe01c5' -> [(Abbreviation,[Ptr "VS214\0"#])]+ '\xe01c6' -> [(Abbreviation,[Ptr "VS215\0"#])]+ '\xe01c7' -> [(Abbreviation,[Ptr "VS216\0"#])]+ '\xe01c8' -> [(Abbreviation,[Ptr "VS217\0"#])]+ '\xe01c9' -> [(Abbreviation,[Ptr "VS218\0"#])]+ '\xe01ca' -> [(Abbreviation,[Ptr "VS219\0"#])]+ '\xe01cb' -> [(Abbreviation,[Ptr "VS220\0"#])]+ '\xe01cc' -> [(Abbreviation,[Ptr "VS221\0"#])]+ '\xe01cd' -> [(Abbreviation,[Ptr "VS222\0"#])]+ '\xe01ce' -> [(Abbreviation,[Ptr "VS223\0"#])]+ '\xe01cf' -> [(Abbreviation,[Ptr "VS224\0"#])]+ '\xe01d0' -> [(Abbreviation,[Ptr "VS225\0"#])]+ '\xe01d1' -> [(Abbreviation,[Ptr "VS226\0"#])]+ '\xe01d2' -> [(Abbreviation,[Ptr "VS227\0"#])]+ '\xe01d3' -> [(Abbreviation,[Ptr "VS228\0"#])]+ '\xe01d4' -> [(Abbreviation,[Ptr "VS229\0"#])]+ '\xe01d5' -> [(Abbreviation,[Ptr "VS230\0"#])]+ '\xe01d6' -> [(Abbreviation,[Ptr "VS231\0"#])]+ '\xe01d7' -> [(Abbreviation,[Ptr "VS232\0"#])]+ '\xe01d8' -> [(Abbreviation,[Ptr "VS233\0"#])]+ '\xe01d9' -> [(Abbreviation,[Ptr "VS234\0"#])]+ '\xe01da' -> [(Abbreviation,[Ptr "VS235\0"#])]+ '\xe01db' -> [(Abbreviation,[Ptr "VS236\0"#])]+ '\xe01dc' -> [(Abbreviation,[Ptr "VS237\0"#])]+ '\xe01dd' -> [(Abbreviation,[Ptr "VS238\0"#])]+ '\xe01de' -> [(Abbreviation,[Ptr "VS239\0"#])]+ '\xe01df' -> [(Abbreviation,[Ptr "VS240\0"#])]+ '\xe01e0' -> [(Abbreviation,[Ptr "VS241\0"#])]+ '\xe01e1' -> [(Abbreviation,[Ptr "VS242\0"#])]+ '\xe01e2' -> [(Abbreviation,[Ptr "VS243\0"#])]+ '\xe01e3' -> [(Abbreviation,[Ptr "VS244\0"#])]+ '\xe01e4' -> [(Abbreviation,[Ptr "VS245\0"#])]+ '\xe01e5' -> [(Abbreviation,[Ptr "VS246\0"#])]+ '\xe01e6' -> [(Abbreviation,[Ptr "VS247\0"#])]+ '\xe01e7' -> [(Abbreviation,[Ptr "VS248\0"#])]+ '\xe01e8' -> [(Abbreviation,[Ptr "VS249\0"#])]+ '\xe01e9' -> [(Abbreviation,[Ptr "VS250\0"#])]+ '\xe01ea' -> [(Abbreviation,[Ptr "VS251\0"#])]+ '\xe01eb' -> [(Abbreviation,[Ptr "VS252\0"#])]+ '\xe01ec' -> [(Abbreviation,[Ptr "VS253\0"#])]+ '\xe01ed' -> [(Abbreviation,[Ptr "VS254\0"#])]+ '\xe01ee' -> [(Abbreviation,[Ptr "VS255\0"#])]+ '\xe01ef' -> [(Abbreviation,[Ptr "VS256\0"#])]++ _ -> mempty
+ test/Main.hs view
@@ -0,0 +1,1 @@+{-# OPTIONS_GHC -F -pgmF hspec-discover #-}
+ test/Unicode/Char/General/NamesSpec.hs view
@@ -0,0 +1,83 @@+{-# LANGUAGE BlockArguments #-}++module Unicode.Char.General.NamesSpec+ ( spec+ ) where++import Unicode.Char.General+import Unicode.Char.General.Names+import Data.Foldable (traverse_)+import Test.Hspec++spec :: Spec+spec = do+ it "name: Test some characters" do+ name minBound `shouldBe` Nothing+ name 'A' `shouldBe` Just "LATIN CAPITAL LETTER A"+ name '\x0080' `shouldBe` Nothing+ name '\x009f' `shouldBe` Nothing+ -- Name with correction+ name '\x01A2' `shouldBe` Just "LATIN CAPITAL LETTER OI"+ name '\x2200' `shouldBe` Just "FOR ALL"+ -- Name generated using pattern (example from UCD file)+ name '\x4e12' `shouldBe` Just "CJK UNIFIED IDEOGRAPH-4E12"+ name '\xfeff' `shouldBe` Just "ZERO WIDTH NO-BREAK SPACE"+ name '\xffff' `shouldBe` Nothing+ -- Name with correction+ name '\x1D0C5' `shouldBe` Just "BYZANTINE MUSICAL SYMBOL FHTORA SKLIRON CHROMA VASIS"+ name '\x1f41d' `shouldBe` Just "HONEYBEE"+ -- Name generated using pattern (example from UCD file)+ name '\x2f89f' `shouldBe` Just "CJK COMPATIBILITY IDEOGRAPH-2F89F"+ -- Last name defined, as of Unicode 14.0.0+ name '\xe01ef' `shouldBe` Just "VARIATION SELECTOR-256"+ name maxBound `shouldBe` Nothing+ it "correctedName: Test some characters" do+ correctedName minBound `shouldBe` Nothing+ correctedName 'A' `shouldBe` Just "LATIN CAPITAL LETTER A"+ correctedName '\x0080' `shouldBe` Nothing+ correctedName '\x009f' `shouldBe` Nothing+ -- Name with correction+ correctedName '\x01A2' `shouldBe` Just "LATIN CAPITAL LETTER GHA"+ correctedName '\x2200' `shouldBe` Just "FOR ALL"+ -- Name generated using pattern (example from UCD file)+ correctedName '\x4e12' `shouldBe` Just "CJK UNIFIED IDEOGRAPH-4E12"+ correctedName '\xfeff' `shouldBe` Just "ZERO WIDTH NO-BREAK SPACE"+ correctedName '\xffff' `shouldBe` Nothing+ -- Name with correction+ correctedName '\x1D0C5' `shouldBe` Just "BYZANTINE MUSICAL SYMBOL FTHORA SKLIRON CHROMA VASIS"+ correctedName '\x1f41d' `shouldBe` Just "HONEYBEE"+ -- Name generated using pattern (example from UCD file)+ correctedName '\x2f89f' `shouldBe` Just "CJK COMPATIBILITY IDEOGRAPH-2F89F"+ -- Last name defined, as of Unicode 14.0.0+ correctedName '\xe01ef' `shouldBe` Just "VARIATION SELECTOR-256"+ correctedName maxBound `shouldBe` Nothing+ it "nameOrAlias: Test some characters" do+ nameOrAlias minBound `shouldBe` Just "NULL"+ nameOrAlias 'A' `shouldBe` Just "LATIN CAPITAL LETTER A"+ nameOrAlias '\x0080' `shouldBe` Just "PADDING CHARACTER"+ nameOrAlias '\x009f' `shouldBe` Just "APPLICATION PROGRAM COMMAND"+ -- Name with correction+ nameOrAlias '\x01A2' `shouldBe` Just "LATIN CAPITAL LETTER OI"+ nameOrAlias '\x2200' `shouldBe` Just "FOR ALL"+ -- Name generated using pattern (example from UCD file)+ nameOrAlias '\x4e12' `shouldBe` Just "CJK UNIFIED IDEOGRAPH-4E12"+ nameOrAlias '\xfeff' `shouldBe` Just "ZERO WIDTH NO-BREAK SPACE"+ nameOrAlias '\xffff' `shouldBe` Nothing+ -- Name with correction+ nameOrAlias '\x1D0C5' `shouldBe` Just "BYZANTINE MUSICAL SYMBOL FHTORA SKLIRON CHROMA VASIS"+ nameOrAlias '\x1f41d' `shouldBe` Just "HONEYBEE"+ -- Name generated using pattern (example from UCD file)+ nameOrAlias '\x2f89f' `shouldBe` Just "CJK COMPATIBILITY IDEOGRAPH-2F89F"+ -- Last name defined, as of Unicode 14.0.0+ nameOrAlias '\xe01ef' `shouldBe` Just "VARIATION SELECTOR-256"+ nameOrAlias maxBound `shouldBe` Nothing+ it "Every defined character has at least a name or an alias" do+ let checkName c = case nameOrAlias c of+ Just _ -> True+ Nothing -> case generalCategory c of+ Surrogate -> True+ PrivateUse -> True+ NotAssigned -> True+ _ -> False+ traverse_ (`shouldSatisfy` checkName)+ [minBound..maxBound]
+ test/export_all_chars.hs view
@@ -0,0 +1,42 @@+{-# LANGUAGE BlockArguments #-}++-------------------------------------------------------------------------------+-- |+-- Description: Export all characters and their properties to a CSV file.+--+-- This is intended to compare Haskell results to other languages.+-------------------------------------------------------------------------------++module Main where++import Data.Char+import Data.Foldable+import Data.Version+import Unicode.Char (unicodeVersion)+import Numeric++import qualified Unicode.Char.General.Names as UNames++main :: IO ()+main = do+ -- First line is Unicode version+ putStrLn (showVersion unicodeVersion)+ -- Second line is CSV header+ putStrLn header+ -- Then all the supported characters+ traverse_ addEntry [minBound..maxBound]++-- | File header+header :: String+header = "Char,Name"++-- | Convert a character to its (short) hexadecimal Unicode codepoint.+mkCodePointHex :: Char -> String+mkCodePointHex c = showHex (ord c) mempty++-- | Make a CSV entry for a char.+addEntry :: Char -> IO ()+addEntry c = do+ putStr (mkCodePointHex c)+ putChar ','+ putStrLn (show (UNames.name c))
+ unicode-data-names.cabal view
@@ -0,0 +1,130 @@+cabal-version: 2.2+name: unicode-data-names+version: 0.1.0+synopsis: Unicode characters names and aliases+description:+ @unicode-data-names@ provides Haskell APIs to access the Unicode+ character /names and aliases/ from the+ <https://www.unicode.org/ucd/ Unicode character database> (UCD).+ .+ The Haskell data structures are generated programmatically from the UCD.+ The latest Unicode version supported by this library is+ @<https://www.unicode.org/versions/Unicode14.0.0/ 14.0.0>@.+homepage: http://github.com/composewell/unicode-data+bug-reports: https://github.com/composewell/unicode-data/issues+license: Apache-2.0+license-file: LICENSE+author: Composewell Technologies and Contributors+maintainer: dev@wismill.eu+copyright: 2022 Composewell Technologies and Contributors+category: Data,Text,Unicode+stability: Experimental+build-type: Simple+tested-with: GHC==8.0.2+ , GHC==8.2.2+ , GHC==8.4.4+ , GHC==8.6.5+ , GHC==8.8.4+ , GHC==8.10.7+ , GHC==9.0.1+ , GHC==9.2.1++extra-source-files:+ Changelog.md+ README.md++source-repository head+ type: git+ location: https://github.com/composewell/unicode-data++flag export-all-chars+ description: Build the export-all-chars executable+ manual: True+ default: False++common default-extensions+ default-extensions:+ BangPatterns+ DeriveGeneric+ MagicHash+ RecordWildCards+ ScopedTypeVariables+ TupleSections+ FlexibleContexts+ LambdaCase++ -- Experimental, may lead to issues+ DeriveAnyClass+ TemplateHaskell+ UnboxedTuples++common compile-options+ ghc-options: -Wall+ -fwarn-identities+ -fwarn-incomplete-record-updates+ -fwarn-incomplete-uni-patterns+ -fwarn-tabs+ default-language: Haskell2010++library+ import: default-extensions, compile-options+ default-language: Haskell2010+ exposed-modules:+ -- The module structure is derived from+ -- https://www.unicode.org/reports/tr44/#Property_Index_Table+ Unicode.Char.General.Names++ -- Generated files+ -- This module structure is largely based on the UCD file names from which+ -- the properties are generated.+ Unicode.Internal.Char.UnicodeData.DerivedName+ Unicode.Internal.Char.UnicodeData.NameAliases++ hs-source-dirs: lib+ build-depends:+ base >= 4.7 && < 4.17++test-suite test+ import: default-extensions, compile-options+ type: exitcode-stdio-1.0+ main-is: Main.hs+ hs-source-dirs:+ test+ other-modules:+ Unicode.Char.General.NamesSpec+ build-depends:+ base >= 4.7 && < 4.17+ , hspec >= 2.0 && < 2.10+ , unicode-data+ , unicode-data-names+ build-tool-depends:+ hspec-discover:hspec-discover >= 2.0 && < 2.10+ default-language: Haskell2010++executable export-all-chars+ import: default-extensions, compile-options+ default-language: Haskell2010+ ghc-options: -O2+ hs-source-dirs: test+ main-is: export_all_chars.hs+ if flag(export-all-chars)+ buildable: True+ build-depends:+ base+ , unicode-data+ , unicode-data-names+ else+ buildable: False++benchmark bench+ import: default-extensions, compile-options+ type: exitcode-stdio-1.0+ hs-source-dirs: bench+ main-is: Main.hs+ build-depends:+ base >= 4.7 && < 4.17,+ deepseq >= 1.1 && < 1.5,+ tasty-bench >= 0.2.5 && < 0.4,+ tasty >= 1.4.1,+ unicode-data-names+ ghc-options: -O2 -fdicts-strict -rtsopts