diff --git a/LICENSE b/LICENSE new file mode 100644 --- /dev/null +++ b/LICENSE @@ -0,0 +1,675 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. + diff --git a/Setup.hs b/Setup.hs new file mode 100644 --- /dev/null +++ b/Setup.hs @@ -0,0 +1,5 @@ +#!/usr/bin/env runhaskell + +import qualified Distribution.Simple + +main = Distribution.Simple.defaultMain diff --git a/changelog b/changelog new file mode 100644 --- /dev/null +++ b/changelog @@ -0,0 +1,39 @@ +2010-11-18 Dr. Alistair Ward + +0.9.0.0 + * First version of the package. +0.9.0.1 + * Reacted to major-number changes in the packages 'ToolShed-0.10.0.0' & 'RegExDot-0.10.0.0'. + * Created "src/RegExChar/" (& then modified module-names accordingly) & "data/" sub-directories. + * Pacified 'hlint'. + * Added a makefile. + * Qualified identifiers used in error-messages. +0.9.0.2 + * Ported to ghc-7.0.1: + Replaced the deprecated module "Test.QuickCheck.Batch", & labelled properties. + * Added the cabal-flag 'threaded', to account for the requirement to link with the threaded runtime when using a threaded version of the library package "RegExDot". +0.9.0.3 + * Parallelised processing where @grecce@ is requested to read multiple input-data files. + * Extracted matching-operation from the "Main" module, into new module "Grecce.Grep". + * Migrated modules required only by the executable 'grecce', from the directory "RegExChar" into the new directory "Grecce". + * Replaced use of @threaded@-flag in source-code, with CABAL CPP-macro. + * Added RPM-spec file. +0.9.0.4 + * Added default values of command-line flags, to usage-message. + * Reacted to migration of "RegExDot.Options" to "ToolShed.Options". + * Used new "ToolShed.Package" module. +0.9.0.5 + * Fixed negated reference to unparenthesised CPP-macro 'MIN_VERSION'. + * Protected reference in "Grecce/QC/QuickChecks.hs", to 'Test.QuickCheck.verboseCheck', using CPP-statement. +0.9.0.6 + * Amended RPM ".spec" file, to install under "/usr", rather than "/usr/local". + * Renamed package from 'RegExChar' to 'regexchar', for compatibility with Debian's ".deb"-format. + * Added directory-structure required to build ".deb" package. +0.9.0.7 + * Refined man-page to be an 'extra-source-file' in .cabal & .spec files. + * Added manually controlled "llvm" flag to "regexchar.cabal". + * Replaced references to the module "ToolShed.Package" with "Distribution.Package". + * Changed identifier for type-parameters, to better reflect its role. + * Removed dependency on archaic module "ToolShed.Arithmetic". + * Reacted to the creation of module "ToolShed.Defaultable". + diff --git a/copyright b/copyright new file mode 100644 --- /dev/null +++ b/copyright @@ -0,0 +1,11 @@ +Author: + Dr. Alistair Ward . + +Copyright: + Copyright (C) 2010-2011 Dr. Alistair Ward. All Rights Reserved. + +Home-page: + http://functionalley.eu + +License: + GNU GENERAL PUBLIC LICENSE Version 3; see '/usr/share/common-licenses/GPL-3' or '/usr/share/doc/licenses/gpl-3.0.txt' where available, or the local packaged file 'LICENSE'. diff --git a/data/Tests.txt b/data/Tests.txt new file mode 100644 --- /dev/null +++ b/data/Tests.txt @@ -0,0 +1,95 @@ +#Copyright (C) 2010 Dr. Alistair Ward +# +#This program is free software: you can redistribute it and/or modify +#it under the terms of the GNU General Public License as published by +#the Free Software Foundation, either version 3 of the License, or +#(at your option) any later version. +# +#This program is distributed in the hope that it will be useful, +#but WITHOUT ANY WARRANTY; without even the implied warranty of +#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +#GNU General Public License for more details. +# +#You should have received a copy of the GNU General Public License +#along with this program. If not, see . + +#AUTHOR: Dr. Alistair Ward +#DESCRIPTION: Designed to be read by the module "FunctionAlley.RegExChar.Assert.RegExOptsChar". +#CAVEAT: Regrettably, "Text.Regex.Posix" actually dies when presented with some of these, so they've been commented-out. + +("^$", [("", [(0,0)])]) + +("^a.[cd][^ef]$", [("abcg", [(0,4)]),("aBdE", [(0,4)])]) +("^a*.*[cd]*[^ef]*$", [("", [(0,0)]),("a", [(0,1)]),("ab", [(0,2)]),("aBc", [(0,3)]),("ac", [(0,2)]),("acd", [(0,3)]),("abcdgh", [(0,6)])]) +("^a*?.*?[cd]*?[^e]*?$", [("", [(0,0)]),("a", [(0,1)]),("ab", [(0,2)]),("aBc", [(0,3)]),("ac", [(0,2)]),("acd", [(0,3)]),("abcdf", [(0,5)])]) +("^a+.+[cd]+[^ef]+$", [("abcg", [(0,4)]),("aaBcG", [(0,5)]),("aBBcg", [(0,5)]),("abBcdG", [(0,6)])]) +("^a+?.+?[cd]+?[^e]+?$", [("abcf", [(0,4)]),("aaBcF", [(0,5)]),("aBBcE", [(0,5)]),("abBcdg", [(0,6)])]) +("^a?.?[cd]?[^ef]?$", [("", [(0,0)]),("a", [(0,1)]),("ab", [(0,2)]),("aBc", [(0,3)]),("ac", [(0,2)]),("acd", [(0,3)]),("abcg", [(0,4)])]) +("^a??.??[cd]??[^ef]??$", [("", [(0,0)]),("a", [(0,1)]),("ab", [(0,2)]),("aBc", [(0,3)]),("ac", [(0,2)]),("acd", [(0,3)]),("abcE", [(0,4)])]) +("^a{2}.{1,2}[cd]{1,}[^ef]{2,3}$", [("aabcEF", [(0,6)]),("aaBbcdEFE", [(0,9)])]) +("^.{1,2}?[cd]{1,}?[^ef]{2,3}?$", [("bcEF", [(0,4)]),("BbcdEFE", [(0,7)])]) + +#("^[^][a][bb]?$", [("xa", [(0,2)]),("xab", [(0,3)])]) #Is this legal ? +#("^[]*$", [("", [(0,0)])]) #Is this legal ? + +#Escape-sequences. +("^\\[]$", [("[]", [(0,2)])]) +("^\\\\+$", [("\\", [(0,1)]),("\\\\", [(0,2)]),("\\\\\\", [(0,3)])]) +("^[\\\\]+$", [("\\", [(0,1)]),("\\\\", [(0,2)]),("\\\\\\", [(0,3)])]) +("\\^\\$", [("^$", [(0,2)])]) + +#Non-greedy. +("^aa?a*a+$", [("aa", [(0,2)]),("aaa", [(0,3)]),("aaaa", [(0,4)])]) +("^a+?[^a]+?a+[^a]+$", [("abab", [(0,4)]),("aabbaabb", [(0,8)]),("aaabbbaaabbb", [(0,12)])]) +("^a.*?$", [("a", [(0,1)]),("ab", [(0,2)]),("abc", [(0,3)])]) +("^.*?a$", [("a", [(0,1)]),("ba", [(0,2)]),("cba", [(0,3)])]) +("^.*?a.*?$", [("a", [(0,1)]),("ab", [(0,2)]),("ba", [(0,2)]),("bab", [(0,3)])]) + +#Shortcuts. +("^\\S\\s+\\S$", [("a \t\r\n b", [(0,7)])]) +("^\\d\\D+\\d$", [("0aA_!@#$%^&*()-=+;:',<.>/?9", [(0,27)])]) #"Text.Regex.Posix" doesn't understand these Perl-shortcuts. +("^\\w+$", [("aA_09", [(0,5)])]) +("^\\W+$", [("!@#$%^&*()-=+;:',<.>/?", [(0,22)])]) +("^[\\d]*[\\D]*$", [("", [(0,0)]),("0123456809", [(0,10)]),("abc", [(0,3)]),("012abc", [(0,6)])]) #"Text.Regex.Posix" doesn't understand these Perl-shortcuts. +("^\\S[\\sx\\d]\\D$", [("_ _", [(0,3)]),("01a", [(0,3)]),("a\t ", [(0,3)]),("_x_", [(0,3)])]) #"Text.Regex.Posix" doesn't understand '\\d' or '\\D'. +("^[^\\d]*$", [("abc", [(0,3)]),("_ \t", [(0,3)])]) + +#Posix Character-classes. +("^[[:xdigit:]]*$", [("", [(0,0)]),("0123456789abcdefABCDEF", [(0,22)])]) +#user error (Text.Regex.Posix.String died: (ReturnCode 4,"Invalid character class name")) +#("^[[:^xdigit:]]*$", [("", [(0,0)]),("ghijklmnopqrstuvwxyzGHIJKLMNOPQRSTUVWXYZ", [(0,40)])]) +#("^[^[:^xdigit:]]*$", [("", [(0,0)]),("0123456789abcdefABCDEF", [(0,22)])]) +#("^[a[:digit:]z]*$", [("", [(0,0)]),("012", [(0,3)]),("z210a", [(0,5)])]) +#("^[^a[:digit:]z]*$", [("", [(0,0)]),("_bc", [(0,3)])]) +#("^[^0[:^digit:]1]*$", [("", [(0,0)]),("23456789", [(0,8)])]) + +#Bracket-expressions. +("^[a-z0-9]$", [("b", [(0,1)]),("y", [(0,1)]),("1", [(0,1)]),("8", [(0,1)])]) +#("^[-b-b-a-]$", [("-", [(0,1)]),("b", [(0,1)]),("a", [(0,1)])]) #user error (Text.Regex.Posix.String died: (ReturnCode 11,"Invalid range end")) + +#Sub-expressions. +("^a+(b?)*c{2,3}$", [("acc", [(0,3),(-1, 0)]),("aabccc", [(0,6),(2,1)]),("abbbcc", [(0,6),(3,1)])]) #For 1st data-set, "Text.Regex.Posix" returns [(0,3),(1,0)]. +("^a+(b?|c+?[de]{1,2})*f$", [("af", [(0,2),(1,0)]),("aabf", [(0,4),(2,1)]),("aaacdf", [(0,6),(3,2)]),("abccdef", [(0,7),(2,4)])]) +("^a+(b?(c|de)+?f)+g$", [("abcfg", [(0,5),(1,3),(2,1)]),("aadecfbcfg", [(0,10),(6,3),(7,1)])]) + +#Alternatives. +("^(a|.|[cd]|[^ef])$", [("a", [(0,1),(0,1)]),("b", [(0,1),(0,1)]),("c", [(0,1),(0,1)]),("E", [(0,1),(0,1)])]) +("^(a*|.*?|[cd]*|[^ef]*?)$", [("", [(0,0),(0,0)]),("aa", [(0,2),(0,2)]),("bb", [(0,2),(0,2)]),("dc", [(0,2),(0,2)]),("EF", [(0,2),(0,2)])]) +("^(a+|.+?|[cd]+|[^ef]+?)$", [("aa", [(0,2),(0,2)]),("bb", [(0,2),(0,2)]),("dc", [(0,2),(0,2)]),("FE", [(0,2),(0,2)])]) +("^(a?|.??|[cd]?|[^ef]??)$", [("", [(0,0),(0,0)]),("a", [(0,1),(0,1)]),("b", [(0,1),(0,1)]),("c", [(0,1),(0,1)]),("E", [(0,1),(0,1)])]) +("^(a{2}|.{1,2}?|[cd]{1,}|[^ef]{2,3}?)$", [("aa", [(0,2),(0,2)]),("b", [(0,1),(0,1)]),("cddc", [(0,4),(0,4)]),("EF", [(0,2),(0,2)])]) + +("^(.*){2}.$", [("a", [(0,1),(0,0)])]) +("^(a|[^a])*$", [("", [(0,0),(-1,0)]),("a", [(0,1),(0,1)]),("b", [(0,1),(0,1)]),("aa", [(0,2),(1,1)]),("ab", [(0,2),(1,1)]),("bb", [(0,2),(1,1)]),("ba", [(0,2),(1,1)])]) #"Text.Regex.Posix" returns [(0,0),(-1,0)] for the 1st data-set, which looks correct. +("^(a|b*)*$", [("", [(0,0),(-1,0)]),("a", [(0,1),(0,1)]),("b", [(0,1),(0,1)]),("aa", [(0,2),(1,1)]),("ab", [(0,2),(1,1)]),("bb", [(0,2),(0,2)]),("ba", [(0,2),(1,1)])]) +("^(a*)+b$", [("b", [(0,1),(0,0)]),("ab", [(0,2),(0,1)])]) +("^(a|bc|def)+$", [("abcdef", [(0,6),(3,3)]),("defbca", [(0,6),(5,1)])]) +("^a(b(c|de|fgh){1,3}i|j(k|lm|nop)+q)*r$", [("abcdefghijklmnopqr", [(0,18),(9,8),(13,3)]),("ar", [(0,2),(-1, 0)]),("abfghdecijnoplmkqr", [(0,18),(9,8),(15,1)]),("ajnoplmkqbfghdecir", [(0,18),(9,8),(15,1)]),("abcijklmnopklmnopqbdeijklmnopklmnopqr", [(0,37),(22,14),(32,3)])]) #"Text.Regex.Posix" returns; [[(0,18),(9,8),(5,3),(13,3)], [(0,2),(-1,0),(-1,0),(-1,0)], [(0,18),(9,8),(7,1),(15,1)], [(0,37),(22,14),(19,2),(32,3)]]. +("^(((a*?)*?)*?|a*b)$", [("b", [(0,1),(0,1)]),("ab", [(0,2),(0,2)]),("aab", [(0,3),(0,3)]),("aaab", [(0,4),(0,4)])]) #"Text.Regex.Posix" returns [(0,1),(-1,0),(-1,0)], ... + +#Internal anchors. +("a*(^b+)*c*", [("abbc", [(0,1),(-1,0)]), ("abb", [(0,1),(-1,0)]), ("bbc", [(0,3),(0,2)]), ("ac", [(0,2),(-1,0)]), ("a", [(0,1),(-1,0)]), ("bb", [(0,2),(0,2)]), ("c", [(0,1),(-1,0)])]) +("a*(b+$)*c*", [("abbc", [(0,1),(-1,0)]), ("abb", [(0,3),(1,2)]), ("bbc", [(2,1),(-1,0)]), ("ac", [(0,2),(-1,0)]), ("a", [(0,1),(-1,0)]), ("bb", [(0,2),(0,2)]), ("c", [(0,1),(-1,0)])]) #3rd fails. +("a*(^b+$)*c*", [("abbc", [(0,1),(-1,0)]), ("abb", [(0,1),(-1,0)]), ("bbc", [(2,1),(-1,0)]), ("ac", [(0,2),(-1,0)]), ("a", [(0,1),(-1,0)]), ("bb", [(0,2),(0,2)]), ("c", [(0,1),(-1,0)])]) #3rd fails. +("a*(^b?|^b+)*c*", [("abbc", [(0,1),(-1,0)]), ("abb", [(0,1),(-1,0)]), ("bbc", [(0,3),(0,2)]), ("ac", [(0,2),(-1,0)]), ("a", [(0,1),(-1,0)]), ("bb", [(0,2),(0,2)]), ("c", [(0,1),(0,0)])]) +("(^a*|b+|c*$)*", [("abbc", [(0,4),(3,1)]), ("abb", [(0,3),(1,2)]), ("bbc", [(0,3),(2,1)]), ("ac", [(0,2),(1,1)]), ("a", [(0,1),(0,1)]), ("bb", [(0,2),(0,2)]), ("c", [(0,1),(0,1)])]) #"Text.Regex.Posix" doesn't return from either 3rd or 6th. diff --git a/data/Tests_ATTBasic.txt b/data/Tests_ATTBasic.txt new file mode 100644 --- /dev/null +++ b/data/Tests_ATTBasic.txt @@ -0,0 +1,190 @@ +#AUTHOR: Dr. Alistair Ward +#DESCRIPTION: Designed to be read by the module "FunctionAlley.RegExChar.Assert.RegExOptsChar". +#Based on + +("abracadabra$", [("abracadabracadabra", [(7,11)])]) +("a...b", [("abababbb", [(2,5)])]) +("XXXXXX", [("..XXXXXX", [(2,6)])]) + +("\\)", [("()", [(1,1)])]) +("a]", [("a]a", [(0,2)])]) +("\\}", [("}", [(0,1)])]) +("\\]", [("]", [(0,1)])]) +("]", [("]", [(0,1)])]) +#("{", [("{", [(0,1)])]) #Text.Regex.Posix dies on this. +("}", [("}", [(0,1)])]) + +("^a", [("ax", [(0,1)])]) +("\\^a", [("a^a", [(1,2)])]) +("a\\^", [("a^", [(0,2)])]) +("a$", [("aa", [(1,1)])]) +("a\\$", [("a$", [(0,2)])]) + +("^$", [("", [(0,0)])]) +("$^", [("", [(0,0)])]) +("a($)", [("aa", [(1,1),(2,0)])]) +("a*(^a)", [("aa", [(0,1),(0,1)])]) + +("(..)*(...)*", [("a", [(0,0)]), ("abcd", [(0,4),(2,2)])]) #1st test returns [(0,0),(0,0),(0,0)] (presumable the /last/ match of the repeatable group doesn't exist). +("(ab|a)(bc|c)", [("abc", [(0,3),(0,2),(2,1)])]) +("(ab)c|abc", [("abc", [(0,3),(0,2)])]) #Returns [(0,3)] (wrong choice amongst potential matches). +("a{0}b", [("ab", [(1,1)])]) + +("(a*)(b?)(b+)b{3}", [("aaabbbbbbb", [(0,10),(0,3),(3,1),(4,3)])]) +("(a*)(b{0,1})(b{1,})b{3}", [("aaabbbbbbb", [(0,10),(0,3),(3,1),(4,3)])]) + +("((a|a)|a)", [("a", [(0,1),(0,1),(0,1)])]) #Returns [(0,1),(0,1)] (wrong choice amongst potential matches). +("(a*)(a|aa)", [("aaaa", [(0,4),(0,3),(3,1)])]) +("a*(a.|aa)", [("aaaa", [(0,4),(2,2)])]) + +("a(b)|c(d)|a(e)f", [("aef", [(0,3),(1,1)])]) +("(a|b)?.*", [("b", [(0,1),(0,1)])]) +("(a|b)c|a(b|c)", [("ac", [(0,2),(0,1)]), ("ab", [(0,2),(1,1)])]) #1st test returns [(0,2),(1,1)] (wrong choice amongst potential matches). +("(a|b)*c|(a|ab)*c", [("abc", [(0,3),(1,1)]), ("xc", [(1,1)])]) #1st test returns [(0,3),(0,2)] (by selecting the 2nd alternative). +("(.a|.b).*|.*(.a|.b)", [("xa", [(0,2),(0,2)])]) +("a?(ab|ba)ab", [("abab", [(0,4),(0,2)])]) +("a?(ac{0}b|ba)ab", [("abab", [(0,4),(0,2)])]) +("ab|abab", [("abbabab", [(0,2)])]) +("aba|bab|bba", [("baaabbbaba", [(5,3)])]) +("aba|bab", [("baaabbbaba", [(6,3)])]) +("(aa|aaa)*|(a|aaaaa)", [("aa", [(0,2),(0,2)])]) +("(a.|.a.)*|(a|.a...)", [("aa", [(0,2),(0,2)])]) +("ab|a", [("xabc", [(1,2)]), ("xxabc", [(2,2)])]) + +#("(Ab|cD)*", [("aBcD", [(0,4),(2,2)])]) #Relies on case-insensitivity. +("[^-]", [("--a", [(2,1)])]) +("[a-]*", [("--a", [(0,3)])]) +("[a-m-]*", [("--amoma--", [(0,4)])]) + +(":::1:::0:|:::1:1:0:", [(":::0:::1:::1:::0:", [(8,9)])]) +(":::1:::0:|:::1:1:1:", [(":::0:::1:::1:::0:", [(8,9)])]) + +("[[:upper:]]", [("A", [(0,1)])]) +("[[:lower:]]+", [("`az{", [(1,2)])]) +("[[:upper:]]+", [("@AZ[", [(1,2)])]) + +("[[-]]", [("[[-]]", [(2,2)])]) +("\n", [("\n", [(0,1)])]) +("[^a]", [("\n", [(0,1)])]) +("\na", [("\na", [(0,2)])]) +("(a)(b)(c)", [("abc", [(0,3),(0,1),(1,1),(2,1)])]) +("xxx", [("xxx", [(0,3)])]) +("(^|[ (,;])((([Ff]eb[^ ]* *|0*2/|\\* */?)0*[6-7]))([^0-9]|$)", [("feb 6,", [(0,6),(0,0),(0,5),(0,5),(0,4),(5,1)]), ("2/7", [(0,3),(0,0),(0,3),(0,3),(0,2),(3,0)]), ("feb 1,Feb 6", [(5,6),(5,1),(6,5),(6,5),(6,4),(11,0)])]) + +("((((((((((((((((((((((((((((((x))))))))))))))))))))))))))))))", [("x", [(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1)])]) +("((((((((((((((((((((((((((((((x))))))))))))))))))))))))))))))*", [("xx", [(0,2),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1),(1,1)])]) + +("a?(ab|ba)*", [("ababababababababababababababababababababababababababababababababababababababababa", [(0,81),(79,2)])]) +("abaa|abbaa|abbbaa|abbbbaa", [("ababbabbbabbbabbbbabbbbaa", [(18,7)]), ("ababbabbbabbbabbbbabaa", [(18,4)])]) +("aaac|aabc|abac|abbc|baac|babc|bbac|bbbc", [("baaabbbabac", [(7,4)])]) + +(".*", [("\x01\xff", [(0,2)])]) +("aaaa|bbbb|cccc|ddddd|eeeeee|fffffff|gggg|hhhh|iiiii|jjjjj|kkkkk|llll", [("XaaaXbbbXcccXdddXeeeXfffXgggXhhhXiiiXjjjXkkkXlllXcbaXaaaa", [(53,4)])]) +("a*a*a*a*a*b", [("aaaaaaaaab", [(0,10)])]) + +("^", [("", [(0,0)]), ("abc", [(0,0)])]) +("$", [("", [(0,0)]), ("abc", [(3,0)])]) +("^$", [("", [(0,0)])]) +("^a$", [("a", [(0,1)])]) + +("abc", [("abc", [(0,3)]), ("xabcy", [(1,3)]), ("ababc", [(2,3)])]) +("ab*c", [("abc", [(0,3)])]) +("ab*bc", [("abc", [(0,3)]), ("abbc", [(0,4)]), ("abbbbc", [(0,6)])]) +("ab+bc", [("abbc", [(0,4)]), ("abbbbc", [(0,6)])]) +("ab?bc", [("abbc", [(0,4)]), ("abc", [(0,3)])]) +("ab?c", [("abc", [(0,3)])]) + +("^abc$", [("abc", [(0,3)])]) +("^abc", [("abcc", [(0,3)])]) +("abc$", [("aabc", [(1,3)])]) + +("a.c", [("abc", [(0,3)]), ("axc", [(0,3)])]) +("a.*c", [("axyzc", [(0,5)])]) + +("a[bc]d", [("abd", [(0,3)])]) +("a[b-d]e", [("ace", [(0,3)])]) +("a[b-d]", [("aac", [(1,2)])]) +("a[-b]", [("a-", [(0,2)])]) +("a[b-]", [("a-", [(0,2)])]) +("a]", [("a]", [(0,2)])]) +("a[]]b", [("a]b", [(0,3)])]) +("a[^bc]d", [("aed", [(0,3)])]) +("a[^-b]c", [("adc", [(0,3)])]) +("a[^]b]c", [("adc", [(0,3)])]) + +("ab|cd", [("abc", [(0,2)]), ("abcd", [(0,2)])]) +("a\\(b", [("a(b", [(0,3)])]) +("a\\(*b", [("ab", [(0,2)]), ("a((b", [(0,4)])]) +("((a))", [("abc", [(0,1),(0,1),(0,1)])]) +("(a)b(c)", [("abc", [(0,3),(0,1),(2,1)])]) +("a+b+c", [("aabbabc", [(4,3)])]) +("a*", [("aaa", [(0,3)])]) + +("(a*)*", [("-", [(0,0),(0,0)])]) +("(a*)+", [("-", [(0,0),(0,0)])]) +("(a*|b)*", [("-", [(0,0),(0,0)])]) +("(a+|b)*", [("ab", [(0,2),(1,1)])]) +("(a+|b)+", [("ab", [(0,2),(1,1)])]) +("(a+|b)?", [("ab", [(0,1),(0,1)])]) +("[^ab]*", [("cde", [(0,3)])]) + +("(^)*", [("-", [(0,0),(0,0)])]) +("a*", [("", [(0,0)])]) + +("([abc])*d", [("abbbcd", [(0,6),(4,1)])]) +("([abc])*bcd", [("abcd", [(0,4),(0,1)])]) + +("a|b|c|d|e", [("e", [(0,1)])]) +("(a|b|c|d|e)f", [("ef", [(0,2),(0,1)])]) + +("((a*|b))*", [("-", [(0,0),(0,0),(0,0)])]) +("abcd*efg", [("abcdefg", [(0,7)])]) +("ab*", [("xabyabbbz", [(1,2)]), ("xayabbbz", [(1,1)])]) +("(ab|cd)e", [("abcde", [(2,3),(2,2)])]) +("[abhgefdc]ij", [("hij", [(0,3)])]) + +("(a|b)c*d", [("abcd", [(1,3),(1,1)])]) +("(ab|ab*)bc", [("abc", [(0,3),(0,1)])]) + +("a([bc]*)c*", [("abc", [(0,3),(1,2)])]) +("a([bc]*)(c*d)", [("abcd", [(0,4),(1,2),(3,1)])]) +("a([bc]+)(c*d)", [("abcd", [(0,4),(1,2),(3,1)])]) +("a([bc]*)(c+d)", [("abcd", [(0,4),(1,1),(2,2)])]) +("a[bcd]*dcdcde", [("adcdcde", [(0,7)])]) +("(ab|a)b*c", [("abc", [(0,3),(0,2)])]) +("((a)(b)c)(d)", [("abcd", [(0,4),(0,3),(0,1),(1,1),(3,1)])]) + +("[A-Za-z_][A-Za-z0-9_]*", [("alpha", [(0,5)])]) +("^a(bc+|b[eh])g|.h$", [("abh", [(1,2)])]) +("(bc+d$|ef*g.|h?i(j|k))", [("effgz", [(0,5),(0,5)]), ("ij", [(0,2),(0,2),(1,1)]), ("reffgz", [(1,5),(1,5)])]) +("(((((((((a)))))))))", [("a", [(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1),(0,1)])]) +("multiple words", [("multiple words yeah", [(0,14)])]) + +("(.*)c(.*)", [("abcde", [(0,5),(0,2),(3,2)])]) +("abcd", [("abcd", [(0,4)])]) +("a(bc)d", [("abcd", [(0,4),(1,2)])]) +("a[-]?c", [("ac", [(0,3)])]) + +("M[ou]'?am+[ae]r .*([AEae]l[- ])?[GKQ]h?[aeu]+([dtz][dhz]?)+af[iy]", [("Muammar Qaddafi", [(0,15),(-1,0),(10,2)]), ("Mo'ammar Gadhafi", [(0,16),(-1,0),(11,2)]), ("Muammar Kaddafi", [(0,15),(-1,0),(10,2)]), ("Muammar Qadhafi", [(0,15),(-1,0),(10,2)]), ("Muammar Gadafi", [(0,14),(-1,0),(10,1)]), ("Mu'ammar Qadafi", [(0,15),(-1,0),(11,1)]), ("Moamar Gaddafi", [(0,14),(-1,0),(9,2)]), ("Mu'ammar Qadhdhafi", [(0,18),(-1,0),(13,2)]), ("Muammar Khaddafi", [(0,16),(-1,0),(11,2)]), ("Muammar Ghaddafy", [(0,16),(-1,0),(11,2)]), ("Muammar Ghadafi", [(0,15),(-1,0),(11,1)]), ("Muammar Ghaddafi", [(0,16),(-1,0),(11,2)]), ("Muamar Kaddafi", [(0,14),(-1,0),(9,2)]), ("Muammar Quathafi", [(0,16),(-1,0),(11,2)]), ("Muammar Gheddafi", [(0,16),(-1,0),(11,2)]), ("Moammar Khadafy", [(0,15),(-1,0),(11,1)]), ("Moammar Qudhafi", [(0,15),(-1,0),(10,2)])]) + +("a+(b|c)*d+", [("aabcdd", [(0,6),(3,1)])]) +("^.+$", [("vivi", [(0,4)])]) +("^(.+)$", [("vivi", [(0,4),(0,4)])]) +("^([^!.]+).att.com!(.+)$", [("gryphon.att.com!eby", [(0,19),(0,7),(16,3)])]) + +("^([^!]+!)?([^!]+)$", [("bas", [(0,3),(-1,0),(0,3)]), ("bar!bas", [(0,7),(0,4),(4,3)]), ("foo!bas", [(0,7),(0,4),(4,3)])]) +("^.+!([^!]+!)([^!]+)$", [("foo!bar!bas", [(0,11),(4,4),(8,3)])]) +("((foo)|(bar))!bas", [("bar!bas", [(0,7),(0,3),(-1,0),(0,3)]), ("foo!bar!bas", [(4,7),(4,3),(-1,0),(4,3)]), ("foo!bas", [(0,7),(0,3),(0,3)])]) #NULL matches aren't found in 1st two tests. +("((foo)|bar)!bas", [("bar!bas", [(0,7),(0,3)]), ("foo!bar!bas", [(4,7),(4,3)]), ("foo!bas", [(0,7),(0,3),(0,3)])]) +("(foo|(bar))!bas", [("bar!bas", [(0,7),(0,3),(0,3)]), ("foo!bar!bas", [(4,7),(4,3),(4,3)]), ("foo!bas", [(0,7),(0,3)])]) +("(foo|bar)!bas", [("bar!bas", [(0,7),(0,3)]), ("foo!bar!bas", [(4,7),(4,3)]), ("foo!bas", [(0,7),(0,3)])]) +("^(([^!]+!)?([^!]+)|.+!([^!]+!)([^!]+))$", [("bas", [(0,3),(0,3),(0,0),(0,3)]), ("bar!bas", [(0,7),(0,7),(0,4),(4,3)]), ("foo!bar!bas", [(0,11),(0,11),(0,0),(0,0),(4,4),(8,3)]), ("foo!bas", [(0,7),(0,7),(0,4),(4,3)])]) #4th fails; doesn't capture NULL groups defined in unselected (1st) alternative. +("^([^!]+!)?([^!]+)$|^.+!([^!]+!)([^!]+)$", [("bas", [(0,3),(-1,0),(0,3)]), ("bar!bas", [(0,7),(0,4),(4,3)]), ("foo!bar!bas", [(0,11),(1,0),(-1,0),(4,4),(8,3)]), ("foo!bas", [(0,7),(0,4),(4,3)])]) #3rd fails. + +(".*(/XXX).*", [("/XXX", [(0,4),(0,4)])]) +(".*(\\\\XXX).*", [("\\XXX", [(0,4),(0,4)])]) +("\\\\XXX", [("\\XXX", [(0,4)])]) +(".*(/000).*", [("/000", [(0,4),(0,4)])]) +(".*(\\\\000).*", [("\\000", [(0,4),(0,4)])]) +("\\\\000", [("\\000", [(0,4)])]) + diff --git a/data/Tests_ATTInterpretation.txt b/data/Tests_ATTInterpretation.txt new file mode 100644 --- /dev/null +++ b/data/Tests_ATTInterpretation.txt @@ -0,0 +1,113 @@ +#AUTHOR: Dr. Alistair Ward +#DESCRIPTION: Designed to be read by the module "FunctionAlley.RegExChar.Assert.RegExOptsChar". +#Based on . + +("a+", [("xaax", [(1,2)])]) +(".(a*).", [("xaax", [(0,4),(1,2)])]) + +("(a?)((ab)?)", [("ab", [(0,2),(0,0),(0,2),(0,2)])]) #Returns [(0,1),(0,1),(1,0),(-1,0)] +("(a?)((ab)?)(b?)", [("ab", [(0,2),(0,1),(1,0),(-1,0),(1,1)])]) +("((a?)((ab)?))(b?)", [("ab", [(0,2),(0,2),(0,0),(0,2),(0,2),(2,0)])]) #Returns [(0,2),(0,1),(0,1),(1,0),(-1,0),(1,1)] +("(a?)(((ab)?)(b?))", [("ab", [(0,2),(0,1),(1,1),(1,0),(-1,0),(1,1)])]) + +("(.?)", [("x", [(0,1),(0,1)])]) +("(.?){1}", [("x", [(0,1),(0,1)])]) +("(.?)(.?)", [("x", [(0,1),(0,1),(1,0)])]) +("(.?){2}", [("x", [(0,1),(1,0)])]) +("(.?)*", [("x", [(0,1),(0,1)])]) + +("(.?.?)", [("xxx", [(0,2),(0,2)])]) +("(.?.?){1}", [("xxx", [(0,2),(0,2)])]) +("(.?.?)(.?.?)", [("xxx", [(0,3),(0,2),(2,1)])]) +("(.?.?){2}", [("xxx", [(0,3),(2,1)])]) +("(.?.?)(.?.?)(.?.?)", [("xxx", [(0,3),(0,2),(2,1),(3,0)])]) +("(.?.?){3}", [("xxx", [(0,3),(3,0)])]) +("(.?.?)*", [("xxx", [(0,3),(2,1)])]) + +("a?((ab)?)(b?)", [("ab", [(0,2),(1,0),(-1,0),(1,1)])]) +("(a?)((ab)?)b?", [("ab", [(0,2),(0,1),(1,0),(-1,0)])]) +("a?((ab)?)b?", [("ab", [(0,2),(1,0),(-1,0)])]) + +("(a*){2}", [("xxxxx", [(0,0),(0,0)])]) + +("(ab?)(b?a)", [("aba", [(0,3),(0,2),(2,1)])]) +("(a|ab)(ba|a)", [("aba", [(0,3),(0,2),(2,1)])]) +("(a|ab|ba)", [("aba", [(0,2),(0,2)])]) +("(a|ab|ba)(a|ab|ba)", [("aba", [(0,3),(0,2),(2,1)])]) +("(a|ab|ba)*", [("aba", [(0,3),(2,1)])]) + +("(aba|a*b)", [("ababa", [(0,3),(0,3)])]) +("(aba|a*b)(aba|a*b)", [("ababa", [(0,5),(0,2),(2,3)])]) #Returns [(0,4),(0,3),(3,1)] +("(aba|a*b)*", [("ababa", [(0,5),(2,3)])]) +("(aba|ab|a)", [("ababa", [(0,3),(0,3)])]) +("(aba|ab|a)(aba|ab|a)", [("ababa", [(0,5),(0,2),(2,3)])]) +("(aba|ab|a)*", [("ababa", [(0,5),(2,3)])]) + +("(a(b)?)", [("aba", [(0,2),(0,2),(1,1)])]) +("(a(b)?)(a(b)?)", [("aba", [(0,3),(0,2),(1,1),(2,1),(-1,0)])]) #Returns [(0,3),(0,2),(1,1),(2,1),(3,0)] +("(a(b)?)+", [("aba", [(0,3),(2,1),(-1,0)])]) #Returns [(0,(0,3)),(1,(2,1)),(2,(3,0))] + +("(.*)(.*)", [("xx", [(0,2),(0,2),(2,0)])]) +(".*(.*)", [("xx", [(0,2),(2,0)])]) + +("(a.*z|b.*y)", [("azbazby", [(0,5),(0,5)])]) +("(a.*z|b.*y)(a.*z|b.*y)", [("azbazby", [(0,7),(0,5),(5,2)])]) +("(a.*z|b.*y)*", [("azbazby", [(0,7),(5,2)])]) + +("(.|..)(.*)", [("ab", [(0,2),(0,2),(2,0)])]) +("((..)*(...)*)", [("xxx", [(0,3),(0,3),(-1,0),(0,3)])]) #Returns [(0,2),(0,2),(0,2),(-1,0)] +("((..)*(...)*)((..)*(...)*)", [("xxx", [(0,3),(0,3),(-1,0),(0,3),(3,0),(-1,0)])]) #Returns [(0,2),(0,2),(0,2),(-1,0),(2,0),(-1,0),(-1,0)] +("((..)*(...)*)*", [("xxx", [(0,3),(0,3),(-1,0),(0,3)])]) #Returns [(0,2),(0,2),(0,2),(-1,0)] + +#Back-references haven't yet been implemented. +#("(a{0,1})*b\1", [("ab", [(0,2),(1,0)])]) +#("(a*)*b\1", [("ab", [(0,2),(1,0)])]) +#("(a*)b\1*", [("ab", [(0,2),(0,1)])]) +#("(a*)*b\1*", [("ab", [(0,2),(1,0)])]) +#("(a{0,1})*b(\1)", [("ab", [(0,2),(1,0),(2,0)])]) +#("(a*)*b(\1)", [("ab", [(0,2),(1,0),(2,0)])]) +#("(a*)b(\1)*", [("ab", [(0,2),(0,1),(0,0)])]) +#("(a*)*b(\1)*", [("ab", [(0,2),(1,0),(2,0)])]) + +#("(a{0,1})*b\1", [("aba", [(0,3),(0,1)])]) +#("(a*)*b\1", [("aba", [(0,3),(0,1)])]) +#("(a*)b\1*", [("aba", [(0,3),(0,1)])]) +#("(a*)*b\1*", [("aba", [(0,3),(0,1)])]) +#("(a*)*b(\1)*", [("aba", [(0,3),(0,1),(2,1)])]) + +#("(a{0,1})*b\1", [("abaa", [(0,3),(0,1)])]) +#("(a*)*b\1", [("abaa", [(0,3),(0,1)])]) +#("(a*)b\1*", [("abaa", [(0,4),(0,1)])]) +#("(a*)*b\1*", [("abaa", [(0,4),(0,1)])]) +#("(a*)*b(\1)*", [("abaa", [(0,4),(0,1),(3,1)])]) + +#("(a{0,1})*b\1", [("aab", [(0,3),(2,0)])]) +#("(a*)*b\1", [("aab", [(0,3),(2,0)])]) +#("(a*)b\1*", [("aab", [(0,3),(0,2)])]) +#("(a*)*b\1*", [("aab", [(0,3),(2,0)])]) +#("(a*)*b(\1)*", [("aab", [(0,3),(2,0),(3,0)])]) + +#("(a{0,1})*b\1", [("aaba", [(0,4),(1,1)])]) +#("(a*)*b\1", [("aaba", [(0,4),(1,1)])]) +#("(a*)b\1*", [("aaba", [(0,3),(0,2)])]) +#("(a*)*b\1*", [("aaba", [(0,4),(1,1)])]) +#("(a*)*b(\1)*", [("aaba", [(0,4),(1,1),(3,1)])]) + +#("(a{0,1})*b\1", [("aabaa", [(0,4),(1,1)])]) +#("(a*)*b\1", [("aabaa", [(0,5),(0,2)])]) +#("(a*)b\1*", [("aabaa", [(0,5),(0,2)])]) +#("(a*)*b\1*", [("aabaa", [(0,5),(0,2)])]) +#("(a*)*b(\1)*", [("aabaa", [(0,5),(0,2),(3,2)])]) + +#("(x)*a\1*", [("a", [(0,1),(1,0)])]) +#("(x)*a(\1)*", [("a", [(0,1),(-1,0),(-1,0)])]) + +("(aa(b(b))?)+", [("aabbaa", [(0,6),(4,2),(-1,0),(-1,0)])]) #Returns [(0,6),(4,2),(6,0)] +("(a(b)?)+", [("aba", [(0,3),(2,1),(-1,0)])]) #Returns [(0,3),(2,1),(3,0)] +("([ab]+)([bc]+)([cd]*)", [("abcd", [(0,4),(0,2),(2,1),(3,1)])]) +#("([ab]*)([bc]*)([cd]*)\1", [("abcdaa", [(0,5),(0,1),(1,2),(3,1)]), ("abcdab", [(0,6),(0,2),(2,1),(3,1)])]) +#("([ab]*)([bc]*)([cd]*)\1*", [("abcdaa", [(0,6),(0,1),(1,2),(3,1)]), ("abcdab", [(0,6),(0,2),(2,1),(3,1)])]) +("^(A([^B]*))?(B(.*))?", [("Aa", [(0,2),(0,2),(1,1)]), ("Bb", [(0,2),(-1,0),(-1,0),(0,2),(1,1)])]) #Returns [(0,2),(0,2),(1,1),(2,0)], [(0,2),(-1,0),(0,2),(1,1)] +#(".*([AB]).*\1", [("ABA", [(0,3),(0,1)])]) +("[^A]*A", [("\nA", [(0,2)])]) + diff --git a/makefile b/makefile new file mode 100644 --- /dev/null +++ b/makefile @@ -0,0 +1,75 @@ +# Copyright (C) 2010 Dr. Alistair Ward +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +.PHONY: all build clean clobber configure copy deb haddock help hlint install lintian prof rpm rpmlint sdist + +all: install + +install: build haddock + @[ -z "$$CABAL_INSTALL_OPTIONS" ] || echo "INFO: CABAL_INSTALL_OPTIONS='$$CABAL_INSTALL_OPTIONS'" + runhaskell Setup.hs $@ $$CABAL_INSTALL_OPTIONS + +prof: + CABAL_CONFIGURE_OPTIONS="--enable-library-profiling --enable-executable-profiling $$CABAL_CONFIGURE_OPTIONS" make install + +copy: build + @[ -z "$$CABAL_COPY_OPTIONS" ] || echo "INFO: CABAL_COPY_OPTIONS='$$CABAL_COPY_OPTIONS'" + runhaskell Setup.hs $@ $$CABAL_COPY_OPTIONS + +build: configure + @[ -z "$$CABAL_BUILD_OPTIONS" ] || echo "INFO: CABAL_BUILD_OPTIONS='$$CABAL_BUILD_OPTIONS'" + runhaskell Setup.hs $@ $$CABAL_BUILD_OPTIONS + +configure: regexchar.cabal Setup.hs + @[ -z "$$CABAL_CONFIGURE_OPTIONS" ] || echo "INFO: CABAL_CONFIGURE_OPTIONS='$$CABAL_CONFIGURE_OPTIONS'" + runhaskell Setup.hs $@ $$CABAL_CONFIGURE_OPTIONS #--user + +haddock: configure + PATH=~/.cabal/bin:$$PATH runhaskell Setup.hs $@ --hyperlink-source --executables #Amend path to find 'HsColour', as required for 'hyperlink-source'. + +hlint: + @$@ src/ + +sdist: configure + runhaskell Setup.hs $@ + +rpm: regexchar.spec + rpmbuild -bb -- $^ + mv -- `find /usr/src/packages/ -type f -name 'regexchar*.rpm' -mtime 0 -print` ./ + +rpmlint: regexchar.spec + @$@ $^ + +deb: build + mkdir -p 'debian/usr/bin/' && strip --strip-all 'dist/build/grecce/grecce' && mv 'dist/build/grecce/grecce' 'debian/usr/bin/' + mkdir -p 'debian/usr/share/doc/regexchar/' && cp 'changelog' 'changelog.Debian' 'LICENSE' 'debian/usr/share/doc/regexchar/' && gzip --best debian/usr/share/doc/regexchar/* + mkdir -p 'debian/usr/share/regexchar/data/' && cp data/*.txt 'debian/usr/share/regexchar/data/' + cp 'copyright' 'debian/usr/share/doc/regexchar/' + mkdir -p 'debian/usr/share/man/man1/' && cp 'man/man1/grecce.1' 'debian/usr/share/man/man1/' && gzip --best debian/usr/share/man/man1/*.1 + cd 'debian' && md5sum `find usr -type f -print` >'DEBIAN/md5sums' + fakeroot dpkg-deb --build debian . + +lintian: + lintian -v *.deb + +clean: + runhaskell Setup.hs $@ + rm -rf 'debian/usr/' 'debian/DEBIAN/md5sums' + +clobber: clean + rm -f -- *.rpm *.deb + +help: + @grep '^[a-zA-Z].*:' makefile | sed -e 's/:.*//' diff --git a/man/man1/grecce.1 b/man/man1/grecce.1 new file mode 100644 --- /dev/null +++ b/man/man1/grecce.1 @@ -0,0 +1,173 @@ +.TH grecce 1 +.SH NAME +\fBgrecce\fR - print lines matching a pattern. +.SH SYNOPSIS +\fBgrecce\fR [\fIOPTIONS\fR] \fIFILE\fR ... +.SH DESCRIPTION +.PP +Searches the named input \fIFILE\fRs (or standard input if no files are named, or if \fIFILE\fR is defined as a single hyphen-minus (\fB-\fR)), +for lines containing a match for \fIregex\fR. +.PP +By default, prints the matching lines. +.SS Supported Regular-expression Syntax +.IP \(bu +\fIAnchors\fR, including around \fIsub-expression\fRs. +.IP \(bu +Standard \fIgreedy\fR & also \fBPerl\fR-style \fInon-greedy\fR, quantification. +.IP \(bu +\fICapture-groups\fR, potentially containing alternative \fIsub-expression\fRs. +.IP \(bu +\fIBracket-expressions\fR +.br +(including \fBPerl\fR-style \fIshortcuts\fR for; \fB\\d\fR, \fB\\D\fR, \fB\\s\fR, \fB\\S\fR, \fB\\w\fR, \fB\\W\fR) +.IP \(bu +Posix \fICharacter-classes\fR; \fB[:alnum:]\fR, \fB[:alpha:]\fR, \fB[:ascii:]\fR, \fB[:blank:]\fR, \fB[:cntrl:]\fR, \fB[:digit:]\fR, \fB[:graph:]\fR, \fB[:lower:]\fR, \fB[:print:]\fR, \fB[:punct:]\fR, \fB[:space:]\fR, \fB[:upper:]\fR, \fB[:word:]\fR, \fB[:xdigit:]\fR +.SH OPTIONS +.SS "General Output Control" +.TP +\fB-c\fR, \fB--count\fR +Print only a count of matching lines per file. +.TP +\fB-l\fR, \fB--files-with-matches\fR +List names of files containing at least one match. +.TP +\fB-L\fR, \fB--files-without-match\fR +List names of files not containing any matches. +.TP +\fB--verbose\fR +.IP \(bu +Shows how the input data was captured by \fIregex\fR. +.IP \(bu +The output is a triple (\fIprefix\fR, data consumed by \fIregex\fR, \fIsuffix\fR): +If data is consumed either before, by, or after \fIregex\fR, then the corresponding section contains \fBJust\fR, otherwise it contains \fBNothing\fR. +.IP \(bu +If the regex matched, the central section, following \fBJust\fR, +details the consumption of each term in the \fIregex\fR, in terms of another triple; (\fIprinted term\fR, \fIoffset\fR, \fIinput-data\fR). +.IP \(bu +Because of the inherently recursive nature of a regex, the output is actually tree-structured. +.IP \(bu +This is an addition to the options available in \fBegrep\fR, & provides a powerful insight into the inner workings of the regex-engine; useful for debugging the regex. +.SS "Output Line Prefix Control" +.TP +\fB-n\fR, \fB--number\fR +Prepend the line-number of the input, to the output. +.SS "Matching Control" +.TP +\fB-e\fR \fIregex\fR, \fB--regexp=\fR\fIregex\fR +Define the regex. +.TP +\fB-f\fR \fIfile-name\fR, \fB--file=\fR\fIfile-name\fR +Read the regex from \fIfile-name\fR. +.TP +\fB-v\fR, \fB--invert-match\fR +Select non-matching lines. +.TP +\fB--complyStrictlyWithPosix=\fR\fIBool\fR +Define the offset of input-data, corresponding to a \fIcapture-group\fR which matched zero times, as the artificial value \fB-1\fR specified by \fBPosix\fR. +This is an addition to the options available in \fBegrep\fR, & is of relatively little use. +.SS Performance +.TP +\fB +RTS -N\fR [\fICores\fR] [\fB-H\fR \fIHeap-size\fR] \fB-RTS\fR +If the executable has been linked with the threaded GHC-runtime, use \fICores\fR simultaneous threads (defaulting to the number of CPU-cores available), +to process alternative sub-expressions and multiple files in parallel; +When specifying this option, it is usually beneficial, given sufficient resources, to suggest a large initial heap-size for the garbage-collector. +.br +. +.PP +These options are rather esoteric & not normally required. +.br +\fIBool\fR can be defined as either "\fBTrue\fR" or "\fBFalse\fR". +.TP +\fB--abortTrialRepetitionsOnInherentFailure=\fR\fIBool\fR +If an alternative can't match, irrespective of the subsequent concatenation, then terminate futile trial repetitions. +.TP +\fB--catchIncompatibleAnchors=\fR\fIBool\fR +Avoid futile trial solutions, involving repetitions of anchored alternatives, which must consume data. +.TP +\fB--checkExistenceOfInelasticTail=\fR\fIBool\fR +If \fIregex\fR ends in one or more meta-data, whose repetition-range is precisely defined, check that this sequence exists at the end of the input data. +.TP +\fB--checkForUnconsumableData=\fR\fIBool\fR +Check whether there's no possibility of consuming some of the input data. +.TP +\fB--moderateGreed=\fR\fIBool\fR +Greedily consume data, only up to the limit beyond which, future requirements would be compromised. +.TP +\fB--permitReorderingOfAlternatives=\fR\fIBool\fR +Permit alternatives to be re-ordered, in an attempt to process the cheapest first. +.TP +\fB--unrollRepeatedSingletonAlternative=\fR\fIBool\fR +Unroll repetitions of singleton alternatives; this doesn't affect the result. +.TP +\fB--useFirstMatchAmongAlternatives=\fR\fIBool\fR +Rather than performing an exhaustive search for the optimal choice amongst alternatives, merely select the first that matches; conform to Perl rather than Posix. +.TP +\fB--validateMinConsumptionOfAlternatives=\fR\fIBool\fR +When the number of repetitions of a group of alternatives is precisely specified, check the availability of the resulting minimum data-requirement. +.SS Match-preference +.TP +\fB--preferAlternativesWhichFeedTheGreedy=\fR\fIBool\fR +Prefer solutions in which the choice of alternatives directs data from non-greedy to greedy consumers. +.TP +\fB--preferAlternativesWhichMimickUnrolling=\fR\fIBool\fR +Prefer solutions in which the choice of alternatives consumes data like the unrolled repeatable group. +.TP +\fB--preferFewerRepeatedAlternatives=\fR\fIBool\fR +Prefer solutions employing fewer repetitions of alternatives, to discourage the capture of null lists. +.SS "Generic Program-information" +.TP +\fB--version\fR +Output version-information & exit. +.TP +\fB-?\fR, \fB--help\fR +Display help & exit. +.TP +\fB--printCurrentSettings\fR +Display the currently defined options & then exit. +.SH EXIT-STATUS +\fB0\fR on success, and >\fB0\fR if an error occurs. +.SH EXAMPLES +.TP +\fBecho\fR 'Function Alley' | \fBgrecce\fR 'n\\sA' +Function Alley +.TP +\fBecho\fR 'Function Alley' | \fBgrecce\fR --verbose '[[:alpha:]]+' +(Just (.*?,0,""),Just [([[:alpha:]]+,0,"Function")],Just (.*,8," Alley")) +.SH Pathological +.TP +\fBecho\fR 'aaa' | \fBgrecce\fR --verbose '^(a?){3}(a){3}$' +(Nothing,Just [[[('a'?,0,"")],[('a'?,0,"")],[('a'?,0,"")]],[[('a',0,"a")],[('a',1,"a")],[('a',2,"a")]]],Nothing) +.TP +\fBecho\fR 'azazaz' | \fBgrecce\fR --verbose '^(.*z){3}$' +(Nothing,Just [[[(.*,0,"a"),('z',1,"z")],[(.*,2,"a"),('z',3,"z")],[(.*,4,"a"),('z',5,"z")]]],Nothing) +.TP +\fBecho\fR 'aaaaaa' | \fBgrecce\fR --verbose '^a*a+a{2,}a{3,}$' +(Nothing,Just [('a'*,0,""),('a'+,0,"a"),('a'{2,},1,"aa"),('a'{3,},3,"aaa")],Nothing) +.TP +\fBecho\fR 'aaaaaaaabc' | \fBgrecce\fR --verbose '(a|a{2})*c' +(Just (.*?,0,"aaaaaaaab"),Just [[],('c',9,"c")],Just (.*,10,"")) +.SH AUTHOR +Written by Dr. Alistair Ward. +.SH BUGS +.SS "REPORTING BUGS" +Report bugs to \fBgrecce\fR \fIat\fR \fBfunctionalley\fR \fIdot\fR \fBeu\fR +.PP +Home page: \fBhttp://functionalley.eu\fR +.SS Known Bugs +There is no support for either \fIBack-references\fR or \fICase-insensitivity\fR. +.PP +The only support for \fIZero-width assertions\fR, is for \fIAnchors\fR. +.PP +File-reading is rather slow, due to the inability to optimise the underlying polymorphic regex-engine for Characters. +.SH COPYRIGHT +Copyright \(co 2010 Dr. Alistair Ward +.PP +This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. +.PP +This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. +.PP +You should have received a copy of the GNU General Public License along with this program. If not, see \fB\fR. +.SH "SEE ALSO" +The full documentation for \fBgrecce\fR is generated by \fBHaddock\fR, & is available in the distribution. +.PP +Built from \fBRegExChar\fR, a specialisation of the polymorphic regex-engine \fBRegExDot\fR. diff --git a/regexchar.cabal b/regexchar.cabal new file mode 100644 --- /dev/null +++ b/regexchar.cabal @@ -0,0 +1,89 @@ +--Package-properties +Name: regexchar +Version: 0.9.0.7 +Cabal-Version: >= 1.6 +Copyright: (C) 2010 Dr. Alistair Ward +License: GPL +License-file: LICENSE +Author: Dr. Alistair Ward +Stability: Stable interface, incomplete features. +Synopsis: A POSIX extended regular-expression engine. +Build-Type: Simple +Description: Provides a POSIX, extended regex-engine, specialised from the underlying polymorphic engine /regexdot/, to merely process character-lists, as is typically required. +Category: Regex, Text +Tested-With: GHC == 6.10, GHC == 6.12, GHC == 7.0 +Homepage: http://functionalley.eu +Maintainer: regexchar functionalley eu +Bug-reports: regexchar functionalley eu +Data-files: data/*.txt +Extra-Source-Files: man/man1/*.1 *.spec changelog copyright makefile + +flag threaded + Description: Link with the threaded runtime. + default: True + +flag llvm + Description: Whether the 'llvm' backend has been installed and is required for code-generation. + manual: True + default: False + +Library + hs-source-dirs: src + + Exposed-modules: + RegExChar.ExtendedRegExChar + RegExChar.MetaChar + RegExChar.RegExOptsChar + + Build-depends: + base == 4.*, + containers, + haskell98, + parsec == 3.*, + regexdot == 0.10.*, + toolshed == 0.11.* + + GHC-options: -Wall + GHC-prof-options: -prof -auto-all -caf-all + + if impl(ghc >= 7.0) && flag(llvm) + GHC-options: -fllvm + +Executable grecce + hs-source-dirs: src + Main-Is: Main.hs + Other-modules: + Grecce.Assert.RegExOptsChar + Grecce.CommandOptions + Grecce.Grep + Grecce.Performance.ExtendedRegEx + Grecce.Performance.ExtendedRegExTest + Grecce.Performance.ExtendedRegExTestsNegative + Grecce.Performance.ExtendedRegExTestsPositive + Grecce.QC.ExtendedRegExChar + Grecce.QC.MetaChar + Grecce.QC.QuickChecks + Grecce.QC.RepeatableMetaChar + + Build-depends: + array, + Cabal >= 1.6 && < 2, + QuickCheck >= 2.4, + regex-base, + regex-posix + + GHC-options: -Wall -O2 + GHC-prof-options: -prof -auto-all -caf-all + + if flag(threaded) + Build-depends: parallel >= 3.0 + GHC-options: -threaded -feager-blackholing + else + Build-depends: parallel + + if impl(ghc >= 7.0) + GHC-options: -rtsopts + + if flag(llvm) + GHC-options: -fllvm + diff --git a/regexchar.spec b/regexchar.spec new file mode 100644 --- /dev/null +++ b/regexchar.spec @@ -0,0 +1,50 @@ +%define package %name-%version +%define tarBall %package.tar.gz +%define _bindir %prefix/bin +%define _libdir %prefix/lib/%package +%define _datadir %prefix/share/%package +%define _docdir %prefix/share/doc/%package +%define _mandir %prefix/share/man + +Summary: A re-implementation of egrep, using a specialisation of a polymorphic regex-engine. +Name: regexchar +Version: 0.9.0.7 +Release: 1 +License: GPLv3 +#From '/usr/share/doc/packages/rpm/GROUPS'. +Group: Applications/Text +Source0: http://functionalley.eu/Downloads/sdist/%tarBall +URL: http://functionalley.eu +Prefix: /usr + +%description +Provides a POSIX extended regex-engine, specialised from an underlying polymorphic engine, to process merely character-lists, as is typically required. + +%prep +rm -rf -- '%package/' +CABAL_CONFIGURE_OPTIONS='--user' make --directory="$OLDPWD/" sdist && mv -- "$OLDPWD/dist/%tarBall" '%_sourcedir/' && tar -zxf '%_sourcedir/%tarBall' #Build & copy the tar-ball into the source-directory, then unpack it into the build-directory. + +%build + +%install +rm -rf -- '%buildroot/' #Clean any previous build. CAVEAT: the user might have specified '--buildroot', for which I can't account. +CABAL_CONFIGURE_OPTIONS='--user --prefix=%buildroot%prefix/' make --directory='%package/' install #Which regrettably also builds 'Haddock', which then requires 'HsColour'. +mkdir -p -- '%buildroot%_mandir/' && mv -- '%_builddir/%package/man/man1' '%buildroot%_mandir/' +mkdir -p -- '%buildroot%_docdir/' && mv -- %_builddir/%package/{changelog,copyright,LICENSE} '%buildroot%_docdir/' +rm -- $(find '%buildroot%_libdir/' \( -name '*.o' -o -name '*.hi' \) -print) +rm -rf -- $(find '%buildroot' -type d -name 'html') #We don't need source-code documentation in a binary package. + +%clean +make --directory='%package/' clobber +rm -- '%_sourcedir/%tarBall' +rm -rf -- '%_builddir/%package' '%buildroot' + +%files +%attr(0755, root, root) %_bindir/grecce +%attr(0644, root, root) %_libdir/ghc-*/libHS%{package}.a +%attr(0644, root, root) %doc %_mandir/man1/*.1.gz +%attr(0644, root, root) %doc %_docdir/changelog +%attr(0644, root, root) %doc %_docdir/copyright +%attr(0644, root, root) %doc %_docdir/LICENSE +%attr(0644, root, root) %_datadir/data/*.txt + diff --git a/src/Grecce/Assert/RegExOptsChar.hs b/src/Grecce/Assert/RegExOptsChar.hs new file mode 100644 --- /dev/null +++ b/src/Grecce/Assert/RegExOptsChar.hs @@ -0,0 +1,92 @@ +{-# LANGUAGE FlexibleContexts #-} +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] + + * Reads lines from the specified file. + + * Each line describes a 'Test'. + + * Each input-data string should match the /regex/ in the specified way. + + * Permits comparison with "Text.Regex.Posix.String". +-} + +module Grecce.Assert.RegExOptsChar( +-- * Types +-- ** Type-synonyms +-- Test, +-- TestDataAndResult, +-- TestFilter, +-- * Functions + findFailures, + findFailuresPosix, + readTests +-- ** Predicates +-- isMatch +) where + +import Control.Applicative((<$>)) +import Text.Regex.Posix() +import qualified Control.Arrow +import qualified RegExChar.ExtendedRegExChar as ExtendedRegExChar +import qualified RegExChar.RegExOptsChar as RegExOptsChar --CAVEAT: beware of the similar name. +import qualified RegExDot.CompilationOptions as CompilationOptions +import qualified RegExDot.ExecutionOptions as ExecutionOptions +import qualified Text.Regex.Base.RegexLike as RegexLike +import qualified Text.Regex.Posix.Wrap + +-- | Composed input data & list of captured 'RegExOptsChar.MatchSpan'. +type TestDataAndResult = (ExtendedRegExChar.InputData, [RegExOptsChar.MatchSpan]) + +-- | Composed of the 'String' from which an unspecified type of regex is read, & a list of input data & results, to which it should respectively be applied & compared. +type Test = (String {-regex-source-}, [TestDataAndResult]) + +-- | Filters a list of 'Test's, & returns a sub-set. +type TestFilter = [Test] -> [Test] + +-- | 'True' if the specified 'ExtendedRegExChar.InputData', is a member of the set defined by the polymorphic 'Text.Regex.Base.RegexLike.RegexLike' parameter. +isMatch :: RegexLike.RegexLike r ExtendedRegExChar.InputData => r -> TestDataAndResult -> Bool +isMatch regex (inputData, matchSpanList) = RegexLike.matchOnce regex inputData == Just (RegExOptsChar.toZeroIndexedArray matchSpanList) + +-- | Returns those 'Test's, which 'RegExOptsChar.RegExOptsChar' fails. +findFailures :: CompilationOptions.CompilationOptions -> ExecutionOptions.ExecutionOptions -> TestFilter +findFailures compilationOptions executionOptions = filter (not . uncurry all . Control.Arrow.first (isMatch . RegexLike.makeRegexOpts compilationOptions executionOptions)) + +-- | Returns those 'Test's, which 'Text.Regex.Posix.Wrap.Regex' fails. +findFailuresPosix :: TestFilter +findFailuresPosix = filter (not . uncurry all . Control.Arrow.first ((isMatch :: Text.Regex.Posix.Wrap.Regex -> TestDataAndResult -> Bool) . RegexLike.makeRegex)) + +{- | + * Read 'Test's from file, ignoring blank lines & comments. + + * To facilitate parsing, each line in the file has the format of @show Test@. +-} +readTests :: String -> IO [Test] +readTests fileName = map ( + \line -> case reads line of + [(test, _)] -> test + _ -> error $ "Grecce.Assert.RegExOptsChar.readTests:\tfailed to read 'Test' from '" ++ line ++ "'" + ) . filter ( + (/= '#') . head --Remove comments. + ) . filter ( + not . null --Remove blank lines. + ) . lines <$> readFile fileName + diff --git a/src/Grecce/CommandOptions.hs b/src/Grecce/CommandOptions.hs new file mode 100644 --- /dev/null +++ b/src/Grecce/CommandOptions.hs @@ -0,0 +1,172 @@ +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] + + * Defines the options, read from the command-line, + & as such assumes that the type-parameter of the underlying polymorphic /regex/-engine is 'Char'. + + * Supplies mutators, which ensure the consistency of the chosen set. +-} + +module Grecce.CommandOptions( +-- * Types +-- ** Type-synonyms + CompilationOptionsMutator, + ExecutionOptionsMutator, +-- Transformer, +-- ** Data-types + CommandOptions(..), +-- * Functions +-- ** Mutators + setCompilationOption, + setCountMatches, + setExecutionOption, + setInvertMatch, + setPrependLineNumbers, + setListFilesWithMatches, + setListFilesWithoutMatch, + setVerbose +) where + +import qualified RegExChar.ExtendedRegExChar as ExtendedRegExChar +import qualified RegExDot.CompilationOptions as CompilationOptions +import qualified RegExDot.ExecutionOptions as ExecutionOptions +import qualified ToolShed.Defaultable as Defaultable +import qualified ToolShed.Options as Options + +-- | The set of command-line options; based on those provided by /egrep/. +data CommandOptions = MkCommandOptions { + countMatches :: Bool, -- ^ Print only a count of matching lines per file. + extendedRegExChar :: Maybe ExtendedRegExChar.ExtendedRegExChar, -- ^ The /regex/. + invertMatch :: Bool, -- ^ Select non-matching lines. + prependLineNumbers :: Bool, -- ^ Prepend the line-number of the matched input-data, to the output. + listFilesWithMatches :: Bool, -- ^ List the names of any files containing at least one match. + listFilesWithoutMatch :: Bool, -- ^ List the names of any files containing zero matches. + verbose :: Bool, -- ^ Show the captured data. + compilationOptions :: CompilationOptions.CompilationOptions, -- ^ Used to define the regex-request. + executionOptions :: ExecutionOptions.ExecutionOptions -- ^ Used to control the regex-runtime. +} deriving Show + +-- | An arbitrary function which transforms the supplied 'ExecutionOptions.ExecutionOptions'. +type Transformer = CommandOptions -> CommandOptions + +instance Defaultable.Defaultable CommandOptions where + defaultValue = Options.blankValue { + countMatches = False, + extendedRegExChar = Nothing, + invertMatch = False, + prependLineNumbers = False, + listFilesWithMatches = False, + listFilesWithoutMatch = False, + verbose = False, + compilationOptions = Defaultable.defaultValue, + executionOptions = Defaultable.defaultValue + } + +instance Options.Options CommandOptions where + blankValue = MkCommandOptions { + countMatches = undefined, + extendedRegExChar = undefined, + invertMatch = undefined, + prependLineNumbers = undefined, + listFilesWithMatches = undefined, + listFilesWithoutMatch = undefined, + verbose = undefined, + compilationOptions = undefined, + executionOptions = undefined + } + +-- | Mutator. +setCountMatches :: Transformer +setCountMatches commandOptions + | ($ commandOptions) `any` [ + prependLineNumbers, + listFilesWithoutMatch, + listFilesWithMatches, + verbose + ] = error "Grecce.CommandOptions.setCountMatches:\t'countMatches' is incompatible with; 'prependLineNumbers', 'listFilesWithoutMatch', 'listFilesWithMatches' & 'verbose'." + | otherwise = commandOptions { countMatches = True } + +-- | Mutator. +setInvertMatch :: Transformer +setInvertMatch commandOptions + | verbose commandOptions = error "Grecce.CommandOptions.setInvertMatch:\t'invertMatch' is incompatible with 'verbose'." + | otherwise = commandOptions { invertMatch = True } + +-- | Mutator. +setPrependLineNumbers :: Transformer +setPrependLineNumbers commandOptions + | ($ commandOptions) `any` [ + countMatches, + listFilesWithoutMatch, + listFilesWithMatches + ] = error "Grecce.CommandOptions.setPrependLineNumbers:\t'prependLineNumbers' is incompatible with 'countMatches', 'listFilesWithoutMatch' & 'listFilesWithMatches'." + | otherwise = commandOptions { prependLineNumbers = True } + +-- | Mutator. +setListFilesWithMatches :: Transformer +setListFilesWithMatches commandOptions + | ($ commandOptions) `any` [ + countMatches, + prependLineNumbers, + listFilesWithoutMatch, + verbose + ] = error "Grecce.CommandOptions.setListFilesWithMatches:\t'listFilesWithMatches' is incompatible with; 'countMatches', 'prependLineNumbers', 'listFilesWithoutMatch' & 'verbose'." + | otherwise = commandOptions { listFilesWithMatches = True } + +-- | Mutator. +setListFilesWithoutMatch :: Transformer +setListFilesWithoutMatch commandOptions + | ($ commandOptions) `any` [ + countMatches, + prependLineNumbers, + listFilesWithMatches, + verbose + ] = error "Grecce.CommandOptions.setListFilesWithoutMatch:\t'listFilesWithoutMatch' is incompatible with; 'countMatches', 'prependLineNumbers', 'listFilesWithMatches' & 'verbose'." + | otherwise = commandOptions { listFilesWithoutMatch = True } + +-- | Mutator, which additionally sets the corresponding 'ExecutionOptions.ExecutionOptions'. +setVerbose :: Transformer +setVerbose commandOptions + | ($ commandOptions) `any` [ + countMatches, + invertMatch, + listFilesWithoutMatch, + listFilesWithMatches + ] = error "Grecce.CommandOptions.setVerbose:\t'verbose' is incompatible with; 'countMatches', 'invertMatch', 'listFilesWithoutMatch' & 'listFilesWithMatches'." + | otherwise = commandOptions { + verbose = True, + executionOptions = ExecutionOptions.setVerbose True $ executionOptions commandOptions + } + +-- | A function which accepts a 'String' & the 'CompilationOptions.CompilationOptions' to transform, using its value. +type CompilationOptionsMutator = String -> CompilationOptions.CompilationOptions -> CompilationOptions.CompilationOptions + +-- | Generic mutator, which accepts a function & the parameter-string it requires, to amend an unspecified field of 'CompilationOptions.CompilationOptions'. +setCompilationOption :: CompilationOptionsMutator -> String -> Transformer +setCompilationOption compilationOptionsMutator s commandOptions = commandOptions { compilationOptions = compilationOptionsMutator s $ compilationOptions commandOptions } + +-- | A function which accepts a 'String' & the 'ExecutionOptions.ExecutionOptions' to transform, using its value. +type ExecutionOptionsMutator = String -> ExecutionOptions.ExecutionOptions -> ExecutionOptions.ExecutionOptions + +-- | Generic mutator, which accepts a function & the parameter-string it requires, to amend an unspecified field of 'ExecutionOptions.ExecutionOptions'. +setExecutionOption :: ExecutionOptionsMutator -> String -> Transformer +setExecutionOption executionOptionsMutator s commandOptions = commandOptions { executionOptions = executionOptionsMutator s $ executionOptions commandOptions } + diff --git a/src/Grecce/Grep.hs b/src/Grecce/Grep.hs new file mode 100644 --- /dev/null +++ b/src/Grecce/Grep.hs @@ -0,0 +1,215 @@ +{-# LANGUAGE CPP #-} +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] Performs the match-operation on the specified file-list, using the user's command-line options. +-} + +module Grecce.Grep ( +-- * Types +-- LineNumber, +-- Matches, +-- ** Type-synonyms +-- * Functions + grep +) where + +import Control.Applicative((<$>)) +import Control.Arrow((&&&)) +import qualified Control.Arrow +import qualified Control.Monad +import qualified Data.List +import qualified Grecce.CommandOptions as CommandOptions +import qualified RegExChar.ExtendedRegExChar as ExtendedRegExChar +import RegExChar.ExtendedRegExChar as ExtendedRegExChar((+~), (=~)) +import qualified RegExDot.RegEx as RegEx +import qualified RegExDot.RegExOpts as RegExOpts +import qualified RegExDot.Result as Result +import qualified System +import qualified System.IO + +#if MIN_VERSION_parallel(3,0,0) +import qualified Control.Parallel.Strategies +#endif + +type LineNumber = Int +type Matches = Int + +-- | Creates a 'RegExOpts.RegExOpts' record from 'CommandOptions.CommandOptions', and uses it to filter the lines read from the list of input-data files, the result of which is then printed. +grep :: + CommandOptions.CommandOptions -- ^ The match-criteria, which may include the name of the file on which to operate. + -> [String] -- ^ A supplementary list of files on which to operate + -> IO () -- ^ Nothing is returned, since the result is printed. +grep commandOptions nonOptions +{- + The algorithm depends fundamentally on whether the data comes from 'stdin' or from file. + This is because of the requirement, in the former case, to deliver instantaneous feedback. + This forces us to print matches as they're found, rather than batching them for return to the caller to print. + If we COULD return 'IO [String]', then 'matchFilterStdIn' could be called instead of 'grep' when (filename == "-"); annoyingly, it just doesn't work. +-} + | any ($ fileNames) [null, all (== "-")] = let + matchFilterStdIn :: LineNumber -> Matches -> IO System.ExitCode + matchFilterStdIn lineNumber matches = {-#SCC "matchFilterStdIn" #-} do + isEOF <- System.IO.isEOF + + if isEOF + then if listFilesWithMatches + then return {-to IO-monad-} $ System.ExitFailure 1 --No match has been found, otherwise we'd have exited earlier. + else if listFilesWithoutMatch + then putStrLn dummyFileName >> return {-to IO-monad-} System.ExitSuccess + else do + Control.Monad.when countMatches $ print matches + + return {-to IO-monad-} $ if matches == 0 + then System.ExitFailure 1 + else System.ExitSuccess + else {-not EOF-} do + line <- getLine --Process line-at-a-time, to provide instant feedback. + + let + lineNumber' :: LineNumber + lineNumber' = lineNumber + 1 + + onSuccess :: String -> IO System.ExitCode + onSuccess line' + | listFilesWithMatches = putStrLn dummyFileName >> return {-to IO-monad-} System.ExitSuccess + | listFilesWithoutMatch = return {-to IO-monad-} $ System.ExitFailure 1 + | otherwise = do + Control.Monad.unless countMatches . putStrLn $ ( + if prependLineNumbers + then showLineNumber lineNumber' + else id + ) line' + + matchFilterStdIn lineNumber' $ matches + 1 --Recurse. + + onFailure, onMismatch :: IO System.ExitCode + onFailure = matchFilterStdIn lineNumber' matches --Recurse + onMismatch + | invertMatch = onSuccess line + | otherwise = onFailure + + if verbose + then let + result :: RegEx.Result Char + result = line +~ regExOpts + in if Result.isMatch result + then if invertMatch + then onFailure + else onSuccess $ show result + else onMismatch + else {-terse-} if line =~ regExOpts + then if invertMatch + then onFailure + else onSuccess line + else onMismatch + where + dummyFileName :: String + dummyFileName = "" + in matchFilterStdIn 0 0 >>= System.exitWith + | otherwise {-file-names provided-} = let + findMatch :: String -> RegEx.Result Char + findMatch = (+~ regExOpts) + + matchFilter :: [String] -> [String] + matchFilter = {-#SCC "matchFilter" #-} ( + if countMatches + then return {-to List-monad-} . show . length --Substitute the list of matches, with its length. + else id + ) . ( + let + isMatch :: String -> Bool + isMatch = {-#SCC "isMatch" #-} (if invertMatch then not else id) . (=~ regExOpts) + in if prependLineNumbers + then map (uncurry showLineNumber) . ( + if verbose + then map ( + Control.Arrow.second show + ) . filter ( + Result.isMatch . snd + ) . map ( + Control.Arrow.second findMatch + ) + else {-terse-} filter (isMatch . snd) + ) . zip [1 :: LineNumber ..] + else {-unnumbered-} if verbose + then map show . filter Result.isMatch . map findMatch + else {-terse-} filter isMatch + ) + + identifySource :: String -> [String] -> [String] + identifySource fileName results + | listFilesWithMatches = [fileName | not isEmpty] {-list-comprehension-} + | listFilesWithoutMatch = [fileName | isEmpty] {-list-comprehension-} + | length (Data.List.nub fileNames) > 1 = (showString fileName . separate) `map` results + | otherwise = results + where + isEmpty :: Bool + isEmpty = null results + in do + results +#if MIN_VERSION_parallel(3,0,0) + <- concat . Control.Parallel.Strategies.parMap Control.Parallel.Strategies.rdeepseq +#else + <- concatMap +#endif + ( + uncurry identifySource . Control.Arrow.second (matchFilter . {-#SCC "lines" #-} lines {-chops '\n'-}) + ) . zip fileNames <$> mapM readFile fileNames + + mapM_ putStrLn results + + System.exitWith $ if null results + then System.ExitFailure 1 + else System.ExitSuccess + where + countMatches, invertMatch, prependLineNumbers, listFilesWithMatches, listFilesWithoutMatch, verbose :: Bool + [countMatches, invertMatch, prependLineNumbers, listFilesWithMatches, listFilesWithoutMatch, verbose] = ($ commandOptions) `map` [ + CommandOptions.countMatches, + CommandOptions.invertMatch, + CommandOptions.prependLineNumbers, + CommandOptions.listFilesWithMatches, + CommandOptions.listFilesWithoutMatch, + CommandOptions.verbose + ] --Divvy-up. + +--Separate the regEx & the names of the input-data files to feed it. + extendedRegExChar :: ExtendedRegExChar.ExtendedRegExChar + fileNames :: [String] + (extendedRegExChar, fileNames) = case CommandOptions.extendedRegExChar commandOptions of + Just c -> (c, nonOptions) + _ + | null nonOptions -> error "Undefined regex." + | otherwise -> read . head &&& tail $ nonOptions + +--Combine the regex & options, to create a complete task-description. + regExOpts :: RegExOpts.RegExOpts ExtendedRegExChar.ExtendedRegExChar + regExOpts = RegExOpts.MkRegExOpts { + RegExOpts.compilationOptions = CommandOptions.compilationOptions commandOptions, + RegExOpts.executionOptions = CommandOptions.executionOptions commandOptions, + RegExOpts.regEx = extendedRegExChar + } + +--Define functions used to format the output. + separate :: ShowS + separate = showChar ':' + + showLineNumber :: LineNumber -> ShowS + showLineNumber n = shows n . separate + diff --git a/src/Grecce/Performance/ExtendedRegEx.hs b/src/Grecce/Performance/ExtendedRegEx.hs new file mode 100644 --- /dev/null +++ b/src/Grecce/Performance/ExtendedRegEx.hs @@ -0,0 +1,144 @@ +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] + + * CPU-times the specified stress-test(s), over a hard-coded list of /regex/-engines, + providing each combination with a varied list of 'Performance.ExtendedRegExTest.TestGenerator's, + & a level of 'Performance.ExtendedRegExTest.TestComplexity' which increases linearly to the specified cap. + + * Either a simple 'Bool' result, or the full data-capture, can be requested. + + [@CAVEATS@] + + * For compatibility with the other regex-engines, uses the specialised instance 'ExtendedRegExChar.ExtendedRegExChar', + rather than the underlying polymorphic 'RegEx.ExtendedRegEx'. + + * "Text.Regex.Posix" stalls forever on 'Performance.ExtendedRegExTestsPositive.OVERLAPPING_ALTERNATIVES', + with 'Performance.ExtendedRegExTest.TestComplexity'=3 & data-capture requested. + + [@TODO@] + + * Utilise the standard interface in "RegExOptsChar", to permit identical treatment of different regex-engines. +-} + +module Grecce.Performance.ExtendedRegEx( +-- * Types +-- ** Type-synonyms +-- GenerateAndRunTest, +-- GetAndRunTestRange, +-- * Constants +-- solitary, +-- * Functions + run +-- tailorTest +) where + +import Control.Applicative((<$>), (<*>)) +import Control.Arrow((&&&)) +import qualified Control.Arrow +import qualified Data.Maybe +import qualified Grecce.Performance.ExtendedRegExTest as Performance.ExtendedRegExTest +import qualified Grecce.Performance.ExtendedRegExTestsNegative as Performance.ExtendedRegExTestsNegative +import qualified Grecce.Performance.ExtendedRegExTestsPositive as Performance.ExtendedRegExTestsPositive +import qualified RegExChar.ExtendedRegExChar as ExtendedRegExChar +import qualified RegExDot.ExecutionOptions as ExecutionOptions +import qualified RegExDot.RegEx as RegEx +import qualified RegExDot.RegExOpts as RegExOpts +import qualified Text.Regex.Posix +import qualified ToolShed.Defaultable as Defaultable +import qualified ToolShed.TimePure as TimePure + +solitary :: Bool +solitary = True --Currently required because of the disastrous performance of "Text.Regex.Posix" on 'Performance.ExtendedRegExTestsPositive.OVERLAPPING_ALTERNATIVES'. + +type GetAndRunTestRange a = [a] -> IO () +type GenerateAndRunTest = Performance.ExtendedRegExTest.TestGenerator Char -> Performance.ExtendedRegExTest.TestComplexity -> IO (ExtendedRegExChar.InputData, [(Double, String)]) + +{- | + * Print the CPU-seconds, required to run the range of 'Performance.ExtendedRegExTest.TestGenerator's corresponding to the specified test-names & a linearly increasing 'Performance.ExtendedRegExTest.TestComplexity'. + + * If the specified maximum 'Performance.ExtendedRegExTest.TestComplexity' is set to the special value '0', then it is increased linearly, without limit. +-} +run :: ExecutionOptions.ExecutionOptions -> Performance.ExtendedRegExTest.TestComplexity -> Either [Performance.ExtendedRegExTestsNegative.TestName] [Performance.ExtendedRegExTestsPositive.TestName] -> IO () +run executionOptions 0 testRange = run executionOptions maxBound testRange --Run the specified test-range, without any upper bound to 'Performance.ExtendedRegExTest.TestComplexity'. +run executionOptions maxTestComplexity (Left negativeTestRange) = runNegativeTestRange negativeTestRange where + runNegativeTestRange :: GetAndRunTestRange Performance.ExtendedRegExTestsNegative.TestName + runNegativeTestRange [] = runNegativeTestRange [minBound .. maxBound] --Run all the tests, to the specified maximum 'Performance.ExtendedRegExTest.TestComplexity'. + runNegativeTestRange testRange = sequence_ $ ( + \complexity testGenerator -> cpuSecondsFor testGenerator complexity >>= print + ) <$> [1 .. maxTestComplexity] <*> Data.Maybe.mapMaybe Performance.ExtendedRegExTest.query testRange where + cpuSecondsFor :: GenerateAndRunTest + cpuSecondsFor testGenerator = ( + \(inputData, (isMismatch, extendedRegExCharString)) -> (,) extendedRegExCharString <$> mapM ( + TimePure.getCPUSeconds . ($ inputData) . (show .) + ) [ + isMismatch, + if solitary + then const True --CAVEAT: should be 'undefined'. + else not . (Text.Regex.Posix.=~ extendedRegExCharString) :: String -> Bool --CAVEAT: . + ] + ) . Control.Arrow.second ( + ( + \extendedRegEx -> ( + RegEx./~ RegExOpts.MkRegExOpts { --CAVEAT: . + RegExOpts.compilationOptions = Defaultable.defaultValue, + RegExOpts.executionOptions = executionOptions, + RegExOpts.regEx = extendedRegEx + } + ) + ) &&& show . ExtendedRegExChar.MkExtendedRegExChar False {-hasNonCapturingTopLevelAlternatives-} + ) . testGenerator + +run executionOptions maxTestComplexity (Right positiveTestRange) = runPositiveTestRange positiveTestRange where + runPositiveTestRange :: GetAndRunTestRange Performance.ExtendedRegExTestsPositive.TestName + runPositiveTestRange [] = runPositiveTestRange [minBound .. maxBound] --Run all the tests, to the specified maximum 'Performance.ExtendedRegExTest.TestComplexity'. + runPositiveTestRange testRange = sequence_ $ ( + \complexity testGenerator -> cpuSecondsFor testGenerator complexity >>= print + ) <$> [1 .. maxTestComplexity] <*> Data.Maybe.mapMaybe Performance.ExtendedRegExTest.query testRange where + cpuSecondsFor :: GenerateAndRunTest + cpuSecondsFor testGenerator = ( + \(inputData, (showMatch, extendedRegExCharString)) -> (,) extendedRegExCharString <$> ( + (TimePure.getCPUSeconds . ($ inputData)) `mapM` [ + showMatch, + if ExecutionOptions.requireMatchList executionOptions + then if solitary + then const "" +-- else (Text.Regex.Posix.=~ extendedRegExCharString) :: String -> String --CAVEAT: . + else (\(_, _, _, s) -> show s) . ((Text.Regex.Posix.=~ extendedRegExCharString) :: String -> (String, String, String, [String])) --CAVEAT: . + else show . if solitary + then const True --CAVEAT: should be 'undefined'. + else (Text.Regex.Posix.=~ extendedRegExCharString) :: String -> Bool --CAVEAT: . + ] + ) + ) . Control.Arrow.second ( + ( + \extendedRegEx -> let + regExOpts :: RegExOpts.RegExOpts (RegEx.ExtendedRegEx Char) + regExOpts = RegExOpts.MkRegExOpts { + RegExOpts.compilationOptions = Defaultable.defaultValue, + RegExOpts.executionOptions = executionOptions, + RegExOpts.regEx = extendedRegEx + } --Similar to 'RegexLike.makeRegexOpts', but without the requirement to read the regex. + in if ExecutionOptions.requireMatchList executionOptions + then show . (RegEx.+~ regExOpts) --CAVEAT: . + else show . (RegEx.=~ regExOpts) --CAVEAT: . + ) &&& show . ExtendedRegExChar.MkExtendedRegExChar False {-hasNonCapturingTopLevelAlternatives-} + ) . testGenerator + diff --git a/src/Grecce/Performance/ExtendedRegExTest.hs b/src/Grecce/Performance/ExtendedRegExTest.hs new file mode 100644 --- /dev/null +++ b/src/Grecce/Performance/ExtendedRegExTest.hs @@ -0,0 +1,51 @@ +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] + + * Defines the types used for performance-testing. + + * Defines an interface to permit queries on repositories of 'TestGenerator's. +-} + +module Grecce.Performance.ExtendedRegExTest( +-- * Type-classes + TestGeneratorRepository(..), +-- * Types +-- ** Type-synonyms + TestComplexity, + Test, + TestGenerator +) where + +import qualified RegExDot.RegEx as RegEx + +-- | Used to specify the level of complexity of /regex/ to generate. +type TestComplexity = Int + +-- | A pair defining the data & the /regex/ to which it should be fed. +type Test a = (RegEx.InputData a, RegEx.ExtendedRegEx a) + +-- | The type of a function, which accepts a 'TestComplexity', & returns a 'Test'. +type TestGenerator a = TestComplexity -> Test a + +-- | Implementer contracts to generate an appropriate 'Test'. +class TestGeneratorRepository a where + query :: a -> Maybe (TestGenerator Char) --CAVEAT: polymorphism reduced, to avoid requirement for non-portable Haskell-extensions. + diff --git a/src/Grecce/Performance/ExtendedRegExTestsNegative.hs b/src/Grecce/Performance/ExtendedRegExTestsNegative.hs new file mode 100644 --- /dev/null +++ b/src/Grecce/Performance/ExtendedRegExTestsNegative.hs @@ -0,0 +1,85 @@ +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] + + * Defines 'Performance.ExtendedRegExTest.TestGenerator's, for 'Performance.ExtendedRegExTest.Test's, which should fail. + + * Implements 'Performance.ExtendedRegExTest.TestGeneratorRepository'. +-} + +module Grecce.Performance.ExtendedRegExTestsNegative( +-- * Types +-- ** Data-types + TestName +) where + +import qualified Control.Arrow +import qualified Data.Maybe +import qualified Grecce.Performance.ExtendedRegExTest as Performance.ExtendedRegExTest +import qualified Grecce.Performance.ExtendedRegExTestsPositive as Performance.ExtendedRegExTestsPositive + +-- | Defines an enumeration of keys, by which to reference 'Performance.ExtendedRegExTest.TestGenerator's. +data TestName = + INTERLEAVED_GREEDY_STAR_VS_MANDATORY + | LINEAR_FRACTAL_NONGREEDY_STAR + | OVERLAPPING_ALTERNATIVES + deriving ( + Bounded, + Enum, + Eq, + Read, + Show + ) + +instance Performance.ExtendedRegExTest.TestGeneratorRepository TestName where + query = ( + `lookup` [ +{- + Explanation: based on 'Performance.ExtendedRegExTestsPositive.INTERLEAVED_GREEDY_STAR_VS_MANDATORY', but with twice the input data following by a mismatch. + Performance: vastly better performance than "Text.Regex.Posix", benefiting from 'ExecutionOptions.checkExistenceOfInelasticTail'. +-} + ( + INTERLEAVED_GREEDY_STAR_VS_MANDATORY, + Control.Arrow.first ((++ "!") . concat . replicate 2) . Data.Maybe.fromJust ( + Performance.ExtendedRegExTest.query Performance.ExtendedRegExTestsPositive.INTERLEAVED_GREEDY_STAR_VS_MANDATORY + ) + ), +{- + Explanation: based on 'Performance.ExtendedRegExTestsPositive.LINEAR_FRACTAL_NONGREEDY_STAR', but with mismatching input data, which causes the time-complexity explodes to O(n!). + Performance: vastly better performance than "Text.Regex.Posix", benefiting from 'ExecutionOptions.checkForUnconsumableData'. +-} + ( + LINEAR_FRACTAL_NONGREEDY_STAR, + Control.Arrow.first (++ "!") . Data.Maybe.fromJust ( + Performance.ExtendedRegExTest.query Performance.ExtendedRegExTestsPositive.LINEAR_FRACTAL_NONGREEDY_STAR + ) + ), +{- + Explanation: based on 'Performance.ExtendedRegExTestsPositive.OVERLAPPING_ALTERNATIVES', but with mismatching input data. + Performance: marginally better performance than "Text.Regex.Posix", benefiting from 'ExecutionOptions.checkForUnconsumableData' & 'ExecutionOptions.permitReorderingOfAlternatives'. +-} + ( + OVERLAPPING_ALTERNATIVES, + Control.Arrow.first (++ "!") . Data.Maybe.fromJust ( + Performance.ExtendedRegExTest.query Performance.ExtendedRegExTestsPositive.OVERLAPPING_ALTERNATIVES + ) + ) + ] + ) diff --git a/src/Grecce/Performance/ExtendedRegExTestsPositive.hs b/src/Grecce/Performance/ExtendedRegExTestsPositive.hs new file mode 100644 --- /dev/null +++ b/src/Grecce/Performance/ExtendedRegExTestsPositive.hs @@ -0,0 +1,329 @@ +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] + + * Defines 'Performance.ExtendedRegExTest.TestGenerator's, for 'Performance.ExtendedRegExTest.Test's, which should pass. + + * Implements 'Performance.ExtendedRegExTest.TestGeneratorRepository'. +-} + +module Grecce.Performance.ExtendedRegExTestsPositive( +-- * Types +-- ** Data-types + TestName(..) +-- * Functions +-- triangularSeries +) where + +import Control.Arrow((&&&)) +import RegExDot.DSL((-:), (?:), (*:), (<~>)) +import RegExDot.RegEx((.*), (.*?)) +import RegExDot.Repeatable((^#->#), (^#->), (^#)) +import qualified Data.Char +import qualified Grecce.Performance.ExtendedRegExTest as Performance.ExtendedRegExTest +import qualified RegExDot.Anchor as Anchor +import qualified RegExDot.BracketExpressionMember as BracketExpressionMember +import qualified RegExDot.Meta as Meta +import qualified RegExDot.RegEx as RegEx +import qualified RegExDot.Repeatable as Repeatable + +-- | Defines an enumeration of keys, by which to reference 'Performance.ExtendedRegExTest.TestGenerator's. +data TestName = + BIPOLAR_GREEDY_OPTIONAL_VS_MANDATORY + | INTERLEAVED_GREEDY_STAR_VS_MANDATORY + | TRIANGULAR_CONSUMPTION + | BINARY_TREE + | REPEATABLE_ALTERNATIVES + | REPEATED_ALTERNATIVES + | LINEAR_FRACTAL_RANGES + | LINEAR_FRACTAL_NONGREEDY_STAR + | BINARY_TREE_FRACTAL + | FUTILE_REPEATABLE_ALTERNATIVES + | OVERLAPPING_ALTERNATIVES + | UNBRIDLED_GREED + | MULTIPLE_CHOICE + deriving ( + Bounded, + Enum, + Eq, + Read, + Show + ) + +instance Performance.ExtendedRegExTest.TestGeneratorRepository TestName where + query = ( + `lookup` [ +{- + "^(?){}(){}$". + Explanation: each of the optional meta-data in the first half of the expression can capture data, but in doing so starves a mandatory meta-datum in the second half of the expression, + presenting the opportunity for O(2^n) time-complexity backtracking. + Performance: vastly better Boolean-result performance than "Text.Regex.Posix". Looks like O(n^2) time-complexity. +-} + ( + BIPOLAR_GREEDY_OPTIONAL_VS_MANDATORY, + (`replicate` testChar) &&& let + exactly :: Performance.ExtendedRegExTest.TestComplexity -> (RegEx.Pattern Char -> RegEx.RepeatablePattern Char) -> RegEx.RepeatablePattern Char + exactly n f = (RegEx.captureGroup . map ((Nothing, Nothing) <~>) . return {-to List-monad-} . return {-to List-monad-} $ f testRequirement) ^# n + in ((Just Anchor.Bow, Just Anchor.Stern) <~>) . (`map` [Repeatable.zeroOrOne, Repeatable.one]) . exactly + ), +{- + "^(.*z){}$". + Explanation: each '.*' can consume all the remaining data, starving the interspersed mandatory meta data, presenting the opportunity for O(2^n) time-complexity backtracking. + Performance: marginally better Boolean-result & data-capture performance than "Text.Regex.Posix" with 'ExecutionOptions.moderateGreed'. Empirically O(n^3) time-complexity. +-} + ( + INTERLEAVED_GREEDY_STAR_VS_MANDATORY, + concat . (`replicate` (testChar : "z")) &&& RegEx.dock . ( + `mkExtendedRegExFromRepeatableSingletonAlternative` ((Nothing, Nothing) <~>) [ + (.*), + RegEx.simply $ Meta.Literal 'z' + ] + ) . flip (^#) + ), +{- + "^*+{2,}{3,} ... a{,}$". + Explanation: each term greedily consumes all the data, starving all the remaining terms, until after much back-tracking, it consumes the just the minimum permissible. + The correct consumption-pattern is triangular, starting at the apex & spreading into a delta. + Performance: vastly better Boolean-result & data-capture performance than "Text.Regex.Posix". +-} + + ( + TRIANGULAR_CONSUMPTION, + (`replicate` testChar) . triangularSeries &&& ((Just Anchor.Bow, Just Anchor.Stern) <~>) . (`take` map (testRequirement ^#->) [0 ..]) . succ + ), +{- + "(|{2})*c". + Explanation: from + Performance: disastrous performance. +-} + ( + BINARY_TREE, + (++ "bc") . (`replicate` testChar) . (2 ^) &&& const ( + (Nothing, Nothing) <~> RegEx.captureGroup ( + map ( + ((Nothing, Nothing) <~>) . return {-to List-monad-} . (testRequirement ^#) . (2 ^) + ) [0 :: Int .. 1] + ) *: RegEx.Require (Meta.Literal 'c') -: [] + ) + ), +{- + "^(?|{2}|{4}|{8}| ... |{2^})*$". + Explanation: all but one of the 'Alternatives' must be used, one on each successive repetition, in order to consume all the 'InputData'. + A correct implementation will match against the longest possible Alternative on each successive repetition, just as if the repetition were unrolled; see 'REPEATED_ALTERNATIVES'. + Performance: vastly worse Boolean-result performance (even with 'ExecutionOptions.permitReorderingOfAlternatives'), + & worse data-capture performance (even with 'ExecutionOptions.useFirstMatchAmongAlternatives'), than "Text.Regex.Posix". +-} + ( + REPEATABLE_ALTERNATIVES, + (`replicate` testChar) . pred {-requires several different Alternatives-} . (2 ^) &&& ( + \i -> ((Just Anchor.Bow, Just Anchor.Stern) <~>) . return {-to List-monad-} . Repeatable.zeroOrMore . RegEx.captureGroup . map ((Nothing, Nothing) <~>) $ [Repeatable.zeroOrOne testRequirement] : map ( + return {-to List-monad-} . (testRequirement ^#) . (2 ^) + ) [1 .. i] + ) + ), +{- + "^(?|{2}|{4}| ... |{2^})(?|{2}|{4}| ... |{2^})(?|{2}|{4}| ... |{2^})$". + Explanation: 'REPEATABLE_ALTERNATIVES' with the repetition unrolled, to permit inspection of individual capture-groups, + revealing the use by "Text.Regex.Posix", of Perl-style "1st-past-the-post" selection between Alternatives. + Performance: much faster than 'REPEATABLE_ALTERNATIVES'. Vastly better Boolean-result & data-capture performance than both 'REPEATABLE_ALTERNATIVES' & "Text.Regex.Posix". +-} + ( + REPEATED_ALTERNATIVES, + (`replicate` testChar) . pred {-requires several different Alternatives-} . (2 ^) &&& ( + \i -> ((Just Anchor.Bow, Just Anchor.Stern) <~>) . replicate i . Repeatable.one . RegEx.captureGroup . map ((Nothing, Nothing) <~>) $ [Repeatable.zeroOrOne testRequirement] : map ( + return {-to List-monad-} . (testRequirement ^#) . (2 ^) + ) [1 .. i] + ) + ), +{- + "^(((({0,2}){0,2}){0,2}){0,2}){0,2} ... $". + Explanation: + Performance: vastly better Boolean-result performance than "Text.Regex.Posix", which exhausts memory at n==10. Better data-capture performance too. + Stack-overflow at 'Performance.ExtendedRegExTest.TestComplexity'=14. +-} + ( + LINEAR_FRACTAL_RANGES, + let + base :: Repeatable.Repetitions + base = 2 --Arbitrarily. + + bounds :: Repeatable.RepetitionBounds + bounds = (0, Just base) + in (`replicate` testChar) . (base ^) &&& RegEx.dock . ( + ( + mkExtendedRegExFromRepeatableSingletonAlternative (^#-># bounds) `iterate` ((Nothing, Nothing) <~>) [testRequirement ^#-># bounds] {-initial value-} + ) !! + ) . pred + ), +{- + "^((((((((((((((*?)*?)*?)*?)*?)*?)*?)*?)*?)*?)*?)*?)*?)*?)*? ... $". + Explanation: interesting because all 'RegEx.Alternatives' are expanded before the first comparison against input data, & the number possible repetitions of each expansion is very vague. + Performance: vastly better Boolean-result performance than "Text.Regex.Posix", but inferior data-capture performance. +-} + ( + LINEAR_FRACTAL_NONGREEDY_STAR, + const (replicate 8 {-arbitrary-} testChar) &&& RegEx.dock . ( + ( + mkExtendedRegExFromRepeatableSingletonAlternative Repeatable.zeroOrMore' `iterate` ((Nothing, Nothing) <~>) [Repeatable.zeroOrMore' testRequirement] {-initial value-} + ) !! + ) + ), +{- + "^((([^]|)|(|[^]))|((|[^])|([^]|)))$". + Explanation: exhibits symmetry in the left & right branches of the binary tree of RegEx.Alternatives, & so the exponentially expanding set of leaves are equivalent & all match the data. + Performance: vastly better Boolean-result & comparable data.capture performance to "Text.Regex.Posix", which exhausts memory at complexity 5 to 6, & to egrep which lasts until complexity==6. +-} + ( + BINARY_TREE_FRACTAL, + let + base :: Repeatable.Repetitions + base = 2 --Arbitrarily. + in (`replicate` testChar) . (base ^) &&& RegEx.dock . head . ( + ( + iterate ( + \extendedRegExList -> ( + ((Nothing, Nothing) <~>) . return {-to List-monad-} . (^# base) . RegEx.captureGroup + ) `map` [extendedRegExList, reverse extendedRegExList] + ) $ map ( + ((Nothing, Nothing) <~>) . return {-to List-monad-} . ($ RegEx.Require $ Meta.Literal testChar) + ) [Repeatable.one, Repeatable.zeroOrOne] + ) !! + ) + ), +{- + "^(1|{2}2|{3}3| ... |{})**$". + Explanation: none of the Alternatives can match, & consequently the group must match zero times, presenting the opportunity for many wasted trial repetitions. + Performance: comparable Boolean-result performance to "Text.Regex.Posix", & marginally superior data-capture performance. +-} + ( + FUTILE_REPEATABLE_ALTERNATIVES, + \alternatives -> ( + replicate 1024 {-arbitrarily-} testChar, + ((Just Anchor.Bow, Just Anchor.Stern) <~>) $ map Repeatable.zeroOrMore [ + RegEx.captureGroup . map ((Nothing, Nothing) <~>) $ ( + \i -> ($ i) `map` [ + (testRequirement ^#), + RegEx.simply . Meta.Literal . Data.Char.chr . (Data.Char.ord '0' +) . (`mod` 10) + ] + ) `map` [1 .. alternatives], + testRequirement + ] + ) + ), +{- + "^a?((ab?)*|(bc?)*|(cd?)*| ...)*$". + Explanation: the zeroeth term matches greedily, consuming the datum mandated by the 1st Alternative, so the 2nd Alternative greedily consumes two data, including the datum mandated by the 3rd, ... + If there're an even number of Alternatives, then n/2 repetitions (one for each), will consume all the input data, but otherwise the final input datum isn't consumed. + The consumption back-tracks rapidly until the zeroeth term yields-up the initial datum, after which the odd Alternatives match consuming all the input data, in (n/2 + 1) repetitions. + This suggests a very tractable O(n) time-complexity, but Posix requires that we select the combination of Alternatives which consume most data, + & since there are two terms in the regex that can consume each input datum, it looks like there should be 2^n candidate solutions; in fact there's just one. + So, all O(Alternatives ^ repetitions) permutations may be searched in vain for a superior solution. + Performance: vastly worse Boolean-result performance than "Text.Regex.Posix", but the latter then stalls indefinitely (CPU-bound) at complexity=[13 (not verbose), 4 (verbose)]. +-} + ( + OVERLAPPING_ALTERNATIVES, + let + bracket :: a -> [a] -> [a] + bracket terminal = (terminal :) . (++ [terminal]) + + zeroOrMoreAlternatives :: [RegEx.Concatenation Char] -> RegEx.Concatenation Char + zeroOrMoreAlternatives = return {-to List-monad-} . Repeatable.zeroOrMore . RegEx.captureGroup . map ((Nothing, Nothing) <~>) + in ( + id &&& ( + ((Just Anchor.Bow, Just Anchor.Stern) <~>) . (RegEx.Require (Meta.Literal testChar) ?:) . zeroOrMoreAlternatives . map ( + \(a, b) -> zeroOrMoreAlternatives $ return {-to List-monad-} [a, b] + ) . init . tail . uncurry zip . ( + map Repeatable.one . init &&& map Repeatable.zeroOrOne . tail + ) . bracket undefined . map ( + RegEx.Require . Meta.Literal + ) + ) + ) . (`take` (filter Data.Char.isAlphaNum $ filter Data.Char.isAscii [minBound .. maxBound])) . succ + ), +{- + "^(a|b|c)*(a.*?c){3}$". + Explanation: zero repetitions of the set of Alternatives on the LHS is the only solution, but because the greedy star, within which they're grouped, permits many more, + & because the RHS's requirement for all the input-data, isn't explicit, there's a temptation to over-consume. + Performance: empirically O(2^n) time-complexity, but linearised by 'ExecutionOptions.moderateGreed'. + Better Boolean-result & data-capture performance than "Text.Regex.Posix". +-} + ( + UNBRIDLED_GREED, + \complexity -> let + complexity' :: Performance.ExtendedRegExTest.TestComplexity + complexity' = 1 + complexity + + testSet :: String + testSet = take complexity' . filter Data.Char.isAlphaNum $ filter Data.Char.isAscii [minBound .. maxBound] + in ( + concat $ replicate complexity' testSet, + ((Just Anchor.Bow, Just Anchor.Stern) <~>) $ ( + RegEx.captureGroup . map ((Nothing, Nothing) <~>) $ map ( + return {-to List-monad-} . RegEx.simply . Meta.Literal + ) testSet + ) *: [ + ( + RegEx.captureGroup . map ((Nothing, Nothing) <~>) . return {-to List-monad-} $ RegEx.Require (Meta.Literal $ head testSet) -: (.*?) : RegEx.Require (Meta.Literal $ last testSet) -: [] + ) ^# complexity' + ] + ) + ), +{- + "^(a*[^a]|[^b]|[^c]| ... |[^z])*$" + Explanation: The first alternative greedily consumes all the input data until finding a mismatch on the last, whilst all the remaining alternatives successfully consume a single input datum. + The same occurs on each successive repetition of the alternatives, of which there are as many as input data, + leading to a lot of wasted time with O( ^ ) time-complexity. + Performance: poor Boolean-result & worse data-capture performance (benefiting from 'ExecutionOptions.useFirstMatchAmongAlternatives'), compared to "Text.Regex.Posix". +-} + ( + MULTIPLE_CHOICE, + const (replicate 8 {-arbitrary-} testChar) &&& ( + ((Just Anchor.Bow, Just Anchor.Stern) <~>) . return {-to List-monad-} . Repeatable.zeroOrMore . RegEx.captureGroup . map ((Nothing, Nothing) <~>) . ( + [ + Repeatable.zeroOrMore testRequirement, + RegEx.simply . Meta.NoneOf $ map BracketExpressionMember.Literal [testChar] + ] : + ) . map ( + return {-to List-monad-} . RegEx.simply . Meta.NoneOf . return {-to List-monad-} . BracketExpressionMember.Literal + ) . ( + `take` tail [testChar ..] + ) + ) + ) + ] + ) where + testChar :: Char + testChar = 'a' + + testRequirement :: RegEx.Pattern Char + testRequirement = RegEx.Require $ Meta.Literal testChar + + mkExtendedRegExFromRepeatableSingletonAlternative :: (RegEx.Pattern Char -> RegEx.RepeatablePattern Char) -> RegEx.ExtendedRegEx Char -> RegEx.ExtendedRegEx Char + mkExtendedRegExFromRepeatableSingletonAlternative makeRepetitions = ((Nothing, Nothing) <~>) . return {-to List-monad-} . makeRepetitions . RegEx.captureGroup . return {-to List-monad-} + +{- | + * The sum of the sequence descending from the specified integer; @n + (n - 1) + (n - 2) + ... + 2 + 1 + 0@. + + * . + + * . +-} +triangularSeries :: Integral i => i -> i +triangularSeries i = i * (i + 1) `div` 2 diff --git a/src/Grecce/QC/ExtendedRegExChar.hs b/src/Grecce/QC/ExtendedRegExChar.hs new file mode 100644 --- /dev/null +++ b/src/Grecce/QC/ExtendedRegExChar.hs @@ -0,0 +1,205 @@ +{-# LANGUAGE CPP #-} +{-# OPTIONS_GHC -fno-warn-orphans #-} +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] Implements 'Test.QuickCheck.Arbitrary' & defines tests based on it. +-} + +module Grecce.QC.ExtendedRegExChar( +-- * Types +-- ** Data-types +-- AlternativesChar(..), +-- PatternChar(..), +-- RepeatablePatternChar(..), +-- ** Type-synonyms +-- Testable, +-- * Functions +-- deconstructAlternativesChar, +-- deconstructPatternChar, +-- deconstructRepeatablePatternChar, + quickChecks +) where + +import Control.Applicative((<$>)) +import Grecce.QC.MetaChar() +import RegExDot.DSL((-:), (?:), (+:), (<~>)) +import RegExDot.RegEx((+~)) +import Test.QuickCheck((==>)) +import qualified RegExChar.ExtendedRegExChar as ExtendedRegExChar --CAVEAT: beware of the similar name. +import qualified RegExChar.MetaChar as MetaChar +import qualified RegExDot.Anchor as Anchor +import qualified RegExDot.BracketExpressionMember as BracketExpressionMember +import qualified RegExDot.Consumer as Consumer +import qualified RegExDot.ConsumptionBounds as ConsumptionBounds +import qualified RegExDot.ConsumptionProfile as ConsumptionProfile +import qualified RegExDot.Meta as Meta +import qualified RegExDot.RegEx as RegEx +import qualified RegExDot.RegExOpts as RegExOpts +import qualified RegExDot.Repeatable as Repeatable +import qualified RegExDot.Result as Result +import qualified Test.QuickCheck +import qualified ToolShed.SelfValidate as SelfValidate + +-- | A specialised instance, required to instantiate 'Test.QuickCheck.Arbitrary'. +newtype AlternativesChar = MkAlternativesChar (RegEx.Alternatives Char) deriving (Eq, Read, Show) + +-- | Accessor. +deconstructAlternativesChar :: AlternativesChar -> RegEx.Alternatives Char +deconstructAlternativesChar (MkAlternativesChar a) = a + +instance SelfValidate.SelfValidator AlternativesChar where + isValid = SelfValidate.isValid . deconstructAlternativesChar + +instance Consumer.Consumer AlternativesChar where + consumptionProfile = Consumer.consumptionProfile . deconstructAlternativesChar + starHeight = Consumer.starHeight . deconstructAlternativesChar + +instance Test.QuickCheck.Arbitrary AlternativesChar where + arbitrary = MkAlternativesChar . RegEx.MkAlternatives . map ExtendedRegExChar.extendedRegEx <$> ( + Test.QuickCheck.elements [1, 2] >>= Test.QuickCheck.vector {-[ExtendedRegExChar]-} + ) --TODO: permit zero alternatives. +#if !(MIN_VERSION_QuickCheck(2,1,0)) + coarbitrary = undefined --CAVEAT: stops warnings from ghc. +#endif + +-- | A specialised instance, required to instantiate 'Test.QuickCheck.Arbitrary'. +newtype PatternChar = MkPatternChar (RegEx.Pattern Char) deriving (Eq, Read, Show) + +-- | Accessor. +deconstructPatternChar :: PatternChar -> RegEx.Pattern Char +deconstructPatternChar (MkPatternChar pattern) = pattern + +instance SelfValidate.SelfValidator PatternChar where + isValid = SelfValidate.isValid . deconstructPatternChar + +instance Consumer.Consumer PatternChar where + consumptionProfile = Consumer.consumptionProfile . deconstructPatternChar + starHeight = Consumer.starHeight . deconstructPatternChar + +instance Test.QuickCheck.Arbitrary PatternChar where + arbitrary = MkPatternChar <$> Test.QuickCheck.frequency [ + (4, RegEx.Require . MetaChar.deconstruct <$> Test.QuickCheck.arbitrary {-MetaChar-}), + (1, RegEx.CaptureGroup . deconstructAlternativesChar <$> Test.QuickCheck.arbitrary {-AlternativesChar-}) + ] --CAVEAT: may recurse forever if 'RegEx.CaptureGroup / RegEx.Require' is too high. +#if ! (MIN_VERSION_QuickCheck(2,1,0)) + coarbitrary = undefined --CAVEAT: stops warnings from ghc. +#endif + +-- | A specialised instance, required to instantiate 'Test.QuickCheck.Arbitrary'. +newtype RepeatablePatternChar = MkRepeatablePatternChar (Repeatable.Repeatable PatternChar) deriving (Eq, Read, Show) + +-- | Accessor. +deconstructRepeatablePatternChar :: RepeatablePatternChar -> Repeatable.Repeatable PatternChar +deconstructRepeatablePatternChar (MkRepeatablePatternChar repeatable) = repeatable + +instance SelfValidate.SelfValidator RepeatablePatternChar where + isValid = SelfValidate.isValid . deconstructRepeatablePatternChar + +instance Consumer.Consumer RepeatablePatternChar where + consumptionProfile = Consumer.consumptionProfile . deconstructRepeatablePatternChar + starHeight = Consumer.starHeight . deconstructRepeatablePatternChar + +instance Test.QuickCheck.Arbitrary RepeatablePatternChar where + arbitrary = do + patternChar <- Test.QuickCheck.arbitrary + fewest <- Test.QuickCheck.elements [0 .. 9] --Could be more, but this is an adequate test. + most <- Test.QuickCheck.oneof [return {-to Gen-monad-} Nothing, Just <$> Test.QuickCheck.elements [max fewest 1 .. 9]] + isGreedy <- Test.QuickCheck.arbitrary {-Bool-} + + let + repetitionBounds :: Repeatable.RepetitionBounds + repetitionBounds = (fewest, most) + + return {-to Gen-monad-} $ MkRepeatablePatternChar Repeatable.MkRepeatable { + Repeatable.base = patternChar, + Repeatable.repetitionBounds = repetitionBounds, + Repeatable.isGreedy = Repeatable.hasPreciseBounds repetitionBounds || isGreedy --Only specify non-greedy where space exists. + } +#if ! (MIN_VERSION_QuickCheck(2,1,0)) + coarbitrary = undefined --CAVEAT: stops warnings from ghc. +#endif + +instance Test.QuickCheck.Arbitrary ExtendedRegExChar.ExtendedRegExChar where + arbitrary = do + hasBowAnchor <- Test.QuickCheck.arbitrary {-Bool-} + concatenation <- map ((deconstructPatternChar <$> {-replace base-}) . deconstructRepeatablePatternChar {-Repeatable.Repeatable PatternChar-}) <$> Test.QuickCheck.vector 2 {-[RepeatablePatternChar]-} + hasSternAnchor <- Test.QuickCheck.arbitrary {-Bool-} + + return {-to Gen-monad-} $ ExtendedRegExChar.MkExtendedRegExChar False RegEx.MkExtendedRegEx { + RegEx.bowAnchor = if hasBowAnchor then Just Anchor.Bow else Nothing, + RegEx.concatenation = concatenation, + RegEx.sternAnchor = if hasSternAnchor then Just Anchor.Stern else Nothing + } + +#if ! (MIN_VERSION_QuickCheck(2,1,0)) + coarbitrary = undefined --CAVEAT: stops warnings from ghc. +#endif + +type Testable = ExtendedRegExChar.ExtendedRegExChar -> Test.QuickCheck.Property + +-- | Defines invariant properties, which must hold for any 'ExtendedRegExChar.ExtendedRegExChar'. +quickChecks :: (Testable -> IO ()) -> IO () +quickChecks checker = quickChecks1 >> quickChecks2 where + quickChecks1 = checker `mapM_` [prop_consumptionProfile, prop_consumptionProfile2, prop_io, prop_io', prop_isValid, prop_starHeight] where + prop_consumptionProfile, prop_consumptionProfile2, prop_io, prop_io', prop_isValid, prop_starHeight :: Testable + prop_consumptionProfile r = Test.QuickCheck.label "prop_consumptionProfile" $ ( + case maybeMaxData of + Nothing -> True + Just m -> m >= minData + ) && ( + not b == all (not . Consumer.getHasSpecificRequirement) (RegEx.concatenation $ ExtendedRegExChar.extendedRegEx r) + ) + where + ConsumptionProfile.MkConsumptionProfile { + ConsumptionProfile.consumptionBounds = (minData, maybeMaxData), + ConsumptionProfile.hasSpecificRequirement = b + } = Consumer.consumptionProfile r + + prop_consumptionProfile2 r = RegEx.isDefined (ExtendedRegExChar.extendedRegEx r) ==> Test.QuickCheck.label "prop_consumptionProfile2" $ or [hasSpecificRequirement, canConsumeAnything] --There's either; a requirement for at least one specific, or we can consume at least one arbitrary; input datum. + where + ConsumptionProfile.MkConsumptionProfile { + ConsumptionProfile.hasSpecificRequirement = hasSpecificRequirement, + ConsumptionProfile.canConsumeAnything = canConsumeAnything + } = Consumer.consumptionProfile r + + prop_io r = Test.QuickCheck.label "prop_io" $ read (show r) == r + prop_io' r = Test.QuickCheck.label "prop_io'" $ read (show r') == r' where r' = ExtendedRegExChar.extendedRegEx r --Check "RegEx.ExtendedRegEx Char" too. + prop_isValid r = Test.QuickCheck.label "prop_isValid" $ SelfValidate.isValid r + prop_starHeight r = Test.QuickCheck.label "prop_starHeight" $ (Consumer.starHeight r == 0) == ConsumptionBounds.isPrecise (Consumer.getConsumptionBounds r) + + quickChecks2 = Test.QuickCheck.quickCheck {-'checker' is technically the wrong type-} `mapM_` [prop_double] where + prop_double :: Double -> Test.QuickCheck.Property + prop_double d = Test.QuickCheck.label "prop_double" $ RegEx.extractDataFromMatchList `fmap` Result.getMatchList result == Just s where + s :: String + s = show $ d * 1e6 --Implementation of show, uses exponential notation, when the number is large. + + result :: RegEx.Result Char + result = s +~ RegExOpts.mkRegEx ( + (Just Anchor.Bow, Just Anchor.Stern) <~> sign ?: digits +: map ( + Repeatable.zeroOrOne . RegEx.captureGroup . map ((Nothing, Nothing) <~>) . return {-to List-monad-} + ) [ + RegEx.Require (Meta.Literal '.') -: digits +: [], + RegEx.Require (Meta.AnyOf $ map BracketExpressionMember.Literal "eE") -: sign ?: digits +: [] + ] + ) where + sign, digits :: RegEx.Pattern Char + sign = RegEx.Require . Meta.AnyOf $ map BracketExpressionMember.Literal "+-" + digits = RegEx.Require . Meta.AnyOf $ map BracketExpressionMember.Literal ['0' .. '9'] + diff --git a/src/Grecce/QC/MetaChar.hs b/src/Grecce/QC/MetaChar.hs new file mode 100644 --- /dev/null +++ b/src/Grecce/QC/MetaChar.hs @@ -0,0 +1,78 @@ +{-# LANGUAGE CPP #-} +{-# OPTIONS_GHC -fno-warn-orphans #-} +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] Implements 'Test.QuickCheck.Arbitrary' & defines tests based on it. + + [@TODO@] Test /Perl-style shortcuts/ & /Posix Character-classes/. +-} + +module Grecce.QC.MetaChar( +-- * Types +-- ** Type-synonyms +-- Testable, +-- * Constants +-- testCharacters, +-- * Functions + quickChecks +) where + +import Control.Applicative((<$>)) +import qualified Data.List +import qualified RegExChar.MetaChar as MetaChar --CAVEAT: beware of the similar name. +import qualified RegExDot.BracketExpression as BracketExpression +import qualified RegExDot.BracketExpressionMember as BracketExpressionMember +import qualified RegExDot.Meta as Meta +import qualified Test.QuickCheck + +-- CAVEAT: module "Test.QuickCheck.Arbitrary" defines an instance of this from package "Quickcheck-2.1.0.1". +#if !(MIN_VERSION_QuickCheck(2,1,0)) +instance Test.QuickCheck.Arbitrary Char where + arbitrary = Test.QuickCheck.elements testCharacters + coarbitrary = undefined --CAVEAT: stops warnings from ghc. +#endif + +testCharacters :: String +testCharacters = filter (`notElem` [Meta.shortcutToken, BracketExpression.negationToken]) [' ' .. '~'] --TODO: permit all characters. + +instance Test.QuickCheck.Arbitrary MetaChar.MetaChar where + arbitrary = Test.QuickCheck.oneof $ map (MetaChar.MkMetaChar <$>) [ + return {-to Gen-monad-} Meta.Any, + Meta.Literal <$> Test.QuickCheck.elements testCharacters, + Meta.AnyOf . map BracketExpressionMember.Literal . Data.List.nub <$> arbitraryCharList 1 16, --Test.QuickCheck.vector 16 {-Char-}, + Meta.NoneOf . map BracketExpressionMember.Literal . Data.List.nub <$> arbitraryCharList 1 4 + ] where + arbitraryCharList :: Int -> Int -> Test.QuickCheck.Gen String + arbitraryCharList i j = do + l <- Test.QuickCheck.arbitrary {-Int-} + + take (min j $ i `max` l) . Data.List.nub . Data.List.filter (`elem` testCharacters) <$> Test.QuickCheck.vector maxBound {-Char-} +#if ! (MIN_VERSION_QuickCheck(2,1,0)) + coarbitrary = undefined --CAVEAT: stops warnings from ghc. +#endif + +type Testable = MetaChar.MetaChar -> Test.QuickCheck.Property + +-- | Defines invariant properties, which must hold for any 'MetaChar.MetaChar'. +quickChecks :: (Testable -> IO ()) -> IO () +quickChecks checker = checker `mapM_` [prop_io] where + prop_io :: Testable + prop_io m = Test.QuickCheck.label "prop_io" $ read (show m) == m + diff --git a/src/Grecce/QC/QuickChecks.hs b/src/Grecce/QC/QuickChecks.hs new file mode 100644 --- /dev/null +++ b/src/Grecce/QC/QuickChecks.hs @@ -0,0 +1,52 @@ +{-# LANGUAGE CPP #-} +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] Calls the /quickChecks/-functions for modules supporting this feature. +-} + +module Grecce.QC.QuickChecks( +-- * Functions + run +) where + +import qualified Grecce.QC.ExtendedRegExChar as QC.ExtendedRegExChar +import qualified Grecce.QC.MetaChar as QC.MetaChar +import qualified Grecce.QC.RepeatableMetaChar as QC.RepeatableMetaChar +import qualified Test.QuickCheck + +{- | Run the /quickChecks/-functions for modules supporting this feature. + The single Boolean flag enables verbose output. +-} +run :: Bool -> IO () +run isVerbose = + putStrLn "MetaChar" >> QC.MetaChar.quickChecks checker + >> putStrLn "RepeatableMetaChar" >> QC.RepeatableMetaChar.quickChecks checker + >> putStrLn "ExtendedRegExChar" >> QC.ExtendedRegExChar.quickChecks checker + where + checker :: Test.QuickCheck.Testable t => t -> IO () + checker + | isVerbose = +#if MIN_VERSION_QuickCheck(2,4,0) + Test.QuickCheck.verboseCheck +#else + error "Grecce.QC.QuickChecks:\t'Test.QuickCheck.verboseCheck' is only available as of 'QuickCheck-2.4'." +#endif + | otherwise = Test.QuickCheck.quickCheck + diff --git a/src/Grecce/QC/RepeatableMetaChar.hs b/src/Grecce/QC/RepeatableMetaChar.hs new file mode 100644 --- /dev/null +++ b/src/Grecce/QC/RepeatableMetaChar.hs @@ -0,0 +1,98 @@ +{-# LANGUAGE CPP #-} +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] Implements 'Test.QuickCheck.Arbitrary' & defines tests based on it. +-} + +module Grecce.QC.RepeatableMetaChar( +-- * Types +-- ** Data-types +-- RepeatableMetaChar, +-- ** Type-synonyms +-- Testable, +-- * Functions +-- deconstruct, + quickChecks +) where + +import Control.Applicative((<$>)) +import Grecce.QC.MetaChar() +import qualified RegExChar.MetaChar as MetaChar +import qualified RegExDot.Consumer as Consumer +import qualified RegExDot.ConsumptionBounds as ConsumptionBounds +import qualified RegExDot.ConsumptionProfile as ConsumptionProfile +import qualified RegExDot.Repeatable as Repeatable +import qualified Test.QuickCheck +import qualified ToolShed.SelfValidate as SelfValidate + +-- | A specialised instance, required to instantiate 'Test.QuickCheck.Arbitrary'. +newtype RepeatableMetaChar = MkRepeatableMetaChar (Repeatable.Repeatable MetaChar.MetaChar) deriving (Eq, Read, Show) + +-- | Accessor. +deconstruct :: RepeatableMetaChar -> Repeatable.Repeatable MetaChar.MetaChar +deconstruct (MkRepeatableMetaChar repeatableMetaChar) = repeatableMetaChar + +instance SelfValidate.SelfValidator RepeatableMetaChar where + isValid = SelfValidate.isValid . deconstruct + +instance Consumer.Consumer RepeatableMetaChar where + consumptionProfile = Consumer.consumptionProfile . deconstruct + starHeight = Consumer.starHeight . deconstruct + +instance Test.QuickCheck.Arbitrary RepeatableMetaChar where + arbitrary = do + metaChar <- Test.QuickCheck.arbitrary + fewest <- Test.QuickCheck.elements [0 .. 9] --Could be more, but this is an adequate test. + most <- Test.QuickCheck.oneof [return Nothing, Just <$> Test.QuickCheck.elements [max fewest 1 .. 9]] + isGreedy <- Test.QuickCheck.arbitrary + + let + repetitionBounds :: Repeatable.RepetitionBounds + repetitionBounds = (fewest, most) + + return {-to Gen-monad-} . MkRepeatableMetaChar $ Repeatable.MkRepeatable { + Repeatable.base = metaChar, + Repeatable.repetitionBounds = repetitionBounds, + Repeatable.isGreedy = Repeatable.hasPreciseBounds repetitionBounds || isGreedy --Only specify 'non-greedy' where space exists. + } +#if !(MIN_VERSION_QuickCheck(2,1,0)) + coarbitrary = undefined --CAVEAT: stops warnings from ghc. +#endif + +type Testable = RepeatableMetaChar -> Test.QuickCheck.Property + +-- | Defines invariant properties, which must hold for any 'RepeatableMetaChar'. +quickChecks :: (Testable -> IO ()) -> IO () +quickChecks checker = checker `mapM_` [prop_consumptionProfile, prop_io, prop_isValid, prop_starHeight] where + prop_consumptionProfile, prop_io, prop_isValid, prop_starHeight :: Testable + prop_consumptionProfile r = Test.QuickCheck.label "prop_consumptionProfile" $ ( + case maybeMaxData of + Nothing -> True + Just m -> m >= minData + ) && ( + b == Consumer.getHasSpecificRequirement (Repeatable.base $ deconstruct r) + ) where ConsumptionProfile.MkConsumptionProfile { + ConsumptionProfile.consumptionBounds = (minData, maybeMaxData), + ConsumptionProfile.hasSpecificRequirement = b + } = Consumer.consumptionProfile r + prop_io (MkRepeatableMetaChar r) = Test.QuickCheck.label "prop_io" $ read (show r) == r + prop_isValid r = Test.QuickCheck.label "prop_isValid" $ SelfValidate.isValid r + prop_starHeight (MkRepeatableMetaChar r) = Test.QuickCheck.label "prop_starHeight" $ Consumer.starHeight r == if ConsumptionBounds.isPrecise (Consumer.getConsumptionBounds r) then 0 else 1 + diff --git a/src/Main.hs b/src/Main.hs new file mode 100644 --- /dev/null +++ b/src/Main.hs @@ -0,0 +1,197 @@ +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] + + * Contains the entry-point to the program. + + * Provides command-line access to "ExtendedRegExChar". + + * Defines the permissible command-line arguments; aping /egrep/, but with the addition of /--verbose/, which enables one to see the details of the data-capture. + + [@TODO@] Coordinate the output of 'printVersion' with the definition in '.cabal'; this may not be feasible. +-} + +module Main( +-- * Type-classes +-- CommandLineAction, +-- * Functions +-- $main + main --Added for Haskell style scanner, 'scan-0.1.0.5'. +) where + +import Control.Applicative((<$>)) +import qualified Data.List +import qualified Distribution.Package +import qualified Distribution.Text +import qualified Distribution.Version +import qualified Grecce.Assert.RegExOptsChar as Assert.RegExOptsChar +import qualified Grecce.CommandOptions as CommandOptions +import qualified Grecce.Grep as Grep +import qualified Grecce.Performance.ExtendedRegEx as Performance.ExtendedRegEx +import qualified Grecce.Performance.ExtendedRegExTestsNegative as Performance.ExtendedRegExTestsNegative +import qualified Grecce.Performance.ExtendedRegExTestsPositive as Performance.ExtendedRegExTestsPositive +import qualified Grecce.QC.QuickChecks as QC.QuickChecks +import qualified RegExDot.CompilationOptions as CompilationOptions +import qualified RegExDot.ExecutionOptions as ExecutionOptions +import qualified System +import qualified System.Console.GetOpt as G +import qualified System.IO +import qualified System.IO.Error +import qualified ToolShed.Defaultable as Defaultable + +-- | Used to thread user-defined command-line options, though the list of functions which implement them. +type CommandLineAction = CommandOptions.CommandOptions -> IO CommandOptions.CommandOptions --Supplied as the type-argument to 'G.OptDescr'. + +{- | + * Parses the command-line arguments, to determine 'CommandOptions.CommandOptions'. + + * Arguments following 'CommandOptions.CommandOptions', are interpreted as a /regex/, followed by the names of the files to which it is applied. +-} +main :: IO () +main = do + progName <- System.getProgName + args <- System.getArgs + + let + usage :: String + usage = "Usage:\t" ++ G.usageInfo progName optDescrList ++ " [, ...]" + +--Define the command-line options, & the 'CommandLineAction's used to handle them. + optDescrList :: [G.OptDescr CommandLineAction] + optDescrList = [ +-- String [String] (G.ArgDescr CommandLineAction) String + G.Option "e" ["regex"] (setExtendedRegExChar `G.ReqArg` "") "Input:\tdefine the regex.", + G.Option "f" ["file"] (readExtendedRegExCharFromFile `G.ReqArg` "") "Input:\tread the regex from file.", + G.Option "c" ["count"] (G.NoArg $ return {-to IO-monad-} . CommandOptions.setCountMatches) ("Output:\tprint only a count of matching lines per file; default '" ++ show (CommandOptions.countMatches Defaultable.defaultValue) ++ "'."), + G.Option "l" ["files-with-matches"] (G.NoArg $ return {-to IO-monad-} . CommandOptions.setListFilesWithMatches) ("Output:\tlist names of files containing at least one match; default '" ++ show (CommandOptions.listFilesWithMatches Defaultable.defaultValue) ++ "'."), + G.Option "L" ["files-without-match"] (G.NoArg $ return {-to IO-monad-} . CommandOptions.setListFilesWithoutMatch) ("Output:\tlist names of files not containing any matches; default '" ++ show (CommandOptions.listFilesWithoutMatch Defaultable.defaultValue) ++ "'."), + G.Option "n" ["number"] (G.NoArg $ return {-to IO-monad-} . CommandOptions.setPrependLineNumbers) ("Output:\tprepend the line-number of the input, to the output; default '" ++ show (CommandOptions.prependLineNumbers Defaultable.defaultValue) ++ "'."), + G.Option "v" ["invert-match"] (G.NoArg $ return {-to IO-monad-} . CommandOptions.setInvertMatch) ("Output:\tselect non-matching lines; default '" ++ show (CommandOptions.invertMatch Defaultable.defaultValue) ++ "'."), + G.Option "" ["version"] (G.NoArg $ const printVersion) "Output:\tprint version-information & then exit.", + G.Option "?" ["help"] (G.NoArg $ const printUsage) "Output:\tdisplay this help-text & then exit.", + G.Option "" ["printCurrentSettings"] (G.NoArg printCurrentSettings) "Output:\tdisplay the currently defined options & then exit.", + G.Option "" ["verbose"] (G.NoArg $ return {-to IO-monad-} . CommandOptions.setVerbose) ("Output:\tsee how the input data was captured by the RegEx; default '" ++ show (CommandOptions.verbose Defaultable.defaultValue) ++ "'."), + G.Option "" ["performance"] ( + extendedRegExCharPerformance `G.ReqArg` "(Int,Either [TestName])" + ) ( + "Test:\tcompare performance with other RegEx-engines, using the specified pair (, Left " ++ show [ + minBound :: Performance.ExtendedRegExTestsNegative.TestName .. maxBound + ] ++ " or Right " ++ show [ + minBound :: Performance.ExtendedRegExTestsPositive.TestName .. maxBound + ] ++ "). NB: specify this option last." + ), + G.Option "" ["runQuickChecks"] (G.NoArg runQuickChecks) "Test:\trun Quick-checks using arbitrary data & then exit. Specify 'verbose' before this option, for detailed output.", + G.Option "" ["assert"] (assertFromFile `G.ReqArg` "") "Test:\trun the assertions, defined in the specified file, & then exit. NB: specify this option last.", + G.Option "" ["assertPosix"] (assertPosixFromFile `G.ReqArg` "") "Test:\trun the assertions, defined in the specified file, against Text.Regex.Posix, & then exit. NB: specify this option last.", + G.Option "" ["abortTrialRepetitionsOnInherentFailure"] (setExecutionOption abortTrialRepetitionsOnInherentFailure `G.ReqArg` "Bool") ("Performance:\tif an alternative can't match, irrespective of the subsequent concatenation, then terminate futile trial repetitions; default '" ++ show (ExecutionOptions.abortTrialRepetitionsOnInherentFailure Defaultable.defaultValue) ++ "'."), +-- G.Option "" ["abortTrialRepetitionsOnZeroConsumption"] (setExecutionOption abortTrialRepetitionsOnZeroConsumption `G.ReqArg` "Bool") ("Performance:\tcheck for zero data-consumption by the n-th repetition of an alternative, before considering (n + 1); default '" ++ show (ExecutionOptions.abortTrialRepetitionsOnZeroConsumption Defaultable.defaultValue) ++ "'."), +-- G.Option "" ["bypassInputDataForLiberalConsumer"] (setExecutionOption bypassInputDataForLiberalConsumer `G.ReqArg` "Bool") ("Performance:\tbypass reading of the unconsumed input data, if the remaining RegEx can consume a sufficient quantity of anything; default '" ++ show (ExecutionOptions.bypassInputDataForLiberalConsumer Defaultable.defaultValue) ++ "'."), + G.Option "" ["catchIncompatibleAnchors"] (setExecutionOption catchIncompatibleAnchors `G.ReqArg` "Bool") ("Performance:\tavoid futile trial solutions, involving repetitions of anchored alternatives, which must consume data; default '" ++ show (ExecutionOptions.catchIncompatibleAnchors Defaultable.defaultValue) ++ "'."), + G.Option "" ["checkExistenceOfInelasticTail"] (setExecutionOption checkExistenceOfInelasticTail `G.ReqArg` "Bool") ("Performance:\tif the regex ends in one or more Meta-data, whose repetition-range is precisely defined, check that this sequence exists at the end of the input data; default '" ++ show (ExecutionOptions.checkExistenceOfInelasticTail Defaultable.defaultValue) ++ "'."), + G.Option "" ["checkForUnconsumableData"] (setExecutionOption checkForUnconsumableData `G.ReqArg` "Bool") ("Performance:\tcheck whether there's no possibility of consuming some of the input data; default '" ++ show (ExecutionOptions.checkForUnconsumableData Defaultable.defaultValue) ++ "'."), + G.Option "" ["complyStrictlyWithPosix"] (setCompilationOption complyStrictlyWithPosix `G.ReqArg` "Bool") ("Output:\tdefine the offset of captured data, corresponding to a sub-expression which matched zero times, as the artificial value -1 specified by Posix; currently only affects results of 'assert'; default '" ++ show (CompilationOptions.complyStrictlyWithPosix Defaultable.defaultValue) ++ "'."), + G.Option "" ["moderateGreed"] (setExecutionOption moderateGreed `G.ReqArg` "Bool") ("Performance:\tgreedily consume data, only up to the limit beyond which, future requirements would be compromised; default '" ++ show (ExecutionOptions.moderateGreed Defaultable.defaultValue) ++ "'."), + G.Option "" ["permitReorderingOfAlternatives"] (setExecutionOption permitReorderingOfAlternatives `G.ReqArg` "Bool") ("Performance:\tpermit alternatives to be re-ordered, in an attempt to process the cheapest first; default '" ++ show (ExecutionOptions.permitReorderingOfAlternatives Defaultable.defaultValue) ++ "'."), + G.Option "" ["preferAlternativesWhichFeedTheGreedy"] (setExecutionOption preferAlternativesWhichFeedTheGreedy `G.ReqArg` "Bool") ("Preference:\tprefer solutions in which the choice of alternatives directs data from non-greedy to greedy consumers; default '" ++ show (ExecutionOptions.preferAlternativesWhichFeedTheGreedy Defaultable.defaultValue) ++ "'."), + G.Option "" ["preferAlternativesWhichMimickUnrolling"] (setExecutionOption preferAlternativesWhichMimickUnrolling `G.ReqArg` "Bool") ("Preference:\tprefer solutions in which the choice of alternatives consumes data like the unrolled repeatable group; default '" ++ show (ExecutionOptions.preferAlternativesWhichMimickUnrolling Defaultable.defaultValue) ++ "'."), + G.Option "" ["preferFewerRepeatedAlternatives"] (setExecutionOption preferFewerRepeatedAlternatives `G.ReqArg` "Bool") ("Preference:\tprefer solutions employing fewer repetitions of alternatives, to discourage the capture of null lists; default '" ++ show (ExecutionOptions.preferFewerRepeatedAlternatives Defaultable.defaultValue) ++ "'."), + G.Option "" ["unrollRepeatedSingletonAlternative"] (setExecutionOption unrollRepeatedSingletonAlternative `G.ReqArg` "Bool") ("Performance:\tunroll repetitions of singleton alternatives; this doesn't affect the result; default '" ++ show (ExecutionOptions.unrollRepeatedSingletonAlternative Defaultable.defaultValue) ++ "'."), + G.Option "" ["useFirstMatchAmongAlternatives"] (setExecutionOption useFirstMatchAmongAlternatives `G.ReqArg` "Bool") ("Performance:\trather than performing an exhaustive search for the optimal choice amongst alternatives, merely select the first that matches; conform to Perl rather than Posix; default '" ++ show (ExecutionOptions.useFirstMatchAmongAlternatives Defaultable.defaultValue) ++ "'."), + G.Option "" ["validateMinConsumptionOfAlternatives"] (setExecutionOption validateMinConsumptionOfAlternatives `G.ReqArg` "Bool") ("Performance:\twhen the number of repetitions of a group of alternatives is precisely specified, check the availability of the resulting minimum data-requirement; default '" ++ show (ExecutionOptions.validateMinConsumptionOfAlternatives Defaultable.defaultValue) ++ "'.") + ] where + printVersion, printUsage :: IO CommandOptions.CommandOptions + + printVersion = System.IO.hPutStrLn System.IO.stderr (Distribution.Text.display packageIdentifier ++ "\n\nCopyright (C) 2010 Dr. Alistair Ward.\nThis program comes with ABSOLUTELY NO WARRANTY.\nThis is free software, and you are welcome to redistribute it under certain conditions.\n\nWritten by Dr. Alistair Ward.") >> System.exitWith System.ExitSuccess where + packageIdentifier :: Distribution.Package.PackageIdentifier + packageIdentifier = Distribution.Package.PackageIdentifier { + Distribution.Package.pkgName = Distribution.Package.PackageName "grecce", + Distribution.Package.pkgVersion = Distribution.Version.Version [0, 9, 0, 7] [] + } + + printUsage = System.IO.hPutStrLn System.IO.stderr usage >> System.exitWith System.ExitSuccess + + printCurrentSettings, runQuickChecks :: CommandLineAction + printCurrentSettings commandOptions = print commandOptions >> System.exitWith System.ExitSuccess + runQuickChecks commandOptions = QC.QuickChecks.run (CommandOptions.verbose commandOptions) >> System.exitWith System.ExitSuccess + + extendedRegExCharPerformance, readExtendedRegExCharFromFile, assertFromFile, assertPosixFromFile, setExtendedRegExChar :: String -> CommandLineAction + extendedRegExCharPerformance s commandOptions = ( + uncurry $ Performance.ExtendedRegEx.run (CommandOptions.executionOptions commandOptions) + ) (read s) >> System.exitWith System.ExitSuccess + + readExtendedRegExCharFromFile s commandOptions = (head . dropWhile null . lines <$> readFile s) >>= (`setExtendedRegExChar` commandOptions) + + assertFromFile fileName commandOptions = do + failedTests <- Assert.RegExOptsChar.findFailures (CommandOptions.compilationOptions commandOptions) (CommandOptions.executionOptions commandOptions) <$> Assert.RegExOptsChar.readTests fileName + + if null failedTests + then System.exitWith System.ExitSuccess + else do + mapM_ print failedTests + + System.exitWith $ System.ExitFailure 2 + + assertPosixFromFile fileName _ = do + failedTests <- Assert.RegExOptsChar.findFailuresPosix <$> Assert.RegExOptsChar.readTests fileName + + if null failedTests + then System.exitWith System.ExitSuccess + else do + mapM_ print failedTests + + System.exitWith $ System.ExitFailure 2 + + setExtendedRegExChar s commandOptions = case CommandOptions.extendedRegExChar commandOptions of + Just e -> error $ "'" ++ show e ++ "' has already been defined" + _ -> return {-to IO-monad-} commandOptions { CommandOptions.extendedRegExChar = Just $ read s } + + abortTrialRepetitionsOnInherentFailure, {-abortTrialRepetitionsOnZeroConsumption, bypassInputDataForLiberalConsumer,-} catchIncompatibleAnchors, checkExistenceOfInelasticTail, checkForUnconsumableData, moderateGreed, permitReorderingOfAlternatives, preferAlternativesWhichFeedTheGreedy, preferAlternativesWhichMimickUnrolling, preferFewerRepeatedAlternatives, unrollRepeatedSingletonAlternative, useFirstMatchAmongAlternatives, validateMinConsumptionOfAlternatives :: CommandOptions.ExecutionOptionsMutator + abortTrialRepetitionsOnInherentFailure s e = e { ExecutionOptions.abortTrialRepetitionsOnInherentFailure = read s } +-- abortTrialRepetitionsOnZeroConsumption s e = e { ExecutionOptions.abortTrialRepetitionsOnZeroConsumption = read s } +-- bypassInputDataForLiberalConsumer s e = e { ExecutionOptions.bypassInputDataForLiberalConsumer = read s } + catchIncompatibleAnchors s e = e { ExecutionOptions.catchIncompatibleAnchors = read s } + checkExistenceOfInelasticTail s e = e { ExecutionOptions.checkExistenceOfInelasticTail = read s } + checkForUnconsumableData s e = e { ExecutionOptions.checkForUnconsumableData = read s } + moderateGreed s e = e { ExecutionOptions.moderateGreed = read s } + permitReorderingOfAlternatives s e = e { ExecutionOptions.permitReorderingOfAlternatives = read s } + preferAlternativesWhichFeedTheGreedy s e = e { ExecutionOptions.preferAlternativesWhichFeedTheGreedy = read s } + preferAlternativesWhichMimickUnrolling s e = e { ExecutionOptions.preferAlternativesWhichMimickUnrolling = read s } + preferFewerRepeatedAlternatives s e = e { ExecutionOptions.preferFewerRepeatedAlternatives = read s } + unrollRepeatedSingletonAlternative s e = e { ExecutionOptions.unrollRepeatedSingletonAlternative = read s } + useFirstMatchAmongAlternatives s e = e { ExecutionOptions.useFirstMatchAmongAlternatives = read s } + validateMinConsumptionOfAlternatives s e = e { ExecutionOptions.validateMinConsumptionOfAlternatives = read s } + + complyStrictlyWithPosix :: CommandOptions.CompilationOptionsMutator + complyStrictlyWithPosix s e = e { CompilationOptions.complyStrictlyWithPosix = read s } + + setCompilationOption :: CommandOptions.CompilationOptionsMutator -> String -> CommandLineAction + setCompilationOption compilationOptionsMutator s = return {-to IO-monad-} . CommandOptions.setCompilationOption compilationOptionsMutator s + + setExecutionOption :: CommandOptions.ExecutionOptionsMutator -> String -> CommandLineAction + setExecutionOption executionOptionsMutator s = return {-to IO-monad-} . CommandOptions.setExecutionOption executionOptionsMutator s + +-- G.getOpt :: G.ArgOrder CommandLineAction -> [G.OptDescr Action] -> [String] -> ([Action], [String], [String]) + case G.getOpt G.RequireOrder optDescrList args of + (commandLineActions, nonOptions, []) -> do + commandOptions <- Data.List.foldl' (>>=) (return {-to IO-monad-} Defaultable.defaultValue) commandLineActions --Sequentially transform the 'CommandOptions', using 'CommandLineAction's corresponding to the specified command-line flags. + + Grep.grep commandOptions nonOptions --Perform the requested match, against the list of input-data files. + (_, _, errors) -> System.IO.Error.ioError . System.IO.Error.userError $ concat errors ++ usage --Throw. + diff --git a/src/RegExChar/ExtendedRegExChar.hs b/src/RegExChar/ExtendedRegExChar.hs new file mode 100644 --- /dev/null +++ b/src/RegExChar/ExtendedRegExChar.hs @@ -0,0 +1,197 @@ +{-# OPTIONS_GHC -fno-warn-orphans #-} +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] An 'RegEx.ExtendedRegEx', which has been specialised for 'Char', to create a tradition non-polymorphic /regex/. +-} + +module RegExChar.ExtendedRegExChar( +-- * Types +-- ** Type-synonyms + ExtendedRegExChar(..), + InputData, +-- * Functions +-- ** Operators + (+~), + (=~), + (/~) +) where + +import Control.Applicative((<$>)) +import Control.Arrow((***)) +import RegExDot.DSL((<~>), (-:)) +import Text.ParserCombinators.Parsec((), (<|>)) +import qualified Control.Monad +import qualified Data.List +import qualified RegExChar.MetaChar as MetaChar +import qualified RegExDot.Anchor as Anchor +import qualified RegExDot.Consumer as Consumer +import qualified RegExDot.RegEx as RegEx +import qualified RegExDot.RegExOpts as RegExOpts +import qualified RegExDot.Repeatable as Repeatable +import qualified Text.ParserCombinators.Parsec as Parsec +import qualified ToolShed.SelfValidate as SelfValidate + +infix 4 +~, =~, /~ --Same as (==) & (/=). + +-- | Specialise a 'RegEx.ExtendedRegEx' for 'Char', & encapsulate it to permit tailored instance-declarations. +data ExtendedRegExChar = MkExtendedRegExChar { + hasNonCapturingTopLevelAlternatives :: Bool, -- ^ The string from which a 'RegEx.ExtendedRegEx' is read, may, if data-capture isn't required, omit explicit delimiters around top-level 'RegEx.Alternatives'. + extendedRegEx :: RegEx.ExtendedRegEx Char +} deriving Eq + +-- | Abbreviation. +type InputData = RegEx.InputData Char + +instance SelfValidate.SelfValidator ExtendedRegExChar where + isValid MkExtendedRegExChar { + hasNonCapturingTopLevelAlternatives = hasNonCapturingTopLevelAlternatives', + extendedRegEx = extendedRegEx' + } = (not hasNonCapturingTopLevelAlternatives' || all (RegEx.isCaptureGroup . Repeatable.base) (RegEx.concatenation extendedRegEx')) && SelfValidate.isValid extendedRegEx' + +instance Consumer.Consumer ExtendedRegExChar where + consumptionProfile = Consumer.consumptionProfile . extendedRegEx + starHeight = Consumer.starHeight . extendedRegEx + +instance RegEx.ShortcutExpander Char where + expand c = error $ "RegExDot.RegEx.ShortcutExpander.expand RegExChar.ExtendedRegExChar:\tunrecognised shortcut '" ++ show c ++ "'." + +instance Read ExtendedRegExChar where + readsPrec _ s + | s == reverse Anchor.tokens = [ + ( + MkExtendedRegExChar { + hasNonCapturingTopLevelAlternatives = False, + extendedRegEx = (Just Anchor.Bow, Just Anchor.Stern) <~> [] + }, + "" + ) + ] --The order of adjacent zero-width assertions is irrelevant. + | otherwise = let + extendedRegExCharParser :: Parsec.Parser ExtendedRegExChar + extendedRegExCharParser = reduce {-correct prior assumption-} <$> alternativesParser {-assume non-capturing top-level Alternatives-} where + reduce :: RegEx.Alternatives Char -> ExtendedRegExChar + reduce alternatives + | RegEx.isSingletonAlternatives alternatives = MkExtendedRegExChar False . head $ RegEx.deconstructAlternatives alternatives + | otherwise = MkExtendedRegExChar True $ (Nothing, Nothing) <~> RegEx.CaptureGroup alternatives -: [] --Infer non-capturing top-level 'RegEx.Alternatives' from the presence of 'RegEx.alternativeExtendedRegExSeparatorToken's. + + alternativesParser :: Parsec.Parser (RegEx.Alternatives Char) + alternativesParser = RegEx.MkAlternatives <$> extendedRegExParser `Parsec.sepBy1` (Parsec.char RegEx.alternativeExtendedRegExSeparatorToken "RegEx.alternativeExtendedRegExSeparatorToken " ++ show RegEx.alternativeExtendedRegExSeparatorToken) where + extendedRegExParser :: Parsec.Parser (RegEx.ExtendedRegEx Char) + extendedRegExParser = do + maybeBowAnchor <- Parsec.option Nothing {-default-} $ (Parsec.char Anchor.bowToken "Anchor.bowToken " ++ show Anchor.bowToken) >> return {-to GenParser-monad-} (Just Anchor.Bow) + repeatableRequirementList <- repeatableRequirementListParser + + ( + do + repeatableCaptureGroup <- Repeatable.repeatableParser . RegEx.CaptureGroup =<< uncurry Parsec.between ( + Parsec.char *** Parsec.char $ RegEx.captureGroupDelimiters + ) alternativesParser {-recurse-} "RegEx.captureGroupDelimiters " ++ show RegEx.captureGroupDelimiters + + extendedRegEx' <- extendedRegExParser --Recurse. + + return {-to GenParser-monad-} $ RegEx.transformExtendedRegEx ((repeatableRequirementList ++) . (repeatableCaptureGroup :)) extendedRegEx' { RegEx.bowAnchor = maybeBowAnchor } + ) <|> ( + do + maybeSternAnchor <- Parsec.option Nothing {-default-} $ (Parsec.char Anchor.sternToken "Anchor.sternToken " ++ show Anchor.sternToken) >> return {-to GenParser-monad-} (Just Anchor.Stern) + + return {-to GenParser-monad-} RegEx.MkExtendedRegEx { + RegEx.bowAnchor = maybeBowAnchor, + RegEx.concatenation = repeatableRequirementList, + RegEx.sternAnchor = maybeSternAnchor + } + ) + where + repeatableRequirementListParser :: Parsec.Parser (RegEx.Concatenation Char) + repeatableRequirementListParser = Parsec.choice [ + Parsec.try . Parsec.lookAhead $ ( + Parsec.char Anchor.sternToken "Anchor.sternToken " ++ show Anchor.sternToken + ) >> ( + ( + Parsec.eof >> return {-to GenParser-monad-} [] + ) <|> ( + Parsec.oneOf [RegEx.alternativeExtendedRegExSeparatorToken, snd RegEx.captureGroupDelimiters] >> return {-to GenParser-monad-} [] + ) + ), + Control.Monad.liftM2 (:) ( + MetaChar.metaCharParser >>= Repeatable.repeatableParser . RegEx.Require . MetaChar.deconstruct + ) repeatableRequirementListParser, {-recurse-} + return {-to GenParser-monad-} [] + ] + + in (error . ("readsPrec RegExChar.ExtendedRegExChar:\tparse-error; " ++) . show) `either` (filter (SelfValidate.isValid . fst) . return) $ Parsec.parse ( + Control.Monad.liftM2 (,) extendedRegExCharParser Parsec.getInput + ) "ExtendedRegExChar" s + +instance Show ExtendedRegExChar where + showsPrec _ MkExtendedRegExChar { + hasNonCapturingTopLevelAlternatives = hasNonCapturingTopLevelAlternatives', + extendedRegEx = RegEx.MkExtendedRegEx { + RegEx.bowAnchor = maybeBowAnchor, + RegEx.concatenation = concatenation', + RegEx.sternAnchor = maybeSternAnchor + } + } = RegEx.showsMaybeAnchor maybeBowAnchor . foldl (.) (showString "") ( + ( + let + showAlternatives :: RegEx.Alternatives Char -> [ShowS] + showAlternatives = Data.List.intersperse (showChar RegEx.alternativeExtendedRegExSeparatorToken) . map (shows . MkExtendedRegExChar False) . RegEx.deconstructAlternatives + in if hasNonCapturingTopLevelAlternatives' + then map ( + \repeatablePattern -> case Repeatable.base repeatablePattern of + RegEx.CaptureGroup alternatives -> foldr (.) (showString "") $ showAlternatives alternatives + _ -> error $ "Show RegExChar.ExtendedRegExChar: unexpected " ++ show repeatablePattern + ) + else map ( + \repeatablePattern -> ( + case Repeatable.base repeatablePattern of + RegEx.Require meta -> shows $ MetaChar.MkMetaChar meta + RegEx.CaptureGroup alternatives -> showChar ( + fst RegEx.captureGroupDelimiters + ) . foldr (.) ( + showChar $ snd RegEx.captureGroupDelimiters --Initial value. + ) ( + showAlternatives alternatives + ) + ) . Repeatable.showSuffix repeatablePattern + ) + ) concatenation' + ) . RegEx.showsMaybeAnchor maybeSternAnchor + +-- | A veneer over the underlying polymorphic operator, 'RegEx.+~'. +(+~) :: + InputData -- ^ The input-data string. + -> RegExOpts.RegExOpts ExtendedRegExChar -- ^ The match-options, parameterised by the regex-specification. + -> RegEx.Result Char +inputData +~ regExOpts = inputData RegEx.+~ fmap extendedRegEx regExOpts --CAVEAT: . + +-- | A veneer over the underlying polymorphic operator, 'RegEx.=~'. +(=~) :: + InputData -- ^ The input-data string. + -> RegExOpts.RegExOpts ExtendedRegExChar -- ^ The match-options, parameterised by the regex-specification. + -> Bool +inputData =~ regExOpts = inputData RegEx.=~ fmap extendedRegEx regExOpts --CAVEAT: . + +-- | Pattern-mismatch operator. +(/~) :: + InputData -- ^ The input-data string. + -> RegExOpts.RegExOpts ExtendedRegExChar -- ^ The match-options, parameterised by the regex-specification. + -> Bool +(/~) inputData = not . (inputData =~) + diff --git a/src/RegExChar/MetaChar.hs b/src/RegExChar/MetaChar.hs new file mode 100644 --- /dev/null +++ b/src/RegExChar/MetaChar.hs @@ -0,0 +1,252 @@ +{-# OPTIONS_GHC -fno-warn-orphans #-} +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] + + * Implements 'Meta.ShortcutExpander' 'Char', to implement some of the traditional /Perl-style shortcuts/: + + [@\\d@] => Any 'Data.Char.isDigit'-character. + + [@\\D@] => Any non-'Data.Char.isDigit' character. + + [@\\s@] => Any 'Data.Char.isSpace'-character. + + [@\\S@] => Any non-'Data.Char.isSpace' character. + + [@\\w@] => Any 'Data.Char.isWord' letter. + + [@\\W@] => Any non-'Data.Char.isWord' letter. + + * Enables one to compose concise regexen, containing any 'Char' that's a member of one of these predefined sets. + + * Though 'Meta.Meta' is polymorphic, & the type-parameter can't be assumed to implement either 'Enum' or 'Ord', + 'Char' actually does, so this module is able to implement ranges within a /Bracket-expression/. + + * Defines specialised instances of 'Read' & 'Show', to cope with /Perl-style shortcuts/, /Posix Character-classes/ & /Bracket-expression/ range-specifications. + + [@TODO@] Use @Environment.getLocale@. Regrettably, this returns in the "IO"-monad, & even it didn't, how does one pass that information to 'Read' ?! +-} + +module RegExChar.MetaChar( +-- * Types +-- AssociationList, +-- Dictionary, +-- ** Type-synonyms + MetaChar(..), +-- * Constants +-- bracketExpressionRangeToken, +-- posixCharacterClassDelimiters, +-- * Functions +-- implementPerlShortcut, + metaCharParser, +-- ** Accessors (Deconstructors) + deconstruct +) where + +import Control.Applicative((<$>)) +import Control.Arrow((***)) +import Text.ParserCombinators.Parsec(()) +import qualified Control.Monad +import qualified Data.Char +import qualified Data.Map +import qualified RegExDot.BracketExpression as BracketExpression +import qualified RegExDot.BracketExpressionMember as BracketExpressionMember +import qualified RegExDot.Consumer as Consumer +import qualified RegExDot.Meta as Meta +import qualified RegExDot.RegEx as RegEx +import qualified RegExDot.Repeatable as Repeatable +import qualified RegExDot.ShowablePredicate as ShowablePredicate +import qualified Text.ParserCombinators.Parsec as Parsec +import qualified ToolShed.SelfValidate as SelfValidate + +-- | Holds a mapping from an escape-sequence & the predicate via which it is implemented. +type AssociationList key = [(key, ShowablePredicate.Predicate Char)] + +-- | Holds a mapping from an escape-sequence & the predicate via which it is implemented. +type Dictionary key = Data.Map.Map key (ShowablePredicate.Predicate Char) + +instance BracketExpressionMember.ShortcutExpander Char where + findPredicate shortcut = ShowablePredicate.MkShowablePredicate [Meta.shortcutToken, shortcut] <$> shortcut `Data.Map.lookup` dictionary where + dictionary :: Dictionary Char + dictionary = Data.Map.fromList $ perlShortcuts ++ invert perlShortcuts where + perlShortcuts :: AssociationList Char + perlShortcuts = [ + ('d', Data.Char.isDigit), + ('s', Data.Char.isSpace), + ('w', \c -> ($ c) `any` [Data.Char.isAlphaNum, (== '_')]) + ] + + invert :: AssociationList Char -> AssociationList Char + invert = map (Data.Char.toUpper *** (not .)) + +instance Meta.ShortcutExpander Char where + expand c = case BracketExpressionMember.findPredicate c of + Just showablePredicate -> Meta.Predicate showablePredicate + _ -> Meta.Literal c + +-- | A specialised instance, encapsulated to permit tailored instance-declarations. +newtype MetaChar = MkMetaChar (Meta.Meta Char) deriving Eq + +-- | Accessor. +deconstruct :: MetaChar -> Meta.Meta Char +deconstruct (MkMetaChar metaChar) = metaChar + +instance SelfValidate.SelfValidator MetaChar where + isValid = SelfValidate.isValid . deconstruct + +instance Consumer.Consumer MetaChar where + consumptionProfile = Consumer.consumptionProfile . deconstruct + starHeight = Consumer.starHeight . deconstruct + +-- | Builds a parser of traditional regex-syntax, which understands 'Char'-specific concepts like /Perl-style shortcuts/ & /Posix Character-classes/. +metaCharParser :: Parsec.Parser MetaChar +metaCharParser = MkMetaChar <$> Parsec.choice [ + (Parsec.char Meta.anyToken "Meta.anyToken " ++ show Meta.anyToken) >> return {-to GenParser-monad-} Meta.Any, + (Parsec.char Meta.shortcutToken "Meta.shortcutToken " ++ show Meta.shortcutToken) >> Meta.expand <$> Parsec.anyChar, + uncurry Parsec.between (Parsec.char *** Parsec.char $ BracketExpression.delimiterTokens) ( + do + let + implementPosixCharacterClass :: String -> Maybe (ShowablePredicate.ShowablePredicate Char) + implementPosixCharacterClass identifier = ShowablePredicate.MkShowablePredicate ( + fst posixCharacterClassDelimiters ++ identifier ++ snd posixCharacterClassDelimiters + ) <$> identifier `Data.Map.lookup` dictionary where + dictionary :: Dictionary String + dictionary = Data.Map.fromList $ posixCharacterClasses ++ invert posixCharacterClasses where + posixCharacterClasses :: AssociationList String + posixCharacterClasses = [ + ("alnum", Data.Char.isAlphaNum), + ("alpha", Data.Char.isAlpha), + ("ascii", Data.Char.isAscii), + ("blank", (`elem` " \t")), + ("cntrl", Data.Char.isControl), + ("digit", Data.Char.isDigit), + ("graph", \c -> not $ ($ c) `any` [Data.Char.isSpace, Data.Char.isControl]), + ("lower", Data.Char.isLower), + ("print", Data.Char.isPrint), + ("punct", Data.Char.isPunctuation), + ("space", Data.Char.isSpace), + ("upper", Data.Char.isUpper), + ("word", \c -> ($ c) `any` [Data.Char.isAlphaNum, (== '_')]), + ("xdigit", Data.Char.isHexDigit) + ] + + invert :: AssociationList String -> AssociationList String + invert = map $ (BracketExpression.negationToken :) *** (not .) + + cTor <- Parsec.option Meta.AnyOf {-default-} $ (Parsec.char BracketExpression.negationToken "BracketExpression.negationToken " ++ show BracketExpression.negationToken) >> return {-to GenParser-monad-} Meta.NoneOf + + literalBracketExpressionTerminator <- Parsec.option [] {-default-} $ return {-to List-monad-} . BracketExpressionMember.Literal <$> ( + Parsec.char (snd BracketExpression.delimiterTokens) "Literal Bracket-expression terminator " ++ show (snd BracketExpression.delimiterTokens) + ) --If the first item in a BracketExpression (or negated BracketExpression) is the terminator-token, then it is treated as a 'BracketExpressionMember.Literal'. + + cTor . (literalBracketExpressionTerminator ++) <$> Parsec.many {-potentially zero-} ( + Parsec.choice [ + ( + do + _ <- Parsec.char Meta.shortcutToken "Meta.shortcutToken " ++ show Meta.shortcutToken + c <- Parsec.anyChar + + return {-to GenParser-monad-} $ case BracketExpressionMember.findPredicate c of + Just showablePredicate -> BracketExpressionMember.Predicate showablePredicate + _ -> BracketExpressionMember.Literal c --Escaped literal. + ) "Perl-style shortcut", + Parsec.try ( + uncurry Parsec.between (Parsec.string *** Parsec.string $ posixCharacterClassDelimiters) ( + do + identifier <- Parsec.many1 $ Parsec.noneOf [head $ snd posixCharacterClassDelimiters] + + case implementPosixCharacterClass identifier of + Just showablePredicate -> return {-to GenParser-monad-} $ BracketExpressionMember.Predicate showablePredicate + _ -> Parsec.unexpected $ "MetaChar.metaCharParser:\tunrecognised Posix Character-class; " ++ show identifier + ) "Posix Character-class " ++ show posixCharacterClassDelimiters + ), --Regurgitate erroneously consumed input. + Parsec.try ( + ( + do + rangeStart <- Parsec.noneOf [snd BracketExpression.delimiterTokens] + _ <- Parsec.char bracketExpressionRangeToken "bracketExpressionRangeToken " ++ show bracketExpressionRangeToken + rangeEnd <- Parsec.noneOf [snd BracketExpression.delimiterTokens] + + return {-to GenParser-monad-} . BracketExpressionMember.Predicate . ShowablePredicate.MkShowablePredicate [ + rangeStart, + bracketExpressionRangeToken, + rangeEnd + ] $ \c -> rangeStart <= c && c <= rangeEnd --Create custom predicate, utilising "Ord Char". + ) "Bracket-expression range" + ), --Regurgitate erroneously consumed input. + BracketExpressionMember.Literal <$> Parsec.noneOf [snd BracketExpression.delimiterTokens] "BracketExpressionMember.Literal" --TODO: the first Char-member can be a literal ']'. + ] "Bracket-expression member" + ) "Bracket-expression member-list" + ) "BracketExpression.delimiterTokens " ++ show BracketExpression.delimiterTokens, + Meta.Literal <$> Parsec.noneOf RegEx.tokens + ] + +instance Read MetaChar where + readsPrec _ = ( + (error . ("readsPrec RegExChar.MetaChar:\tparse-error; " ++) . show) `either` return + ) . Parsec.parse ( + Control.Monad.liftM2 (,) metaCharParser Parsec.getInput + ) "MetaChar" + +-- | The token used to signify an ordered range of members in a /Bracket-expression/. +bracketExpressionRangeToken :: Char +bracketExpressionRangeToken = '-' + +-- | The delimiters of a /Posix Character-class/. +posixCharacterClassDelimiters :: (String, String) +posixCharacterClassDelimiters = ("[:", ":]") + +instance Show MetaChar where + showsPrec _ (MkMetaChar Meta.Any) = showChar Meta.anyToken + showsPrec _ (MkMetaChar (Meta.Literal c)) = ( + if c `elem` [ + fst BracketExpression.delimiterTokens, + fst Repeatable.rangeDelimiters, + Repeatable.oneOrMoreToken, + Repeatable.zeroOrMoreToken, + Repeatable.zeroOrOneToken, + Meta.anyToken, + Meta.shortcutToken + ] ++ RegEx.tokens + then showChar Meta.shortcutToken + else id + ) . showChar c + showsPrec _ (MkMetaChar (Meta.AnyOf bracketExpression)) = showChar ( + fst BracketExpression.delimiterTokens + ) . showString ( + foldr ( + \e -> case e of + BracketExpressionMember.Predicate showablePredicate -> shows showablePredicate + BracketExpressionMember.Literal literal -> ( + if literal `elem` [ + bracketExpressionRangeToken, --CAVEAT: only unambiguously literal when at the start or end of a "BracketExpression". + Meta.shortcutToken, + snd BracketExpression.delimiterTokens --CAVEAT: only unambiguously literal when at the start of a "BracketExpression". + ] + then showChar Meta.shortcutToken + else id + ) . showChar literal + ) ( + showChar (snd BracketExpression.delimiterTokens) "" --Initial value. + ) bracketExpression + ) + showsPrec _ (MkMetaChar (Meta.NoneOf bracketExpression)) = showChar x . showChar BracketExpression.negationToken . showString xs where (x : xs) = show . MkMetaChar $ Meta.AnyOf bracketExpression + showsPrec _ (MkMetaChar m) = shows m + diff --git a/src/RegExChar/RegExOptsChar.hs b/src/RegExChar/RegExOptsChar.hs new file mode 100644 --- /dev/null +++ b/src/RegExChar/RegExOptsChar.hs @@ -0,0 +1,218 @@ +{-# LANGUAGE MultiParamTypeClasses, TypeSynonymInstances, FlexibleInstances, FlexibleContexts #-} +{-# OPTIONS_GHC -fno-warn-orphans #-} + +{- + Copyright (C) 2010 Dr. Alistair Ward + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +-} +{- | + [@AUTHOR@] Dr. Alistair Ward + + [@DESCRIPTION@] Creates a /back-end/ implementation, conforming to . + + [@CAVEATS@] + + * The underlying polymorphic (in terms of the base-type of the list of input data) 'RegEx.ExtendedRegEx'-engine is never going to be a drop-in replacement for other /regex/-engines, + so this standard interface has only been implemented for traditional input data-type ['Char']. + + * The standard interface requires many Haskell-extensions, but since this implementation is just a thin layer over the underlying portable polymorphic 'RegEx.ExtendedRegEx'-engine, + the latter can still be used directly, where any of these extensions are unavailable. +-} + +module RegExChar.RegExOptsChar( +-- * Types +-- ** Type-synonyms + MatchSpan, +-- MatchDataSpan, + RegExOptsChar, +-- * Functions +-- exciseNonCapturingTopLevelAlternatives, +-- matchUntilFailure, +-- toMatchDataSpanList, + toZeroIndexedArray, +-- ** Operators + (=~) +-- ** Predicates +-- complyStrictlyWithPosix, +-- hasNonCapturingTopLevelAlternatives +) where + +import Control.Applicative((<$>)) +import Text.Regex.Base.Context() --Instance-declarations. +import qualified Data.Array +import qualified RegExChar.ExtendedRegExChar as ExtendedRegExChar +import qualified RegExDot.CompilationOptions as CompilationOptions +import qualified RegExDot.ConsumptionBounds as ConsumptionBounds +import qualified RegExDot.DataSpan as DataSpan +import qualified RegExDot.DataSpanTree as DataSpanTree +import qualified RegExDot.ExecutionOptions as ExecutionOptions +import qualified RegExDot.RegEx as RegEx +import qualified RegExDot.RegExOpts as RegExOpts +import qualified RegExDot.Result as Result +import qualified RegExDot.Tree as Tree +import qualified Text.Regex.Base.RegexLike as RegexLike +import qualified ToolShed.ListPlus as ListPlus +import qualified ToolShed.Defaultable as Defaultable +import qualified ToolShed.Options as Options + +infix 4 =~ --Same as (==). + +-- | Defines a specific instance of the polymorphic base-type. +type RegExOptsChar = RegExOpts.RegExOpts ExtendedRegExChar.ExtendedRegExChar + +-- | Convenience accessor-function. +hasNonCapturingTopLevelAlternatives :: RegExOptsChar -> Bool +hasNonCapturingTopLevelAlternatives = ExtendedRegExChar.hasNonCapturingTopLevelAlternatives . RegExOpts.regEx + +-- | Convenience accessor-function. +complyStrictlyWithPosix :: RegExOptsChar -> Bool +complyStrictlyWithPosix = CompilationOptions.complyStrictlyWithPosix . RegExOpts.compilationOptions + +instance RegexLike.RegexOptions RegExOptsChar CompilationOptions.CompilationOptions ExecutionOptions.ExecutionOptions where + blankCompOpt = Options.blankValue + blankExecOpt = Options.blankValue + defaultCompOpt = Defaultable.defaultValue + defaultExecOpt = Defaultable.defaultValue + setExecOpts e r = r { RegExOpts.executionOptions = e } + getExecOpts = RegExOpts.executionOptions + +--Newer versions of this 'Text.Regex.Base.RegexLike.RegexMaker' have additional monadic methods, which can fail on parse-errors. +instance RegexLike.RegexMaker RegExOptsChar CompilationOptions.CompilationOptions ExecutionOptions.ExecutionOptions String where + makeRegexOpts c e source = RegExOpts.MkRegExOpts { + RegExOpts.compilationOptions = c, + RegExOpts.executionOptions = e, + RegExOpts.regEx = read source + } + +-- | The offset and length of the 'RegEx.InputData' consumed in one 'RegEx.Match'. +type MatchSpan = (RegexLike.MatchOffset, RegexLike.MatchLength) + +{- | + * The 'MatchSpan' augmented by the consumed 'ExtendedRegExChar.InputData' to which it refers. + + * Similar to 'DataSpan.DataSpan'. +-} +type MatchDataSpan = (ExtendedRegExChar.InputData, MatchSpan) + +{- | + * Convert the 'RegEx.MatchList', into the 'MatchDataSpan's required by "Text.Regex.Base.RegexLike". + + * Treat the whole 'ExtendedRegExChar.ExtendedRegExChar' as an additional implicit capture-group. + + * CAVEAT: @DataSpanTree.extractCaptureGroups True@ may return some 'DataSpan.DataSpan's containing the artificial offset @-1@ specified by Posix => don't use this as the basis for any arithmetic. +-} +toMatchDataSpanList :: + Bool + -> ConsumptionBounds.DataLength -- ^ The offset to use for zero-length match. + -> RegEx.MatchList Char -- ^ The list of matches. + -> [MatchDataSpan] +toMatchDataSpanList _ offset [] = [DataSpan.empty offset] --The whole regex matched, but consumed nothing. +toMatchDataSpanList strictPosixCompliance _ matchList = DataSpanTree.extractCaptureGroups strictPosixCompliance . return {-to List-monad-} . Tree.Node . return {-to List-monad-} $ DataSpanTree.toTreeList matchList + +-- | Optionally remove the 2nd element from the list, where it represents the data captured by a groups of top-level 'Alternatives', which weren't explicitly delimited & therefore are non-capturing. +exciseNonCapturingTopLevelAlternatives :: RegExOptsChar -> [MatchDataSpan] -> [MatchDataSpan] +exciseNonCapturingTopLevelAlternatives regExOptsChar + | hasNonCapturingTopLevelAlternatives regExOptsChar = ListPlus.excise 1 --The zeroeth represents the whole regex, the 1st represents the top-level non-capturing group of Alternatives, & the remainder represent explicitly delimited groups. + | otherwise = id + +-- | Convert the specified list, into a zero-indexed array. +toZeroIndexedArray :: [e] -> Data.Array.Array Int e +toZeroIndexedArray l = Data.Array.listArray (0, length l - 1) l + +{- | + * Repeatedly apply the 'RegExOptsChar' to the 'ExtendedRegExChar.InputData', forwarding unmatched input data to the next match-attempt, until it's all been consumed. + + * The offsets, from the start of the input data, of all matches after the first, are shifted to account for input data already consumed by previous matches. +-} +matchUntilFailure :: + RegExOptsChar -- ^ The match-criteria. + -> ExtendedRegExChar.InputData -- ^ The input-data, to be consumed by repeated matching. + -> [RegEx.MatchList Char] +matchUntilFailure regExOptsChar inputData = shiftOffsets 0 $ matchUntilFailure' inputData where + matchUntilFailure' :: ExtendedRegExChar.InputData -> [RegEx.MatchList Char] + matchUntilFailure' unmatchedInputData = case unmatchedInputData ExtendedRegExChar.+~ regExOptsChar of --CAVEAT: . + (_, Just matchList, maybeSternAnchorResult) -> matchList : if null (RegEx.extractDataFromMatchList matchList) then [] else matchUntilFailure' (RegEx.extractDataFromMatch' maybeSternAnchorResult) + _ -> [] + + shiftOffsets :: ConsumptionBounds.DataLength -> [RegEx.MatchList Char] -> [RegEx.MatchList Char] + shiftOffsets offset (matchList : matchLists) = let + shiftedMatchList :: RegEx.MatchList Char + shiftedMatchList = RegEx.shiftMatchList offset matchList + in shiftedMatchList : shiftOffsets (DataSpan.after . last . DataSpanTree.flattenTreeList 0 $ DataSpanTree.toTreeList shiftedMatchList) matchLists {-recurse-} + shiftOffsets _ _ = [] + +instance RegexLike.RegexLike RegExOptsChar ExtendedRegExChar.InputData where +{- E.g.: + ("xabcdxabdxxadx" RegExOptsChar.=~ "a(b(c?))*d") :: [Text.Regex.Base.RegexLike.MatchArray] + [array (0,2) [(0,(1,4)),(1,(2,2)),(2,(3,1))],array (0,2) [(0,(6,3)),(1,(7,1)),(2,(8,0))],array (0,1) [(0,(11,2)),(1,(-1,0))]] +-} +-- matchAll :: RegExOptsChar -> ExtendedRegExChar.InputData -> [RegexLike.MatchArray] + matchAll regExOptsChar = map (toZeroIndexedArray . map snd {-span-} . exciseNonCapturingTopLevelAlternatives regExOptsChar . toMatchDataSpanList (complyStrictlyWithPosix regExOptsChar) 0) . matchUntilFailure regExOptsChar + +{- E.g.: + ("xabcdxabdxxadx" RegExOptsChar.=~ "a(b(c?))*d") :: [Text.Regex.Base.RegexLike.MatchText String] + [array (0,2) [(0,("abcd",(1,4))),(1,("bc",(2,2))),(2,("c",(3,1)))],array (0,2) [(0,("abd",(6,3))),(1,("b",(7,1))),(2,("",(8,0)))],array (0,1) [(0,("ad",(11,2))),(1,("",(-1,0)))]] +-} +-- matchAllText :: RegExOptsChar -> ExtendedRegExChar.InputData -> [RegexLike.MatchText ExtendedRegExChar.InputData] + matchAllText regExOptsChar = map (toZeroIndexedArray . exciseNonCapturingTopLevelAlternatives regExOptsChar . toMatchDataSpanList (complyStrictlyWithPosix regExOptsChar) 0) . matchUntilFailure regExOptsChar + +{- E.g.: + ("xabcdx" RegExOptsChar.=~ "a(b(c?))*d") :: Text.Regex.Base.RegexLike.MatchArray + array (0,2) [(0,(1,4)),(1,(2,2)),(2,(3,1))] +-} +-- matchOnce :: RegExOptsChar -> ExtendedRegExChar.InputData -> Maybe RegexLike.MatchArray + matchOnce regExOptsChar inputData = toZeroIndexedArray . map snd {-span-} . exciseNonCapturingTopLevelAlternatives regExOptsChar . toMatchDataSpanList ( + complyStrictlyWithPosix regExOptsChar + ) ( + RegEx.externalMatchLength $ Result.getPreMatch extendedRegExResult + ) <$> Result.getMatchList extendedRegExResult where + extendedRegExResult :: RegEx.Result Char + extendedRegExResult = inputData ExtendedRegExChar.+~ regExOptsChar --CAVEAT: . + +{- E.g.: + ("xabcdx" RegExOptsChar.=~ "a(b(c?))*d") :: (String, Text.Regex.Base.RegexLike.MatchText String, String) + ("x",array (0,2) [(0,("abcd",(1,4))),(1,("bc",(2,2))),(2,("c",(3,1)))],"x") +-} +-- matchOnceText :: RegExOptsChar -> ExtendedRegExChar.InputData -> Maybe (ExtendedRegExChar.InputData, MatchText ExtendedRegExChar.InputData, ExtendedRegExChar.InputData) + matchOnceText regExOptsChar inputData = case inputData ExtendedRegExChar.+~ regExOptsChar of --CAVEAT: . + (maybeBowAnchorResult, Just matchList, maybeSternAnchorResult) -> Just ( + RegEx.extractDataFromMatch' maybeBowAnchorResult, + toZeroIndexedArray . exciseNonCapturingTopLevelAlternatives regExOptsChar $ toMatchDataSpanList ( + complyStrictlyWithPosix regExOptsChar + ) ( + RegEx.externalMatchLength maybeBowAnchorResult + ) matchList, + RegEx.extractDataFromMatch' maybeSternAnchorResult + ) + _ -> Nothing + +{- E.g.: + ("xabcdx" RegExOptsChar.=~ "a(b(c?))*d") :: Bool + True +-} +-- matchTest :: RegExOptsChar -> ExtendedRegExChar.InputData -> Bool + matchTest = flip (ExtendedRegExChar.=~) --CAVEAT: . + +{- | + * Match-operator. + + * The polymorphic return-type is resolved by the caller's "RegexLike.RegexContext". +-} +(=~) :: RegexLike.RegexContext RegExOptsChar ExtendedRegExChar.InputData target + => ExtendedRegExChar.InputData -- ^ The input data. + -> String -- ^ The string from which to read the regex-specification. + -> target -- ^ The polymorphic return-type. +inputData =~ s = (RegexLike.makeRegex s :: RegExOptsChar) `RegexLike.match` inputData +