diff --git a/LICENSE b/LICENSE new file mode 100644 --- /dev/null +++ b/LICENSE @@ -0,0 +1,661 @@ + GNU AFFERO GENERAL PUBLIC LICENSE + Version 3, 19 November 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU Affero General Public License is a free, copyleft license for +software and other kinds of works, specifically designed to ensure +cooperation with the community in the case of network server software. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +our General Public Licenses are intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + Developers that use our General Public Licenses protect your rights +with two steps: (1) assert copyright on the software, and (2) offer +you this License which gives you legal permission to copy, distribute +and/or modify the software. + + A secondary benefit of defending all users' freedom is that +improvements made in alternate versions of the program, if they +receive widespread use, become available for other developers to +incorporate. Many developers of free software are heartened and +encouraged by the resulting cooperation. However, in the case of +software used on network servers, this result may fail to come about. +The GNU General Public License permits making a modified version and +letting the public access it on a server without ever releasing its +source code to the public. + + The GNU Affero General Public License is designed specifically to +ensure that, in such cases, the modified source code becomes available +to the community. It requires the operator of a network server to +provide the source code of the modified version running there to the +users of that server. Therefore, public use of a modified version, on +a publicly accessible server, gives the public access to the source +code of the modified version. + + An older license, called the Affero General Public License and +published by Affero, was designed to accomplish similar goals. This is +a different license, not a version of the Affero GPL, but Affero has +released a new version of the Affero GPL which permits relicensing under +this license. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU Affero General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Remote Network Interaction; Use with the GNU General Public License. + + Notwithstanding any other provision of this License, if you modify the +Program, your modified version must prominently offer all users +interacting with it remotely through a computer network (if your version +supports such interaction) an opportunity to receive the Corresponding +Source of your version by providing access to the Corresponding Source +from a network server at no charge, through some standard or customary +means of facilitating copying of software. This Corresponding Source +shall include the Corresponding Source for any work covered by version 3 +of the GNU General Public License that is incorporated pursuant to the +following paragraph. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the work with which it is combined will remain governed by version +3 of the GNU General Public License. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU Affero General Public License from time to time. Such new versions +will be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU Affero General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU Affero General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU Affero General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If your software can interact with users remotely through a computer +network, you should also make sure that it provides a way for users to +get its source. For example, if your program is a web application, its +interface could display a "Source" link that leads users to an archive +of the code. There are many ways you could offer source, and different +solutions will be better for different programs; see section 13 for the +specific requirements. + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU AGPL, see +. diff --git a/Setup.hs b/Setup.hs new file mode 100644 --- /dev/null +++ b/Setup.hs @@ -0,0 +1,2 @@ +import Distribution.Simple +main = defaultMain diff --git a/ShellCheck.cabal b/ShellCheck.cabal new file mode 100644 --- /dev/null +++ b/ShellCheck.cabal @@ -0,0 +1,34 @@ +Name: ShellCheck +Version: 0.3.0 +Synopsis: Shell script analysis tool +License: OtherLicense +License-file: LICENSE +Category: Static Analysis +Author: Vidar Holen +Maintainer: vidar@vidarholen.net +Homepage: http://www.shellcheck.net/ +Build-Type: Simple +Cabal-Version: >= 1.6 +Bug-reports: https://github.com/koalaman/shellcheck/issues +Description: + The goals of ShellCheck are: + . + * To point out and clarify typical beginner's syntax issues, + that causes a shell to give cryptic error messages. + . + * To point out and clarify typical intermediate level semantic problems, + that causes a shell to behave strangely and counter-intuitively. + . + * To point out subtle caveats, corner cases and pitfalls, that may cause an + advanced user's otherwise working script to fail under future circumstances. + +source-repository head + type: git + location: git://github.com/koalaman/shellcheck.git + +library + build-depends: base >= 4, base < 5, parsec, containers, regex-compat, mtl, directory, json + exposed-modules: ShellCheck.AST, ShellCheck.Data, ShellCheck.Parser, ShellCheck.Analytics, ShellCheck.Simple + +executable shellcheck + main-is: shellcheck.hs diff --git a/ShellCheck/AST.hs b/ShellCheck/AST.hs new file mode 100644 --- /dev/null +++ b/ShellCheck/AST.hs @@ -0,0 +1,344 @@ +{- + This file is part of ShellCheck. + http://www.vidarholen.net/contents/shellcheck + + ShellCheck is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + ShellCheck is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see . +-} +module ShellCheck.AST where + +import Control.Monad +import Control.Monad.Identity +import qualified Text.Regex as Re + +data Id = Id Int deriving (Show, Eq, Ord) + +data Quoted = Quoted | Unquoted deriving (Show, Eq) +data Dashed = Dashed | Undashed deriving (Show, Eq) +data AssignmentMode = Assign | Append deriving (Show, Eq) + +data Token = + TA_Base Id String Token + | TA_Binary Id String Token Token + | TA_Expansion Id Token + | TA_Literal Id String + | TA_Sequence Id [Token] + | TA_Trinary Id Token Token Token + | TA_Unary Id String Token + | TA_Variable Id String + | TC_And Id ConditionType String Token Token + | TC_Binary Id ConditionType String Token Token + | TC_Group Id ConditionType Token + | TC_Noary Id ConditionType Token + | TC_Or Id ConditionType String Token Token + | TC_Unary Id ConditionType String Token + | T_AND_IF Id + | T_AndIf Id (Token) (Token) + | T_Arithmetic Id Token + | T_Array Id [Token] + | T_Assignment Id AssignmentMode String (Maybe Token) Token + | T_Backgrounded Id Token + | T_Backticked Id [Token] + | T_Bang Id + | T_Banged Id Token + | T_BraceExpansion Id String + | T_BraceGroup Id [Token] + | T_CLOBBER Id + | T_Case Id + | T_CaseExpression Id Token [([Token],[Token])] + | T_Condition Id ConditionType Token + | T_DGREAT Id + | T_DLESS Id + | T_DLESSDASH Id + | T_DSEMI Id + | T_Do Id + | T_DollarArithmetic Id Token + | T_DollarBraced Id Token + | T_DollarBracket Id Token + | T_DollarDoubleQuoted Id [Token] + | T_DollarExpansion Id [Token] + | T_DollarSingleQuoted Id String + | T_Done Id + | T_DoubleQuoted Id [Token] + | T_EOF Id + | T_Elif Id + | T_Else Id + | T_Esac Id + | T_Extglob Id String [Token] + | T_FdRedirect Id String Token + | T_Fi Id + | T_For Id + | T_ForArithmetic Id Token Token Token [Token] + | T_ForIn Id String [Token] [Token] + | T_Function Id String Token + | T_GREATAND Id + | T_Glob Id String + | T_Greater Id + | T_HereDoc Id Dashed Quoted String [Token] + | T_HereString Id Token + | T_If Id + | T_IfExpression Id [([Token],[Token])] [Token] + | T_In Id + | T_IoFile Id Token Token + | T_LESSAND Id + | T_LESSGREAT Id + | T_Lbrace Id + | T_Less Id + | T_Literal Id String + | T_Lparen Id + | T_NEWLINE Id + | T_NormalWord Id [Token] + | T_OR_IF Id + | T_OrIf Id (Token) (Token) + | T_Pipeline Id [Token] + | T_ProcSub Id String [Token] + | T_Rbrace Id + | T_Redirecting Id [Token] Token + | T_Rparen Id + | T_Script Id String [Token] + | T_Select Id + | T_SelectIn Id String [Token] [Token] + | T_Semi Id + | T_SimpleCommand Id [Token] [Token] + | T_SingleQuoted Id String + | T_Subshell Id [Token] + | T_Then Id + | T_Until Id + | T_UntilExpression Id [Token] [Token] + | T_While Id + | T_WhileExpression Id [Token] [Token] + | T_Annotation Id [Annotation] Token + deriving (Show) + +data Annotation = DisableComment Integer deriving (Show, Eq) +data ConditionType = DoubleBracket | SingleBracket deriving (Show, Eq) + +-- I apologize for nothing! +lolHax s = Re.subRegex (Re.mkRegex "(Id [0-9]+)") (show s) "(Id 0)" +instance Eq Token where + (==) a b = (lolHax a) == (lolHax b) + + +analyze :: Monad m => (Token -> m ()) -> (Token -> m ()) -> (Token -> Token) -> Token -> m Token +analyze f g i t = + round t + where + round t = do + f t + newT <- delve t + g t + return . i $ newT + roundAll = mapM round + + roundMaybe Nothing = return Nothing + roundMaybe (Just v) = do + s <- round v + return (Just s) + + dl l v = do + x <- roundAll l + return $ v x + dll l m v = do + x <- roundAll l + y <- roundAll m + return $ v x m + d1 t v = do + x <- round t + return $ v x + d2 t1 t2 v = do + x <- round t1 + y <- round t2 + return $ v x y + + delve (T_NormalWord id list) = dl list $ T_NormalWord id + delve (T_DoubleQuoted id list) = dl list $ T_DoubleQuoted id + delve (T_DollarDoubleQuoted id list) = dl list $ T_DollarDoubleQuoted id + delve (T_DollarExpansion id list) = dl list $ T_DollarExpansion id + delve (T_Backticked id list) = dl list $ T_Backticked id + delve (T_DollarArithmetic id c) = d1 c $ T_DollarArithmetic id + delve (T_DollarBracket id c) = d1 c $ T_DollarBracket id + delve (T_IoFile id op file) = d2 op file $ T_IoFile id + delve (T_HereString id word) = d1 word $ T_HereString id + delve (T_FdRedirect id v t) = d1 t $ T_FdRedirect id v + delve (T_Assignment id mode var index value) = do + a <- roundMaybe index + b <- round value + return $ T_Assignment id mode var a b + delve (T_Array id t) = dl t $ T_Array id + delve (T_Redirecting id redirs cmd) = do + a <- roundAll redirs + b <- round cmd + return $ T_Redirecting id a b + delve (T_SimpleCommand id vars cmds) = dll vars cmds $ T_SimpleCommand id + delve (T_Pipeline id l) = dl l $ T_Pipeline id + delve (T_Banged id l) = d1 l $ T_Banged id + delve (T_AndIf id t u) = d2 t u $ T_AndIf id + delve (T_OrIf id t u) = d2 t u $ T_OrIf id + delve (T_Backgrounded id l) = d1 l $ T_Backgrounded id + delve (T_Subshell id l) = dl l $ T_Subshell id + delve (T_ProcSub id typ l) = dl l $ T_ProcSub id typ + delve (T_Arithmetic id c) = d1 c $ T_Arithmetic id + delve (T_IfExpression id conditions elses) = do + newConds <- mapM (\(c, t) -> do + x <- mapM round c + y <- mapM round t + return (x,y) + ) conditions + newElses <- roundAll elses + return $ T_IfExpression id newConds newElses + delve (T_BraceGroup id l) = dl l $ T_BraceGroup id + delve (T_WhileExpression id c l) = dll c l $ T_WhileExpression id + delve (T_UntilExpression id c l) = dll c l $ T_UntilExpression id + delve (T_ForIn id v w l) = dll w l $ T_ForIn id v + delve (T_SelectIn id v w l) = dll w l $ T_SelectIn id v + delve (T_CaseExpression id word cases) = do + newWord <- round word + newCases <- mapM (\(c, t) -> do + x <- mapM round c + y <- mapM round t + return (x,y) + ) cases + return $ T_CaseExpression id newWord newCases + + delve (T_ForArithmetic id a b c group) = do + x <- round a + y <- round b + z <- round c + list <- mapM round group + return $ T_ForArithmetic id x y z list + + delve (T_Script id s l) = dl l $ T_Script id s + delve (T_Function id name body) = d1 body $ T_Function id name + delve (T_Condition id typ token) = d1 token $ T_Condition id typ + delve (T_Extglob id str l) = dl l $ T_Extglob id str + delve (T_DollarBraced id op) = d1 op $ T_DollarBraced id + delve (T_HereDoc id d q str l) = dl l $ T_HereDoc id d q str + + delve (TC_And id typ str t1 t2) = d2 t1 t2 $ TC_And id typ str + delve (TC_Or id typ str t1 t2) = d2 t1 t2 $ TC_Or id typ str + delve (TC_Group id typ token) = d1 token $ TC_Group id typ + delve (TC_Binary id typ op lhs rhs) = d2 lhs rhs $ TC_Binary id typ op + delve (TC_Unary id typ op token) = d1 token $ TC_Unary id typ op + delve (TC_Noary id typ token) = d1 token $ TC_Noary id typ + + delve (TA_Binary id op t1 t2) = d2 t1 t2 $ TA_Binary id op + delve (TA_Unary id op t1) = d1 t1 $ TA_Unary id op + delve (TA_Sequence id l) = dl l $ TA_Sequence id + delve (TA_Trinary id t1 t2 t3) = do + a <- round t1 + b <- round t2 + c <- round t3 + return $ TA_Trinary id a b c + delve (TA_Expansion id t) = d1 t $ TA_Expansion id + delve (TA_Base id b t) = d1 t $ TA_Base id b + delve (T_Annotation id anns t) = d1 t $ T_Annotation id anns + delve t = return t + +getId t = case t of + T_AND_IF id -> id + T_OR_IF id -> id + T_DSEMI id -> id + T_Semi id -> id + T_DLESS id -> id + T_DGREAT id -> id + T_LESSAND id -> id + T_GREATAND id -> id + T_LESSGREAT id -> id + T_DLESSDASH id -> id + T_CLOBBER id -> id + T_If id -> id + T_Then id -> id + T_Else id -> id + T_Elif id -> id + T_Fi id -> id + T_Do id -> id + T_Done id -> id + T_Case id -> id + T_Esac id -> id + T_While id -> id + T_Until id -> id + T_For id -> id + T_Select id -> id + T_Lbrace id -> id + T_Rbrace id -> id + T_Lparen id -> id + T_Rparen id -> id + T_Bang id -> id + T_In id -> id + T_NEWLINE id -> id + T_EOF id -> id + T_Less id -> id + T_Greater id -> id + T_SingleQuoted id _ -> id + T_Literal id _ -> id + T_NormalWord id _ -> id + T_DoubleQuoted id _ -> id + T_DollarExpansion id _ -> id + T_DollarBraced id _ -> id + T_DollarArithmetic id _ -> id + T_BraceExpansion id _ -> id + T_IoFile id _ _ -> id + T_HereDoc id _ _ _ _ -> id + T_HereString id _ -> id + T_FdRedirect id _ _ -> id + T_Assignment id _ _ _ _ -> id + T_Array id _ -> id + T_Redirecting id _ _ -> id + T_SimpleCommand id _ _ -> id + T_Pipeline id _ -> id + T_Banged id _ -> id + T_AndIf id _ _ -> id + T_OrIf id _ _ -> id + T_Backgrounded id _ -> id + T_IfExpression id _ _ -> id + T_Subshell id _ -> id + T_BraceGroup id _ -> id + T_WhileExpression id _ _ -> id + T_UntilExpression id _ _ -> id + T_ForIn id _ _ _ -> id + T_SelectIn id _ _ _ -> id + T_CaseExpression id _ _ -> id + T_Function id _ _ -> id + T_Arithmetic id _ -> id + T_Script id _ _ -> id + T_Condition id _ _ -> id + T_Extglob id _ _ -> id + T_Backticked id _ -> id + TC_And id _ _ _ _ -> id + TC_Or id _ _ _ _ -> id + TC_Group id _ _ -> id + TC_Binary id _ _ _ _ -> id + TC_Unary id _ _ _ -> id + TC_Noary id _ _ -> id + TA_Binary id _ _ _ -> id + TA_Unary id _ _ -> id + TA_Sequence id _ -> id + TA_Variable id _ -> id + TA_Trinary id _ _ _ -> id + TA_Expansion id _ -> id + TA_Literal id _ -> id + TA_Base id _ _ -> id + T_ProcSub id _ _ -> id + T_Glob id _ -> id + T_ForArithmetic id _ _ _ _ -> id + T_DollarSingleQuoted id _ -> id + T_DollarDoubleQuoted id _ -> id + T_DollarBracket id _ -> id + T_Annotation id _ _ -> id + +blank :: Monad m => Token -> m () +blank = const $ return () +doAnalysis f t = analyze f blank id t +doStackAnalysis startToken endToken t = analyze startToken endToken id t +doTransform i t = runIdentity $ analyze blank blank i t + diff --git a/ShellCheck/Analytics.hs b/ShellCheck/Analytics.hs new file mode 100644 --- /dev/null +++ b/ShellCheck/Analytics.hs @@ -0,0 +1,1985 @@ +{- + This file is part of ShellCheck. + http://www.vidarholen.net/contents/shellcheck + + ShellCheck is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + ShellCheck is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see . +-} +module ShellCheck.Analytics where + +import ShellCheck.AST +import ShellCheck.Data +import ShellCheck.Parser +import Control.Monad +import Control.Monad.State +import qualified Data.Map as Map +import Data.Char +import Data.Functor +import Data.List +import Data.Maybe +import Debug.Trace +import Text.Regex +import Data.Maybe + +data Shell = Ksh | Zsh | Sh | Bash + deriving (Show, Eq) + +genericChecks = [ + runBasicAnalysis (\x -> mapM_ (flip ($) x) basicChecks) + ,runBasicTreeAnalysis treeChecks + ,subshellAssignmentCheck + ,checkSpacefulness + ,checkQuotesInLiterals + ,checkShebang + ,checkFunctionsUsedExternally + ,checkUnusedAssignments + ,checkWrongArithmeticAssignment + ] + +checksFor Sh = map runBasicAnalysis [ + checkBashisms + ,checkTimeParameters + ,checkCdAndBack Sh + ] +checksFor Ksh = map runBasicAnalysis [ + checkEchoSed + ,checkCdAndBack Ksh + ] +checksFor Zsh = map runBasicAnalysis [ + checkTimeParameters + ,checkEchoSed + ,checkCdAndBack Zsh + ] +checksFor Bash = map runBasicAnalysis [ + checkTimeParameters + ,checkBraceExpansionVars + ,checkEchoSed + ,checkCdAndBack Bash + ] + +runAllAnalytics root m = addToMap notes m + where shell = determineShell root + notes = checkList ((checksFor shell) ++ genericChecks) root + +checkList l t = concatMap (\f -> f t) l +addToMap list map = foldr (\(id,note) m -> Map.adjust (\(Metadata pos notes) -> Metadata pos (note:notes)) id m) map list + +prop_determineShell0 = determineShell (T_Script (Id 0) "#!/bin/sh" []) == Sh +prop_determineShell1 = determineShell (T_Script (Id 0) "#!/usr/bin/env ksh" []) == Ksh +prop_determineShell2 = determineShell (T_Script (Id 0) "" []) == Bash +determineShell (T_Script _ shebang _) = normalize $ shellFor shebang + where shellFor s | "/env " `isInfixOf` s = head ((drop 1 $ words s)++[""]) + shellFor s = reverse . takeWhile (/= '/') . reverse $ s + normalize "sh" = Sh + normalize "ksh" = Ksh + normalize "zsh" = Zsh + normalize "bash" = Bash + normalize _ = Bash + +runBasicAnalysis f t = snd $ runState (doAnalysis f t) [] +basicChecks = [ + checkUuoc + ,checkPipePitfalls + ,checkForInQuoted + ,checkForInLs + ,checkShorthandIf + ,checkDollarStar + ,checkUnquotedDollarAt + ,checkStderrRedirect + ,checkUnquotedN + ,checkNumberComparisons + ,checkSingleBracketOperators + ,checkDoubleBracketOperators + ,checkNoaryWasBinary + ,checkConstantNoary + ,checkForDecimals + ,checkDivBeforeMult + ,checkArithmeticDeref + ,checkArithmeticBadOctal + ,checkComparisonAgainstGlob + ,checkPrintfVar + ,checkCommarrays + ,checkOrNeq + ,checkEchoWc + ,checkConstantIfs + ,checkTr + ,checkPipedAssignment + ,checkAssignAteCommand + ,checkUuoe + ,checkFindNameGlob + ,checkGrepRe + ,checkNeedlessCommands + ,checkQuotedCondRegex + ,checkForInCat + ,checkFindExec + ,checkValidCondOps + ,checkGlobbedRegex + ,checkTrapQuotes + ,checkTestRedirects + ,checkIndirectExpansion + ,checkSudoRedirect + ,checkPS1Assignments + ,checkBackticks + ,checkInexplicablyUnquoted + ,checkTildeInQuotes + ,checkLonelyDotDash + ,checkSpuriousExec + ,checkSpuriousExpansion + ,checkUnusedEchoEscapes + ,checkDollarBrackets + ,checkSshHereDoc + ,checkSshCommandString + ,checkGlobsAsOptions + ,checkWhileReadPitfalls + ,checkArithmeticOpCommand + ,checkCharRangeGlob + ] +treeChecks = [ + checkUnquotedExpansions + ,checkSingleQuotedVariables + ,checkRedirectToSame + ,checkPrefixAssignmentReference + ] + + +runBasicTreeAnalysis checks token = + checkList (map runTree checks) token + where + parentTree = getParentTree token + runTree f t = runBasicAnalysis (flip f $ parentTree) t + +filterByAnnotation token metadataMap = + Map.mapWithKey removeVals metadataMap + where + removeVals id (Metadata pos notes) = + Metadata pos $ filter (not . shouldIgnore id . numFor) notes + numFor (Note _ code _) = code + shouldIgnore id num = + any (shouldIgnoreFor num) $ getPath parents (T_Bang id) + shouldIgnoreFor num (T_Annotation _ anns _) = + any hasNum anns + where + hasNum (DisableComment ts) = num == ts + shouldIgnoreFor _ _ = False + parents = getParentTree token + +addNoteFor id note = modify ((id, note):) +warn id code note = addNoteFor id $ Note WarningC code $ note +err id code note = addNoteFor id $ Note ErrorC code $ note +info id code note = addNoteFor id $ Note InfoC code $ note +style id code note = addNoteFor id $ Note StyleC code $ note + +isVariableStartChar x = x == '_' || x >= 'a' && x <= 'z' || x >= 'A' && x <= 'Z' +isVariableChar x = isVariableStartChar x || x >= '0' && x <= '9' + +prop_isVariableName1 = isVariableName "_fo123" +prop_isVariableName2 = not $ isVariableName "4" +prop_isVariableName3 = not $ isVariableName "test: " +isVariableName (x:r) = isVariableStartChar x && all isVariableChar r +isVariableName _ = False + +willSplit x = + case x of + T_DollarBraced _ _ -> True + T_DollarExpansion _ _ -> True + T_Backticked _ _ -> True + T_BraceExpansion _ s -> True + T_Glob _ _ -> True + T_Extglob _ _ _ -> True + T_NormalWord _ l -> any willSplit l + _ -> False + +isGlob (T_Extglob _ _ _) = True +isGlob (T_Glob _ _) = True +isGlob (T_NormalWord _ l) = any isGlob l +isGlob _ = False + +wouldHaveBeenGlob s = '*' `elem` s + +isConfusedGlobRegex ('*':_) = True +isConfusedGlobRegex [x,'*'] | x /= '\\' = True +isConfusedGlobRegex _ = False + +isPotentiallyConfusedGlobRegex = + let re = mkRegex "[a-z1-9]\\*" in + isJust . matchRegex re + +matches string regex = isJust $ matchRegex regex string + +headOrDefault _ (a:_) = a +headOrDefault def _ = def + +isConstant token = + case token of + T_NormalWord _ l -> all isConstant l + T_DoubleQuoted _ l -> all isConstant l + T_SingleQuoted _ _ -> True + T_Literal _ _ -> True + _ -> False + +isEmpty token = + case token of + T_NormalWord _ l -> all isEmpty l + T_DoubleQuoted _ l -> all isEmpty l + T_SingleQuoted _ "" -> True + T_Literal _ "" -> True + _ -> False + +makeSimple (T_NormalWord _ [f]) = f +makeSimple (T_Redirecting _ _ f) = f +makeSimple (T_Annotation _ _ f) = f +makeSimple t = t +simplify = doTransform makeSimple + +deadSimple (T_NormalWord _ l) = [concat (concatMap (deadSimple) l)] +deadSimple (T_DoubleQuoted _ l) = [(concat (concatMap (deadSimple) l))] +deadSimple (T_SingleQuoted _ s) = [s] +deadSimple (T_DollarBraced _ _) = ["${VAR}"] +deadSimple (T_DollarArithmetic _ _) = ["${VAR}"] +deadSimple (T_DollarExpansion _ _) = ["${VAR}"] +deadSimple (T_Backticked _ _) = ["${VAR}"] +deadSimple (T_Glob _ s) = [s] +deadSimple (T_Pipeline _ [x]) = deadSimple x +deadSimple (T_Literal _ x) = [x] +deadSimple (T_SimpleCommand _ vars words) = concatMap (deadSimple) words +deadSimple (T_Redirecting _ _ foo) = deadSimple foo +deadSimple (T_DollarSingleQuoted _ s) = [s] +deadSimple (T_Annotation _ _ s) = deadSimple s +deadSimple _ = [] + +(!!!) list i = + case drop i list of + [] -> Nothing + (r:_) -> Just r + +verify f s = checkBasic f s == Just True +verifyNot f s = checkBasic f s == Just False +verifyFull f s = checkFull f s == Just True +verifyNotFull f s = checkFull f s == Just False +verifyTree f s = checkTree f s == Just True +verifyNotTree f s = checkTree f s == Just False + +checkBasic f s = checkFull (runBasicAnalysis f) s +checkTree f s = checkFull (runBasicTreeAnalysis [f]) s +checkFull f s = case parseShell "-" s of + (ParseResult (Just (t, m)) _) -> Just . not . null $ f t + _ -> Nothing + + +prop_checkEchoWc3 = verify checkEchoWc "n=$(echo $foo | wc -c)" +checkEchoWc (T_Pipeline id [a, b]) = + when (acmd == ["echo", "${VAR}"]) $ + case bcmd of + ["wc", "-c"] -> countMsg + ["wc", "-m"] -> countMsg + _ -> return () + where + acmd = deadSimple a + bcmd = deadSimple b + countMsg = style id 2000 $ "See if you can use ${#variable} instead." +checkEchoWc _ = return () + +prop_checkEchoSed1 = verify checkEchoSed "FOO=$(echo \"$cow\" | sed 's/foo/bar/g')" +prop_checkEchoSed2 = verify checkEchoSed "rm $(echo $cow | sed -e 's,foo,bar,')" +checkEchoSed (T_Pipeline id [a, b]) = + when (acmd == ["echo", "${VAR}"]) $ + case bcmd of + ["sed", v] -> checkIn v + ["sed", "-e", v] -> checkIn v + _ -> return () + where + sedRe = mkRegex "^s(.)(.*)\\1(.*)\\1g?$" + acmd = deadSimple a + bcmd = deadSimple b + checkIn s = + case matchRegex sedRe s of + Just _ -> style id 2001 $ "See if you can use ${variable//search/replace} instead." + _ -> return () +checkEchoSed _ = return () + +prop_checkPipedAssignment1 = verify checkPipedAssignment "A=ls | grep foo" +prop_checkPipedAssignment2 = verifyNot checkPipedAssignment "A=foo cmd | grep foo" +prop_checkPipedAssignment3 = verifyNot checkPipedAssignment "A=foo" +checkPipedAssignment (T_Pipeline _ (T_Redirecting _ _ (T_SimpleCommand id (_:_) []):_:_)) = + warn id 2036 "If you wanted to assign the output of the pipeline, use a=$(b | c) ." +checkPipedAssignment _ = return () + +prop_checkAssignAteCommand1 = verify checkAssignAteCommand "A=ls -l" +prop_checkAssignAteCommand2 = verify checkAssignAteCommand "A=ls --sort=$foo" +prop_checkAssignAteCommand3 = verify checkAssignAteCommand "A=cat foo | grep bar" +prop_checkAssignAteCommand4 = verifyNot checkAssignAteCommand "A=foo ls -l" +prop_checkAssignAteCommand5 = verifyNot checkAssignAteCommand "PAGER=cat grep bar" +checkAssignAteCommand (T_SimpleCommand id ((T_Assignment _ _ _ _ assignmentTerm):[]) (firstWord:_)) = + when ("-" `isPrefixOf` (concat $ deadSimple firstWord) || + (isCommonCommand (getLiteralString assignmentTerm) && not (isCommonCommand (getLiteralString firstWord)))) $ + warn id 2037 "To assign the output of a command, use var=$(cmd) ." + where + isCommonCommand (Just s) = s `elem` commonCommands + isCommonCommand _ = False +checkAssignAteCommand _ = return () + +prop_checkArithmeticOpCommand1 = verify checkArithmeticOpCommand "i=i + 1" +prop_checkArithmeticOpCommand2 = verify checkArithmeticOpCommand "foo=bar * 2" +prop_checkArithmeticOpCommand3 = verifyNot checkArithmeticOpCommand "foo + opts" +checkArithmeticOpCommand (T_SimpleCommand id ((T_Assignment _ _ _ _ _):[]) (firstWord:_)) = + fromMaybe (return ()) $ check <$> getGlobOrLiteralString firstWord + where + check op = + when (op `elem` ["+", "-", "*", "/"]) $ + warn (getId firstWord) 2099 $ + "Use $((..)) for arithmetics, e.g. i=$((i " ++ op ++ " 2))" +checkArithmeticOpCommand _ = return () + +prop_checkWrongArit = verifyFull checkWrongArithmeticAssignment "i=i+1" +prop_checkWrongArit2 = verifyFull checkWrongArithmeticAssignment "n=2; i=n*2" +checkWrongArithmeticAssignment t = runBasicAnalysis f t + where + regex = mkRegex "^([_a-zA-Z][_a-zA-Z0-9]*)([+*-]).+$" + flow = getVariableFlow t + references = foldl (flip ($)) Map.empty (map insertRef flow) + insertRef (Assignment (_, _, name, _)) = + Map.insert name () + insertRef _ = id + + getNormalString (T_NormalWord _ words) = do + parts <- foldl (liftM2 (\x y -> x ++ [y])) (Just []) $ map getLiterals words + return $ concat parts + getNormalString _ = Nothing + + getLiterals (T_Literal _ s) = return s + getLiterals (T_Glob _ s) = return s + getLiterals _ = Nothing + + f (T_SimpleCommand id ((T_Assignment _ _ _ _ val):[]) []) = + fromMaybe (return ()) $ do + str <- getNormalString val + match <- matchRegex regex str + var <- match !!! 0 + op <- match !!! 1 + Map.lookup var references + return $ do + warn (getId val) 2100 $ + "Use $((..)) for arithmetics, e.g. i=$((i " ++ op ++ " 2))" + f _ = return () + +prop_checkUuoc1 = verify checkUuoc "cat foo | grep bar" +prop_checkUuoc2 = verifyNot checkUuoc "cat * | grep bar" +prop_checkUuoc3 = verify checkUuoc "cat $var | grep bar" +prop_checkUuoc4 = verifyNot checkUuoc "cat $var" +checkUuoc (T_Pipeline _ ((T_Redirecting _ _ cmd):_:_)) = checkCommand "cat" f cmd + where + f [word] = when (isSimple word) $ + style (getId word) 2002 "Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead." + f _ = return () + isSimple (T_NormalWord _ parts) = all isSimple parts + isSimple (T_DollarBraced _ _) = True + isSimple x = not $ willSplit x +checkUuoc _ = return () + +prop_checkNeedlessCommands = verify checkNeedlessCommands "foo=$(expr 3 + 2)" +prop_checkNeedlessCommands2 = verify checkNeedlessCommands "foo=`echo \\`expr 3 + 2\\``" +prop_checkNeedlessCommands3 = verifyNot checkNeedlessCommands "foo=$(expr foo : regex)" +checkNeedlessCommands cmd@(T_SimpleCommand id _ _) | + cmd `isCommand` "expr" && (not $ ":" `elem` deadSimple cmd) = + style id 2003 "expr is antiquated. Consider rewriting this using $((..)), ${} or [[ ]]." +checkNeedlessCommands _ = return () + +prop_checkPipePitfalls3 = verify checkPipePitfalls "ls | grep -v mp3" +prop_checkPipePitfalls4 = verifyNot checkPipePitfalls "find . -print0 | xargs -0 foo" +prop_checkPipePitfalls5 = verifyNot checkPipePitfalls "ls -N | foo" +prop_checkPipePitfalls6 = verify checkPipePitfalls "find . | xargs foo" +checkPipePitfalls (T_Pipeline id commands) = do + for ["find", "xargs"] $ + \(find:xargs:_) -> let args = deadSimple xargs in + when (not $ hasShortParameter args '0') $ + warn (getId find) 2038 "Use either 'find .. -print0 | xargs -0 ..' or 'find .. -exec .. +' to allow for non-alphanumeric filenames." + + for ["?", "echo"] $ + \(_:echo:_) -> info (getId echo) 2008 "echo doesn't read from stdin, are you sure you should be piping to it?" + + for' ["ps", "grep"] $ + \x -> info x 2009 "Consider using pgrep instead of grepping ps output." + + didLs <- liftM or . sequence $ [ + for' ["ls", "grep"] $ + \x -> warn x 2010 "Don't use ls | grep. Use a glob or a for loop with a condition to allow non-alphanumeric filenames.", + for' ["ls", "xargs"] $ + \x -> warn x 2011 "Use 'find .. -print0 | xargs -0 ..' or 'find .. -exec .. +' to allow non-alphanumeric filenames." + ] + when (not didLs) $ do + for ["ls", "?"] $ + \(ls:_) -> (when (not $ hasShortParameter (deadSimple ls) 'N') $ + info (getId ls) 2012 "Use find instead of ls to better handle non-alphanumeric filenames.") + return () + where + for l f = + let indices = indexOfSublists l (map (headOrDefault "" . deadSimple) commands) + in do + mapM_ f (map (\n -> take (length l) $ drop n $ commands) indices) + return . not . null $ indices + for' l f = for l (first f) + first func (x:_) = func (getId x) + first _ _ = return () + hasShortParameter list char = any (\x -> "-" `isPrefixOf` x && char `elem` x) list +checkPipePitfalls _ = return () + +indexOfSublists sub all = f 0 all + where + f _ [] = [] + f n a@(r:rest) = + let others = f (n+1) rest in + if match sub a + then n:others + else others + match ("?":r1) (_:r2) = match r1 r2 + match (x1:r1) (x2:r2) | x1 == x2 = match r1 r2 + match [] _ = True + match _ _ = False + + +bracedString l = concat $ deadSimple l +isMagicInQuotes (T_DollarBraced _ l) | '@' `elem` (bracedString l) = True +isMagicInQuotes _ = False + +prop_checkShebang1 = verifyFull checkShebang "#!/usr/bin/env bash -x\necho cow" +prop_checkShebang2 = verifyNotFull checkShebang "#! /bin/sh -l " +checkShebang (T_Script id sb _) = + if (length $ words sb) > 2 then + let note = Note ErrorC 2096 $ "On most OS, shebangs can only specify a single parameter." + in [(id, note)] + else [] + +prop_checkBashisms = verify checkBashisms "while read a; do :; done < <(a)" +prop_checkBashisms2 = verify checkBashisms "[ foo -nt bar ]" +prop_checkBashisms3 = verify checkBashisms "echo $((i++))" +prop_checkBashisms4 = verify checkBashisms "rm !(*.hs)" +prop_checkBashisms5 = verify checkBashisms "source file" +prop_checkBashisms6 = verify checkBashisms "[ \"$a\" == 42 ]" +prop_checkBashisms7 = verify checkBashisms "echo ${var[1]}" +prop_checkBashisms8 = verify checkBashisms "echo ${!var[@]}" +prop_checkBashisms9 = verify checkBashisms "echo ${!var*}" +prop_checkBashisms10= verify checkBashisms "echo ${var:4:12}" +prop_checkBashisms11= verifyNot checkBashisms "echo ${var:-4}" +prop_checkBashisms12= verify checkBashisms "echo ${var//foo/bar}" +prop_checkBashisms13= verify checkBashisms "exec -c env" +prop_checkBashisms14= verify checkBashisms "echo -n \"Foo: \"" +prop_checkBashisms15= verify checkBashisms "let n++" +prop_checkBashisms16= verify checkBashisms "echo $RANDOM" +prop_checkBashisms17= verify checkBashisms "echo $((RANDOM%6+1))" +prop_checkBashisms18= verify checkBashisms "foo &> /dev/null" +checkBashisms = bashism + where + errMsg id s = err id 2040 $ "#!/bin/sh was specified, so " ++ s ++ " is not supported, even when sh is actually bash." + warnMsg id s = warn id 2039 $ "#!/bin/sh was specified, but " ++ s ++ " is not standard." + bashism (T_ProcSub id _ _) = errMsg id "process substitution" + bashism (T_Extglob id _ _) = warnMsg id "extglob" + bashism (T_DollarSingleQuoted id _) = warnMsg id "$'..'" + bashism (T_DollarDoubleQuoted id _) = warnMsg id "$\"..\"" + bashism (T_ForArithmetic id _ _ _ _) = warnMsg id "arithmetic for loop" + bashism (T_Arithmetic id _) = warnMsg id "standalone ((..))" + bashism (T_DollarBracket id _) = warnMsg id "$[..] in place of $((..))" + bashism (T_SelectIn id _ _ _) = warnMsg id "select loop" + bashism (T_BraceExpansion id _) = warnMsg id "brace expansion" + bashism (T_Condition id DoubleBracket _) = warnMsg id "[[ ]]" + bashism (T_HereString id _) = warnMsg id "here-string" + bashism (TC_Binary id SingleBracket op _ _) + | op `elem` [ "-nt", "-ef", "\\<", "\\>", "==" ] = + warnMsg id op + bashism (TA_Unary id op _) + | op `elem` [ "|++", "|--", "++|", "--|"] = + warnMsg id (filter (/= '|') op) + bashism t@(T_SimpleCommand id _ _) + | t `isCommand` "source" = + warnMsg id "'source' in place of '.'" + bashism (T_FdRedirect id "&" (T_IoFile _ (T_Greater _) _)) = warnMsg id "&>" + bashism (T_DollarBraced id token) = + mapM_ check expansion + where + str = concat $ deadSimple token + check (regex, feature) = + when (isJust $ matchRegex regex str) $ warnMsg id feature + + bashism t@(T_SimpleCommand _ _ (cmd:arg:_)) + | t `isCommand` "echo" && "-" `isPrefixOf` argString = + when (not $ "--" `isPrefixOf` argString) $ -- echo "-------" + warnMsg (getId arg) "echo flag" + where argString = (concat $ deadSimple arg) + bashism t@(T_SimpleCommand _ _ (cmd:arg:_)) + | t `isCommand` "exec" && "-" `isPrefixOf` (concat $ deadSimple arg) = + warnMsg (getId arg) "exec flag" + bashism t@(T_SimpleCommand id _ _) + | t `isCommand` "let" = warnMsg id "'let'" + bashism t@(TA_Variable id "RANDOM") = + warnMsg id "RANDOM" + + bashism _ = return() + + varChars="_0-9a-zA-Z" + expansion = let re = mkRegex in [ + (re $ "^[" ++ varChars ++ "]+\\[.*\\]$", "array references"), + (re $ "^![" ++ varChars ++ "]+\\[[*@]]$", "array key expansion"), + (re $ "^![" ++ varChars ++ "]+[*@]$", "name matching prefix"), + (re $ "^[" ++ varChars ++ "]+:[^-=?+]", "string indexing"), + (re $ "^[" ++ varChars ++ "]+(\\[.*\\])?/", "string replacement"), + (re $ "^RANDOM$", "$RANDOM") + ] + +prop_checkForInQuoted = verify checkForInQuoted "for f in \"$(ls)\"; do echo foo; done" +prop_checkForInQuoted2 = verifyNot checkForInQuoted "for f in \"$@\"; do echo foo; done" +prop_checkForInQuoted2a = verifyNot checkForInQuoted "for f in *.mp3; do echo foo; done" +prop_checkForInQuoted2b = verify checkForInQuoted "for f in \"*.mp3\"; do echo foo; done" +prop_checkForInQuoted3 = verify checkForInQuoted "for f in 'find /'; do true; done" +prop_checkForInQuoted4 = verify checkForInQuoted "for f in 1,2,3; do true; done" +prop_checkForInQuoted5 = verify checkForInQuoted "for f in ls; do true; done" +checkForInQuoted (T_ForIn _ f [T_NormalWord _ [word@(T_DoubleQuoted id list)]] _) = + when (any (\x -> willSplit x && not (isMagicInQuotes x)) list + || (getLiteralString word >>= (return . wouldHaveBeenGlob)) == Just True) $ + err id 2066 $ "Since you double quoted this, it will not word split, and the loop will only run once." +checkForInQuoted (T_ForIn _ f [T_NormalWord _ [T_SingleQuoted id s]] _) = + warn id 2041 $ "This is a literal string. To run as a command, use $(" ++ s ++ ")." +checkForInQuoted (T_ForIn _ f [T_NormalWord _ [T_Literal id s]] _) = + if ',' `elem` s + then warn id 2042 $ "Use spaces, not commas, to separate loop elements." + else warn id 2043 $ "This loop will only run once, with " ++ f ++ "='" ++ s ++ "'." +checkForInQuoted _ = return () + +prop_checkForInCat1 = verify checkForInCat "for f in $(cat foo); do stuff; done" +prop_checkForInCat1a= verify checkForInCat "for f in `cat foo`; do stuff; done" +prop_checkForInCat2 = verify checkForInCat "for f in $(cat foo | grep lol); do stuff; done" +prop_checkForInCat2a= verify checkForInCat "for f in `cat foo | grep lol`; do stuff; done" +prop_checkForInCat3 = verifyNot checkForInCat "for f in $(cat foo | grep bar | wc -l); do stuff; done" +checkForInCat (T_ForIn _ f [T_NormalWord _ w] _) = mapM_ checkF w + where + checkF (T_DollarExpansion id [T_Pipeline _ r]) + | all isLineBased r = + info id 2013 "To read lines rather than words, pipe/redirect to a 'while read' loop." + checkF (T_Backticked id cmds) = checkF (T_DollarExpansion id cmds) + checkF _ = return () + isLineBased cmd = any (cmd `isCommand`) + ["grep", "fgrep", "egrep", "sed", "cat", "awk", "cut", "sort"] +checkForInCat _ = return () + +prop_checkForInLs = verify checkForInLs "for f in $(ls *.mp3); do mplayer \"$f\"; done" +prop_checkForInLs2 = verify checkForInLs "for f in `ls *.mp3`; do mplayer \"$f\"; done" +prop_checkForInLs3 = verify checkForInLs "for f in `find / -name '*.mp3'`; do mplayer \"$f\"; done" +checkForInLs t = try t + where + try (T_ForIn _ f [T_NormalWord _ [T_DollarExpansion id [x]]] _) = + check id f x + try (T_ForIn _ f [T_NormalWord _ [T_Backticked id [x]]] _) = + check id f x + try _ = return () + check id f x = + case deadSimple x of + ("ls":n) -> + let warntype = if any ("-" `isPrefixOf`) n then warn else err in + warntype id 2045 $ "Iterate over globs whenever possible (e.g. 'for f in */*.wav'), as for loops over ls will fail for filenames like 'my file*.txt'." + ("find":_) -> warn id 2044 $ "Use find -exec or a while read loop instead, as for loops over find will fail for filenames like 'my file*.txt'." + _ -> return () + + +prop_checkFindExec1 = verify checkFindExec "find / -name '*.php' -exec rm {};" +prop_checkFindExec2 = verify checkFindExec "find / -exec touch {} && ls {} \\;" +prop_checkFindExec3 = verify checkFindExec "find / -execdir cat {} | grep lol +" +prop_checkFindExec4 = verifyNot checkFindExec "find / -name '*.php' -exec foo {} +" +prop_checkFindExec5 = verifyNot checkFindExec "find / -execdir bash -c 'a && b' \\;" +prop_checkFindExec6 = verify checkFindExec "find / -type d -execdir rm *.jpg \\;" +checkFindExec cmd@(T_SimpleCommand _ _ t@(h:r)) | cmd `isCommand` "find" = do + c <- broken r False + when c $ do + let wordId = getId $ last t in + err wordId 2067 "Missing ';' or + terminating -exec. You can't use |/||/&&, and ';' has to be a separate, quoted argument." + + where + broken [] v = return v + broken (w:r) v = do + when v $ (mapM_ warnFor $ fromWord w) + case getLiteralString w of + Just "-exec" -> broken r True + Just "-execdir" -> broken r True + Just "+" -> broken r False + Just ";" -> broken r False + _ -> broken r v + + shouldWarn x = + case x of + T_DollarExpansion _ _ -> True + T_Backticked _ _ -> True + T_Glob _ _ -> True + T_Extglob _ _ _ -> True + _ -> False + + warnFor x = + if shouldWarn x + then info (getId x) 2014 "This will expand once before find runs, not per file found." + else return () + + fromWord (T_NormalWord _ l) = l + fromWord _ = [] +checkFindExec _ = return () + + +prop_checkUnquotedExpansions1 = verifyTree checkUnquotedExpansions "rm $(ls)" +prop_checkUnquotedExpansions1a= verifyTree checkUnquotedExpansions "rm `ls`" +prop_checkUnquotedExpansions2 = verifyTree checkUnquotedExpansions "rm foo$(date)" +prop_checkUnquotedExpansions3 = verifyTree checkUnquotedExpansions "[ $(foo) == cow ]" +prop_checkUnquotedExpansions3a= verifyTree checkUnquotedExpansions "[ ! $(foo) ]" +prop_checkUnquotedExpansions4 = verifyNotTree checkUnquotedExpansions "[[ $(foo) == cow ]]" +prop_checkUnquotedExpansions5 = verifyNotTree checkUnquotedExpansions "for f in $(cmd); do echo $f; done" +prop_checkUnquotedExpansions6 = verifyNotTree checkUnquotedExpansions "$(cmd)" +prop_checkUnquotedExpansions7 = verifyNotTree checkUnquotedExpansions "cat << foo\n$(ls)\nfoo" +checkUnquotedExpansions t tree = + check t + where + check t@(T_DollarExpansion _ _) = examine t + check t@(T_Backticked _ _) = examine t + check _ = return () + examine t = + unless (inUnquotableContext tree t || usedAsCommandName tree t) $ + warn (getId t) 2046 "Quote this to prevent word splitting." + + +prop_checkRedirectToSame = verifyTree checkRedirectToSame "cat foo > foo" +prop_checkRedirectToSame2 = verifyTree checkRedirectToSame "cat lol | sed -e 's/a/b/g' > lol" +prop_checkRedirectToSame3 = verifyNotTree checkRedirectToSame "cat lol | sed -e 's/a/b/g' > foo.bar && mv foo.bar lol" +prop_checkRedirectToSame4 = verifyNotTree checkRedirectToSame "foo /dev/null > /dev/null" +prop_checkRedirectToSame5 = verifyNotTree checkRedirectToSame "foo > bar 2> bar" +checkRedirectToSame s@(T_Pipeline _ list) parents = + mapM_ (\l -> (mapM_ (\x -> doAnalysis (checkOccurences x) l) (getAllRedirs list))) list + where checkOccurences t@(T_NormalWord exceptId x) u@(T_NormalWord newId y) = + when (exceptId /= newId + && x == y + && not (isOutput t && isOutput u) + && not (special t)) $ do + let note = Note InfoC 2094 $ "Make sure not to read and write the same file in the same pipeline." + addNoteFor newId $ note + addNoteFor exceptId $ note + checkOccurences _ _ = return () + getAllRedirs l = concatMap (\(T_Redirecting _ ls _) -> concatMap getRedirs ls) l + getRedirs (T_FdRedirect _ _ (T_IoFile _ op file)) = + case op of T_Greater _ -> [file] + T_Less _ -> [file] + T_DGREAT _ -> [file] + _ -> [] + getRedirs _ = [] + special x = "/dev/" `isPrefixOf` (concat $ deadSimple x) + isOutput t = + case drop 1 $ getPath parents t of + (T_IoFile _ op _):_ -> + case op of + T_Greater _ -> True + T_DGREAT _ -> True + _ -> False + _ -> False +checkRedirectToSame _ _ = return () + + +prop_checkShorthandIf = verify checkShorthandIf "[[ ! -z file ]] && scp file host || rm file" +prop_checkShorthandIf2 = verifyNot checkShorthandIf "[[ ! -z file ]] && { scp file host || echo 'Eek'; }" +prop_checkShorthandIf3 = verifyNot checkShorthandIf "foo && bar || echo baz" +prop_checkShorthandIf4 = verifyNot checkShorthandIf "foo && a=b || a=c" +checkShorthandIf (T_AndIf id _ (T_OrIf _ _ (T_Pipeline _ t))) + | not $ isOk t = + info id 2015 "Note that A && B || C is not if-then-else. C may run when A is true." + where + isOk [t] = isAssignment t || (fromMaybe False $ do + name <- getCommandBasename t + return $ name `elem` ["echo", "exit"]) + isOk _ = False +checkShorthandIf _ = return () + + +prop_checkDollarStar = verify checkDollarStar "for f in $*; do ..; done" +checkDollarStar (T_NormalWord _ [(T_DollarBraced id l)]) | (bracedString l) == "*" = + warn id 2048 $ "Use \"$@\" (with quotes) to prevent whitespace problems." +checkDollarStar _ = return () + + +prop_checkUnquotedDollarAt = verify checkUnquotedDollarAt "ls $@" +prop_checkUnquotedDollarAt1= verifyNot checkUnquotedDollarAt "ls ${#@}" +prop_checkUnquotedDollarAt2 = verify checkUnquotedDollarAt "ls ${foo[@]}" +prop_checkUnquotedDollarAt3 = verifyNot checkUnquotedDollarAt "ls ${#foo[@]}" +prop_checkUnquotedDollarAt4 = verifyNot checkUnquotedDollarAt "ls \"$@\"" +prop_checkUnquotedDollarAt5 = verifyNot checkUnquotedDollarAt "ls ${foo/@/ at }" +checkUnquotedDollarAt (T_NormalWord _ [T_DollarBraced id l]) = + let string = bracedString l + failing = err id 2068 $ "Add double quotes around ${" ++ string ++ "}, otherwise it's just like $* and breaks on spaces." + in do + when ("@" `isPrefixOf` string) failing + when (not ("#" `isPrefixOf` string) && "[@]" `isInfixOf` string) failing +checkUnquotedDollarAt _ = return () + +prop_checkStderrRedirect = verify checkStderrRedirect "test 2>&1 > cow" +prop_checkStderrRedirect2 = verifyNot checkStderrRedirect "test > cow 2>&1" +checkStderrRedirect (T_Redirecting _ [ + T_FdRedirect id "2" (T_IoFile _ (T_GREATAND _) (T_NormalWord _ [T_Literal _ "1"])), + T_FdRedirect _ _ (T_IoFile _ op _) + ] _) = case op of + T_Greater _ -> error + T_DGREAT _ -> error + _ -> return () + where error = err id 2069 $ "The order of the 2>&1 and the redirect matters. The 2>&1 has to be last." +checkStderrRedirect _ = return () + +lt x = trace ("FAILURE " ++ (show x)) x +ltt t x = trace ("FAILURE " ++ (show t)) x + + +prop_checkSingleQuotedVariables = verifyTree checkSingleQuotedVariables "echo '$foo'" +prop_checkSingleQuotedVariables2 = verifyTree checkSingleQuotedVariables "echo 'lol$1.jpg'" +prop_checkSingleQuotedVariables3 = verifyNotTree checkSingleQuotedVariables "sed 's/foo$/bar/'" +prop_checkSingleQuotedVariables3a= verifyTree checkSingleQuotedVariables "sed 's/${foo}/bar/'" +prop_checkSingleQuotedVariables3b= verifyTree checkSingleQuotedVariables "sed 's/$(echo cow)/bar/'" +prop_checkSingleQuotedVariables3c= verifyTree checkSingleQuotedVariables "sed 's/$((1+foo))/bar/'" +prop_checkSingleQuotedVariables4 = verifyNotTree checkSingleQuotedVariables "awk '{print $1}'" +prop_checkSingleQuotedVariables5 = verifyNotTree checkSingleQuotedVariables "trap 'echo $SECONDS' EXIT" +checkSingleQuotedVariables t@(T_SingleQuoted id s) parents = + case matchRegex re s of + Just [] -> unless (probablyOk t) $ info id 2016 $ "Expressions don't expand in single quotes, use double quotes for that." + _ -> return () + where + probablyOk t = fromMaybe False $ do + cmd <- getClosestCommand parents t + name <- getCommandBasename cmd + return $ name `elem` [ + "trap" + ,"sh" + ,"bash" + ,"ksh" + ,"zsh" + ,"ssh" + ] + || "awk" `isSuffixOf` name + || "perl" `isPrefixOf` name + + re = mkRegex "\\$[{(0-9a-zA-Z_]" +checkSingleQuotedVariables _ _ = return () + + +prop_checkUnquotedN = verify checkUnquotedN "if [ -n $foo ]; then echo cow; fi" +prop_checkUnquotedN2 = verify checkUnquotedN "[ -n $cow ]" +prop_checkUnquotedN3 = verifyNot checkUnquotedN "[[ -n $foo ]] && echo cow" +checkUnquotedN (T_Condition _ SingleBracket (TC_Unary _ SingleBracket "-n" (T_NormalWord id [t]))) | willSplit t = + err id 2070 "Always true because you failed to quote. Use [[ ]] instead." +checkUnquotedN _ = return () + +prop_checkNumberComparisons1 = verify checkNumberComparisons "[[ $foo < 3 ]]" +prop_checkNumberComparisons2 = verify checkNumberComparisons "[[ 0 >= $(cmd) ]]" +prop_checkNumberComparisons3 = verifyNot checkNumberComparisons "[[ $foo ]] > 3" +prop_checkNumberComparisons4 = verify checkNumberComparisons "[[ $foo > 2.72 ]]" +prop_checkNumberComparisons5 = verify checkNumberComparisons "[[ $foo -le 2.72 ]]" +prop_checkNumberComparisons6 = verify checkNumberComparisons "[[ 3.14 = $foo ]]" +checkNumberComparisons (TC_Binary id typ op lhs rhs) = do + when (op `elem` ["<", ">", "<=", ">=", "\\<", "\\>", "\\<=", "\\>="]) $ do + when (isNum lhs || isNum rhs) $ err id 2071 $ "\"" ++ op ++ "\" is for string comparisons. Use " ++ (eqv op) ++" ." + mapM_ checkDecimals [lhs, rhs] + + when (op `elem` ["-lt", "-gt", "-le", "-ge", "-eq", "=", "=="]) $ do + mapM_ checkDecimals [lhs, rhs] + + where + checkDecimals hs = when (isFraction hs) $ err (getId hs) 2072 $ decimalError + decimalError = "Decimals are not supported. Either use integers only, or use bc or awk to compare." + isNum t = case deadSimple t of [v] -> all isDigit v + _ -> False + isFraction t = case deadSimple t of [v] -> isJust $ matchRegex floatRegex v + _ -> False + eqv ('\\':s) = eqv s + eqv "<" = "-lt" + eqv ">" = "-gt" + eqv "<=" = "-le" + eqv ">=" = "-ge" + eqv _ = "the numerical equivalent" + floatRegex = mkRegex "^[0-9]+\\.[0-9]+$" +checkNumberComparisons _ = return () + +prop_checkSingleBracketOperators1 = verify checkSingleBracketOperators "[ test =~ foo ]" +prop_checkSingleBracketOperators2 = verify checkSingleBracketOperators "[ $foo > $bar ]" +prop_checkSingleBracketOperators3 = verifyNot checkSingleBracketOperators "[[ foo < bar ]]" +prop_checkSingleBracketOperators5 = verify checkSingleBracketOperators "until [ $n <= $z ]; do echo foo; done" +checkSingleBracketOperators (TC_Binary id typ op lhs rhs) + | typ == SingleBracket && op `elem` ["<", ">", "<=", ">="] = + err id 2073 $ "Can't use " ++ op ++" in [ ]. Escape it or use [[..]]." +checkSingleBracketOperators (TC_Binary id typ op lhs rhs) + | typ == SingleBracket && op == "=~" = + err id 2074 $ "Can't use " ++ op ++" in [ ]. Use [[..]] instead." +checkSingleBracketOperators _ = return () + +prop_checkDoubleBracketOperators1 = verify checkDoubleBracketOperators "[[ 3 \\< 4 ]]" +prop_checkDoubleBracketOperators3 = verifyNot checkDoubleBracketOperators "[[ foo < bar ]]" +checkDoubleBracketOperators x@(TC_Binary id typ op lhs rhs) + | typ == DoubleBracket && op `elem` ["\\<", "\\>", "\\<=", "\\>="] = + err id 2075 $ "Escaping " ++ op ++" is required in [..], but invalid in [[..]]" +checkDoubleBracketOperators _ = return () + +prop_checkQuotedCondRegex1 = verify checkQuotedCondRegex "[[ $foo =~ \"bar\" ]]" +prop_checkQuotedCondRegex2 = verify checkQuotedCondRegex "[[ $foo =~ 'cow' ]]" +prop_checkQuotedCondRegex3 = verifyNot checkQuotedCondRegex "[[ $foo =~ $foo ]]" +checkQuotedCondRegex (TC_Binary _ _ "=~" _ rhs) = + case rhs of + T_NormalWord id [T_DoubleQuoted _ _] -> error id + T_NormalWord id [T_SingleQuoted _ _] -> error id + _ -> return () + where + error id = err id 2076 $ "Don't quote rhs of =~, it'll match literally rather than as a regex." +checkQuotedCondRegex _ = return () + +prop_checkGlobbedRegex1 = verify checkGlobbedRegex "[[ $foo =~ *foo* ]]" +prop_checkGlobbedRegex2 = verify checkGlobbedRegex "[[ $foo =~ f* ]]" +prop_checkGlobbedRegex2a = verify checkGlobbedRegex "[[ $foo =~ \\#* ]]" +prop_checkGlobbedRegex3 = verifyNot checkGlobbedRegex "[[ $foo =~ $foo ]]" +prop_checkGlobbedRegex4 = verifyNot checkGlobbedRegex "[[ $foo =~ ^c.* ]]" +checkGlobbedRegex (TC_Binary _ DoubleBracket "=~" _ rhs) = + let s = concat $ deadSimple rhs in + if isConfusedGlobRegex s + then warn (getId rhs) 2049 $ "=~ is for regex. Use == for globs." + else return () +checkGlobbedRegex _ = return () + + +prop_checkConstantIfs1 = verify checkConstantIfs "[[ foo != bar ]]" +prop_checkConstantIfs2 = verify checkConstantIfs "[[ n -le 4 ]]" +prop_checkConstantIfs3 = verify checkConstantIfs "[[ $n -le 4 && n -ge 2 ]]" +prop_checkConstantIfs4 = verifyNot checkConstantIfs "[[ $n -le 3 ]]" +prop_checkConstantIfs5 = verifyNot checkConstantIfs "[[ $n -le $n ]]" +checkConstantIfs (TC_Binary id typ op lhs rhs) + | op `elem` [ "==", "!=", "<=", ">=", "-eq", "-ne", "-lt", "-le", "-gt", "-ge", "=~", ">", "<", "="] = do + when (isJust lLit && isJust rLit) $ warn id 2050 $ "This expression is constant. Did you forget the $ on a variable?" + where + lLit = getLiteralString lhs + rLit = getLiteralString rhs +checkConstantIfs _ = return () + +prop_checkNoaryWasBinary = verify checkNoaryWasBinary "[[ a==$foo ]]" +prop_checkNoaryWasBinary2 = verify checkNoaryWasBinary "[ $foo=3 ]" +prop_checkNoaryWasBinary3 = verify checkNoaryWasBinary "[ $foo!=3 ]" +checkNoaryWasBinary (TC_Noary _ _ t@(T_NormalWord id l)) | not $ isConstant t = do + let str = concat $ deadSimple t + when ('=' `elem` str) $ err id 2077 $ "You need spaces around the comparison operator." +checkNoaryWasBinary _ = return () + +prop_checkConstantNoary = verify checkConstantNoary "[[ '$(foo)' ]]" +prop_checkConstantNoary2 = verify checkConstantNoary "[ \"-f lol\" ]" +prop_checkConstantNoary3 = verify checkConstantNoary "[[ cmd ]]" +prop_checkConstantNoary4 = verify checkConstantNoary "[[ ! cmd ]]" +checkConstantNoary (TC_Noary _ _ t@(T_NormalWord id _)) | isConstant t = do + err id 2078 $ "This expression is constant. Did you forget a $ somewhere?" +checkConstantNoary _ = return () + +prop_checkBraceExpansionVars = verify checkBraceExpansionVars "echo {1..$n}" +checkBraceExpansionVars (T_BraceExpansion id s) | '$' `elem` s = + warn id 2051 $ "Bash doesn't support variables in brace expansions." +checkBraceExpansionVars _ = return () + +prop_checkForDecimals = verify checkForDecimals "((3.14*c))" +checkForDecimals (TA_Literal id s) | any (== '.') s = do + err id 2079 $ "(( )) doesn't support decimals. Use bc or awk." +checkForDecimals _ = return () + +prop_checkDivBeforeMult = verify checkDivBeforeMult "echo $((c/n*100))" +prop_checkDivBeforeMult2 = verifyNot checkDivBeforeMult "echo $((c*100/n))" +checkDivBeforeMult (TA_Binary _ "*" (TA_Binary id "/" _ _) _) = do + info id 2017 $ "Increase precision by replacing a/b*c with a*c/b." +checkDivBeforeMult _ = return () + +prop_checkArithmeticDeref = verify checkArithmeticDeref "echo $((3+$foo))" +prop_checkArithmeticDeref2 = verify checkArithmeticDeref "cow=14; (( s+= $cow ))" +prop_checkArithmeticDeref3 = verifyNot checkArithmeticDeref "cow=1/40; (( s+= ${cow%%/*} ))" +prop_checkArithmeticDeref4 = verifyNot checkArithmeticDeref "(( ! $? ))" +prop_checkArithmeticDeref5 = verifyNot checkArithmeticDeref "(($1))" +prop_checkArithmeticDeref6 = verifyNot checkArithmeticDeref "(( ${a[$i]} ))" +checkArithmeticDeref (TA_Expansion _ (T_DollarBraced id l)) | not . excepting $ bracedString l = + style id 2004 $ "Don't use $ on variables in (( ))." + where + excepting [] = True + excepting s = (any (`elem` "/.:#%?*@[]") s) || (isDigit $ head s) +checkArithmeticDeref _ = return () + +prop_checkArithmeticBadOctal1 = verify checkArithmeticBadOctal "(( 0192 ))" +prop_checkArithmeticBadOctal2 = verifyNot checkArithmeticBadOctal "(( 0x192 ))" +prop_checkArithmeticBadOctal3 = verifyNot checkArithmeticBadOctal "(( 1 ^ 0777 ))" +checkArithmeticBadOctal (TA_Base id "0" (TA_Literal _ str)) | '9' `elem` str || '8' `elem` str = + err id 2080 $ "Numbers with leading 0 are considered octal." +checkArithmeticBadOctal _ = return () + +prop_checkComparisonAgainstGlob = verify checkComparisonAgainstGlob "[[ $cow == $bar ]]" +prop_checkComparisonAgainstGlob2 = verifyNot checkComparisonAgainstGlob "[[ $cow == \"$bar\" ]]" +prop_checkComparisonAgainstGlob3 = verify checkComparisonAgainstGlob "[ $cow = *foo* ]" +prop_checkComparisonAgainstGlob4 = verifyNot checkComparisonAgainstGlob "[ $cow = foo ]" +checkComparisonAgainstGlob (TC_Binary _ DoubleBracket op _ (T_NormalWord id [T_DollarBraced _ _])) | op == "=" || op == "==" = + warn id 2053 $ "Quote the rhs of = in [[ ]] to prevent glob interpretation." +checkComparisonAgainstGlob (TC_Binary _ SingleBracket op _ word) + | (op == "=" || op == "==") && isGlob word = + err (getId word) 2081 $ "[ .. ] can't match globs. Use [[ .. ]] or grep." +checkComparisonAgainstGlob _ = return () + +prop_checkCommarrays1 = verify checkCommarrays "a=(1, 2)" +prop_checkCommarrays2 = verify checkCommarrays "a+=(1,2,3)" +prop_checkCommarrays3 = verifyNot checkCommarrays "cow=(1 \"foo,bar\" 3)" +checkCommarrays (T_Array id l) = + if any ("," `isSuffixOf`) (concatMap deadSimple l) || (length $ filter (==',') (concat $ concatMap deadSimple l)) > 1 + then warn id 2054 "Use spaces, not commas, to separate array elements." + else return () +checkCommarrays _ = return () + +prop_checkOrNeq1 = verify checkOrNeq "if [[ $lol -ne cow || $lol -ne foo ]]; then echo foo; fi" +prop_checkOrNeq2 = verify checkOrNeq "(( a!=lol || a!=foo ))" +prop_checkOrNeq3 = verify checkOrNeq "[ \"$a\" != lol || \"$a\" != foo ]" +prop_checkOrNeq4 = verifyNot checkOrNeq "[ a != $cow || b != $foo ]" +-- This only catches the most idiomatic cases. Fixme? +checkOrNeq (TC_Or id typ op (TC_Binary _ _ op1 word1 _) (TC_Binary _ _ op2 word2 _)) + | word1 == word2 && (op1 == op2 && (op1 == "-ne" || op1 == "!=")) = + warn id 2055 $ "You probably wanted " ++ (if typ == SingleBracket then "-a" else "&&") ++ " here." +checkOrNeq (TA_Binary id "||" (TA_Binary _ "!=" word1 _) (TA_Binary _ "!=" word2 _)) + | word1 == word2 = + warn id 2056 "You probably wanted && here." +checkOrNeq _ = return () + + +allModifiedVariables t = snd $ runState (doAnalysis (\x -> modify $ (++) (getModifiedVariables x)) t) [] + +prop_checkValidCondOps1 = verify checkValidCondOps "[[ a -xz b ]]" +prop_checkValidCondOps2 = verify checkValidCondOps "[ -M a ]" +prop_checkValidCondOps2a= verifyNot checkValidCondOps "[ 3 \\> 2 ]" +prop_checkValidCondOps3 = verifyNot checkValidCondOps "[ 1 = 2 -a 3 -ge 4 ]" +prop_checkValidCondOps4 = verifyNot checkValidCondOps "[[ ! -v foo ]]" +checkValidCondOps (TC_Binary id _ s _ _) + | not (s `elem` ["-nt", "-ot", "-ef", "==", "!=", "<=", ">=", "-eq", "-ne", "-lt", "-le", "-gt", "-ge", "=~", ">", "<", "=", "\\<", "\\>", "\\<=", "\\>="]) = + warn id 2057 "Unknown binary operator." +checkValidCondOps (TC_Unary id _ s _) + | not (s `elem` [ "!", "-a", "-b", "-c", "-d", "-e", "-f", "-g", "-h", "-L", "-k", "-p", "-r", "-s", "-S", "-t", "-u", "-w", "-x", "-O", "-G", "-N", "-z", "-n", "-o", "-v", "-R"]) = + warn id 2058 "Unknown unary operator." +checkValidCondOps _ = return () + +--- Context seeking + +getParentTree t = + snd . snd $ runState (doStackAnalysis pre post t) ([], Map.empty) + where + pre t = modify (\(l, m) -> (t:l, m)) + post t = do + ((_:rest), map) <- get + case rest of [] -> put (rest, map) + (x:_) -> put (rest, Map.insert (getId t) x map) + +getTokenMap t = + snd $ runState (doAnalysis f t) (Map.empty) + where + f t = modify (Map.insert (getId t) t) + + +inUnquotableContext tree t = + case t of + TC_Noary _ DoubleBracket _ -> True + TC_Unary _ DoubleBracket _ _ -> True + TC_Binary _ DoubleBracket _ _ _ -> True + TA_Unary _ _ _ -> True + TA_Binary _ _ _ _ -> True + TA_Trinary _ _ _ _ -> True + TA_Expansion _ _ -> True + T_Assignment _ _ _ _ _ -> True + T_Redirecting _ _ _ -> + any (isCommand t) ["local", "declare", "typeset", "export"] + T_DoubleQuoted _ _ -> True + T_CaseExpression _ _ _ -> True + T_HereDoc _ _ _ _ _ -> True + T_ForIn _ _ _ _ -> True -- Pragmatically assume it's desirable here + x -> case Map.lookup (getId x) tree of + Nothing -> False + Just parent -> inUnquotableContext tree parent + +isParamTo tree cmd t = + go t + where + go x = case Map.lookup (getId x) tree of + Nothing -> False + Just parent -> check parent + check t = + case t of + T_SingleQuoted _ _ -> go t + T_DoubleQuoted _ _ -> go t + T_NormalWord _ _ -> go t + T_SimpleCommand _ _ _ -> isCommand t cmd + T_Redirecting _ _ _ -> isCommand t cmd + _ -> False + +getClosestCommand tree t = + msum . map getCommand $ getPath tree t + where + getCommand t@(T_Redirecting _ _ _) = return t + getCommand _ = Nothing + +usedAsCommandName tree token = go (getId token) (tail $ getPath tree token) + where + go currentId ((T_NormalWord id [word]):rest) + | currentId == (getId word) = go id rest + go currentId ((T_DoubleQuoted id [word]):rest) + | currentId == (getId word) = go id rest + go currentId ((T_SimpleCommand _ _ (word:_)):_) + | currentId == (getId word) = True + go _ _ = False + +-- A list of the element and all its parents +getPath tree t = t : + case Map.lookup (getId t) tree of + Nothing -> [] + Just parent -> getPath tree parent + +--- Command specific checks + +checkCommand str f t@(T_SimpleCommand id _ (cmd:rest)) = + if t `isCommand` str then f rest else return () +checkCommand _ _ _ = return () + +checkUnqualifiedCommand str f t@(T_SimpleCommand id _ (cmd:rest)) = + if t `isUnqualifiedCommand` str then f rest else return () +checkUnqualifiedCommand _ _ _ = return () + +getLiteralString = getLiteralStringExt (const Nothing) + +getGlobOrLiteralString = getLiteralStringExt f + where + f (T_Glob _ str) = return str + f _ = Nothing + +getLiteralStringExt more t = g t + where + allInList l = let foo = map g l in if all isJust foo then return $ concat (catMaybes foo) else Nothing + g s@(T_DoubleQuoted _ l) = allInList l + g s@(T_DollarDoubleQuoted _ l) = allInList l + g s@(T_NormalWord _ l) = allInList l + g (T_SingleQuoted _ s) = return s + g (T_Literal _ s) = return s + g x = more x + +isLiteral t = isJust $ getLiteralString t + +isCommand token str = isCommandMatch token (\cmd -> cmd == str || ("/" ++ str) `isSuffixOf` cmd) +isUnqualifiedCommand token str = isCommandMatch token (\cmd -> cmd == str) + +isCommandMatch token matcher = fromMaybe False $ do + cmd <- getCommandName token + return $ matcher cmd + +getCommandName (T_Redirecting _ _ w) = + getCommandName w +getCommandName (T_SimpleCommand _ _ (w:_)) = + getLiteralString w +getCommandName (T_Annotation _ _ t) = getCommandName t +getCommandName _ = Nothing + +getCommandBasename = liftM basename . getCommandName +basename = reverse . (takeWhile (/= '/')) . reverse + +isAssignment (T_Annotation _ _ w) = isAssignment w +isAssignment (T_Redirecting _ _ w) = isAssignment w +isAssignment (T_SimpleCommand _ (w:_) []) = True +isAssignment _ = False + +prop_checkPrintfVar1 = verify checkPrintfVar "printf \"Lol: $s\"" +prop_checkPrintfVar2 = verifyNot checkPrintfVar "printf 'Lol: $s'" +prop_checkPrintfVar3 = verify checkPrintfVar "printf -v cow $(cmd)" +prop_checkPrintfVar4 = verifyNot checkPrintfVar "printf \"%${count}s\" var" +checkPrintfVar = checkUnqualifiedCommand "printf" f where + f (dashv:var:rest) | getLiteralString dashv == (Just "-v") = f rest + f (format:params) = check format + f _ = return () + check format = + if '%' `elem` (concat $ deadSimple format) || isLiteral format + then return () + else warn (getId format) 2059 $ + "Don't use variables in the printf format string. Use printf \"..%s..\" \"$foo\"." + +prop_checkUuoe1 = verify checkUuoe "echo $(date)" +prop_checkUuoe1a= verify checkUuoe "echo `date`" +prop_checkUuoe2 = verify checkUuoe "echo \"$(date)\"" +prop_checkUuoe2a= verify checkUuoe "echo \"`date`\"" +prop_checkUuoe3 = verifyNot checkUuoe "echo \"The time is $(date)\"" +checkUuoe = checkUnqualifiedCommand "echo" f where + msg id = style id 2005 "Useless echo? Instead of 'echo $(cmd)', just use 'cmd'." + f [T_NormalWord id [(T_DollarExpansion _ _)]] = msg id + f [T_NormalWord id [T_DoubleQuoted _ [(T_DollarExpansion _ _)]]] = msg id + f [T_NormalWord id [(T_Backticked _ _)]] = msg id + f [T_NormalWord id [T_DoubleQuoted _ [(T_Backticked _ _)]]] = msg id + f _ = return () + +prop_checkTr1 = verify checkTr "tr [a-f] [A-F]" +prop_checkTr2 = verify checkTr "tr 'a-z' 'A-Z'" +prop_checkTr2a= verify checkTr "tr '[a-z]' '[A-Z]'" +prop_checkTr3 = verifyNot checkTr "tr -d '[:lower:]'" +prop_checkTr3a= verifyNot checkTr "tr -d '[:upper:]'" +prop_checkTr3b= verifyNot checkTr "tr -d '|/_[:upper:]'" +prop_checkTr4 = verifyNot checkTr "ls [a-z]" +prop_checkTr5 = verify checkTr "tr foo bar" +prop_checkTr6 = verify checkTr "tr 'hello' 'world'" +prop_checkTr8 = verifyNot checkTr "tr aeiou _____" +prop_checkTr9 = verifyNot checkTr "a-z n-za-m" +prop_checkTr10= verifyNot checkTr "tr --squeeze-repeats rl lr" +prop_checkTr11= verifyNot checkTr "tr abc '[d*]'" +checkTr = checkCommand "tr" (mapM_ f) + where + f w | isGlob w = do -- The user will go [ab] -> '[ab]' -> 'ab'. Fixme? + warn (getId w) 2060 $ "Quote parameters to tr to prevent glob expansion." + f word = case getLiteralString word of + Just "a-z" -> info (getId word) 2018 "Use '[:lower:]' to support accents and foreign alphabets." + Just "A-Z" -> info (getId word) 2019 "Use '[:upper:]' to support accents and foreign alphabets." + + Just s -> do -- Eliminate false positives by only looking for dupes in SET2? + when ((not $ "-" `isPrefixOf` s || "[:" `isInfixOf` s) && duplicated s) $ + info (getId word) 2020 "tr replaces sets of chars, not words (mentioned due to duplicates)." + + unless ("[:" `isPrefixOf` s) $ + when ("[" `isPrefixOf` s && "]" `isSuffixOf` s && (length s > 2) && (not $ '*' `elem` s)) $ + info (getId word) 2021 "Don't use [] around ranges in tr, it replaces literal square brackets." + Nothing -> return () + + duplicated s = + let relevant = filter isAlpha s + in not $ relevant == nub relevant + + +prop_checkFindNameGlob1 = verify checkFindNameGlob "find / -name *.php" +prop_checkFindNameGlob2 = verify checkFindNameGlob "find / -type f -ipath *(foo)" +prop_checkFindNameGlob3 = verifyNot checkFindNameGlob "find * -name '*.php'" +checkFindNameGlob = checkCommand "find" f where + acceptsGlob (Just s) = s `elem` [ "-ilname", "-iname", "-ipath", "-iregex", "-iwholename", "-lname", "-name", "-path", "-regex", "-wholename" ] + acceptsGlob _ = False + f [] = return () + f [x] = return () + f (a:b:r) = do + when (acceptsGlob (getLiteralString a) && isGlob b) $ do + let (Just s) = getLiteralString a + warn (getId b) 2061 $ "Quote the parameter to " ++ s ++ " so the shell won't interpret it." + f (b:r) + + +prop_checkGrepRe1 = verify checkGrepRe "cat foo | grep *.mp3" +prop_checkGrepRe2 = verify checkGrepRe "grep -Ev cow*test *.mp3" +prop_checkGrepRe3 = verify checkGrepRe "grep --regex=*.mp3 file" +prop_checkGrepRe4 = verifyNot checkGrepRe "grep foo *.mp3" +prop_checkGrepRe5 = verifyNot checkGrepRe "grep-v --regex=moo *" +prop_checkGrepRe6 = verifyNot checkGrepRe "grep foo \\*.mp3" +prop_checkGrepRe7 = verify checkGrepRe "grep *foo* file" +prop_checkGrepRe8 = verify checkGrepRe "ls | grep foo*.jpg" +prop_checkGrepRe9 = verifyNot checkGrepRe "grep '[0-9]*' file" + +checkGrepRe = checkCommand "grep" f where + -- --regex=*(extglob) doesn't work. Fixme? + skippable (Just s) = not ("--regex=" `isPrefixOf` s) && "-" `isPrefixOf` s + skippable _ = False + f [] = return () + f (x:r) | skippable (getLiteralString x) = f r + f (re:_) = do + when (isGlob re) $ do + warn (getId re) 2062 $ "Quote the grep pattern so the shell won't interpret it." + let string = concat $ deadSimple re + if isConfusedGlobRegex string then + warn (getId re) 2063 $ "Grep uses regex, but this looks like a glob." + else + if (isPotentiallyConfusedGlobRegex string) + then info (getId re) 2022 "Note that c* does not mean \"c followed by anything\" in regex." + else return () + + +prop_checkTrapQuotes1 = verify checkTrapQuotes "trap \"echo $num\" INT" +prop_checkTrapQuotes1a= verify checkTrapQuotes "trap \"echo `ls`\" INT" +prop_checkTrapQuotes2 = verifyNot checkTrapQuotes "trap 'echo $num' INT" +prop_checkTrapQuotes3 = verify checkTrapQuotes "trap \"echo $((1+num))\" EXIT DEBUG" +checkTrapQuotes = checkCommand "trap" f where + f (x:_) = checkTrap x + f _ = return () + checkTrap (T_NormalWord _ [T_DoubleQuoted _ rs]) = mapM_ checkExpansions rs + checkTrap _ = return () + warning id = warn id 2064 $ "Use single quotes, otherwise this expands now rather than when signalled." + checkExpansions (T_DollarExpansion id _) = warning id + checkExpansions (T_Backticked id _) = warning id + checkExpansions (T_DollarBraced id _) = warning id + checkExpansions (T_DollarArithmetic id _) = warning id + checkExpansions _ = return () + +prop_checkTimeParameters1 = verify checkTimeParameters "time -f lol sleep 10" +prop_checkTimeParameters2 = verifyNot checkTimeParameters "time sleep 10" +prop_checkTimeParameters3 = verifyNot checkTimeParameters "time -p foo" +checkTimeParameters = checkUnqualifiedCommand "time" f where + f (x:_) = let s = concat $ deadSimple x in + if "-" `isPrefixOf` s && s /= "-p" then + info (getId x) 2023 "The shell may override 'time' as seen in man time(1). Use 'command time ..' for that one." + else return () + f _ = return () + +prop_checkTestRedirects1 = verify checkTestRedirects "test 3 > 1" +prop_checkTestRedirects2 = verifyNot checkTestRedirects "test 3 \\> 1" +prop_checkTestRedirects3 = verify checkTestRedirects "/usr/bin/test $var > $foo" +checkTestRedirects (T_Redirecting id redirs@(redir:_) cmd) | cmd `isCommand` "test" = + warn (getId redir) 2065 $ "This is interpretted as a shell file redirection, not a comparison." +checkTestRedirects _ = return () + +prop_checkSudoRedirect1 = verify checkSudoRedirect "sudo echo 3 > /proc/file" +prop_checkSudoRedirect2 = verify checkSudoRedirect "sudo cmd < input" +prop_checkSudoRedirect3 = verify checkSudoRedirect "sudo cmd >> file" +prop_checkSudoRedirect4 = verify checkSudoRedirect "sudo cmd &> file" +prop_checkSudoRedirect5 = verifyNot checkSudoRedirect "sudo cmd 2>&1" +prop_checkSudoRedirect6 = verifyNot checkSudoRedirect "sudo cmd 2> log" +prop_checkSudoRedirect7 = verifyNot checkSudoRedirect "sudo cmd > /dev/null 2>&1" +checkSudoRedirect (T_Redirecting _ redirs cmd) | cmd `isCommand` "sudo" = + mapM_ warnAbout redirs + where + warnAbout (T_FdRedirect _ s (T_IoFile id op file)) + | (s == "" || s == "&") && (not $ special file) = + case op of + T_Less _ -> + info (getId op) 2024 $ + "sudo doesn't affect redirects. Use sudo cat file | .." + T_Greater _ -> + warn (getId op) 2024 $ + "sudo doesn't affect redirects. Use ..| sudo tee file" + T_DGREAT _ -> + warn (getId op) 2024 $ + "sudo doesn't affect redirects. Use .. | sudo tee -a file" + _ -> return () + warnAbout _ = return () + special file = (concat $ deadSimple file) == "/dev/null" +checkSudoRedirect _ = return () + +prop_checkPS11 = verify checkPS1Assignments "PS1='\\033[1;35m\\$ '" +prop_checkPS11a= verify checkPS1Assignments "export PS1='\\033[1;35m\\$ '" +prop_checkPSf2 = verify checkPS1Assignments "PS1='\\h \\e[0m\\$ '" +prop_checkPS13 = verify checkPS1Assignments "PS1=$'\\x1b[c '" +prop_checkPS14 = verify checkPS1Assignments "PS1=$'\\e[3m; '" +prop_checkPS14a= verify checkPS1Assignments "export PS1=$'\\e[3m; '" +prop_checkPS15 = verifyNot checkPS1Assignments "PS1='\\[\\033[1;35m\\]\\$ '" +prop_checkPS16 = verifyNot checkPS1Assignments "PS1='\\[\\e1m\\e[1m\\]\\$ '" +prop_checkPS17 = verifyNot checkPS1Assignments "PS1='e033x1B'" +prop_checkPS18 = verifyNot checkPS1Assignments "PS1='\\[\\e\\]'" +checkPS1Assignments (T_Assignment _ _ "PS1" _ word) = warnFor word + where + warnFor word = + let contents = concat $ deadSimple word in + when (containsUnescaped contents) $ + info (getId word) 2025 "Make sure all escape sequences are enclosed in \\[..\\] to prevent line wrapping issues" + containsUnescaped s = + let unenclosed = subRegex enclosedRegex s "" in + isJust $ matchRegex escapeRegex unenclosed + enclosedRegex = mkRegex "\\\\\\[.*\\\\\\]" -- FIXME: shouldn't be eager + escapeRegex = mkRegex "\\\\x1[Bb]|\\\\e|\x1B|\\\\033" +checkPS1Assignments _ = return () + +prop_checkBackticks1 = verify checkBackticks "echo `foo`" +prop_checkBackticks2 = verifyNot checkBackticks "echo $(foo)" +checkBackticks (T_Backticked id _) = + style id 2006 "Use $(..) instead of deprecated `..`" +checkBackticks _ = return () + +prop_checkIndirectExpansion1 = verify checkIndirectExpansion "${foo$n}" +prop_checkIndirectExpansion2 = verifyNot checkIndirectExpansion "${foo//$n/lol}" +prop_checkIndirectExpansion3 = verify checkIndirectExpansion "${$#}" +prop_checkIndirectExpansion4 = verify checkIndirectExpansion "${var${n}_$((i%2))}" +prop_checkIndirectExpansion5 = verifyNot checkIndirectExpansion "${bar}" +checkIndirectExpansion (T_DollarBraced i (T_NormalWord _ contents)) = + when (isIndirection contents) $ + err i 2082 "To expand via indirection, use name=\"foo$n\"; echo \"${!name}\"." + where + isIndirection vars = + let list = catMaybes (map isIndirectionPart vars) in + not (null list) && all id list + isIndirectionPart t = + case t of T_DollarExpansion _ _ -> Just True + T_Backticked _ _ -> Just True + T_DollarBraced _ _ -> Just True + T_DollarArithmetic _ _ -> Just True + T_Literal _ s -> if all isVariableChar s + then Nothing + else Just False + _ -> Just False + +checkIndirectExpansion _ = return () + +prop_checkInexplicablyUnquoted1 = verify checkInexplicablyUnquoted "echo 'var='value';'" +prop_checkInexplicablyUnquoted2 = verifyNot checkInexplicablyUnquoted "'foo'*" +prop_checkInexplicablyUnquoted3 = verifyNot checkInexplicablyUnquoted "wget --user-agent='something'" +checkInexplicablyUnquoted (T_NormalWord id tokens) = mapM_ check (tails tokens) + where + check ((T_SingleQuoted _ _):(T_Literal id str):_) + | all isAlphaNum str = + info id 2026 $ "This word is outside of quotes. Did you intend to 'nest '\"'single quotes'\"' instead'? " + + check ((T_DoubleQuoted _ _):trapped:(T_DoubleQuoted _ _):_) = + case trapped of + T_DollarExpansion id _ -> warnAbout id + T_DollarBraced id _ -> warnAbout id + _ -> return () + + check _ = return () + warnAbout id = + info id 2027 $ "Surrounding quotes actually unquotes this (\"inside\"$outside\"inside\"). Did you forget your quote level?" +checkInexplicablyUnquoted _ = return () + +prop_checkTildeInQuotes1 = verify checkTildeInQuotes "var=\"~/out.txt\"" +prop_checkTildeInQuotes2 = verify checkTildeInQuotes "foo > '~/dir'" +prop_checkTildeInQuotes4 = verifyNot checkTildeInQuotes "~/file" +prop_checkTildeInQuotes5 = verifyNot checkTildeInQuotes "echo '/~foo/cow'" +prop_checkTildeInQuotes6 = verifyNot checkTildeInQuotes "awk '$0 ~ /foo/'" +checkTildeInQuotes = check + where + verify id ('~':_) = warn id 2088 "Note that ~ does not expand in quotes." + verify _ _ = return () + check (T_NormalWord _ ((T_SingleQuoted id str):_)) = + verify id str + check (T_NormalWord _ ((T_DoubleQuoted _ ((T_Literal id str):_)):_)) = + verify id str + check _ = return () + +prop_checkLonelyDotDash1 = verify checkLonelyDotDash "./ file" +prop_checkLonelyDotDash2 = verifyNot checkLonelyDotDash "./file" +checkLonelyDotDash t@(T_Redirecting id _ _) + | isUnqualifiedCommand t "./" = + err id 2083 "Don't add spaces after the slash in './file'." +checkLonelyDotDash _ = return () + + +prop_checkSpuriousExec1 = verify checkSpuriousExec "exec foo; true" +prop_checkSpuriousExec2 = verify checkSpuriousExec "if a; then exec b; exec c; fi" +prop_checkSpuriousExec3 = verifyNot checkSpuriousExec "echo cow; exec foo" +prop_checkSpuriousExec4 = verifyNot checkSpuriousExec "if a; then exec b; fi" +prop_checkSpuriousExec5 = verifyNot checkSpuriousExec "exec > file; cmd" +prop_checkSpuriousExec6 = verify checkSpuriousExec "exec foo > file; cmd" +prop_checkSpuriousExec7 = verifyNot checkSpuriousExec "exec file; echo failed; exit 3" +checkSpuriousExec = doLists + where + doLists (T_Script _ _ cmds) = doList cmds + doLists (T_BraceGroup _ cmds) = doList cmds + doLists (T_WhileExpression _ _ cmds) = doList cmds + doLists (T_UntilExpression _ _ cmds) = doList cmds + doLists (T_ForIn _ _ _ cmds) = doList cmds + doLists (T_IfExpression _ thens elses) = do + mapM_ (\(_, l) -> doList l) thens + doList elses + doLists _ = return () + + stripCleanup = reverse . dropWhile cleanup . reverse + cleanup (T_Pipeline _ [cmd]) = + isCommandMatch cmd (`elem` ["echo", "exit"]) + cleanup _ = False + + doList = doList' . stripCleanup + doList' t@(current:following:_) = do + commentIfExec current + doList (tail t) + doList' _ = return () + + commentIfExec (T_Pipeline id list) = + mapM_ commentIfExec $ take 1 list + commentIfExec (T_Redirecting _ _ f@( + T_SimpleCommand id _ (cmd:arg:_))) = + when (f `isUnqualifiedCommand` "exec") $ + warn (id) 2093 $ + "Remove \"exec \" if script should continue after this command." + commentIfExec _ = return () + + +prop_checkSpuriousExpansion1 = verify checkSpuriousExpansion "if $(true); then true; fi" +prop_checkSpuriousExpansion2 = verify checkSpuriousExpansion "while \"$(cmd)\"; do :; done" +prop_checkSpuriousExpansion3 = verifyNot checkSpuriousExpansion "$(cmd) --flag1 --flag2" +prop_checkSpuriousExpansion4 = verify checkSpuriousExpansion "$((i++))" +checkSpuriousExpansion (T_SimpleCommand _ _ [T_NormalWord _ [word]]) = check word + where + check word = case word of + T_DollarExpansion id _ -> + warn id 2091 "Remove surrounding $() to avoid executing output." + T_Backticked id _ -> + warn id 2092 "Remove backticks to avoid executing output." + T_DollarArithmetic id _ -> + err id 2084 "Remove '$' or use '_=$((expr))' to avoid executing output." + T_DoubleQuoted id [subword] -> check subword + _ -> return () +checkSpuriousExpansion _ = return () + + +prop_checkUnusedEchoEscapes1 = verify checkUnusedEchoEscapes "echo 'foo\\nbar\\n'" +prop_checkUnusedEchoEscapes2 = verifyNot checkUnusedEchoEscapes "echo -e 'foi\\nbar'" +prop_checkUnusedEchoEscapes3 = verify checkUnusedEchoEscapes "echo \"n:\\t42\"" +prop_checkUnusedEchoEscapes4 = verifyNot checkUnusedEchoEscapes "echo lol" +checkUnusedEchoEscapes = checkCommand "echo" f + where + isDashE = mkRegex "^-.*e" + hasEscapes = mkRegex "\\\\[rnt]" + f (arg:_) | (concat $ deadSimple arg) `matches` isDashE = return () + f args = mapM_ checkEscapes args + + checkEscapes (T_NormalWord _ args) = + mapM_ checkEscapes args + checkEscapes (T_DoubleQuoted id args) = + mapM_ checkEscapes args + checkEscapes (T_Literal id str) = examine id str + checkEscapes (T_SingleQuoted id str) = examine id str + checkEscapes _ = return () + + examine id str = + when (str `matches` hasEscapes) $ + info id 2028 "echo won't expand escape sequences. Consider printf." + + +prop_checkDollarBrackets1 = verify checkDollarBrackets "echo $[1+2]" +prop_checkDollarBrackets2 = verifyNot checkDollarBrackets "echo $((1+2))" +checkDollarBrackets (T_DollarBracket id _) = + style id 2007 "Use $((..)) instead of deprecated $[..]" +checkDollarBrackets _ = return () + +prop_checkSshHereDoc1 = verify checkSshHereDoc "ssh host << foo\necho $PATH\nfoo" +prop_checkSshHereDoc2 = verifyNot checkSshHereDoc "ssh host << 'foo'\necho $PATH\nfoo" +checkSshHereDoc (T_Redirecting _ redirs cmd) + | cmd `isCommand` "ssh" = + mapM_ checkHereDoc redirs + where + hasVariables = mkRegex "[`$]" + checkHereDoc (T_FdRedirect _ _ (T_HereDoc id _ Unquoted token tokens)) + | not (all isConstant tokens) = + warn id 2087 $ "Quote '" ++ token ++ "' to make here document expansions happen on the server side rather than on the client." + checkHereDoc _ = return () +checkSshHereDoc _ = return () + +-- This is hard to get right without properly parsing ssh args +prop_checkSshCmdStr1 = verify checkSshCommandString "ssh host \"echo $PS1\"" +prop_checkSshCmdStr2 = verifyNot checkSshCommandString "ssh host \"ls foo\"" +prop_checkSshCmdStr3 = verifyNot checkSshCommandString "ssh \"$host\"" +checkSshCommandString = checkCommand "ssh" f + where + nonOptions args = + filter (\x -> not $ "-" `isPrefixOf` (concat $ deadSimple x)) args + f args = + case nonOptions args of + (hostport:r@(_:_)) -> checkArg $ last r + _ -> return () + checkArg (T_NormalWord _ [T_DoubleQuoted id parts]) = + case filter (not . isConstant) parts of + [] -> return () + (x:_) -> info (getId x) 2029 $ + "Note that, unescaped, this expands on the client side." + checkArg _ = return () + + +--- Subshell detection +prop_subshellAssignmentCheck = verifyFull subshellAssignmentCheck "cat foo | while read bar; do a=$bar; done; echo \"$a\"" +prop_subshellAssignmentCheck2 = verifyNotFull subshellAssignmentCheck "while read bar; do a=$bar; done < file; echo \"$a\"" +prop_subshellAssignmentCheck3 = verifyFull subshellAssignmentCheck "( A=foo; ); rm $A" +prop_subshellAssignmentCheck4 = verifyNotFull subshellAssignmentCheck "( A=foo; rm $A; )" +prop_subshellAssignmentCheck5 = verifyFull subshellAssignmentCheck "cat foo | while read cow; do true; done; echo $cow;" +prop_subshellAssignmentCheck6 = verifyFull subshellAssignmentCheck "( export lol=$(ls); ); echo $lol;" +prop_subshellAssignmentCheck6a= verifyFull subshellAssignmentCheck "( typeset -a lol=a; ); echo $lol;" +prop_subshellAssignmentCheck7 = verifyFull subshellAssignmentCheck "cmd | while read foo; do (( n++ )); done; echo \"$n lines\"" +prop_subshellAssignmentCheck8 = verifyFull subshellAssignmentCheck "n=3 & echo $((n++))" +prop_subshellAssignmentCheck9 = verifyFull subshellAssignmentCheck "read n & n=foo$n" +prop_subshellAssignmentCheck10 = verifyFull subshellAssignmentCheck "(( n <<= 3 )) & (( n |= 4 )) &" +prop_subshellAssignmentCheck11 = verifyFull subshellAssignmentCheck "cat /etc/passwd | while read line; do let n=n+1; done\necho $n" +prop_subshellAssignmentCheck12 = verifyFull subshellAssignmentCheck "cat /etc/passwd | while read line; do let ++n; done\necho $n" +subshellAssignmentCheck t = + let flow = getVariableFlow t + check = findSubshelled flow [("oops",[])] Map.empty + in snd $ runState check [] + + +data Scope = SubshellScope String | NoneScope deriving (Show, Eq) +data StackData = + StackScope Scope + | StackScopeEnd + -- (Base expression, specific position, var name, assigned values) + | Assignment (Token, Token, String, DataSource) + | Reference (Token, Token, String) + deriving (Show, Eq) +data DataSource = DataFrom [Token] | DataExternal + deriving (Show, Eq) + +data VariableState = Dead Token String | Alive deriving (Show, Eq) + +leadType t = + case t of + T_DollarExpansion _ _ -> SubshellScope "$(..) expansion" + T_Backticked _ _ -> SubshellScope "`..` expansion" + T_Backgrounded _ _ -> SubshellScope "backgrounding &" + T_Subshell _ _ -> SubshellScope "(..) group" + -- This considers the whole pipeline one subshell. Consider fixing. + T_Pipeline _ (_:_:[]) -> SubshellScope "pipeline" + _ -> NoneScope + + +getModifiedVariables t = + case t of + T_SimpleCommand _ vars [] -> + concatMap (\x -> case x of + T_Assignment id _ name _ w -> + [(x, x, name, DataFrom [w])] + _ -> [] + ) vars + c@(T_SimpleCommand _ _ _) -> + getModifiedVariableCommand c + + TA_Unary _ "++|" (TA_Variable id name) -> [(t, t, name, DataFrom [t])] + TA_Unary _ "|++" (TA_Variable id name) -> [(t, t, name, DataFrom [t])] + TA_Binary _ op (TA_Variable id name) rhs -> + if any (==op) ["=", "*=", "/=", "%=", "+=", "-=", "<<=", ">>=", "&=", "^=", "|="] + then [(t, t, name, DataFrom [rhs])] + else [] + + --Points to 'for' rather than variable + T_ForIn id str words _ -> [(t, t, str, DataFrom words)] + T_SelectIn id str words _ -> [(t, t, str, DataFrom words)] + _ -> [] + +-- Consider 'export' a reference, since it makes the var available +getReferencedVariableCommand base@(T_SimpleCommand _ _ ((T_NormalWord _ ((T_Literal _ x):_)):rest)) = + case x of + "export" -> concatMap getReference rest + _ -> [] + where + getReference t@(T_Assignment _ _ name _ value) = [(t, t, name)] + getReference t@(T_NormalWord _ [T_Literal _ name]) | not ("-" `isPrefixOf` name) = [(t, t, name)] + getReference _ = [] + +getReferencedVariableCommand _ = [] + +getModifiedVariableCommand base@(T_SimpleCommand _ _ ((T_NormalWord _ ((T_Literal _ x):_)):rest)) = + filter (\(_,_,s,_) -> not ("-" `isPrefixOf` s)) $ + case x of + "read" -> concatMap getLiteral rest + "let" -> concatMap letParamToLiteral rest + + "export" -> concatMap getModifierParam rest + "declare" -> concatMap getModifierParam rest + "typeset" -> concatMap getModifierParam rest + "local" -> concatMap getModifierParam rest + + _ -> [] + where + stripEquals s = let rest = dropWhile (/= '=') s in + if rest == "" then "" else tail rest + stripEqualsFrom (T_NormalWord id1 ((T_Literal id2 s):rs)) = + (T_NormalWord id1 ((T_Literal id2 (stripEquals s)):rs)) + stripEqualsFrom (T_NormalWord id1 [T_DoubleQuoted id2 [T_Literal id3 s]]) = + (T_NormalWord id1 [T_DoubleQuoted id2 [T_Literal id3 (stripEquals s)]]) + stripEqualsFrom t = t + + getLiteral t@(T_NormalWord _ [T_Literal _ s]) = + [(base, t, s, DataExternal)] + getLiteral t@(T_NormalWord _ [T_SingleQuoted _ s]) = + [(base, t, s, DataExternal)] + getLiteral t@(T_NormalWord _ [T_DoubleQuoted _ [T_Literal id s]]) = + [(base, t, s, DataExternal)] + getLiteral x = [] + + getModifierParam t@(T_Assignment _ _ name _ value) = + [(base, t, name, DataFrom [value])] + getModifierParam _ = [] + + letParamToLiteral token = + if var == "" + then [] + else [(base, token, var, DataFrom [stripEqualsFrom token])] + where var = takeWhile (isVariableChar) $ dropWhile (\x -> x `elem` "+-") $ concat $ deadSimple token +getModifiedVariableCommand _ = [] + +-- TODO: +getBracedReference s = takeWhile (\x -> not $ x `elem` ":[#%/^,") $ dropWhile (`elem` "#!") s + +getReferencedVariables t = + case t of + T_DollarBraced id l -> map (\x -> (t, t, x)) $ [getBracedReference $ bracedString l] + TA_Variable id str -> [(t, t, str)] + T_Assignment id Append str _ _ -> [(t, t, str)] + x -> getReferencedVariableCommand x + +getVariableFlow t = + let (_, stack) = runState (doStackAnalysis startScope endScope t) [] + in reverse stack + where + startScope t = + let scopeType = leadType t + in do + when (scopeType /= NoneScope) $ modify ((StackScope scopeType):) + if assignFirst t then setWritten t else return () + + endScope t = + let scopeType = leadType t + in do + setRead t + if assignFirst t then return () else setWritten t + when (scopeType /= NoneScope) $ modify ((StackScopeEnd):) + + assignFirst (T_ForIn _ _ _ _) = True + assignFirst (T_SelectIn _ _ _ _) = True + assignFirst _ = False + + setRead t = + let read = getReferencedVariables t + in mapM_ (\v -> modify ((Reference v):)) read + + setWritten t = + let written = getModifiedVariables t + in mapM_ (\v -> modify ((Assignment v):)) written + +findSubshelled [] _ _ = return () +findSubshelled ((Assignment x@(_, _, str, _)):rest) ((reason,scope):lol) deadVars = + findSubshelled rest ((reason, x:scope):lol) $ Map.insert str Alive deadVars +findSubshelled ((Reference (_, readToken, str)):rest) scopes deadVars = do + case Map.findWithDefault Alive str deadVars of + Alive -> return () + Dead writeToken reason -> do + info (getId writeToken) 2030 $ "Modification of " ++ str ++ " is local (to subshell caused by "++ reason ++")." + info (getId readToken) 2031 $ str ++ " was modified in a subshell. That change might be lost." + findSubshelled rest scopes deadVars + +findSubshelled ((StackScope (SubshellScope reason)):rest) scopes deadVars = + findSubshelled rest ((reason,[]):scopes) deadVars + +findSubshelled ((StackScopeEnd):rest) ((reason, scope):oldScopes) deadVars = + findSubshelled rest oldScopes $ + foldl (\m (_, token, var, _) -> + Map.insert var (Dead token reason) m) deadVars scope + +doVariableFlowAnalysis readFunc writeFunc empty t = fst $ runState ( + foldM (\list x -> do { l <- doFlow x; return $ l ++ list; }) [] flow + ) empty + where + flow = getVariableFlow t + doFlow (Reference (base, token, name)) = + readFunc base token name + doFlow (Assignment (base, token, name, values)) = + writeFunc base token name values + doFlow _ = return [] + +---- Check whether variables could have spaces/globs +prop_checkSpacefulness0 = verifyFull checkSpacefulness "for f in *.mp3; do echo $f; done" +prop_checkSpacefulness1 = verifyFull checkSpacefulness "a='cow moo'; echo $a" +prop_checkSpacefulness2 = verifyNotFull checkSpacefulness "a='cow moo'; [[ $a ]]" +prop_checkSpacefulness3 = verifyNotFull checkSpacefulness "a='cow*.mp3'; echo \"$a\"" +prop_checkSpacefulness4 = verifyFull checkSpacefulness "for f in *.mp3; do echo $f; done" +prop_checkSpacefulness4a= verifyNotFull checkSpacefulness "foo=3; foo=$(echo $foo)" +prop_checkSpacefulness5 = verifyFull checkSpacefulness "a='*'; b=$a; c=lol${b//foo/bar}; echo $c" +prop_checkSpacefulness6 = verifyFull checkSpacefulness "a=foo$(lol); echo $a" +prop_checkSpacefulness7 = verifyFull checkSpacefulness "a=foo\\ bar; rm $a" +prop_checkSpacefulness8 = verifyNotFull checkSpacefulness "a=foo\\ bar; a=foo; rm $a" +prop_checkSpacefulnessA = verifyFull checkSpacefulness "rm $1" +prop_checkSpacefulnessB = verifyFull checkSpacefulness "rm ${10//foo/bar}" +prop_checkSpacefulnessC = verifyNotFull checkSpacefulness "(( $1 + 3 ))" +prop_checkSpacefulnessD = verifyNotFull checkSpacefulness "if [[ $2 -gt 14 ]]; then true; fi" +prop_checkSpacefulnessE = verifyNotFull checkSpacefulness "foo=$3 env" +prop_checkSpacefulnessF = verifyNotFull checkSpacefulness "local foo=$1" +prop_checkSpacefulnessG = verifyNotFull checkSpacefulness "declare foo=$1" +prop_checkSpacefulnessH = verifyFull checkSpacefulness "echo foo=$1" +prop_checkSpacefulnessI = verifyNotFull checkSpacefulness "$1 --flags" +prop_checkSpacefulnessJ = verifyFull checkSpacefulness "echo $PWD" + +checkSpacefulness t = + doVariableFlowAnalysis readF writeF (Map.fromList defaults) t + where + defaults = zip variablesWithoutSpaces (repeat False) + + hasSpaces name = do + map <- get + return $ Map.findWithDefault True name map + + setSpaces name bool = do + modify $ Map.insert name bool + + readF _ token name = do + spaced <- hasSpaces name + if spaced + && (not $ "@" `isPrefixOf` name) -- There's another warning for this + && (not $ isCounting token) + && (not $ inUnquotableContext parents token) + && (not $ usedAsCommandName parents token) + then return [(getId token, Note InfoC 2086 warning)] + else return [] + where + warning = "Double quote to prevent globbing and word splitting." + + writeF _ _ name DataExternal = do + setSpaces name True + return [] + + writeF _ _ name (DataFrom vals) = do + map <- get + setSpaces name + (isSpacefulWord (\x -> Map.findWithDefault True x map) vals) + return [] + + parents = getParentTree t + + isCounting (T_DollarBraced id token) = + case concat $ deadSimple token of + '#':_ -> True + _ -> False + isCounting _ = False + + isSpacefulWord :: (String -> Bool) -> [Token] -> Bool + isSpacefulWord f words = any (isSpaceful f) words + isSpaceful :: (String -> Bool) -> Token -> Bool + isSpaceful spacefulF x = + case x of + T_DollarExpansion _ _ -> True + T_Backticked _ _ -> True + T_Glob _ _ -> True + T_Extglob _ _ _ -> True + T_Literal _ s -> s `containsAny` globspace + T_SingleQuoted _ s -> s `containsAny` globspace + T_DollarBraced _ l -> spacefulF $ getBracedReference $ bracedString l + T_NormalWord _ w -> isSpacefulWord spacefulF w + T_DoubleQuoted _ w -> isSpacefulWord spacefulF w + _ -> False + where + globspace = "*? \t\n" + containsAny s chars = any (\c -> c `elem` s) chars + + +prop_checkQuotesInLiterals1 = verifyFull checkQuotesInLiterals "param='--foo=\"bar\"'; app $param" +prop_checkQuotesInLiterals1a= verifyFull checkQuotesInLiterals "param=\"--foo='lolbar'\"; app $param" +prop_checkQuotesInLiterals2 = verifyNotFull checkQuotesInLiterals "param='--foo=\"bar\"'; app \"$param\"" +prop_checkQuotesInLiterals3 =verifyNotFull checkQuotesInLiterals "param=('--foo='); app \"${param[@]}\"" +prop_checkQuotesInLiterals4 = verifyNotFull checkQuotesInLiterals "param=\"don't bother with this one\"; app $param" +prop_checkQuotesInLiterals5 = verifyNotFull checkQuotesInLiterals "param=\"--foo='lolbar'\"; eval app $param" +checkQuotesInLiterals t = + doVariableFlowAnalysis readF writeF Map.empty t + where + getQuotes name = get >>= (return . Map.lookup name) + setQuotes name ref = modify $ Map.insert name ref + deleteQuotes = modify . Map.delete + parents = getParentTree t + quoteRegex = mkRegex "\"|([= ]|^)'|'( |$)" + containsQuotes s = isJust $ matchRegex quoteRegex s + + -- Just catch the most blatant cases of foo='--cow="lol bert"'; cmd $foo, since that's 99% + writeF _ _ name (DataFrom values) = do + let quotedVars = filter (\v -> containsQuotes (concat $ deadSimple v)) values + case quotedVars of + [] -> deleteQuotes name + x:_ -> setQuotes name (getId x) + return [] + writeF _ _ _ _ = return [] + + readF _ expr name = do + assignment <- getQuotes name + if isJust assignment + && not (isParamTo parents "eval" expr) + && not (inUnquotableContext parents expr) + then return [ + (fromJust assignment, + Note WarningC 2089 "Word splitting will treat quotes as literals. Use an array."), + (getId expr, + Note WarningC 2090 "Embedded quotes in this variable will not be respected.") + ] + else return [] + + +prop_checkFunctionsUsedExternally1 = + verifyFull checkFunctionsUsedExternally "foo() { :; }; sudo foo" +prop_checkFunctionsUsedExternally2 = + verifyFull checkFunctionsUsedExternally "alias f='a'; xargs -n 1 f" +prop_checkFunctionsUsedExternally3 = + verifyNotFull checkFunctionsUsedExternally "f() { :; }; echo f" +checkFunctionsUsedExternally t = + runBasicAnalysis checkCommand t + where + invokingCmds = [ + "chroot", + "find", + "screen", + "ssh", + "su", + "sudo", + "xargs" + ] + checkCommand t@(T_SimpleCommand _ _ (cmd:args)) = + let name = fromMaybe "" $ getCommandBasename t in + when (name `elem` invokingCmds) $ + mapM_ (checkArg name) args + checkCommand _ = return () + + functions = Map.fromList $ runBasicAnalysis findFunctions t + findFunctions (T_Function id name _) = modify ((name, id):) + findFunctions t@(T_SimpleCommand id _ (_:args)) + | t `isUnqualifiedCommand` "alias" = mapM_ getAlias args + findFunctions _ = return () + getAlias arg = + let string = concat $ deadSimple arg + in when ('=' `elem` string) $ + modify ((takeWhile (/= '=') string, getId arg):) + checkArg cmd arg = + case Map.lookup (concat $ deadSimple arg) functions of + Nothing -> return () + Just id -> do + warn (getId arg) 2033 $ + "Shell functions can't be passed to external commands." + info id 2032 $ + "Use own script or sh -c '..' to run this from " ++ cmd ++ "." + +prop_checkUnused0 = verifyNotFull checkUnusedAssignments "var=foo; echo $var" +prop_checkUnused1 = verifyFull checkUnusedAssignments "var=foo; echo $bar" +prop_checkUnused2 = verifyNotFull checkUnusedAssignments "var=foo; export var;" +prop_checkUnused3 = verifyFull checkUnusedAssignments "for f in *; do echo '$f'; done" +prop_checkUnused4 = verifyFull checkUnusedAssignments "local i=0" +prop_checkUnused5 = verifyNotFull checkUnusedAssignments "read lol; echo $lol" +prop_checkUnused6 = verifyNotFull checkUnusedAssignments "var=4; (( var++ ))" +prop_checkUnused7 = verifyNotFull checkUnusedAssignments "var=2; $((var))" +prop_checkUnused8 = verifyFull checkUnusedAssignments "var=2; var=3;" +prop_checkUnused9 = verifyNotFull checkUnusedAssignments "read ''" +prop_checkUnused10= verifyNotFull checkUnusedAssignments "read -p 'test: '" +prop_checkUnused11= verifyNotFull checkUnusedAssignments "bar=5; export foo[$bar]=3" +prop_checkUnused12= verifyNotFull checkUnusedAssignments "read foo; echo ${!foo}" +checkUnusedAssignments t = snd $ runState (mapM_ checkAssignment flow) [] + where + flow = getVariableFlow t + references = foldl (flip ($)) defaultMap (map insertRef flow) + insertRef (Reference (base, token, name)) = + Map.insert name () + insertRef _ = id + + checkAssignment (Assignment (_, token, name, _)) | isVariableName name = + case Map.lookup name references of + Just _ -> return () + Nothing -> do + info (getId token) 2034 $ + name ++ " appears unused. Verify it or export it." + checkAssignment _ = return () + + defaultMap = Map.fromList $ zip internalVariables $ repeat () + +prop_checkGlobsAsOptions1 = verify checkGlobsAsOptions "rm *.txt" +prop_checkGlobsAsOptions2 = verify checkGlobsAsOptions "ls ??.*" +prop_checkGlobsAsOptions3 = verifyNot checkGlobsAsOptions "rm -- *.txt" +checkGlobsAsOptions (T_SimpleCommand _ _ args) = + mapM_ check $ takeWhile (not . isEndOfArgs) args + where + check v@(T_NormalWord _ ((T_Glob id s):_)) | s == "*" || s == "?" = + info id 2035 $ + "Use ./" ++ (concat $ deadSimple v) + ++ " so names with dashes won't become options." + check _ = return () + + isEndOfArgs t = + case concat $ deadSimple t of + "--" -> True + ":::" -> True + "::::" -> True + _ -> False + +checkGlobsAsOptions _ = return () + + +prop_checkWhileReadPitfalls1 = verify checkWhileReadPitfalls "while read foo; do ssh $foo uptime; done < file" +prop_checkWhileReadPitfalls2 = verifyNot checkWhileReadPitfalls "while read -u 3 foo; do ssh $foo uptime; done 3< file" +prop_checkWhileReadPitfalls3 = verifyNot checkWhileReadPitfalls "while true; do ssh host uptime; done" +prop_checkWhileReadPitfalls4 = verifyNot checkWhileReadPitfalls "while read foo; do ssh $foo hostname < /dev/null; done" +prop_checkWhileReadPitfalls5 = verifyNot checkWhileReadPitfalls "while read foo; do echo ls | ssh $foo; done" +prop_checkWhileReadPitfalls6 = verifyNot checkWhileReadPitfalls "while read foo <&3; do ssh $foo; done 3< foo" + +checkWhileReadPitfalls (T_WhileExpression id [command] contents) + | isStdinReadCommand command = do + mapM_ checkMuncher contents + where + munchers = [ "ssh", "ffmpeg", "mplayer" ] + + isStdinReadCommand (T_Pipeline _ [T_Redirecting id redirs cmd]) = + let plaintext = deadSimple cmd + in head (plaintext ++ [""]) == "read" + && (not $ "-u" `elem` plaintext) + && all (not . stdinRedirect) redirs + isStdinReadCommand _ = False + + checkMuncher (T_Pipeline _ ((T_Redirecting _ redirs cmd):_)) = do + let name = fromMaybe "" $ getCommandBasename cmd + when ((not . any stdinRedirect $ redirs) && (name `elem` munchers)) $ do + info id 2095 $ + name ++ " may swallow stdin, preventing this loop from working properly." + warn (getId cmd) 2095 $ + "Add < /dev/null to prevent " ++ name ++ " from swallowing stdin." + checkMuncher _ = return () + + stdinRedirect (T_FdRedirect _ fd _) + | fd == "" || fd == "0" = True + stdinRedirect _ = False +checkWhileReadPitfalls _ = return () + + +prop_checkPrefixAssign1 = verifyTree checkPrefixAssignmentReference "var=foo echo $var" +prop_checkPrefixAssign2 = verifyNotTree checkPrefixAssignmentReference "var=$(echo $var) cmd" +checkPrefixAssignmentReference t@(T_DollarBraced id value) tree = + check path + where + name = getBracedReference $ bracedString value + path = getPath tree t + idPath = map getId path + + check [] = return () + check (t:rest) = + case t of + T_SimpleCommand _ vars (_:_) -> mapM_ checkVar vars + otherwise -> check rest + checkVar (T_Assignment aId mode aName Nothing value) | + aName == name && (not $ aId `elem` idPath) = do + warn aId 2097 "This assignment is only seen by the forked process." + warn id 2098 "This expansion will not see the mentioned assignment." + checkVar _ = return () + +checkPrefixAssignmentReference _ _ = return () + +prop_checkCharRangeGlob1 = verify checkCharRangeGlob "ls *[:digit:].jpg" +prop_checkCharRangeGlob2 = verifyNot checkCharRangeGlob "ls *[[:digit:]].jpg" +prop_checkCharRangeGlob3 = verify checkCharRangeGlob "ls [10-15]" +prop_checkCharRangeGlob4 = verifyNot checkCharRangeGlob "ls [a-zA-Z]" +checkCharRangeGlob (T_Glob id str) | isCharClass str = + if ":" `isPrefixOf` contents + && ":" `isSuffixOf` contents + && contents /= ":" + then warn id 2101 "Named class needs outer [], e.g. [[:digit:]]." + else + if (not $ '[' `elem` contents) && hasDupes + then info id 2102 "Ranges can only match single chars (mentioned due to duplicates)." + else return () + where + isCharClass str = "[" `isPrefixOf` str && "]" `isSuffixOf` str + contents = drop 1 . take ((length str) - 1) $ str + hasDupes = any (>1) . map length . group . sort . filter (/= '-') $ contents +checkCharRangeGlob _ = return () + + + +prop_checkCdAndBack1 = verify (checkCdAndBack Sh) "for f in *; do cd $f; git pull; cd ..; done" +prop_checkCdAndBack2 = verifyNot (checkCdAndBack Sh) "for f in *; do cd $f || continue; git pull; cd ..; done" +prop_checkCdAndBack3 = verifyNot (checkCdAndBack Sh) "while [[ $PWD != / ]]; do cd ..; done" +checkCdAndBack shell = doLists + where + doLists (T_ForIn _ _ _ cmds) = doList cmds + doLists (T_WhileExpression _ _ cmds) = doList cmds + doLists (T_UntilExpression _ _ cmds) = doList cmds + doLists (T_IfExpression _ thens elses) = do + mapM_ (\(_, l) -> doList l) thens + doList elses + doLists _ = return () + + isCdRevert t = + case deadSimple t of + ["cd", p] -> p `elem` ["..", "-"] + _ -> False + + getCmd (T_Annotation id _ x) = getCmd x + getCmd (T_Pipeline id [x]) = getCommandName x + getCmd _ = Nothing + + doList list = + let cds = filter ((== Just "cd") . getCmd) list in + when (length cds >= 2 && isCdRevert (last cds)) $ + warn (getId $ head cds) 2103 message + + message = + if shell == Bash || shell == Zsh + then "Consider using ( subshell ), 'cd foo||exit', or pushd/popd instead." + else "Consider using ( subshell ) or 'cd foo||exit' instead." diff --git a/ShellCheck/Data.hs b/ShellCheck/Data.hs new file mode 100644 --- /dev/null +++ b/ShellCheck/Data.hs @@ -0,0 +1,74 @@ +module ShellCheck.Data where + +internalVariables = [ + -- Generic + "", "_", + + -- Bash + "BASH", "BASHOPTS", "BASHPID", "BASH_ALIASES", "BASH_ARGC", + "BASH_ARGV", "BASH_CMDS", "BASH_COMMAND", "BASH_EXECUTION_STRING", + "BASH_LINENO", "BASH_REMATCH", "BASH_SOURCE", "BASH_SUBSHELL", + "BASH_VERSINFO", "BASH_VERSION", "COMP_CWORD", "COMP_KEY", + "COMP_LINE", "COMP_POINT", "COMP_TYPE", "COMP_WORDBREAKS", + "COMP_WORDS", "COPROC", "DIRSTACK", "EUID", "FUNCNAME", "GROUPS", + "HISTCMD", "HOSTNAME", "HOSTTYPE", "LINENO", "MACHTYPE", "MAPFILE", + "OLDPWD", "OPTARG", "OPTIND", "OSTYPE", "PIPESTATUS", "PPID", "PWD", + "RANDOM", "READLINE_LINE", "READLINE_POINT", "REPLY", "SECONDS", + "SHELLOPTS", "SHLVL", "UID", "BASH_ENV", "BASH_XTRACEFD", "CDPATH", + "COLUMNS", "COMPREPLY", "EMACS", "ENV", "FCEDIT", "FIGNORE", + "FUNCNEST", "GLOBIGNORE", "HISTCONTROL", "HISTFILE", "HISTFILESIZE", + "HISTIGNORE", "HISTSIZE", "HISTTIMEFORMAT", "HOME", "HOSTFILE", "IFS", + "IGNOREEOF", "INPUTRC", "LANG", "LC_ALL", "LC_COLLATE", "LC_CTYPE", + "LC_MESSAGES", "LC_NUMERIC", "LINES", "MAIL", "MAILCHECK", "MAILPATH", + "OPTERR", "PATH", "POSIXLY_CORRECT", "PROMPT_COMMAND", + "PROMPT_DIRTRIM", "PS1", "PS2", "PS3", "PS4", "SHELL", "TIMEFORMAT", + "TMOUT", "TMPDIR", "auto_resume", "histchars", + + -- Zsh + "ARGV0", "BAUD", "cdpath", "COLUMNS", "CORRECT_IGNORE", + "DIRSTACKSIZE", "ENV", "FCEDIT", "fignore", "fpath", "histchars", + "HISTCHARS", "HISTFILE", "HISTSIZE", "HOME", "IFS", "KEYBOARD_HACK", + "KEYTIMEOUT", "LANG", "LC_ALL", "LC_COLLATE", "LC_CTYPE", + "LC_MESSAGES", "LC_NUMERIC", "LC_TIME", "LINES", "LISTMAX", + "LOGCHECK", "MAIL", "MAILCHECK", "mailpath", "manpath", "module_path", + "NULLCMD", "path", "POSTEDIT", "PROMPT", "PROMPT2", "PROMPT3", + "PROMPT4", "prompt", "PROMPT_EOL_MARK", "PS1", "PS2", "PS3", "PS4", + "psvar", "READNULLCMD", "REPORTTIME", "REPLY", "reply", "RPROMPT", + "RPS1", "RPROMPT2", "RPS2", "SAVEHIST", "SPROMPT", "STTY", "TERM", + "TERMINFO", "TIMEFMT", "TMOUT", "TMPPREFIX", "watch", "WATCHFMT", + "WORDCHARS", "ZBEEP", "ZDOTDIR", "ZLE_LINE_ABORTED", + "ZLE_REMOVE_SUFFIX_CHARS", "ZLE_SPACE_SUFFIX_CHARS" + ] + +variablesWithoutSpaces = [ + "$", "-", "?", "!", + "BASHPID", "BASH_ARGC", "BASH_LINENO", "BASH_SUBSHELL", "EUID", "LINENO", + "OPTIND", "PPID", "RANDOM", "SECONDS", "SHELLOPTS", "SHLVL", "UID", + "COLUMNS", "HISTFILESIZE", "HISTSIZE", "LINES" + ] + +commonCommands = [ + "admin", "alias", "ar", "asa", "at", "awk", "basename", "batch", + "bc", "bg", "break", "c99", "cal", "cat", "cd", "cflow", "chgrp", + "chmod", "chown", "cksum", "cmp", "colon", "comm", "command", + "compress", "continue", "cp", "crontab", "csplit", "ctags", "cut", + "cxref", "date", "dd", "delta", "df", "diff", "dirname", "dot", + "du", "echo", "ed", "env", "eval", "ex", "exec", "exit", "expand", + "export", "expr", "fc", "fg", "file", "find", "fold", "fort77", + "fuser", "gencat", "get", "getconf", "getopts", "grep", "hash", + "head", "iconv", "ipcrm", "ipcs", "jobs", "join", "kill", "lex", + "link", "ln", "locale", "localedef", "logger", "logname", "lp", + "ls", "m4", "mailx", "make", "man", "mesg", "mkdir", "mkfifo", + "more", "mv", "newgrp", "nice", "nl", "nm", "nohup", "od", "paste", + "patch", "pathchk", "pax", "pr", "printf", "prs", "ps", "pwd", + "qalter", "qdel", "qhold", "qmove", "qmsg", "qrerun", "qrls", + "qselect", "qsig", "qstat", "qsub", "read", "readonly", "renice", + "return", "rm", "rmdel", "rmdir", "sact", "sccs", "sed", "set", + "sh", "shift", "sleep", "sort", "split", "strings", "strip", "stty", + "tabs", "tail", "talk", "tee", "test", "time", "times", "touch", + "tput", "tr", "trap", "tsort", "tty", "type", "ulimit", "umask", + "unalias", "uname", "uncompress", "unexpand", "unget", "uniq", + "unlink", "unset", "uucp", "uudecode", "uuencode", "uustat", "uux", + "val", "vi", "wait", "wc", "what", "who", "write", "xargs", "yacc", + "zcat" + ] diff --git a/ShellCheck/Parser.hs b/ShellCheck/Parser.hs new file mode 100644 --- /dev/null +++ b/ShellCheck/Parser.hs @@ -0,0 +1,1919 @@ +{- + This file is part of ShellCheck. + http://www.vidarholen.net/contents/shellcheck + + ShellCheck is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + ShellCheck is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see . +-} +{-# LANGUAGE NoMonomorphismRestriction #-} + +module ShellCheck.Parser (Note(..), Severity(..), parseShell, ParseResult(..), ParseNote(..), notesFromMap, Metadata(..), sortNotes) where + +import ShellCheck.AST +import ShellCheck.Data +import Text.Parsec +import Debug.Trace +import Control.Monad +import Data.Char +import Data.List (isPrefixOf, isInfixOf, isSuffixOf, partition, sortBy, intercalate, nub) +import qualified Data.Map as Map +import qualified Control.Monad.State as Ms +import Data.Maybe +import Prelude hiding (readList) +import System.IO +import Text.Parsec.Error +import GHC.Exts (sortWith) + +lastError = 1074 + +backslash = char '\\' +linefeed = (optional carriageReturn) >> char '\n' +singleQuote = char '\'' <|> unicodeSingleQuote +doubleQuote = char '"' <|> unicodeDoubleQuote +variableStart = upper <|> lower <|> oneOf "_" +variableChars = upper <|> lower <|> digit <|> oneOf "_" +functionChars = variableChars <|> oneOf ":+-.?" +specialVariable = oneOf "@*#?-$!" +tokenDelimiter = oneOf "&|;<> \t\n\r" <|> nbsp +quotableChars = "|&;<>()\\ '\t\n\r\xA0" ++ doubleQuotableChars +quotable = nbsp <|> unicodeDoubleQuote <|> oneOf quotableChars +bracedQuotable = oneOf "}\"$`'" +doubleQuotableChars = "\"$`\x201C\x201D" +doubleQuotable = unicodeDoubleQuote <|> oneOf doubleQuotableChars +whitespace = oneOf " \t\n" <|> carriageReturn <|> nbsp +linewhitespace = oneOf " \t" <|> nbsp + +extglobStartChars = "?*@!+" +extglobStart = oneOf extglobStartChars + +prop_spacing = isOk spacing " \\\n # Comment" +spacing = do + x <- many (many1 linewhitespace <|> (try $ string "\\\n")) + optional readComment + return $ concat x + +prop_allspacing = isOk allspacing "#foo" +prop_allspacing2 = isOk allspacing " #foo\n # bar\n#baz\n" +prop_allspacing3 = isOk allspacing "#foo\n#bar\n#baz\n" +allspacing = do + s <- spacing + more <- option False (linefeed >> return True) + if more then do + rest <- allspacing + return $ s ++ "\n" ++ rest + else + return s + +allspacingOrFail = do + s <- allspacing + when (null s) $ fail "Expected spaces" + +unicodeDoubleQuote = do + pos <- getPosition + char '\x201C' <|> char '\x201D' + parseProblemAt pos WarningC 1015 "This is a unicode double quote. Delete and retype it." + return '"' + +unicodeSingleQuote = do + pos <- getPosition + char '\x2018' <|> char '\x2019' + parseProblemAt pos WarningC 1016 "This is a unicode single quote. Delete and retype it." + return '"' + +carriageReturn = do + parseNote ErrorC 1017 "Literal carriage return. Run script through tr -d '\\r' ." + char '\r' + +nbsp = do + parseNote ErrorC 1018 "This is a  . Delete it and retype as space." + char '\xA0' + return ' ' + +--------- Message/position annotation on top of user state +data Note = Note Severity Code String deriving (Show, Eq) +data ParseNote = ParseNote SourcePos Severity Code String deriving (Show, Eq) +data Metadata = Metadata SourcePos [Note] deriving (Show) +data Severity = ErrorC | WarningC | InfoC | StyleC deriving (Show, Eq, Ord) +data Context = ContextName SourcePos String | ContextAnnotation [Annotation] +type Code = Integer + +codeForNote (Note _ code _) = code +codeForParseNote (ParseNote _ _ code _) = code + +initialState = (Id $ -1, Map.empty, []) + +getInitialMeta pos = Metadata pos [] + +getLastId = do + (id, _, _) <- getState + return id + +getNextIdAt sourcepos = do + (id, map, notes) <- getState + let newId = incId id + let newMap = Map.insert newId (getInitialMeta sourcepos) map + putState (newId, newMap, notes) + return newId + where incId (Id n) = (Id $ n+1) + +getNextId = do + pos <- getPosition + getNextIdAt pos + +modifyMap f = do + (id, map, parsenotes) <- getState + putState (id, f map, parsenotes) + +getMap = do + (_, map, _) <- getState + return map + +getParseNotes = do + (_, _, notes) <- getState + return notes + +addParseNote n = do + irrelevant <- shouldIgnoreCode (codeForParseNote n) + when (not irrelevant) $ do + (a, b, notes) <- getState + putState (a, b, n:notes) + +shouldIgnoreCode code = do + context <- getCurrentContexts + return $ any disabling context + where + disabling (ContextAnnotation list) = + any disabling' list + disabling _ = False + disabling' (DisableComment n) = code == n + +-- Store potential parse problems outside of parsec +parseProblem level code msg = do + pos <- getPosition + parseProblemAt pos level code msg + +setCurrentContexts c = do + Ms.modify (\(list, _) -> (list, c)) + +getCurrentContexts = do + (_, context) <- Ms.get + return context + +popContext = do + v <- getCurrentContexts + if not $ null v + then do + let (a:r) = v + setCurrentContexts r + return [a] + else + return [] + +pushContext c = do + v <- getCurrentContexts + setCurrentContexts (c:v) + +parseProblemAt pos level code msg = do + irrelevant <- shouldIgnoreCode code + when (not irrelevant) $ + Ms.modify (\(list, current) -> ((ParseNote pos level code msg):list, current)) + +-- Store non-parse problems inside +addNoteFor id note = modifyMap $ Map.adjust (\(Metadata pos notes) -> Metadata pos (note:notes)) id + +addNote note = do + id <- getLastId + addNoteFor id note + +parseNote c l a = do + pos <- getPosition + parseNoteAt pos c l a + +parseNoteAt pos c l a = addParseNote $ ParseNote pos c l a + +--------- Convenient combinators +thenSkip main follow = do + r <- main + optional follow + return r + +unexpecting s p = try $ do + (try p >> unexpected s) <|> return () + +notFollowedBy2 = unexpecting "keyword/token" + +disregard x = x >> return () + +reluctantlyTill p end = do + (lookAhead ((disregard $ try end) <|> eof) >> return []) <|> do + x <- p + more <- reluctantlyTill p end + return $ x:more + <|> return [] + +reluctantlyTill1 p end = do + notFollowedBy2 end + x <- p + more <- reluctantlyTill p end + return $ x:more + +attempting rest branch = do + ((try branch) >> rest) <|> rest + +orFail parser stuff = do + try (disregard parser) <|> (disregard stuff >> fail "nope") + +wasIncluded p = option False (p >> return True) + +acceptButWarn parser level code note = do + optional $ try (do + pos <- getPosition + parser + parseProblemAt pos level code note + ) + +withContext entry p = do + pushContext entry + do + v <- p + popContext + return v + <|> do -- p failed without consuming input, abort context + popContext + fail $ "" + +called s p = do + pos <- getPosition + withContext (ContextName pos s) p + +withAnnotations anns p = + withContext (ContextAnnotation anns) p + +readConditionContents single = do + readCondContents `attempting` (lookAhead $ do + pos <- getPosition + s <- many1 letter + when (s `elem` commonCommands) $ + parseProblemAt pos WarningC 1009 "Use 'if cmd; then ..' to check exit code, or 'if [[ $(cmd) == .. ]]' to check output.") + + where + typ = if single then SingleBracket else DoubleBracket + readCondBinaryOp = try $ do + id <- getNextId + op <- (choice $ (map tryOp ["==", "!=", "<=", ">=", "=~", ">", "<", "=", "\\<=", "\\>=", "\\<", "\\>"])) <|> otherOp + hardCondSpacing + return op + where + tryOp s = try $ do + id <- getNextId + string s + return $ TC_Binary id typ s + otherOp = try $ do + id <- getNextId + s <- readOp + when (s == "-a" || s == "-o") $ fail "Wrong operator" + return $ TC_Binary id typ s + + readCondUnaryExp = do + op <- readCondUnaryOp + pos <- getPosition + (do + arg <- readCondWord + return $ op arg) + <|> (do + parseProblemAt pos ErrorC 1019 $ "Expected this to be an argument to the unary condition." + fail "oops") + + readCondUnaryOp = try $ do + id <- getNextId + s <- readOp + hardCondSpacing + return $ TC_Unary id typ s + + readOp = try $ do + char '-' + s <- many1 letter + return ('-':s) + + readCondWord = do + notFollowedBy2 (try (spacing >> (string "]"))) + x <- readNormalWord + pos <- getPosition + when (endedWith "]" x) $ do + parseProblemAt pos ErrorC 1020 $ + "You need a space before the " ++ (if single then "]" else "]]") ++ "." + when (single && endedWith ")" x) $ do + parseProblemAt pos ErrorC 1021 $ + "You need a space before the \\)" + disregard spacing + return x + where endedWith str (T_NormalWord id s@(_:_)) = + case (last s) of T_Literal id s -> str `isSuffixOf` s + _ -> False + endedWith _ _ = False + + readCondAndOp = do + id <- getNextId + x <- try (string "&&" <|> string "-a") + when (single && x == "&&") $ addNoteFor id $ Note ErrorC 1022 "You can't use && inside [..]. Use [[..]] instead." + when (not single && x == "-a") $ addNoteFor id $ Note ErrorC 1023 "In [[..]], use && instead of -a." + softCondSpacing + return $ TC_And id typ x + + readCondOrOp = do + id <- getNextId + x <- try (string "||" <|> string "-o") + when (single && x == "||") $ addNoteFor id $ Note ErrorC 1024 "You can't use || inside [..]. Use [[..]] instead." + when (not single && x == "-o") $ addNoteFor id $ Note ErrorC 1025 "In [[..]], use || instead of -o." + softCondSpacing + return $ TC_Or id typ x + + readCondNoaryOrBinary = do + id <- getNextId + x <- readCondWord `attempting` (do + pos <- getPosition + lookAhead (char '[') + parseProblemAt pos ErrorC 1026 $ if single + then "If grouping expressions inside [..], use \\( ..\\)." + else "If grouping expressions inside [[..]], use ( .. )." + ) + (do + pos <- getPosition + isRegex <- regexOperatorAhead + op <- readCondBinaryOp + y <- if isRegex + then readRegex + else readCondWord <|> ( (parseProblemAt pos ErrorC 1027 $ "Expected another argument for this operator.") >> mzero) + return (x `op` y) + ) <|> (return $ TC_Noary id typ x) + + readCondGroup = do + id <- getNextId + pos <- getPosition + lparen <- try $ string "(" <|> string "\\(" + when (single && lparen == "(") $ parseProblemAt pos ErrorC 1028 "In [..] you have to escape (). Use [[..]] instead." + when (not single && lparen == "\\(") $ parseProblemAt pos ErrorC 1029 "In [[..]] you shouldn't escape ()." + if single then hardCondSpacing else disregard spacing + x <- readCondContents + cpos <- getPosition + rparen <- string ")" <|> string "\\)" + if single then hardCondSpacing else disregard spacing + when (single && rparen == ")") $ parseProblemAt cpos ErrorC 1030 "In [..] you have to escape (). Use [[..]] instead." + when (not single && rparen == "\\)") $ parseProblemAt cpos ErrorC 1031 "In [[..]] you shouldn't escape ()." + when (isEscaped lparen `xor` isEscaped rparen) $ parseProblemAt pos ErrorC 1032 "Did you just escape one half of () but not the other?" + return $ TC_Group id typ x + where + isEscaped ('\\':_) = True + isEscaped _ = False + xor x y = x && not y || not x && y + + -- Currently a bit of a hack since parsing rules are obscure + regexOperatorAhead = (lookAhead $ do + try (string "=~") <|> try (string "~=") + return True) + <|> return False + readRegex = called "regex" $ do + id <- getNextId + parts <- many1 ( + readGroup <|> + readSingleQuoted <|> + readDoubleQuoted <|> + readDollarExpression <|> + readNormalLiteral "( " <|> + readPipeLiteral <|> + readGlobLiteral) + disregard spacing + return $ T_NormalWord id parts + where + readGlobLiteral = do + id <- getNextId + s <- many1 (extglobStart <|> oneOf "[]$") + return $ T_Literal id s + readGroup = called "regex grouping" $ do + id <- getNextId + char '(' + parts <- many (readGroup <|> readSingleQuoted <|> readDoubleQuoted <|> readDollarExpression <|> readRegexLiteral <|> readGlobLiteral) + char ')' + return $ T_NormalWord id parts + readRegexLiteral = do + id <- getNextId + str <- readGenericLiteral1 (singleQuote <|> doubleQuotable <|> oneOf "()") + return $ T_Literal id str + readPipeLiteral = do + id <- getNextId + str <- string "|" + return $ T_Literal id str + + readCondTerm = readCondNot <|> readCondExpr + readCondNot = do + id <- getNextId + char '!' + softCondSpacing + expr <- readCondExpr + return $ TC_Unary id typ "!" expr + + readCondExpr = + readCondGroup <|> readCondUnaryExp <|> readCondNoaryOrBinary + + readCondOr = chainl1 readCondAnd readCondAndOp + readCondAnd = chainl1 readCondTerm readCondOrOp + readCondContents = readCondOr + + + +prop_a1 = isOk readArithmeticContents " n++ + ++c" +prop_a2 = isOk readArithmeticContents "$N*4-(3,2)" +prop_a3 = isOk readArithmeticContents "n|=2<<1" +prop_a4 = isOk readArithmeticContents "n &= 2 **3" +prop_a5 = isOk readArithmeticContents "1 |= 4 && n >>= 4" +prop_a6 = isOk readArithmeticContents " 1 | 2 ||3|4" +prop_a7 = isOk readArithmeticContents "3*2**10" +prop_a8 = isOk readArithmeticContents "3" +prop_a9 = isOk readArithmeticContents "a^!-b" +prop_aA = isOk readArithmeticContents "! $?" +prop_aB = isOk readArithmeticContents "10#08 * 16#f" +prop_aC = isOk readArithmeticContents "\"$((3+2))\" + '37'" +prop_aD = isOk readArithmeticContents "foo[9*y+x]++" +readArithmeticContents = + readSequence + where + spacing = many whitespace + + splitBy x ops = chainl1 x (readBinary ops) + readBinary ops = readComboOp ops TA_Binary + readComboOp op token = do + id <- getNextId + op <- choice (map (\x -> try $ do + s <- string x + notFollowedBy2 $ oneOf "&|<>=" + return s + ) op) + spacing + return $ token id op + + readVar = do + id <- getNextId + x <- readVariableName + y <- readArrayIndex <|> return "" + optional spacing + return $ TA_Variable id (x ++ y) + + -- Doesn't help with foo[foo] + readArrayIndex = do + char '[' + x <- many1 $ noneOf "]" + char ']' + return $ "[" ++ x ++ "]" + + readExpansion = do + id <- getNextId + x <- readNormalDollar + spacing + return $ TA_Expansion id x + + readGroup = do + char '(' + s <- readSequence + char ')' + spacing + return s + + readNumber = do + id <- getNextId + num <- many1 $ oneOf "0123456789." + return $ TA_Literal id (num) + + readBased = getArbitrary <|> getHex <|> getOct + where + getThing prefix litchars = try $ do + id <- getNextId + x <- prefix + t <- readExpansion <|> (do + i <- getNextId + stuff <- many1 litchars + return $ TA_Literal i stuff) + return $ TA_Base id x t + + getArbitrary = getThing arbitrary variableChars + getHex = getThing hex hexDigit + getOct = getThing oct digit + + arbitrary = try $ do + b <- many1 digit + s <- char '#' + return (b ++ [s]) + hex = try $ do + z <- char '0' + x <- oneOf "xX" + return (z:x:[]) + oct = string "0" + + readArithTerm = readBased <|> readArithTermUnit + readArithTermUnit = readGroup <|> readExpansion <|> readQuoted <|> readNumber <|> readVar + + readQuoted = readDoubleQuoted <|> readSingleQuoted + + readSequence = do + spacing + id <- getNextId + l <- readAssignment `sepBy` (char ',' >> spacing) + return $ TA_Sequence id l + + readAssignment = readTrinary `splitBy` ["=", "*=", "/=", "%=", "+=", "-=", "<<=", ">>=", "&=", "^=", "|="] + readTrinary = do + let part = readLogicalOr + x <- part + do + id <- getNextId + string "?" + spacing + y <- part + string ":" + spacing + z <- part + return $ TA_Trinary id x y z + <|> + return x + + readLogicalOr = readLogicalAnd `splitBy` ["||"] + readLogicalAnd = readBitOr `splitBy` ["&&"] + readBitOr = readBitXor `splitBy` ["|"] + readBitXor = readBitAnd `splitBy` ["^"] + readBitAnd = readEquated `splitBy` ["&"] + readEquated = readCompared `splitBy` ["==", "!="] + readCompared = readShift `splitBy` ["<=", ">=", "<", ">"] + readShift = readAddition `splitBy` ["<<", ">>"] + readAddition = readMultiplication `splitBy` ["+", "-"] + readMultiplication = readExponential `splitBy` ["*", "/", "%"] + readExponential = readAnyNegated `splitBy` ["**"] + + readAnyNegated = readNegated <|> readAnySigned + readNegated = do + id <- getNextId + op <- oneOf "!~" + spacing + x <- readAnySigned + return $ TA_Unary id [op] x + + readAnySigned = readSigned <|> readAnycremented + readSigned = do + id <- getNextId + op <- choice (map readSignOp "+-") + spacing + x <- readAnycremented + return $ TA_Unary id [op] x + where + readSignOp c = try $ do + char c + notFollowedBy2 $ char c + spacing + return c + + readAnycremented = readNormalOrPostfixIncremented <|> readPrefixIncremented + readPrefixIncremented = do + id <- getNextId + op <- try $ string "++" <|> string "--" + spacing + x <- readArithTerm + return $ TA_Unary id (op ++ "|") x + + readNormalOrPostfixIncremented = do + x <- readArithTerm + spacing + do + id <- getNextId + op <- try $ string "++" <|> string "--" + spacing + return $ TA_Unary id ("|" ++ op) x + <|> + return x + + + +prop_readCondition = isOk readCondition "[ \\( a = b \\) -a \\( c = d \\) ]" +prop_readCondition2 = isOk readCondition "[[ (a = b) || (c = d) ]]" +prop_readCondition3 = isOk readCondition "[[ $c = [[:alpha:].~-] ]]" +prop_readCondition4 = isOk readCondition "[[ $c =~ *foo* ]]" +prop_readCondition5 = isOk readCondition "[[ $c =~ f( ]] )* ]]" +prop_readCondition5a= isOk readCondition "[[ $c =~ a(b) ]]" +prop_readCondition5b= isOk readCondition "[[ $c =~ f( ($var ]]) )* ]]" +prop_readCondition6 = isOk readCondition "[[ $c =~ ^[yY]$ ]]" +prop_readCondition7 = isOk readCondition "[[ ${line} =~ ^[[:space:]]*# ]]" +prop_readCondition8 = isOk readCondition "[[ $l =~ ogg|flac ]]" +prop_readCondition9 = isOk readCondition "[ foo -a -f bar ]" +readCondition = called "test expression" $ do + opos <- getPosition + id <- getNextId + open <- (try $ string "[[") <|> (string "[") + let single = open == "[" + condSpacingMsg False $ if single + then "You need spaces after the opening [ and before the closing ]." + else "You need spaces after the opening [[ and before the closing ]]." + condition <- readConditionContents single + + cpos <- getPosition + close <- (try $ string "]]") <|> (string "]") + when (open == "[[" && close /= "]]") $ parseProblemAt cpos ErrorC 1033 "Did you mean ]] ?" + when (open == "[" && close /= "]" ) $ parseProblemAt opos ErrorC 1034 "Did you mean [[ ?" + spacing + many readCmdWord -- Read and throw away remainders to get then/do warnings. Fixme? + return $ T_Condition id (if single then SingleBracket else DoubleBracket) condition + + +hardCondSpacing = condSpacingMsg False "You need a space here." +softCondSpacing = condSpacingMsg True "You need a space here." +condSpacingMsg soft msg = do + pos <- getPosition + space <- spacing + when (null space) $ (if soft then parseNoteAt else parseProblemAt) pos ErrorC 1035 msg + +readAnnotationPrefix = do + char '#' + many linewhitespace + string "shellcheck" + +prop_readAnnotation1 = isOk readAnnotation "# shellcheck disable=1234,5678\n" +prop_readAnnotation2 = isOk readAnnotation "# shellcheck disable=SC1234 disable=SC5678\n" +readAnnotation = called "shellcheck annotation" $ do + try readAnnotationPrefix + many1 linewhitespace + values <- many1 (readDisable) + linefeed + many linewhitespace + return $ concat values + where + readDisable = forKey "disable" $ do + readCode `sepBy` char ',' + where + readCode = do + optional $ string "SC" + int <- many1 digit + return $ DisableComment (read int) + forKey s p = do + try $ string s + char '=' + value <- p + many linewhitespace + return value + +readAnnotations = do + annotations <- many (readAnnotation `thenSkip` allspacing) + return $ concat annotations + +readComment = do + unexpecting "shellcheck annotation" readAnnotationPrefix + char '#' + many $ noneOf "\r\n" + +prop_readNormalWord = isOk readNormalWord "'foo'\"bar\"{1..3}baz$(lol)" +prop_readNormalWord2 = isOk readNormalWord "foo**(foo)!!!(@@(bar))" +prop_readNormalWord3 = isOk readNormalWord "foo#" +prop_readNormalWord4 = isOk readNormalWord "$\"foo\"$'foo\nbar'" +readNormalWord = readNormalishWord "" + +readNormalishWord end = do + id <- getNextId + pos <- getPosition + x <- many1 (readNormalWordPart end) + checkPossibleTermination pos x + return $ T_NormalWord id x + +checkPossibleTermination pos [T_Literal _ x] = + if x `elem` ["do", "done", "then", "fi", "esac", "}"] + then parseProblemAt pos WarningC 1010 $ "Use semicolon or linefeed before '" ++ x ++ "' (or quote to make it literal)." + else return () +checkPossibleTermination _ _ = return () + +readNormalWordPart end = do + checkForParenthesis + readSingleQuoted <|> readDoubleQuoted <|> readGlob <|> readNormalDollar <|> readBraced <|> readBackTicked <|> readProcSub <|> (readNormalLiteral end) + where + checkForParenthesis = do + return () `attempting` do + pos <- getPosition + lookAhead $ char '(' + parseProblemAt pos ErrorC 1036 "'(' is invalid here. Did you forget to escape it?" + + +readSpacePart = do + id <- getNextId + x <- many1 whitespace + return $ T_Literal id x + +readDollarBracedWord = do + id <- getNextId + list <- many readDollarBracedPart + return $ T_NormalWord id list + +readDollarBracedPart = readSingleQuoted <|> readDoubleQuoted <|> readExtglob <|> readNormalDollar <|> readBackTicked <|> readDollarBracedLiteral + +readDollarBracedLiteral = do + id <- getNextId + vars <- (readBraceEscaped <|> (anyChar >>= \x -> return [x])) `reluctantlyTill1` bracedQuotable + return $ T_Literal id $ concat vars + +prop_readProcSub1 = isOk readProcSub "<(echo test | wc -l)" +prop_readProcSub2 = isOk readProcSub "<( if true; then true; fi )" +readProcSub = called "process substitution" $ do + id <- getNextId + dir <- try $ do + x <- oneOf "<>" + char '(' + return [x] + allspacing + list <- readCompoundList + allspacing + char ')' + return $ T_ProcSub id dir list + +prop_readSingleQuoted = isOk readSingleQuoted "'foo bar'" +prop_readSingleQuoted2 = isWarning readSingleQuoted "'foo bar\\'" +prop_readsingleQuoted3 = isWarning readSingleQuoted "\x2018hello\x2019" +readSingleQuoted = called "single quoted string" $ do + id <- getNextId + singleQuote + s <- readSingleQuotedPart `reluctantlyTill` singleQuote + pos <- getPosition + singleQuote "end of single quoted string" + + let string = concat s + return (T_SingleQuoted id string) `attempting` do + x <- lookAhead anyChar + when (isAlpha x && not (null string) && isAlpha (last string)) $ parseProblemAt pos WarningC 1011 "This apostrophe terminated the single quoted string!" + +readSingleQuotedLiteral = do + singleQuote + strs <- many1 readSingleQuotedPart + singleQuote + return $ concat strs + +readSingleQuotedPart = + readSingleEscaped + <|> (many1 $ noneOf "'\\\x2018\x2019") + +prop_readBackTicked = isOk readBackTicked "`ls *.mp3`" +prop_readBackTicked2 = isOk readBackTicked "`grep \"\\\"\"`" +readBackTicked = called "backtick expansion" $ do + id <- getNextId + pos <- getPosition + char '`' + subStart <- getPosition + subString <- readGenericLiteral "`" + char '`' + -- Result positions may be off due to escapes + result <- subParse subStart readCompoundList (unEscape subString) + return $ T_Backticked id result + where + unEscape [] = [] + unEscape ('\\':x:rest) | x `elem` "$`\\" = x : unEscape rest + unEscape ('\\':'\n':rest) = unEscape rest + unEscape (c:rest) = c : unEscape rest + +subParse pos parser input = do + lastPosition <- getPosition + lastInput <- getInput + setPosition pos + setInput input + result <- parser + setInput lastInput + setPosition lastPosition + return result + +prop_readDoubleQuoted = isOk readDoubleQuoted "\"Hello $FOO\"" +prop_readDoubleQuoted2 = isOk readDoubleQuoted "\"$'\"" +prop_readDoubleQuoted3 = isWarning readDoubleQuoted "\x201Chello\x201D" +readDoubleQuoted = called "double quoted string" $ do + id <- getNextId + doubleQuote + x <- many doubleQuotedPart + doubleQuote "end of double quoted string" + return $ T_DoubleQuoted id x + +doubleQuotedPart = readDoubleLiteral <|> readDoubleQuotedDollar <|> readBackTicked + +readDoubleQuotedLiteral = do + doubleQuote + x <- readDoubleLiteral + doubleQuote + return x + +readDoubleLiteral = do + id <- getNextId + s <- many1 readDoubleLiteralPart + return $ T_Literal id (concat s) + +readDoubleLiteralPart = do + x <- many1 $ (readDoubleEscaped <|> (many1 $ noneOf ('\\':doubleQuotableChars))) + return $ concat x + +readNormalLiteral end = do + id <- getNextId + s <- many1 (readNormalLiteralPart end) + return $ T_Literal id (concat s) + +prop_readGlob1 = isOk readGlob "*" +prop_readGlob2 = isOk readGlob "[^0-9]" +prop_readGlob3 = isOk readGlob "[a[:alpha:]]" +prop_readGlob4 = isOk readGlob "[[:alnum:]]" +prop_readGlob5 = isOk readGlob "[^[:alpha:]1-9]" +readGlob = readExtglob <|> readSimple <|> readClass <|> readGlobbyLiteral + where + readSimple = do + id <- getNextId + c <- oneOf "*?" + return $ T_Glob id [c] + -- Doesn't handle weird things like [^]a] and [$foo]. fixme? + readClass = try $ do + id <- getNextId + char '[' + s <- many1 (predefined <|> (liftM return $ letter <|> digit <|> oneOf globchars)) + char ']' + return $ T_Glob id $ "[" ++ (concat s) ++ "]" + where + globchars = "^-_:?*.,!~@#$%=+{}/~" + predefined = do + try $ string "[:" + s <- many1 letter + string ":]" + return $ "[:" ++ s ++ ":]" + + readGlobbyLiteral = do + id <- getNextId + c <- extglobStart <|> char '[' + return $ T_Literal id [c] + +readNormalLiteralPart end = do + readNormalEscaped <|> (many1 $ noneOf (end ++ quotableChars ++ extglobStartChars ++ "[")) + +readNormalEscaped = called "escaped char" $ do + pos <- getPosition + backslash + do + next <- (quotable <|> oneOf "?*@!+[]{}.,") + return $ if next == '\n' then "" else [next] + <|> + do + next <- anyChar + case escapedChar next of + Just name -> parseNoteAt pos WarningC 1012 $ "\\" ++ [next] ++ " is just literal '" ++ [next] ++ "' here. For " ++ name ++ ", use \"$(printf \"\\" ++ [next] ++ "\")\"." + Nothing -> parseNoteAt pos InfoC 1001 $ "This \\" ++ [next] ++ " will be a regular '" ++ [next] ++ "' in this context." + return [next] + where + escapedChar 'n' = Just "line feed" + escapedChar 't' = Just "tab" + escapedChar 'r' = Just "carriage return" + escapedChar _ = Nothing + + +prop_readExtglob1 = isOk readExtglob "!(*.mp3)" +prop_readExtglob2 = isOk readExtglob "!(*.mp3|*.wmv)" +prop_readExtglob4 = isOk readExtglob "+(foo \\) bar)" +prop_readExtglob5 = isOk readExtglob "+(!(foo *(bar)))" +prop_readExtglob6 = isOk readExtglob "*(((||))|())" +prop_readExtglob7 = isOk readExtglob "*(<>)" +prop_readExtglob8 = isOk readExtglob "@(|*())" +readExtglob = called "extglob" $ do + id <- getNextId + c <- try $ do + f <- extglobStart + char '(' + return f + contents <- readExtglobPart `sepBy` (char '|') + char ')' + return $ T_Extglob id [c] contents + +readExtglobPart = do + id <- getNextId + x <- many (readExtglobGroup <|> readNormalWordPart "" <|> readSpacePart <|> readExtglobLiteral) + return $ T_NormalWord id x + where + readExtglobGroup = do + id <- getNextId + char '(' + contents <- readExtglobPart `sepBy` (char '|') + char ')' + return $ T_Extglob id "" contents + readExtglobLiteral = do + id <- getNextId + str <- many1 (oneOf "<>#;&") + return $ T_Literal id str + + +readSingleEscaped = do + s <- backslash + let attempt level code p msg = do { try $ parseNote level code msg; x <- p; return [s,x]; } + + do { + x <- lookAhead singleQuote; + parseProblem InfoC 1003 "Are you trying to escape that single quote? echo 'You'\\''re doing it wrong'."; + return [s]; + } + <|> attempt InfoC 1004 linefeed "You don't break lines with \\ in single quotes, it results in literal backslash-linefeed." + <|> do + x <- anyChar + return [s,x] + + +readDoubleEscaped = do + bs <- backslash + (linefeed >> return "") + <|> (doubleQuotable >>= return . return) + <|> (anyChar >>= (return . \x -> [bs, x])) + +readBraceEscaped = do + bs <- backslash + (linefeed >> return "") + <|> (bracedQuotable >>= return . return) + <|> (anyChar >>= (return . \x -> [bs, x])) + + +readGenericLiteral endChars = do + strings <- many (readGenericEscaped <|> (many1 $ noneOf ('\\':endChars))) + return $ concat strings + +readGenericLiteral1 endExp = do + strings <- (readGenericEscaped <|> (anyChar >>= \x -> return [x])) `reluctantlyTill1` endExp + return $ concat strings + +readGenericEscaped = do + backslash + x <- anyChar + return $ if x == '\n' then [] else ['\\', x] + +prop_readBraced = isOk readBraced "{1..4}" +prop_readBraced2 = isOk readBraced "{foo,bar,\"baz lol\"}" +readBraced = try $ do + let strip (T_Literal _ s) = return ("\"" ++ s ++ "\"") + id <- getNextId + char '{' + str <- many1 ((readDoubleQuotedLiteral >>= (strip)) <|> readGenericLiteral1 (oneOf "}\"" <|> whitespace)) + char '}' + return $ T_BraceExpansion id $ concat str + +readNormalDollar = readDollarExpression <|> readDollarDoubleQuote <|> readDollarSingleQuote <|> readDollarLonely +readDoubleQuotedDollar = readDollarExpression <|> readDollarLonely +readDollarExpression = readDollarArithmetic <|> readDollarBracket <|> readDollarBraced <|> readDollarExpansion <|> readDollarVariable + +prop_readDollarSingleQuote = isOk readDollarSingleQuote "$'foo\\\'lol'" +readDollarSingleQuote = called "$'..' expression" $ do + id <- getNextId + try $ string "$'" + str <- readGenericLiteral "'" + char '\'' + return $ T_DollarSingleQuoted id str + +prop_readDollarDoubleQuote = isOk readDollarDoubleQuote "$\"hello\"" +readDollarDoubleQuote = do + lookAhead . try $ string "$\"" + id <- getNextId + char '$' + doubleQuote + x <- many doubleQuotedPart + doubleQuote "end of translated double quoted string" + return $ T_DollarDoubleQuoted id x + + +prop_readDollarArithmetic = isOk readDollarArithmetic "$(( 3 * 4 +5))" +prop_readDollarArithmetic2 = isOk readDollarArithmetic "$(((3*4)+(1*2+(3-1))))" +readDollarArithmetic = called "$((..)) expression" $ do + id <- getNextId + try (string "$((") + c <- readArithmeticContents + string "))" + return (T_DollarArithmetic id c) + +readDollarBracket = called "$[..] expression" $ do + id <- getNextId + try (string "$[") + c <- readArithmeticContents + string "]" + return (T_DollarBracket id c) + +readArithmeticExpression = called "((..)) command" $ do + id <- getNextId + try (string "((") + c <- readArithmeticContents + string "))" + return (T_Arithmetic id c) + +prop_readDollarBraced1 = isOk readDollarBraced "${foo//bar/baz}" +prop_readDollarBraced2 = isOk readDollarBraced "${foo/'{cow}'}" +prop_readDollarBraced3 = isOk readDollarBraced "${foo%%$(echo cow})}" +prop_readDollarBraced4 = isOk readDollarBraced "${foo#\\}}" +readDollarBraced = called "parameter expansion" $ do + id <- getNextId + try (string "${") + word <- readDollarBracedWord + char '}' + return $ T_DollarBraced id word + +prop_readDollarExpansion = isOk readDollarExpansion "$(echo foo; ls\n)" +readDollarExpansion = called "command expansion" $ do + id <- getNextId + try (string "$(") + cmds <- readCompoundList + char ')' "end of $(..) expression" + return $ (T_DollarExpansion id cmds) + +prop_readDollarVariable = isOk readDollarVariable "$@" +readDollarVariable = do + id <- getNextId + let singleCharred p = do + n <- p + value <- wrap [n] + return (T_DollarBraced id value) `attempting` do + pos <- getPosition + num <- lookAhead $ many1 p + parseNoteAt pos ErrorC 1037 $ "$" ++ (n:num) ++ " is equivalent to ${" ++ [n] ++ "}"++ num ++"." + + let positional = singleCharred digit + let special = singleCharred specialVariable + + let regular = do + name <- readVariableName + value <- wrap name + return $ T_DollarBraced id value + + try $ char '$' >> (positional <|> special <|> regular) + + where + wrap s = do + x <- getNextId + y <- getNextId + return $ T_NormalWord x [T_Literal y s] + +readVariableName = do + f <- variableStart + rest <- many variableChars + return (f:rest) + +readDollarLonely = do + id <- getNextId + pos <- getPosition + char '$' + n <- lookAhead (anyChar <|> (eof >> return '_')) + when (n /= '\'') $ parseNoteAt pos StyleC 1000 "$ is not used specially and should therefore be escaped." + return $ T_Literal id "$" + +prop_readHereDoc = isOk readHereDoc "<< foo\nlol\ncow\nfoo" +prop_readHereDoc2 = isWarning readHereDoc "<<- EOF\n cow\n EOF" +prop_readHereDoc3 = isOk readHereDoc "<< foo\n$\"\nfoo" +prop_readHereDoc4 = isOk readHereDoc "<< foo\n`\nfoo" +prop_readHereDoc5 = isOk readHereDoc "<<- !foo\nbar\n!foo" +prop_readHereDoc6 = isOk readHereDoc "<< foo\\ bar\ncow\nfoo bar" +readHereDoc = called "here document" $ do + fid <- getNextId + pos <- getPosition + try $ string "<<" + dashed <- (char '-' >> return Dashed) <|> return Undashed + tokenPosition <- getPosition + sp <- spacing + optional $ do + try . lookAhead $ char '(' + let message = "Shells are space sensitive. Use '< <(cmd)', not '<<" ++ sp ++ "(cmd)'." + parseProblemAt pos ErrorC 1038 message + hid <- getNextId + (quoted, endToken) <- + (readDoubleQuotedLiteral >>= return . (\x -> (Quoted, stripLiteral x))) + <|> (readSingleQuotedLiteral >>= return . (\x -> (Quoted, x))) + <|> (readToken >>= (\x -> return (Unquoted, x))) + spacing + + startPos <- getPosition + hereData <- anyChar `reluctantlyTill` do + linefeed + spacing + string endToken + disregard linefeed <|> eof + + do + linefeed + spaces <- spacing + verifyHereDoc dashed quoted spaces hereData + string endToken + parsedData <- parseHereData quoted startPos hereData + return $ T_FdRedirect fid "" $ T_HereDoc hid dashed quoted endToken parsedData + `attempting` (eof >> debugHereDoc tokenPosition endToken hereData) + + where + stripLiteral (T_Literal _ x) = x + stripLiteral (T_SingleQuoted _ x) = x + + readToken = do + liftM concat $ many1 (escaped <|> quoted <|> normal) + where + quoted = liftM stripLiteral readDoubleQuotedLiteral <|> readSingleQuotedLiteral + normal = anyChar `reluctantlyTill1` (whitespace <|> oneOf ";&)'\"\\") + escaped = do -- surely the user must be doing something wrong at this point + char '\\' + c <- anyChar + return [c] + + parseHereData Quoted startPos hereData = do + id <- getNextIdAt startPos + return $ [T_Literal id hereData] + + parseHereData Unquoted startPos hereData = do + subParse startPos readHereData hereData + + readHereData = many $ try readNormalDollar <|> try readBackTicked <|> readHereLiteral + + readHereLiteral = do + id <- getNextId + chars <- many1 $ noneOf "`$" + return $ T_Literal id chars + + verifyHereDoc dashed quoted spacing hereInfo = do + when (dashed == Undashed && spacing /= "") $ + parseNote ErrorC 1039 "Use <<- instead of << if you want to indent the end token." + when (dashed == Dashed && filter (/= '\t') spacing /= "" ) $ + parseNote ErrorC 1040 "When using <<-, you can only indent with tabs." + return () + + debugHereDoc pos endToken doc = + if endToken `isInfixOf` doc + then + let lookAt line = when (endToken `isInfixOf` line) $ + parseProblemAt pos ErrorC 1041 ("Close matches include '" ++ line ++ "' (!= '" ++ endToken ++ "').") + in do + parseProblemAt pos ErrorC 1042 ("Found '" ++ endToken ++ "' further down, but not entirely by itself.") + mapM_ lookAt (lines doc) + else if (map toLower endToken) `isInfixOf` (map toLower doc) + then parseProblemAt pos ErrorC 1043 ("Found " ++ endToken ++ " further down, but with wrong casing.") + else parseProblemAt pos ErrorC 1044 ("Couldn't find end token `" ++ endToken ++ "' in the here document.") + + +readFilename = readNormalWord +readIoFileOp = choice [g_LESSAND, g_GREATAND, g_DGREAT, g_LESSGREAT, g_CLOBBER, redirToken '<' T_Less, redirToken '>' T_Greater ] + +prop_readIoFile = isOk readIoFile ">> \"$(date +%YYmmDD)\"" +readIoFile = called "redirection" $ do + id <- getNextId + op <- readIoFileOp + spacing + file <- readFilename + return $ T_FdRedirect id "" $ T_IoFile id op file + +readIoNumber = try $ do + x <- many1 digit <|> string "&" + lookAhead readIoFileOp + return x + +prop_readIoNumberRedirect = isOk readIoNumberRedirect "3>&2" +prop_readIoNumberRedirect2 = isOk readIoNumberRedirect "2> lol" +prop_readIoNumberRedirect3 = isOk readIoNumberRedirect "4>&-" +prop_readIoNumberRedirect4 = isOk readIoNumberRedirect "&> lol" +readIoNumberRedirect = do + id <- getNextId + n <- readIoNumber + op <- readHereString <|> readHereDoc <|> readIoFile + let actualOp = case op of T_FdRedirect _ "" x -> x + spacing + return $ T_FdRedirect id n actualOp + +readIoRedirect = choice [ readIoNumberRedirect, readHereString, readHereDoc, readIoFile ] `thenSkip` spacing + +readRedirectList = many1 readIoRedirect + +prop_readHereString = isOk readHereString "<<< \"Hello $world\"" +readHereString = called "here string" $ do + id <- getNextId + try $ string "<<<" + spacing + id2 <- getNextId + word <- readNormalWord + return $ T_FdRedirect id "" $ T_HereString id2 word + +readNewlineList = many1 ((newline <|> carriageReturn) `thenSkip` spacing) +readLineBreak = optional readNewlineList + +prop_roflol = isWarning readScript "a &; b" +prop_roflol2 = isOk readScript "a & b" +readSeparatorOp = do + notFollowedBy2 (g_AND_IF <|> g_DSEMI) + notFollowedBy2 (string "&>") + f <- (try $ do + char '&' + spacing + pos <- getPosition + char ';' + parseProblemAt pos ErrorC 1045 "It's not 'foo &; bar', just 'foo & bar'." + return '&' + ) <|> char ';' <|> char '&' + spacing + return f + +readSequentialSep = (disregard $ g_Semi >> readLineBreak) <|> (disregard readNewlineList) +readSeparator = + do + separator <- readSeparatorOp + readLineBreak + return separator + <|> + do + readNewlineList + return '\n' + +makeSimpleCommand id1 id2 prefix cmd suffix = + let + (preAssigned, preRest) = partition assignment prefix + (preRedirected, preRest2) = partition redirection preRest + (postRedirected, postRest) = partition redirection suffix + + redirs = preRedirected ++ postRedirected + assigns = preAssigned + args = cmd ++ preRest2 ++ postRest + in + T_Redirecting id1 redirs $ T_SimpleCommand id2 assigns args + where + assignment (T_Assignment _ _ _ _ _) = True + assignment _ = False + redirection (T_FdRedirect _ _ _) = True + redirection _ = False + + +prop_readSimpleCommand = isOk readSimpleCommand "echo test > file" +prop_readSimpleCommand2 = isOk readSimpleCommand "cmd &> file" +prop_readSimpleCommand3 = isOk readSimpleCommand "export foo=(bar baz)" +prop_readSimpleCommand4 = isOk readSimpleCommand "typeset -a foo=(lol)" +prop_readSimpleCommand5 = isOk readSimpleCommand "time if true; then echo foo; fi" +prop_readSimpleCommand6 = isOk readSimpleCommand "time -p ( ls -l; )" +readSimpleCommand = called "simple command" $ do + id1 <- getNextId + id2 <- getNextId + prefix <- option [] readCmdPrefix + cmd <- option Nothing $ do { f <- readCmdName; return $ Just f; } + when (null prefix && isNothing cmd) $ fail "No command" + case cmd of + Nothing -> return $ makeSimpleCommand id1 id2 prefix [] [] + Just cmd -> do + suffix <- option [] $ + if isModifierCommand cmd + then readModifierSuffix + else if isTimeCommand cmd + then readTimeSuffix + else readCmdSuffix + return $ makeSimpleCommand id1 id2 prefix [cmd] suffix + where + isModifierCommand (T_NormalWord _ [T_Literal _ s]) = + s `elem` ["declare", "export", "local", "readonly", "typeset"] + isModifierCommand _ = False + -- Might not belong in T_SimpleCommand. Fixme? + isTimeCommand (T_NormalWord _ [T_Literal _ "time"]) = True + isTimeCommand _ = False + +prop_readPipeline = isOk readPipeline "! cat /etc/issue | grep -i ubuntu" +prop_readPipeline2 = isWarning readPipeline "!cat /etc/issue | grep -i ubuntu" +prop_readPipeline3 = isOk readPipeline "for f; do :; done|cat" +readPipeline = do + unexpecting "keyword/token" readKeyword + do + (T_Bang id) <- g_Bang + pipe <- readPipeSequence + return $ T_Banged id pipe + <|> do + readPipeSequence + +prop_readAndOr = isOk readAndOr "grep -i lol foo || exit 1" +prop_readAndOr1 = isOk readAndOr "# shellcheck disable=1\nfoo" +prop_readAndOr2 = isOk readAndOr "# shellcheck disable=1\n# lol\n# shellcheck disable=3\nfoo" +readAndOr = do + aid <- getNextId + annotations <- readAnnotations + + andOr <- withAnnotations annotations $ do + chainr1 readPipeline $ do + op <- g_AND_IF <|> g_OR_IF + readLineBreak + return $ case op of T_AND_IF id -> T_AndIf id + T_OR_IF id -> T_OrIf id + + return $ if null annotations + then andOr + else T_Annotation aid annotations andOr + +readTerm = do + allspacing + m <- readAndOr + readTerm' m + +readTerm' current = + do + id <- getNextId + sep <- readSeparator + more <- (option (T_EOF id) readAndOr) + case more of (T_EOF _) -> return [transformWithSeparator id sep current] + _ -> do + list <- readTerm' more + return $ (transformWithSeparator id sep current : list) + <|> + return [current] + +transformWithSeparator i '&' = T_Backgrounded i +transformWithSeparator i _ = id + + +readPipeSequence = do + id <- getNextId + list <- readCommand `sepBy1` (readPipe `thenSkip` (spacing >> readLineBreak)) + spacing + return $ T_Pipeline id list + +readPipe = do + notFollowedBy2 g_OR_IF + char '|' `thenSkip` spacing + +readCommand = (readCompoundCommand <|> readSimpleCommand) + +readCmdName = do + f <- readNormalWord + spacing + return f + +readCmdWord = do + f <- readNormalWord + spacing + return f + +prop_readIfClause = isOk readIfClause "if false; then foo; elif true; then stuff; more stuff; else cows; fi" +prop_readIfClause2 = isWarning readIfClause "if false; then; echo oo; fi" +prop_readIfClause3 = isWarning readIfClause "if false; then true; else; echo lol; fi" +readIfClause = called "if expression" $ do + id <- getNextId + pos <- getPosition + (condition, action) <- readIfPart + elifs <- many readElifPart + elses <- option [] readElsePart + + g_Fi `orFail` do + parseProblemAt pos ErrorC 1046 "Couldn't find 'fi' for this 'if'." + parseProblem ErrorC 1047 "Expected 'fi' matching previously mentioned 'if'." + + return $ T_IfExpression id ((condition, action):elifs) elses + + +verifyNotEmptyIf s = + optional (do + emptyPos <- getPosition + try . lookAhead $ (g_Fi <|> g_Elif <|> g_Else) + parseProblemAt emptyPos ErrorC 1048 $ "Can't have empty " ++ s ++ " clauses (use 'true' as a no-op).") +readIfPart = do + pos <- getPosition + g_If + allspacing + condition <- readTerm + + optional (do + try . lookAhead $ g_Fi + parseProblemAt pos ErrorC 1049 "Did you forget the 'then' for this 'if'?") + + called "then clause" $ do + g_Then `orFail` parseProblem ErrorC 1050 "Expected 'then'." + + acceptButWarn g_Semi ErrorC 1051 "No semicolons directly after 'then'." + allspacing + verifyNotEmptyIf "then" + + action <- readTerm + return (condition, action) + +readElifPart = called "elif clause" $ do + pos <- getPosition + g_Elif + allspacing + condition <- readTerm + g_Then + acceptButWarn g_Semi ErrorC 1052 "No semicolons directly after 'then'." + allspacing + verifyNotEmptyIf "then" + action <- readTerm + return (condition, action) + +readElsePart = called "else clause" $ do + g_Else + acceptButWarn g_Semi ErrorC 1053 "No semicolons directly after 'else'." + allspacing + verifyNotEmptyIf "else" + readTerm + +prop_readSubshell = isOk readSubshell "( cd /foo; tar cf stuff.tar * )" +readSubshell = called "explicit subshell" $ do + id <- getNextId + char '(' + allspacing + list <- readCompoundList + allspacing + char ')' + return $ T_Subshell id list + +prop_readBraceGroup = isOk readBraceGroup "{ a; b | c | d; e; }" +prop_readBraceGroup2 = isWarning readBraceGroup "{foo;}" +readBraceGroup = called "brace group" $ do + id <- getNextId + char '{' + allspacingOrFail <|> parseProblem ErrorC 1054 "You need a space after the '{'." + optional $ do + pos <- getPosition + lookAhead $ char '}' + parseProblemAt pos ErrorC 1055 "You need at least one command here. Use 'true;' as a no-op." + list <- readTerm + char '}' <|> do + parseProblem ErrorC 1056 "Expected a '}'. If you have one, try a ; or \\n in front of it." + fail "Unable to parse" + return $ T_BraceGroup id list + +prop_readWhileClause = isOk readWhileClause "while [[ -e foo ]]; do sleep 1; done" +readWhileClause = called "while loop" $ do + pos <- getPosition + (T_While id) <- g_While + condition <- readTerm + statements <- readDoGroup pos + return $ T_WhileExpression id condition statements + +prop_readUntilClause = isOk readUntilClause "until kill -0 $PID; do sleep 1; done" +readUntilClause = called "until loop" $ do + pos <- getPosition + (T_Until id) <- g_Until + condition <- readTerm + statements <- readDoGroup pos + return $ T_UntilExpression id condition statements + +readDoGroup loopPos = do + pos <- getPosition + optional (do + try . lookAhead $ g_Done + parseProblemAt loopPos ErrorC 1057 "Did you forget the 'do' for this loop?") + + g_Do `orFail` parseProblem ErrorC 1058 "Expected 'do'." + + acceptButWarn g_Semi ErrorC 1059 "No semicolons directly after 'do'." + allspacing + + optional (do + try . lookAhead $ g_Done + parseProblemAt loopPos ErrorC 1060 "Can't have empty do clauses (use 'true' as a no-op).") + + commands <- readCompoundList + g_Done `orFail` do + parseProblemAt pos ErrorC 1061 "Couldn't find 'done' for this 'do'." + parseProblem ErrorC 1062 "Expected 'done' matching previously mentioned 'do'." + return commands + + +prop_readForClause = isOk readForClause "for f in *; do rm \"$f\"; done" +prop_readForClause3 = isOk readForClause "for f; do foo; done" +prop_readForClause4 = isOk readForClause "for((i=0; i<10; i++)); do echo $i; done" +prop_readForClause5 = isOk readForClause "for ((i=0;i<10 && n>x;i++,--n))\ndo \necho $i\ndone" +prop_readForClause6 = isOk readForClause "for ((;;))\ndo echo $i\ndone" +prop_readForClause7 = isOk readForClause "for ((;;)) do echo $i\ndone" +prop_readForClause8 = isOk readForClause "for ((;;)) ; do echo $i\ndone" +prop_readForClause9 = isOk readForClause "for i do true; done" +readForClause = called "for loop" $ do + pos <- getPosition + (T_For id) <- g_For + spacing + typ <- (readRegular <|> readArithmetic) + group <- readDoGroup pos + typ id group + where + readArithmetic = called "arithmetic for condition" $ do + try $ string "((" + x <- readArithmeticContents + char ';' >> spacing + y <- readArithmeticContents + char ';' >> spacing + z <- readArithmeticContents + spacing + string "))" + spacing + optional $ readSequentialSep >> spacing + return $ \id group -> (return $ T_ForArithmetic id x y z group) + + readRegular = do + name <- readVariableName + spacing + values <- readInClause <|> (optional readSequentialSep >> return []) + return $ \id group -> (return $ T_ForIn id name values group) + +prop_readSelectClause1 = isOk readSelectClause "select foo in *; do echo $foo; done" +prop_readSelectClause2 = isOk readSelectClause "select foo; do echo $foo; done" +readSelectClause = called "select loop" $ do + pos <- getPosition + (T_Select id) <- g_Select + spacing + typ <- readRegular + group <- readDoGroup pos + typ id group + where + readRegular = do + name <- readVariableName + spacing + values <- readInClause <|> (readSequentialSep >> return []) + return $ \id group -> (return $ T_SelectIn id name values group) + +readInClause = do + g_In + things <- (readCmdWord) `reluctantlyTill` + (disregard (g_Semi) <|> disregard linefeed <|> disregard g_Do) + + do { + lookAhead (g_Do); + parseNote ErrorC 1063 "You need a line feed or semicolon before the 'do'."; + } <|> do { + optional $ g_Semi; + disregard allspacing; + } + + return things + +prop_readCaseClause = isOk readCaseClause "case foo in a ) lol; cow;; b|d) fooo; esac" +prop_readCaseClause2 = isOk readCaseClause "case foo\n in * ) echo bar;; esac" +readCaseClause = called "case expression" $ do + id <- getNextId + g_Case + word <- readNormalWord + allspacing + g_In + readLineBreak + list <- readCaseList + g_Esac + return $ T_CaseExpression id word list + +readCaseList = many readCaseItem + +readCaseItem = called "case item" $ do + notFollowedBy2 g_Esac + optional g_Lparen + spacing + pattern <- readPattern + g_Rparen + readLineBreak + list <- ((lookAhead g_DSEMI >> return []) <|> readCompoundList) + (g_DSEMI <|> lookAhead (readLineBreak >> g_Esac)) `attempting` do + pos <- getPosition + lookAhead g_Rparen + parseProblemAt pos ErrorC 1074 + "Did you forget the ;; after the previous case item?" + readLineBreak + return (pattern, list) + +prop_readFunctionDefinition = isOk readFunctionDefinition "foo() { command foo --lol \"$@\"; }" +prop_readFunctionDefinition1 = isOk readFunctionDefinition "foo (){ command foo --lol \"$@\"; }" +prop_readFunctionDefinition2 = isWarning readFunctionDefinition "function foo() { command foo --lol \"$@\"; }" +prop_readFunctionDefinition3 = isWarning readFunctionDefinition "function foo { lol; }" +prop_readFunctionDefinition4 = isWarning readFunctionDefinition "foo(a, b) { true; }" +prop_readFunctionDefinition5 = isOk readFunctionDefinition ":(){ :|:;}" +prop_readFunctionDefinition6 = isOk readFunctionDefinition "?(){ foo; }" +prop_readFunctionDefinition7 = isOk readFunctionDefinition "..(){ cd ..; }" +prop_readFunctionDefinition8 = isOk readFunctionDefinition "foo() (ls)" +readFunctionDefinition = called "function" $ do + id <- getNextId + name <- try readFunctionSignature + allspacing + (disregard (lookAhead $ oneOf "{(") <|> parseProblem ErrorC 1064 "Expected a { to open the function definition.") + group <- readBraceGroup <|> readSubshell + return $ T_Function id name group + + +readFunctionSignature = do + readWithFunction <|> readWithoutFunction + where + readWithFunction = do + pos <- getPosition + try $ do + string "function" + whitespace + parseProblemAt pos InfoC 1005 "Drop the keyword 'function'. It's optional in Bash but invalid in other shells." + spacing + name <- readFunctionName + optional spacing + pos <- getPosition + readParens <|> do + parseProblemAt pos InfoC 1006 "Include '()' after the function name (in addition to dropping 'function')." + return name + + readWithoutFunction = try $ do + name <- readFunctionName + optional spacing + readParens + return name + + readParens = do + g_Lparen + optional spacing + g_Rparen <|> do + parseProblem ErrorC 1065 "Trying to declare parameters? Don't. Use () and refer to params as $1, $2.." + many $ noneOf "\n){" + g_Rparen + return () + + readFunctionName = many1 functionChars + + + +readPattern = (readNormalWord `thenSkip` spacing) `sepBy1` (char '|' `thenSkip` spacing) + + +prop_readCompoundCommand = isOk readCompoundCommand "{ echo foo; }>/dev/null" +readCompoundCommand = do + id <- getNextId + cmd <- choice [ readBraceGroup, readArithmeticExpression, readSubshell, readCondition, readWhileClause, readUntilClause, readIfClause, readForClause, readSelectClause, readCaseClause, readFunctionDefinition] + optional spacing + redirs <- many readIoRedirect + when (not . null $ redirs) $ optional $ do + lookAhead $ try (spacing >> needsSeparator) + parseProblem WarningC 1013 "Bash requires ; or \\n here, after redirecting nested compound commands." + return $ T_Redirecting id redirs $ cmd + where + needsSeparator = choice [ g_Then, g_Else, g_Elif, g_Fi, g_Do, g_Done, g_Esac, g_Rbrace ] + + +readCompoundList = readTerm + +readCmdPrefix = many1 (readIoRedirect <|> readAssignmentWord) +readCmdSuffix = many1 (readIoRedirect <|> readCmdWord) +readModifierSuffix = many1 (readIoRedirect <|> readAssignmentWord <|> readCmdWord) +readTimeSuffix = do + flags <- many readFlag + pipeline <- readPipeline + return $ flags ++ [pipeline] + where + -- This fails for quoted variables and such. Fixme? + readFlag = do + lookAhead $ char '-' + readCmdWord + +prop_readAssignmentWord = isOk readAssignmentWord "a=42" +prop_readAssignmentWord2 = isOk readAssignmentWord "b=(1 2 3)" +prop_readAssignmentWord3 = isWarning readAssignmentWord "$b = 13" +prop_readAssignmentWord4 = isWarning readAssignmentWord "b = $(lol)" +prop_readAssignmentWord5 = isOk readAssignmentWord "b+=lol" +prop_readAssignmentWord6 = isWarning readAssignmentWord "b += (1 2 3)" +prop_readAssignmentWord7 = isOk readAssignmentWord "a[3$n'']=42" +prop_readAssignmentWord8 = isOk readAssignmentWord "a[4''$(cat foo)]=42" +prop_readAssignmentWord9 = isOk readAssignmentWord "IFS= " +prop_readAssignmentWord0 = isWarning readAssignmentWord "foo$n=42" +readAssignmentWord = try $ do + id <- getNextId + pos <- getPosition + optional (char '$' >> parseNote ErrorC 1066 "Don't use $ on the left side of assignments.") + variable <- readVariableName + optional (readNormalDollar >> parseNoteAt pos ErrorC + 1067 "For indirection, use (associative) arrays or 'read \"var$n\" <<< \"value\"'") + index <- optionMaybe readArrayIndex + space <- spacing + pos <- getPosition + op <- readAssignmentOp + space2 <- spacing + if space == "" && space2 /= "" + then do + when (variable /= "IFS") $ + parseNoteAt pos InfoC 1007 $ "Note that 'var= value' (with space after equals sign) is similar to 'var=\"\"; value'." + value <- readEmptyLiteral + return $ T_Assignment id op variable index value + else do + when (space /= "" || space2 /= "") $ + parseNoteAt pos ErrorC 1068 "Don't put spaces around the = in assignments." + value <- readArray <|> readNormalWord + spacing + return $ T_Assignment id op variable index value + where + readAssignmentOp = + (string "+=" >> return Append) <|> (string "=" >> return Assign) + readEmptyLiteral = do + id <- getNextId + return $ T_Literal id "" + +-- This is only approximate. Fixme? +-- * Bash allows foo[' ' "" $(true) 2 ``]=var +-- * foo[bar] dereferences bar +readArrayIndex = do + char '[' + optional space + x <- readNormalishWord "]" + optional space + char ']' + return x + +readArray = called "array assignment" $ do + id <- getNextId + char '(' + allspacing + words <- (readNormalWord `thenSkip` allspacing) `reluctantlyTill` (char ')') + char ')' + return $ T_Array id words + +tryToken s t = try $ do + id <- getNextId + string s + spacing + return $ t id + +redirToken c t = try $ do + id <- getNextId + char c + notFollowedBy2 $ char '(' + return $ t id + +tryWordToken s t = tryParseWordToken (string s) t `thenSkip` spacing +tryParseWordToken parser t = try $ do + id <- getNextId + parser + optional (do + try . lookAhead $ char '[' + parseProblem ErrorC 1069 "You need a space before the [.") + try $ lookAhead (keywordSeparator) + return $ t id + +g_AND_IF = tryToken "&&" T_AND_IF +g_OR_IF = tryToken "||" T_OR_IF +g_DSEMI = tryToken ";;" T_DSEMI +g_DLESS = tryToken "<<" T_DLESS +g_DGREAT = tryToken ">>" T_DGREAT +g_LESSAND = tryToken "<&" T_LESSAND +g_GREATAND = tryToken ">&" T_GREATAND +g_LESSGREAT = tryToken "<>" T_LESSGREAT +g_DLESSDASH = tryToken "<<-" T_DLESSDASH +g_CLOBBER = tryToken ">|" T_CLOBBER +g_OPERATOR = g_AND_IF <|> g_OR_IF <|> g_DSEMI <|> g_DLESSDASH <|> g_DLESS <|> g_DGREAT <|> g_LESSAND <|> g_GREATAND <|> g_LESSGREAT + +g_If = tryWordToken "if" T_If +g_Then = tryWordToken "then" T_Then +g_Else = tryWordToken "else" T_Else +g_Elif = tryWordToken "elif" T_Elif +g_Fi = tryWordToken "fi" T_Fi +g_Do = tryWordToken "do" T_Do +g_Done = tryWordToken "done" T_Done +g_Case = tryWordToken "case" T_Case +g_Esac = tryWordToken "esac" T_Esac +g_While = tryWordToken "while" T_While +g_Until = tryWordToken "until" T_Until +g_For = tryWordToken "for" T_For +g_Select = tryWordToken "select" T_Select +g_In = tryWordToken "in" T_In +g_Lbrace = tryWordToken "{" T_Lbrace +g_Rbrace = tryWordToken "}" T_Rbrace + +g_Lparen = tryToken "(" T_Lparen +g_Rparen = tryToken ")" T_Rparen +g_Bang = do + id <- getNextId + char '!' + softCondSpacing + return $ T_Bang id + + +g_Semi = do + notFollowedBy2 g_DSEMI + tryToken ";" T_Semi + +keywordSeparator = + eof <|> disregard whitespace <|> (disregard $ oneOf ";()[<>&|") + +readKeyword = choice [ g_Then, g_Else, g_Elif, g_Fi, g_Do, g_Done, g_Esac, g_Rbrace, g_Rparen, g_DSEMI ] + +ifParse p t f = do + (lookAhead (try p) >> t) <|> f + +readShebang = do + try $ string "#!" + str <- many $ noneOf "\r\n" + optional carriageReturn + optional linefeed + return str + +prop_readScript1 = isOk readScript "#!/bin/bash\necho hello world\n" +prop_readScript2 = isWarning readScript "#!/bin/bash\r\necho hello world\n" +prop_readScript3 = isWarning readScript "#!/bin/bash\necho hello\xA0world" +prop_readScript4 = isWarning readScript "#!/usr/bin/perl\nfoo=(" +readScript = do + id <- getNextId + pos <- getPosition + sb <- option "" readShebang + verifyShell pos (getShell sb) + if (isValidShell $ getShell sb) /= Just False + then + do { + allspacing; + commands <- readTerm; + eof <|> (parseProblem ErrorC 1070 "Parsing stopped here because of parsing errors."); + return $ T_Script id sb commands; + } <|> do { + parseProblem WarningC 1014 "Couldn't read any commands."; + return $ T_Script id sb $ [T_EOF id]; + } + else do + many anyChar + return $ T_Script id sb $ [T_EOF id]; + + where + basename s = reverse . takeWhile (/= '/') . reverse $ s + getShell sb = + case words sb of + [] -> "" + [x] -> basename x + (first:second:_) -> + if basename first == "env" + then second + else basename first + + verifyShell pos s = + case isValidShell s of + Just True -> return () + Just False -> parseProblemAt pos ErrorC 1071 "ShellCheck only supports Bourne based shell scripts, sorry!" + Nothing -> parseProblemAt pos InfoC 1008 "This shebang was unrecognized. Note that ShellCheck only handles Bourne based shells." + + isValidShell s = + let good = s == "" || any (`isPrefixOf` s) goodShells + bad = any (`isPrefixOf` s) badShells + in + if good + then Just True + else if bad + then Just False + else Nothing + + goodShells = [ + "sh", + "bash", + "ksh", + "zsh" + ] + badShells = [ + "awk", + "csh", + "perl", + "python", + "ruby", + "tcsh" + ] + +rp p filename contents = Ms.runState (runParserT p initialState filename contents) ([], []) + +isWarning p s = (fst cs) && (not . null . snd $ cs) where cs = checkString p s +isOk p s = (fst cs) && (null . snd $ cs) where cs = checkString p s + +checkString parser string = + case rp (parser >> eof >> getState) "-" string of + (Right (tree, map, notes), (problems, _)) -> (True, (notesFromMap map) ++ notes ++ problems) + (Left _, (n, _)) -> (False, n) + +parseWithNotes parser = do + item <- parser + map <- getMap + parseNotes <- getParseNotes + return (item, map, nub . sortNotes $ parseNotes) + +toParseNotes (Metadata pos list) = map (\(Note level code note) -> ParseNote pos level code note) list +notesFromMap map = Map.fold (\x -> (++) (toParseNotes x)) [] map + +getAllNotes result = (concatMap (notesFromMap . snd) (maybeToList . parseResult $ result)) ++ (parseNotes result) + +compareNotes (ParseNote pos1 level1 _ s1) (ParseNote pos2 level2 _ s2) = compare (pos1, level1) (pos2, level2) +sortNotes = sortBy compareNotes + + +data ParseResult = ParseResult { parseResult :: Maybe (Token, Map.Map Id Metadata), parseNotes :: [ParseNote] } deriving (Show) + +makeErrorFor parsecError = + ParseNote (errorPos parsecError) ErrorC 1072 $ getStringFromParsec $ errorMessages parsecError + +getStringFromParsec errors = + case map snd $ sortWith fst $ map f errors of + r -> (intercalate " " $ take 1 $ nub r) ++ " Fix any mentioned problems and try again." + where f err = + case err of + UnExpect s -> (1, unexpected s) + SysUnExpect s -> (2, unexpected s) + Expect s -> (3, "Expected " ++ s ++ ".") + Message s -> (4, s ++ ".") + wut "" = "eof" + wut x = x + unexpected s = "Unexpected " ++ (wut s) ++ "." + +parseShell filename contents = do + case rp (parseWithNotes readScript) filename contents of + (Right (script, map, notes), (parsenotes, _)) -> ParseResult (Just (script, map)) (nub $ sortNotes $ notes ++ parsenotes) + (Left err, (p, context)) -> ParseResult Nothing (nub $ sortNotes $ p ++ (notesForContext context) ++ ([makeErrorFor err])) + + where + isName (ContextName _ _) = True + isName _ = False + notesForContext list = zipWith ($) [first, second] $ filter isName list + first (ContextName pos str) = ParseNote pos ErrorC 1073 $ + "Couldn't parse this " ++ str ++ "." + second (ContextName pos str) = ParseNote pos InfoC 1009 $ + "The mentioned parser error was in this " ++ str ++ "." + +lt x = trace (show x) x +ltt t x = trace (show t) x diff --git a/ShellCheck/Simple.hs b/ShellCheck/Simple.hs new file mode 100644 --- /dev/null +++ b/ShellCheck/Simple.hs @@ -0,0 +1,67 @@ +{- + This file is part of ShellCheck. + http://www.vidarholen.net/contents/shellcheck + + ShellCheck is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + ShellCheck is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see . +-} +module ShellCheck.Simple (shellCheck, ShellCheckComment, scLine, scColumn, scSeverity, scCode, scMessage) where + +import ShellCheck.Parser +import ShellCheck.Analytics +import Data.Maybe +import Text.Parsec.Pos +import Data.List + + +prop_findsParseIssue = + let comments = shellCheck "echo \"$12\"" in + (length comments) == 1 && (scCode $ head comments) == 1037 +prop_commentDisablesParseIssue1 = + null $ shellCheck "#shellcheck disable=SC1037\necho \"$12\"" +prop_commentDisablesParseIssue2 = + null $ shellCheck "#shellcheck disable=SC1037\n#lol\necho \"$12\"" + +prop_findsAnalysisIssue = + let comments = shellCheck "echo $1" in + (length comments) == 1 && (scCode $ head comments) == 2086 +prop_commentDisablesAnalysisIssue1 = + null $ shellCheck "#shellcheck disable=SC2086\necho $1" +prop_commentDisablesAnalysisIssue2 = + null $ shellCheck "#shellcheck disable=SC2086\n#lol\necho $1" + +shellCheck :: String -> [ShellCheckComment] +shellCheck script = + let (ParseResult result notes) = parseShell "-" script in + let allNotes = notes ++ (concat $ maybeToList $ do + (tree, map) <- result + let newMap = runAllAnalytics tree map + return $ notesFromMap $ filterByAnnotation tree newMap + ) + in + map formatNote $ nub $ sortNotes allNotes + +data ShellCheckComment = ShellCheckComment { scLine :: Int, scColumn :: Int, scSeverity :: String, scCode :: Int, scMessage :: String } + +instance Show ShellCheckComment where + show c = concat ["(", show $ scLine c, ",", show $ scColumn c, ") ", scSeverity c, ": ", show (scCode c), " ", scMessage c] + +severityToString s = + case s of + ErrorC -> "error" + WarningC -> "warning" + InfoC -> "info" + StyleC -> "style" + +formatNote (ParseNote pos severity code text) = + ShellCheckComment (sourceLine pos) (sourceColumn pos) (severityToString severity) (fromIntegral code) text diff --git a/shellcheck.hs b/shellcheck.hs new file mode 100644 --- /dev/null +++ b/shellcheck.hs @@ -0,0 +1,270 @@ +{- + This file is part of ShellCheck. + http://www.vidarholen.net/contents/shellcheck + + ShellCheck is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + ShellCheck is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see . +-} +import Control.Exception +import Control.Monad +import Data.Char +import GHC.Exts +import GHC.IO.Device +import Prelude hiding (catch) +import ShellCheck.Simple +import System.Console.GetOpt +import System.Directory +import System.Environment +import System.Exit +import System.IO +import Text.JSON +import qualified Data.Map as Map + +data Flag = Flag String String + +header = "Usage: shellcheck [OPTIONS...] FILES..." +options = [ + Option ['f'] ["format"] + (ReqArg (Flag "format") "FORMAT") "output format", + Option ['e'] ["exclude"] + (ReqArg (Flag "exclude") "CODE1,CODE2..") "exclude types of warnings" + ] + +printErr = hPutStrLn stderr + +syntaxFailure = ExitFailure 3 +supportFailure = ExitFailure 4 + +instance JSON ShellCheckComment where + showJSON c = makeObj [ + ("line", showJSON $ scLine c), + ("column", showJSON $ scColumn c), + ("level", showJSON $ scSeverity c), + ("code", showJSON $ scCode c), + ("message", showJSON $ scMessage c) + ] + readJSON = undefined + +parseArguments argv = + case getOpt Permute options argv of + (opts, files, []) -> + if not $ null files + then + return $ Just (opts, files) + else do + printErr "No files specified.\n" + printErr $ usageInfo header options + exitWith syntaxFailure + + (_, _, errors) -> do + printErr $ (concat errors) ++ "\n" ++ usageInfo header options + exitWith syntaxFailure + +formats = Map.fromList [ + ("json", forJson), + ("gcc", forGcc), + ("checkstyle", forCheckstyle), + ("tty", forTty) + ] + +forTty options files = do + output <- mapM doFile files + return $ and output + where + clear = ansi 0 + ansi n = "\x1B[" ++ (show n) ++ "m" + + colorForLevel "error" = 31 -- red + colorForLevel "warning" = 33 -- yellow + colorForLevel "info" = 32 -- green + colorForLevel "style" = 32 -- green + colorForLevel "message" = 1 -- bold + colorForLevel "source" = 0 -- none + colorForLevel _ = 0 -- none + + colorComment level comment = (ansi $ colorForLevel level) ++ comment ++ clear + + doFile path = do + contents <- readContents path + doInput path contents + + doInput filename contents = do + let fileLines = lines contents + let lineCount = length fileLines + let comments = getComments options contents + let groups = groupWith scLine comments + colorFunc <- getColorFunc + mapM_ (\x -> do + let lineNum = scLine (head x) + let line = if lineNum < 1 || lineNum > lineCount + then "" + else fileLines !! (lineNum - 1) + putStrLn "" + putStrLn $ colorFunc "message" ("In " ++ filename ++" line " ++ (show $ lineNum) ++ ":") + putStrLn (colorFunc "source" line) + mapM (\c -> putStrLn (colorFunc (scSeverity c) $ cuteIndent c)) x + putStrLn "" + ) groups + return $ null comments + + cuteIndent comment = + (replicate ((scColumn comment) - 1) ' ') ++ "^-- " ++ (code $ scCode comment) ++ ": " ++ (scMessage comment) + + code code = "SC" ++ (show code) + + getColorFunc = do + term <- hIsTerminalDevice stdout + return $ if term then colorComment else const id + +-- This totally ignores the filenames. Fixme? +forJson options files = do + comments <- liftM concat $ mapM (commentsFor options) files + putStrLn $ encodeStrict $ comments + return . null $ comments + +-- Mimic GCC "file:line:col: (error|warning|note): message" format +forGcc options files = do + files <- mapM process files + return $ and files + where + process file = do + contents <- readContents file + let comments = makeNonVirtual (getComments options contents) contents + mapM_ (putStrLn . format file) comments + return $ null comments + + format filename c = concat [ + filename, ":", + show $ scLine c, ":", + show $ scColumn c, ": ", + case scSeverity c of + "error" -> "error" + "warning" -> "warning" + _ -> "note", + ": ", + concat . lines $ scMessage c, + " [SC", show $ scCode c, "]" + ] + +-- Checkstyle compatible output. A bit of a hack to avoid XML dependencies +forCheckstyle options files = do + putStrLn "" + putStrLn "" + statuses <- mapM (\x -> process x `catch` report) files + putStrLn "" + return $ and statuses + where + process file = do + comments <- commentsFor options file + putStrLn (formatFile file comments) + return $ null comments + report error = do + printErr $ show (error :: SomeException) + return False + + severity "error" = "error" + severity "warning" = "warning" + severity _ = "info" + attr s v = concat [ s, "='", escape v, "' " ] + escape msg = concatMap escape' msg + escape' c = if isOk c then [c] else "&#" ++ (show $ ord c) ++ ";" + isOk x = any ($x) [isAsciiUpper, isAsciiLower, isDigit, (`elem` " ./")] + + formatFile name comments = concat [ + "\n", + concatMap format comments, + "" + ] + + format c = concat [ + "\n" + ] + +commentsFor options file = + liftM (getComments options) $ readContents file + +getComments options contents = + excludeCodes (getExclusions options) $ shellCheck contents + +readContents file = if file == "-" then getContents else readFile file + +-- Realign comments from a tabstop of 8 to 1 +makeNonVirtual comments contents = + map fix comments + where + ls = lines contents + fix c = c { scColumn = real (ls !! (scLine c - 1)) 0 0 (scColumn c) } + real _ r v target | target <= v = r + real [] r v _ = r -- should never happen + real ('\t':rest) r v target = + real rest (r+1) (v + 8 - (v `mod` 8)) target + real (_:rest) r v target = real rest (r+1) (v+1) target + +getOption [] _ def = def +getOption ((Flag var val):_) name _ | name == var = val +getOption (_:rest) flag def = getOption rest flag def + +getOptions options name = + map (\(Flag _ val) -> val) . filter (\(Flag var _) -> var == name) $ options + +split char str = + split' str [] + where + split' (a:rest) element = + if a == char + then (reverse element) : split' rest [] + else split' rest (a:element) + split' [] element = [reverse element] + +getExclusions options = + let elements = concatMap (split ',') $ getOptions options "exclude" + clean = dropWhile (not . isDigit) + in + map (Prelude.read . clean) elements :: [Int] + +excludeCodes codes comments = + filter (not . hasCode) comments + where + hasCode c = scCode c `elem` codes + +main = do + args <- getArgs + parsedArgs <- parseArguments args + code <- do + status <- process parsedArgs + return $ if status then ExitSuccess else ExitFailure 1 + `catch` return + `catch` \err -> do + printErr $ show (err :: SomeException) + return $ ExitFailure 2 + exitWith code + +process Nothing = return False +process (Just (options, files)) = + let format = getOption options "format" "tty" in + case Map.lookup format formats of + Nothing -> do + printErr $ "Unknown format " ++ format + printErr $ "Supported formats:" + mapM_ (printErr . write) $ Map.keys formats + exitWith supportFailure + where write s = " " ++ s + Just f -> do + f options files +