diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,12 @@ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), +and this project adheres to the [Haskell Package Versioning Policy](https://pvp.haskell.org/). + +## [1.0.0] - 2026-10-20 + +### Added + +- Initial release. diff --git a/LICENCE b/LICENCE new file mode 100644 --- /dev/null +++ b/LICENCE @@ -0,0 +1,287 @@ + EUROPEAN UNION PUBLIC LICENCE v. 1.2 + EUPL © the European Union 2007, 2016 + +This European Union Public Licence (the ‘EUPL’) applies to the Work (as defined +below) which is provided under the terms of this Licence. Any use of the Work, +other than as authorised under this Licence is prohibited (to the extent such +use is covered by a right of the copyright holder of the Work). + +The Work is provided under the terms of this Licence when the Licensor (as +defined below) has placed the following notice immediately following the +copyright notice for the Work: + + Licensed under the EUPL + +or has expressed by any other means his willingness to license under the EUPL. + +1. Definitions + +In this Licence, the following terms have the following meaning: + +- ‘The Licence’: this Licence. + +- ‘The Original Work’: the work or software distributed or communicated by the + Licensor under this Licence, available as Source Code and also as Executable + Code as the case may be. + +- ‘Derivative Works’: the works or software that could be created by the + Licensee, based upon the Original Work or modifications thereof. This Licence + does not define the extent of modification or dependence on the Original Work + required in order to classify a work as a Derivative Work; this extent is + determined by copyright law applicable in the country mentioned in Article 15. + +- ‘The Work’: the Original Work or its Derivative Works. + +- ‘The Source Code’: the human-readable form of the Work which is the most + convenient for people to study and modify. + +- ‘The Executable Code’: any code which has generally been compiled and which is + meant to be interpreted by a computer as a program. + +- ‘The Licensor’: the natural or legal person that distributes or communicates + the Work under the Licence. + +- ‘Contributor(s)’: any natural or legal person who modifies the Work under the + Licence, or otherwise contributes to the creation of a Derivative Work. + +- ‘The Licensee’ or ‘You’: any natural or legal person who makes any usage of + the Work under the terms of the Licence. + +- ‘Distribution’ or ‘Communication’: any act of selling, giving, lending, + renting, distributing, communicating, transmitting, or otherwise making + available, online or offline, copies of the Work or providing access to its + essential functionalities at the disposal of any other natural or legal + person. + +2. Scope of the rights granted by the Licence + +The Licensor hereby grants You a worldwide, royalty-free, non-exclusive, +sublicensable licence to do the following, for the duration of copyright vested +in the Original Work: + +- use the Work in any circumstance and for all usage, +- reproduce the Work, +- modify the Work, and make Derivative Works based upon the Work, +- communicate to the public, including the right to make available or display + the Work or copies thereof to the public and perform publicly, as the case may + be, the Work, +- distribute the Work or copies thereof, +- lend and rent the Work or copies thereof, +- sublicense rights in the Work or copies thereof. + +Those rights can be exercised on any media, supports and formats, whether now +known or later invented, as far as the applicable law permits so. + +In the countries where moral rights apply, the Licensor waives his right to +exercise his moral right to the extent allowed by law in order to make effective +the licence of the economic rights here above listed. + +The Licensor grants to the Licensee royalty-free, non-exclusive usage rights to +any patents held by the Licensor, to the extent necessary to make use of the +rights granted on the Work under this Licence. + +3. Communication of the Source Code + +The Licensor may provide the Work either in its Source Code form, or as +Executable Code. If the Work is provided as Executable Code, the Licensor +provides in addition a machine-readable copy of the Source Code of the Work +along with each copy of the Work that the Licensor distributes or indicates, in +a notice following the copyright notice attached to the Work, a repository where +the Source Code is easily and freely accessible for as long as the Licensor +continues to distribute or communicate the Work. + +4. Limitations on copyright + +Nothing in this Licence is intended to deprive the Licensee of the benefits from +any exception or limitation to the exclusive rights of the rights owners in the +Work, of the exhaustion of those rights or of other applicable limitations +thereto. + +5. Obligations of the Licensee + +The grant of the rights mentioned above is subject to some restrictions and +obligations imposed on the Licensee. Those obligations are the following: + +Attribution right: The Licensee shall keep intact all copyright, patent or +trademarks notices and all notices that refer to the Licence and to the +disclaimer of warranties. The Licensee must include a copy of such notices and a +copy of the Licence with every copy of the Work he/she distributes or +communicates. The Licensee must cause any Derivative Work to carry prominent +notices stating that the Work has been modified and the date of modification. + +Copyleft clause: If the Licensee distributes or communicates copies of the +Original Works or Derivative Works, this Distribution or Communication will be +done under the terms of this Licence or of a later version of this Licence +unless the Original Work is expressly distributed only under this version of the +Licence — for example by communicating ‘EUPL v. 1.2 only’. The Licensee +(becoming Licensor) cannot offer or impose any additional terms or conditions on +the Work or Derivative Work that alter or restrict the terms of the Licence. + +Compatibility clause: If the Licensee Distributes or Communicates Derivative +Works or copies thereof based upon both the Work and another work licensed under +a Compatible Licence, this Distribution or Communication can be done under the +terms of this Compatible Licence. For the sake of this clause, ‘Compatible +Licence’ refers to the licences listed in the appendix attached to this Licence. +Should the Licensee's obligations under the Compatible Licence conflict with +his/her obligations under this Licence, the obligations of the Compatible +Licence shall prevail. + +Provision of Source Code: When distributing or communicating copies of the Work, +the Licensee will provide a machine-readable copy of the Source Code or indicate +a repository where this Source will be easily and freely available for as long +as the Licensee continues to distribute or communicate the Work. + +Legal Protection: This Licence does not grant permission to use the trade names, +trademarks, service marks, or names of the Licensor, except as required for +reasonable and customary use in describing the origin of the Work and +reproducing the content of the copyright notice. + +6. Chain of Authorship + +The original Licensor warrants that the copyright in the Original Work granted +hereunder is owned by him/her or licensed to him/her and that he/she has the +power and authority to grant the Licence. + +Each Contributor warrants that the copyright in the modifications he/she brings +to the Work are owned by him/her or licensed to him/her and that he/she has the +power and authority to grant the Licence. + +Each time You accept the Licence, the original Licensor and subsequent +Contributors grant You a licence to their contributions to the Work, under the +terms of this Licence. + +7. Disclaimer of Warranty + +The Work is a work in progress, which is continuously improved by numerous +Contributors. It is not a finished work and may therefore contain defects or +‘bugs’ inherent to this type of development. + +For the above reason, the Work is provided under the Licence on an ‘as is’ basis +and without warranties of any kind concerning the Work, including without +limitation merchantability, fitness for a particular purpose, absence of defects +or errors, accuracy, non-infringement of intellectual property rights other than +copyright as stated in Article 6 of this Licence. + +This disclaimer of warranty is an essential part of the Licence and a condition +for the grant of any rights to the Work. + +8. Disclaimer of Liability + +Except in the cases of wilful misconduct or damages directly caused to natural +persons, the Licensor will in no event be liable for any direct or indirect, +material or moral, damages of any kind, arising out of the Licence or of the use +of the Work, including without limitation, damages for loss of goodwill, work +stoppage, computer failure or malfunction, loss of data or any commercial +damage, even if the Licensor has been advised of the possibility of such damage. +However, the Licensor will be liable under statutory product liability laws as +far such laws apply to the Work. + +9. Additional agreements + +While distributing the Work, You may choose to conclude an additional agreement, +defining obligations or services consistent with this Licence. However, if +accepting obligations, You may act only on your own behalf and on your sole +responsibility, not on behalf of the original Licensor or any other Contributor, +and only if You agree to indemnify, defend, and hold each Contributor harmless +for any liability incurred by, or claims asserted against such Contributor by +the fact You have accepted any warranty or additional liability. + +10. Acceptance of the Licence + +The provisions of this Licence can be accepted by clicking on an icon ‘I agree’ +placed under the bottom of a window displaying the text of this Licence or by +affirming consent in any other similar way, in accordance with the rules of +applicable law. Clicking on that icon indicates your clear and irrevocable +acceptance of this Licence and all of its terms and conditions. + +Similarly, you irrevocably accept this Licence and all of its terms and +conditions by exercising any rights granted to You by Article 2 of this Licence, +such as the use of the Work, the creation by You of a Derivative Work or the +Distribution or Communication by You of the Work or copies thereof. + +11. Information to the public + +In case of any Distribution or Communication of the Work by means of electronic +communication by You (for example, by offering to download the Work from a +remote location) the distribution channel or media (for example, a website) must +at least provide to the public the information requested by the applicable law +regarding the Licensor, the Licence and the way it may be accessible, concluded, +stored and reproduced by the Licensee. + +12. Termination of the Licence + +The Licence and the rights granted hereunder will terminate automatically upon +any breach by the Licensee of the terms of the Licence. + +Such a termination will not terminate the licences of any person who has +received the Work from the Licensee under the Licence, provided such persons +remain in full compliance with the Licence. + +13. Miscellaneous + +Without prejudice of Article 9 above, the Licence represents the complete +agreement between the Parties as to the Work. + +If any provision of the Licence is invalid or unenforceable under applicable +law, this will not affect the validity or enforceability of the Licence as a +whole. Such provision will be construed or reformed so as necessary to make it +valid and enforceable. + +The European Commission may publish other linguistic versions or new versions of +this Licence or updated versions of the Appendix, so far this is required and +reasonable, without reducing the scope of the rights granted by the Licence. New +versions of the Licence will be published with a unique version number. + +All linguistic versions of this Licence, approved by the European Commission, +have identical value. Parties can take advantage of the linguistic version of +their choice. + +14. Jurisdiction + +Without prejudice to specific agreement between parties, + +- any litigation resulting from the interpretation of this License, arising + between the European Union institutions, bodies, offices or agencies, as a + Licensor, and any Licensee, will be subject to the jurisdiction of the Court + of Justice of the European Union, as laid down in article 272 of the Treaty on + the Functioning of the European Union, + +- any litigation arising between other parties and resulting from the + interpretation of this License, will be subject to the exclusive jurisdiction + of the competent court where the Licensor resides or conducts its primary + business. + +15. Applicable Law + +Without prejudice to specific agreement between parties, + +- this Licence shall be governed by the law of the European Union Member State + where the Licensor has his seat, resides or has his registered office, + +- this licence shall be governed by Belgian law if the Licensor has no seat, + residence or registered office inside a European Union Member State. + +Appendix + +‘Compatible Licences’ according to Article 5 EUPL are: + +- GNU General Public License (GPL) v. 2, v. 3 +- GNU Affero General Public License (AGPL) v. 3 +- Open Software License (OSL) v. 2.1, v. 3.0 +- Eclipse Public License (EPL) v. 1.0 +- CeCILL v. 2.0, v. 2.1 +- Mozilla Public Licence (MPL) v. 2 +- GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3 +- Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for + works other than software +- European Union Public Licence (EUPL) v. 1.1, v. 1.2 +- Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or Strong + Reciprocity (LiLiQ-R+). + +The European Commission may update this Appendix to later versions of the above +licences without producing a new version of the EUPL, as long as they provide +the rights granted in Article 2 of this Licence and protect the covered Source +Code from exclusive appropriation. + +All other changes or additions to this Appendix require the production of a new +EUPL version. diff --git a/app/Config.hs b/app/Config.hs new file mode 100644 --- /dev/null +++ b/app/Config.hs @@ -0,0 +1,40 @@ +module Config where + +import Data.List.NonEmpty (NonEmpty, some1) +import Data.Version (showVersion) +import Options.Applicative +import Paths_fluent_lint qualified +import Prelude + +-- | What the command line asks for. +data Config = Config + { lenient :: Bool + , reference :: Maybe FilePath + , files :: NonEmpty FilePath + } + +-- | The options and arguments of the command line. +parser :: Parser Config +parser = do + lenient <- + switch $ + long "lenient" + <> help "Warn about syntax errors instead of failing" + reference <- + optional . strOption $ + long "reference" + <> metavar "FILE" + <> help "The file whose messages every file must match" + files <- some1 . strArgument $ metavar "FILE" <> help "The files to check" + pure Config{..} + +parserInfo :: ParserInfo Config +parserInfo = + info (helper <*> versioner <*> parser) . header $ + "Read Fluent files and validate their contents for errors or missing keys." + where + versioner = simpleVersioner $ "fluent-lint " <> showVersion Paths_fluent_lint.version + +-- | Reads the command line. +get :: IO Config +get = execParser parserInfo diff --git a/app/Document.hs b/app/Document.hs new file mode 100644 --- /dev/null +++ b/app/Document.hs @@ -0,0 +1,64 @@ +module Document where + +import Data.Functor ((<&>)) +import Data.List ((!?)) +import Data.List qualified as List +import Data.Map.Strict (Map) +import Data.Map.Strict qualified as Map +import Data.Maybe (fromMaybe) +import Data.Set (Set) +import Data.Set qualified as Set +import Data.Text (Text) +import Data.Text qualified as Text +import Data.Text.IO qualified as Text +import Effectful +import Language.Fluent.AST +import Language.Fluent.Parser (parseResourceLenient) +import System.IO (IOMode (ReadMode), hSetEncoding, utf8, withFile) +import Prelude + +-- | A Fluent file, with its text and what was parsed from it. +data Document = Document + { path :: FilePath + , source :: Text + , messages :: Map Text (Set Text) + , errors :: [ParseError] + } + +-- | Reads and parses a file, which is UTF-8 whatever the locale is. +read :: (IOE :> es) => FilePath -> Eff es Document +read path = do + source <- liftIO . withFile path ReadMode $ \handle -> + hSetEncoding handle utf8 *> Text.hGetContents handle + let resource = parseResourceLenient source + pure + Document + { messages = + Map.fromList + [ (name, Set.fromList [name | Attribute (Identifier name) _ <- attributes]) + | MessageEntry Message{id = Identifier name, attributes} <- resource.entries + ] + , errors = [e | JunkEntry _ e <- resource.entries] + , .. + } + +-- | Renders syntax errors, each with its source line and an arrow under the column. +renderErrors :: Document -> String +renderErrors Document{source = Text.lines -> source, ..} = + List.intercalate "\n\n" $ + errors <&> \ParseError{..} -> + let number = show line + shown = + Text.unpack + . Text.dropWhileEnd (== '\r') + . fromMaybe Text.empty + $ source !? (line - 1) + in List.intercalate + "\n" + [ path <> ":" <> number <> ":" <> show column <> ": " <> reason + , number <> " | " <> shown + , blank number <> " | " <> blank (take (column - 1) shown) <> "^" + ] + where + blank :: String -> String + blank = fmap \c -> if c == '\t' then c else ' ' diff --git a/app/Lint.hs b/app/Lint.hs new file mode 100644 --- /dev/null +++ b/app/Lint.hs @@ -0,0 +1,40 @@ +module Lint where + +import Control.Monad (unless, when) +import Data.Foldable (for_) +import Data.Map.Strict qualified as Map +import Data.Set qualified as Set +import Document (Document (..)) +import Document qualified +import Effectful (Eff, (:>)) +import Effectful.Hspec (Hspec, expectationFailure, it, pendingWith, shouldBe) +import Prelude + +-- | A syntax error fails the file, which is then not compared with the reference. +strict :: (Hspec :> es) => Maybe Document -> Document -> Eff es () +strict reference document@Document{errors} = do + it "parses" . unless (null errors) . expectationFailure $ Document.renderErrors document + when (null errors) . for_ reference $ flip matches document + +-- | A syntax error is a warning, and the file is still compared with the reference. +lenient :: (Hspec :> es) => Maybe Document -> Document -> Eff es () +lenient reference document@Document{errors} = do + it "parses" . unless (null errors) . pendingWith $ Document.renderErrors document + for_ reference $ flip matches document + +-- | Checks that the file has the messages and attributes of the reference. +matches :: (Hspec :> es) => Document -> Document -> Eff es () +matches reference document = do + it "has every expected message" $ + Map.keys (reference.messages `Map.difference` document.messages) + `shouldBe` [] + it "has no unexpected messages" $ + Map.keys (document.messages `Map.difference` reference.messages) + `shouldBe` [] + it "has every expected attribute" $ + [ name <> "." <> attribute + | (name, missing) <- + Map.toList $ Map.intersectionWith Set.difference reference.messages document.messages + , attribute <- Set.toList missing + ] + `shouldBe` [] diff --git a/app/Main.hs b/app/Main.hs new file mode 100644 --- /dev/null +++ b/app/Main.hs @@ -0,0 +1,22 @@ +module Main where + +import Config (Config (..)) +import Config qualified +import Data.Containers.ListUtils (nubOrd) +import Data.Foldable (for_, toList) +import Document qualified +import Effectful +import Effectful.Hspec (describe, runHspec) +import Lint qualified +import System.Environment (withArgs) +import Prelude + +main :: IO () +main = do + Config{..} <- Config.get + withArgs [] . runEff . runHspec $ do + documents <- mapM Document.read . nubOrd . filter (`notElem` reference) . toList $ files + reference <- mapM Document.read reference + for_ (toList reference <> documents) \document -> + describe document.path $ + (if lenient then Lint.lenient else Lint.strict) reference document diff --git a/fluent-lint.cabal b/fluent-lint.cabal new file mode 100644 --- /dev/null +++ b/fluent-lint.cabal @@ -0,0 +1,64 @@ +cabal-version: 3.0 +name: fluent-lint +version: 1.0.0 +synopsis: Static analysis tool for validating Project Fluent translation files +description: + A command-line tool that parses Project Fluent translation files and checks + for syntax errors and consistency issues. + +license: EUPL-1.2 +license-file: LICENCE +author: IDA +maintainer: IDA +homepage: https://digital-autonomy.institute +bug-reports: https://issues.digital-autonomy.institute +category: Language +build-type: Simple +extra-doc-files: + CHANGELOG.md + +common common + default-language: GHC2021 + ghc-options: + -Weverything + -Wno-all-missed-specialisations + -Wno-missing-export-lists + -Wno-missing-import-lists + -Wno-missing-kind-signatures + -Wno-missing-local-signatures + -Wno-missing-safe-haskell-mode + -Wno-name-shadowing + -Wno-unsafe + + default-extensions: + ApplicativeDo + BlockArguments + DerivingStrategies + LambdaCase + NoImplicitPrelude + OverloadedRecordDot + OverloadedStrings + RecordWildCards + ViewPatterns + +executable fluent-lint + import: common + hs-source-dirs: app + main-is: Main.hs + autogen-modules: + Paths_fluent_lint + + other-modules: + Config + Document + Lint + Paths_fluent_lint + + build-depends: + base >=4.19 && <5, + containers >=0.6 && <0.9, + effectful >=2.6 && <2.7, + fluent-syntax >=1.1 && <1.2, + hspec-effectful >=1.1 && <1.2, + optparse-applicative >=0.18 && <0.20, + text >=2.1 && <2.3,