ShellCheck 0.9.0 → 0.11.0
raw patch · 25 files changed
Files
- CHANGELOG.md +53/−0
- README.md +19/−6
- ShellCheck.cabal +12/−12
- shellcheck.1.md +23/−4
- shellcheck.hs +62/−19
- src/ShellCheck/AST.hs +7/−5
- src/ShellCheck/ASTLib.hs +35/−16
- src/ShellCheck/Analytics.hs +426/−198
- src/ShellCheck/AnalyzerLib.hs +48/−27
- src/ShellCheck/CFG.hs +68/−56
- src/ShellCheck/CFGAnalysis.hs +30/−15
- src/ShellCheck/Checker.hs +55/−0
- src/ShellCheck/Checks/Commands.hs +38/−34
- src/ShellCheck/Checks/ControlFlow.hs +1/−1
- src/ShellCheck/Checks/Custom.hs +1/−1
- src/ShellCheck/Checks/ShellSupport.hs +172/−33
- src/ShellCheck/Data.hs +9/−1
- src/ShellCheck/Formatter/CheckStyle.hs +4/−4
- src/ShellCheck/Formatter/Diff.hs +8/−2
- src/ShellCheck/Formatter/GCC.hs +5/−5
- src/ShellCheck/Formatter/JSON1.hs +4/−4
- src/ShellCheck/Formatter/TTY.hs +7/−7
- src/ShellCheck/Interface.hs +21/−9
- src/ShellCheck/Parser.hs +129/−78
- test/shellcheck.hs +20/−17
CHANGELOG.md view
@@ -1,3 +1,56 @@+## v0.11.0 - 2025-08-03+### Added+- SC2327/SC2328: Warn about capturing the output of redirected commands.+- SC2329: Warn when (non-escaping) functions are never invoked.+- SC2330: Warn about unsupported glob matches with [[ .. ]] in BusyBox.+- SC2331: Suggest using standard -e instead of unary -a in tests.+- SC2332: Warn about `[ ! -o opt ]` being unconditionally true in Bash.+- SC3062: Warn about bashism `[ -o opt ]`.+- Optional `avoid-negated-conditions`: suggest replacing `[ ! a -eq b ]`+ with `[ a -ne b ]`, and similar for -ge/-lt/=/!=/etc (SC2335).+- Precompiled binaries for Linux riscv64 (linux.riscv64)++### Changed+- SC2002 about Useless Use Of Cat is now disabled by default. It can be+ re-enabled with `--enable=useless-use-of-cat` or equivalent directive.+- SC2236/SC2237 about replacing `[ ! -n .. ]` with `[ -z ]` and vice versa+ is now optional under `avoid-negated-conditions`.+- SC2015 about `A && B || C` no longer triggers when B is a test command.+- SC3012: Do not warn about `\<` and `\>` in test/[] as specified in POSIX.1-2024+- Diff output now uses / as path separator on Windows++### Fixed+- SC2218 about function use-before-define is now more accurate.+- SC2317 about unreachable commands is now less spammy for nested ones.+- SC2292, optional suggestion for [[ ]], now triggers for Busybox.+- Updates for Bash 5.3, including `${| cmd; }` and `source -p`++### Removed+- SC3013: removed since the operators `-ot/-nt/-ef` are specified in POSIX.1-2024+++## v0.10.0 - 2024-03-07+### Added+- Precompiled binaries for macOS ARM64 (darwin.aarch64)+- Added support for busybox sh+- Added flag --rcfile to specify an rc file by name.+- Added `extended-analysis=true` directive to enable/disable dataflow analysis+ (with a corresponding --extended-analysis flag).+- SC2324: Warn when x+=1 appends instead of increments+- SC2325: Warn about multiple `!`s in dash/sh.+- SC2326: Warn about `foo | ! bar` in bash/dash/sh.+- SC3012: Warn about lexicographic-compare bashism in test like in [ ]+- SC3013: Warn bashism `test _ -op/-nt/-ef _` like in [ ]+- SC3014: Warn bashism `test _ == _` like in [ ]+- SC3015: Warn bashism `test _ =~ _` like in [ ]+- SC3016: Warn bashism `test -v _` like in [ ]+- SC3017: Warn bashism `test -a _` like in [ ]++### Fixed+- source statements with here docs now work correctly+- "(Array.!): undefined array element" error should no longer occur++ ## v0.9.0 - 2022-12-12 ### Added - SC2316: Warn about 'local readonly foo' and similar (thanks, patrickxia!)
README.md view
@@ -77,7 +77,7 @@ * Sublime, through [SublimeLinter](https://github.com/SublimeLinter/SublimeLinter-shellcheck). -* Atom, through [Linter](https://github.com/AtomLinter/linter-shellcheck).+* Pulsar Edit (former Atom), through [linter-shellcheck-pulsar](https://github.com/pulsar-cooperative/linter-shellcheck-pulsar). * VSCode, through [vscode-shellcheck](https://github.com/timonwong/vscode-shellcheck). @@ -110,8 +110,11 @@ * [Codacy](https://www.codacy.com/) * [Code Climate](https://codeclimate.com/) * [Code Factor](https://www.codefactor.io/)+* [Codety](https://www.codety.io/) via the [Codety Scanner](https://github.com/codetyio/codety-scanner) * [CircleCI](https://circleci.com) via the [ShellCheck Orb](https://circleci.com/orbs/registry/orb/circleci/shellcheck) * [Github](https://github.com/features/actions) (only Linux)+* [Trunk Code Quality](https://trunk.io/code-quality) (universal linter; [allows you to explicitly version your shellcheck install](https://github.com/trunk-io/plugins/blob/bcbb361dcdbe4619af51ea7db474d7fb87540d20/.trunk/trunk.yaml#L32)) via the [shellcheck plugin](https://github.com/trunk-io/plugins/blob/main/linters/shellcheck/plugin.yaml)+* [CodeRabbit](https://coderabbit.ai/) Most other services, including [GitLab](https://about.gitlab.com/), let you install ShellCheck yourself, either through the system's package manager (see [Installing](#installing)),@@ -193,6 +196,12 @@ C:\> choco install shellcheck ``` +Or Windows (via [winget](https://github.com/microsoft/winget-pkgs)):++```cmd+C:\> winget install --id koalaman.shellcheck+```+ Or Windows (via [scoop](http://scoop.sh)): ```cmd@@ -221,17 +230,26 @@ nix-env -iA nixpkgs.shellcheck ``` +Using the [Flox package manager](https://flox.dev/)+```sh+flox install shellcheck+```+ Alternatively, you can download pre-compiled binaries for the latest release here: * [Linux, x86_64](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.linux.x86_64.tar.xz) (statically linked) * [Linux, armv6hf](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.linux.armv6hf.tar.xz), i.e. Raspberry Pi (statically linked) * [Linux, aarch64](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.linux.aarch64.tar.xz) aka ARM64 (statically linked)+* [macOS, aarch64](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.darwin.aarch64.tar.xz) * [macOS, x86_64](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.darwin.x86_64.tar.xz) * [Windows, x86](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.zip) or see the [GitHub Releases](https://github.com/koalaman/shellcheck/releases) for other releases (including the [latest](https://github.com/koalaman/shellcheck/releases/tag/latest) meta-release for daily git builds). +There are currently no official binaries for Apple Silicon, but third party builds are available via+[ShellCheck for Visual Studio Code](https://github.com/vscode-shellcheck/shellcheck-binaries/releases).+ Distro packages already come with a `man` page. If you are building from source, it can be installed with: ```console@@ -299,10 +317,6 @@ $ cabal install -Or if you intend to run the tests:-- $ cabal install --enable-tests- This will compile ShellCheck and install it to your `~/.cabal/bin` directory. Add this directory to your `PATH` (for bash, add this to your `~/.bashrc`):@@ -548,4 +562,3 @@ * The wiki has [long form descriptions](https://github.com/koalaman/shellcheck/wiki/Checks) for each warning, e.g. [SC2221](https://github.com/koalaman/shellcheck/wiki/SC2221). * ShellCheck does not attempt to enforce any kind of formatting or indenting style, so also check out [shfmt](https://github.com/mvdan/sh)!-
ShellCheck.cabal view
@@ -1,5 +1,5 @@ Name: ShellCheck-Version: 0.9.0+Version: 0.11.0 Synopsis: Shell script analysis tool License: GPL-3 License-file: LICENSE@@ -46,21 +46,21 @@ semigroups build-depends: -- The lower bounds are based on GHC 7.10.3- -- The upper bounds are based on GHC 9.4.3- aeson >= 1.4.0 && < 2.2,+ -- The upper bounds are based on GHC 9.12.1+ aeson >= 1.4.0 && < 2.3, array >= 0.5.1 && < 0.6, base >= 4.8.0.0 && < 5,- bytestring >= 0.10.6 && < 0.12,- containers >= 0.5.6 && < 0.7,- deepseq >= 1.4.1 && < 1.5,- Diff >= 0.4.0 && < 0.5,- fgl >= 5.7.0 && < 5.9,- filepath >= 1.4.0 && < 1.5,- mtl >= 2.2.2 && < 2.3,+ bytestring >= 0.10.6 && < 0.13,+ containers >= 0.5.6 && < 0.9,+ deepseq >= 1.4.1 && < 1.6,+ Diff >= 0.4.0 && < 1.1,+ fgl (>= 5.7.0 && < 5.8.1.0) || (>= 5.8.1.1 && < 5.9),+ filepath >= 1.4.0 && < 1.6,+ mtl >= 2.2.2 && < 2.4, parsec >= 3.1.14 && < 3.2,- QuickCheck >= 2.14.2 && < 2.15,+ QuickCheck >= 2.14.2 && < 2.17, regex-tdfa >= 1.2.0 && < 1.4,- transformers >= 0.4.2 && < 0.6,+ transformers >= 0.4.2 && < 0.7, -- getXdgDirectory from 1.2.3.0 directory >= 1.2.3 && < 1.4,
shellcheck.1.md view
@@ -56,6 +56,13 @@ options are cumulative, but all the codes can be specified at once, comma-separated as a single argument. +**--extended-analysis=true/false**++: Enable/disable Dataflow Analysis to identify more issues (default true). If+ ShellCheck uses too much CPU/RAM when checking scripts with several+ thousand lines of code, extended analysis can be disabled with this flag+ or a directive. This flag overrides directives and rc files.+ **-f** *FORMAT*, **--format=***FORMAT* : Specify the output format of shellcheck, which prints its results in the@@ -71,6 +78,11 @@ : Don't try to look for .shellcheckrc configuration files. +**--rcfile** *RCFILE*++: Prefer the specified configuration file over searching for one+ in the default locations.+ **-o**\ *NAME1*[,*NAME2*...],\ **--enable=***NAME1*[,*NAME2*...] : Enable optional checks. The special name *all* enables all of them.@@ -85,7 +97,8 @@ **-s**\ *shell*,\ **--shell=***shell* -: Specify Bourne shell dialect. Valid values are *sh*, *bash*, *dash* and *ksh*.+: Specify Bourne shell dialect. Valid values are *sh*, *bash*, *dash*, *ksh*,+ and *busybox*. The default is to deduce the shell from the file's `shell` directive, shebang, or `.bash/.bats/.dash/.ksh` extension, in that order. *sh* refers to POSIX `sh` (not the system's), and will warn of portability issues.@@ -243,6 +256,12 @@ : Enable an optional check by name, as listed with **--list-optional**. Only file-wide `enable` directives are considered. +**extended-analysis**+: Set to true/false to enable/disable dataflow analysis. Specifying+ `# shellcheck extended-analysis=false` in particularly large (2000+ line)+ auto-generated scripts will reduce ShellCheck's resource usage at the+ expense of certain checks. Extended analysis is enabled by default.+ **external-sources** : Set to `true` in `.shellcheckrc` to always allow ShellCheck to open arbitrary files from 'source' statements (the way most tools do).@@ -298,7 +317,7 @@ disable=SC2236 If no `.shellcheckrc` is found in any of the parent directories, ShellCheck-will look in `~/.shellcheckrc` followed by the XDG config directory+will look in `~/.shellcheckrc` followed by the `$XDG_CONFIG_HOME` (usually `~/.config/shellcheckrc`) on Unix, or `%APPDATA%/shellcheckrc` on Windows. Only the first file found will be used. @@ -378,10 +397,10 @@ # COPYRIGHT -Copyright 2012-2022, Vidar Holen and contributors.+Copyright 2012-2025, Vidar Holen and contributors. Licensed under the GNU General Public License version 3 or later, see https://gnu.org/licenses/gpl.html # SEE ALSO -sh(1) bash(1)+sh(1), bash(1), dash(1), ksh(1)
shellcheck.hs view
@@ -76,7 +76,8 @@ externalSources :: Bool, sourcePaths :: [FilePath], formatterOptions :: FormatterOptions,- minSeverity :: Severity+ minSeverity :: Severity,+ rcfile :: Maybe FilePath } defaultOptions = Options {@@ -86,7 +87,8 @@ formatterOptions = newFormatterOptions { foColorOption = ColorAuto },- minSeverity = StyleC+ minSeverity = StyleC,+ rcfile = Nothing } usageHeader = "Usage: shellcheck [OPTIONS...] FILES..."@@ -100,6 +102,8 @@ (ReqArg (Flag "include") "CODE1,CODE2..") "Consider only given types of warnings", Option "e" ["exclude"] (ReqArg (Flag "exclude") "CODE1,CODE2..") "Exclude types of warnings",+ Option "" ["extended-analysis"]+ (ReqArg (Flag "extended-analysis") "bool") "Perform dataflow analysis (default true)", Option "f" ["format"] (ReqArg (Flag "format") "FORMAT") $ "Output format (" ++ formatList ++ ")",@@ -107,6 +111,9 @@ (NoArg $ Flag "list-optional" "true") "List checks disabled by default", Option "" ["norc"] (NoArg $ Flag "norc" "true") "Don't look for .shellcheckrc files",+ Option "" ["rcfile"]+ (ReqArg (Flag "rcfile") "RCFILE")+ "Prefer the specified configuration file over searching for one", Option "o" ["enable"] (ReqArg (Flag "enable") "check1,check2..") "List of optional checks to enable (or 'all')",@@ -115,7 +122,7 @@ "Specify path when looking for sourced files (\"SCRIPTDIR\" for script's dir)", Option "s" ["shell"] (ReqArg (Flag "shell") "SHELLNAME")- "Specify dialect (sh, bash, dash, ksh)",+ "Specify dialect (sh, bash, dash, ksh, busybox)", Option "S" ["severity"] (ReqArg (Flag "severity") "SEVERITY") "Minimum severity of errors to consider (error, warning, info, style)",@@ -252,9 +259,9 @@ else SomeProblems parseEnum name value list =- case filter ((== value) . fst) list of- [(name, value)] -> return value- [] -> do+ case lookup value list of+ Just value -> return value+ Nothing -> do printErr $ "Unknown value for --" ++ name ++ ". " ++ "Valid options are: " ++ (intercalate ", " $ map fst list) throwError SupportFailure@@ -367,6 +374,11 @@ } } + Flag "rcfile" str -> do+ return options {+ rcfile = Just str+ }+ Flag "enable" value -> let cs = checkSpec options in return options { checkSpec = cs {@@ -374,6 +386,14 @@ } } + Flag "extended-analysis" str -> do+ value <- parseBool str+ return options {+ checkSpec = (checkSpec options) {+ csExtendedAnalysis = Just value+ }+ }+ -- This flag is handled specially in 'process' Flag "format" _ -> return options @@ -391,12 +411,20 @@ throwError SyntaxFailure return (Prelude.read num :: Integer) + parseBool str = do+ case str of+ "true" -> return True+ "false" -> return False+ _ -> do+ printErr $ "Invalid boolean, expected true/false: " ++ str+ throwError SyntaxFailure+ ioInterface :: Options -> [FilePath] -> IO (SystemInterface IO) ioInterface options files = do inputs <- mapM normalize files cache <- newIORef emptyCache configCache <- newIORef ("", Nothing)- return SystemInterface {+ return (newSystemInterface :: SystemInterface IO) { siReadFile = get cache inputs, siFindSource = findSourceFile inputs (sourcePaths options), siGetConfig = getConfig configCache@@ -441,19 +469,34 @@ fallback :: FilePath -> IOException -> IO FilePath fallback path _ = return path + -- Returns the name and contents of .shellcheckrc for the given file- getConfig cache filename = do- path <- normalize filename- let dir = takeDirectory path- (previousPath, result) <- readIORef cache- if dir == previousPath- then return result- else do- paths <- getConfigPaths dir- result <- findConfig paths- writeIORef cache (dir, result)- return result+ getConfig cache filename =+ case rcfile options of+ Just file -> do+ -- We have a specified rcfile. Ignore normal rcfile resolution.+ (path, result) <- readIORef cache+ if path == "/"+ then return result+ else do+ result <- readConfig file+ when (isNothing result) $+ hPutStrLn stderr $ "Warning: unable to read --rcfile " ++ file+ writeIORef cache ("/", result)+ return result + Nothing -> do+ path <- normalize filename+ let dir = takeDirectory path+ (previousPath, result) <- readIORef cache+ if dir == previousPath+ then return result+ else do+ paths <- getConfigPaths dir+ result <- findConfig paths+ writeIORef cache (dir, result)+ return result+ findConfig paths = case paths of (file:rest) -> do@@ -490,7 +533,7 @@ where handler :: FilePath -> IOException -> IO (String, Bool) handler file err = do- putStrLn $ file ++ ": " ++ show err+ hPutStrLn stderr $ file ++ ": " ++ show err return ("", True) andM a b arg = do
src/ShellCheck/AST.hs view
@@ -31,6 +31,7 @@ data Quoted = Quoted | Unquoted deriving (Show, Eq) data Dashed = Dashed | Undashed deriving (Show, Eq)+data Piped = Piped | Unpiped deriving (Show, Eq) data AssignmentMode = Assign | Append deriving (Show, Eq) newtype FunctionKeyword = FunctionKeyword Bool deriving (Show, Eq) newtype FunctionParentheses = FunctionParentheses Bool deriving (Show, Eq)@@ -84,7 +85,7 @@ | Inner_T_DollarDoubleQuoted [t] | Inner_T_DollarExpansion [t] | Inner_T_DollarSingleQuoted String- | Inner_T_DollarBraceCommandExpansion [t]+ | Inner_T_DollarBraceCommandExpansion Piped [t] | Inner_T_Done | Inner_T_DoubleQuoted [t] | Inner_T_EOF@@ -138,7 +139,7 @@ | Inner_T_WhileExpression [t] [t] | Inner_T_Annotation [Annotation] t | Inner_T_Pipe String- | Inner_T_CoProc (Maybe String) t+ | Inner_T_CoProc (Maybe Token) t | Inner_T_CoProcBody t | Inner_T_Include t | Inner_T_SourceCommand t t@@ -152,6 +153,7 @@ | ShellOverride String | SourcePath String | ExternalSources Bool+ | ExtendedAnalysis Bool deriving (Show, Eq) data ConditionType = DoubleBracket | SingleBracket deriving (Show, Eq) @@ -205,7 +207,7 @@ pattern T_Arithmetic id c = OuterToken id (Inner_T_Arithmetic c) pattern T_Array id t = OuterToken id (Inner_T_Array t) pattern TA_Sequence id l = OuterToken id (Inner_TA_Sequence l)-pattern TA_Parentesis id t = OuterToken id (Inner_TA_Parenthesis t)+pattern TA_Parenthesis id t = OuterToken id (Inner_TA_Parenthesis t) pattern T_Assignment id mode var indices value = OuterToken id (Inner_T_Assignment mode var indices value) pattern TA_Trinary id t1 t2 t3 = OuterToken id (Inner_TA_Trinary t1 t2 t3) pattern TA_Unary id op t1 = OuterToken id (Inner_TA_Unary op t1)@@ -227,7 +229,7 @@ pattern TC_Or id typ str t1 t2 = OuterToken id (Inner_TC_Or typ str t1 t2) pattern TC_Unary id typ op token = OuterToken id (Inner_TC_Unary typ op token) pattern T_DollarArithmetic id c = OuterToken id (Inner_T_DollarArithmetic c)-pattern T_DollarBraceCommandExpansion id list = OuterToken id (Inner_T_DollarBraceCommandExpansion list)+pattern T_DollarBraceCommandExpansion id pipe list = OuterToken id (Inner_T_DollarBraceCommandExpansion pipe list) pattern T_DollarBraced id braced op = OuterToken id (Inner_T_DollarBraced braced op) pattern T_DollarBracket id c = OuterToken id (Inner_T_DollarBracket c) pattern T_DollarDoubleQuoted id list = OuterToken id (Inner_T_DollarDoubleQuoted list)@@ -258,7 +260,7 @@ pattern T_UntilExpression id c l = OuterToken id (Inner_T_UntilExpression c l) pattern T_WhileExpression id c l = OuterToken id (Inner_T_WhileExpression c l) -{-# COMPLETE T_AND_IF, T_Bang, T_Case, TC_Empty, T_CLOBBER, T_DGREAT, T_DLESS, T_DLESSDASH, T_Do, T_DollarSingleQuoted, T_Done, T_DSEMI, T_Elif, T_Else, T_EOF, T_Esac, T_Fi, T_For, T_Glob, T_GREATAND, T_Greater, T_If, T_In, T_Lbrace, T_Less, T_LESSAND, T_LESSGREAT, T_Literal, T_Lparen, T_NEWLINE, T_OR_IF, T_ParamSubSpecialChar, T_Pipe, T_Rbrace, T_Rparen, T_Select, T_Semi, T_SingleQuoted, T_Then, T_UnparsedIndex, T_Until, T_While, TA_Assignment, TA_Binary, TA_Expansion, T_AndIf, T_Annotation, T_Arithmetic, T_Array, TA_Sequence, TA_Parentesis, T_Assignment, TA_Trinary, TA_Unary, TA_Variable, T_Backgrounded, T_Backticked, T_Banged, T_BatsTest, T_BraceExpansion, T_BraceGroup, TC_And, T_CaseExpression, TC_Binary, TC_Group, TC_Nullary, T_Condition, T_CoProcBody, T_CoProc, TC_Or, TC_Unary, T_DollarArithmetic, T_DollarBraceCommandExpansion, T_DollarBraced, T_DollarBracket, T_DollarDoubleQuoted, T_DollarExpansion, T_DoubleQuoted, T_Extglob, T_FdRedirect, T_ForArithmetic, T_ForIn, T_Function, T_HereDoc, T_HereString, T_IfExpression, T_Include, T_IndexedElement, T_IoDuplicate, T_IoFile, T_NormalWord, T_OrIf, T_Pipeline, T_ProcSub, T_Redirecting, T_Script, T_SelectIn, T_SimpleCommand, T_SourceCommand, T_Subshell, T_UntilExpression, T_WhileExpression #-}+{-# COMPLETE T_AND_IF, T_Bang, T_Case, TC_Empty, T_CLOBBER, T_DGREAT, T_DLESS, T_DLESSDASH, T_Do, T_DollarSingleQuoted, T_Done, T_DSEMI, T_Elif, T_Else, T_EOF, T_Esac, T_Fi, T_For, T_Glob, T_GREATAND, T_Greater, T_If, T_In, T_Lbrace, T_Less, T_LESSAND, T_LESSGREAT, T_Literal, T_Lparen, T_NEWLINE, T_OR_IF, T_ParamSubSpecialChar, T_Pipe, T_Rbrace, T_Rparen, T_Select, T_Semi, T_SingleQuoted, T_Then, T_UnparsedIndex, T_Until, T_While, TA_Assignment, TA_Binary, TA_Expansion, T_AndIf, T_Annotation, T_Arithmetic, T_Array, TA_Sequence, TA_Parenthesis, T_Assignment, TA_Trinary, TA_Unary, TA_Variable, T_Backgrounded, T_Backticked, T_Banged, T_BatsTest, T_BraceExpansion, T_BraceGroup, TC_And, T_CaseExpression, TC_Binary, TC_Group, TC_Nullary, T_Condition, T_CoProcBody, T_CoProc, TC_Or, TC_Unary, T_DollarArithmetic, T_DollarBraceCommandExpansion, T_DollarBraced, T_DollarBracket, T_DollarDoubleQuoted, T_DollarExpansion, T_DoubleQuoted, T_Extglob, T_FdRedirect, T_ForArithmetic, T_ForIn, T_Function, T_HereDoc, T_HereString, T_IfExpression, T_Include, T_IndexedElement, T_IoDuplicate, T_IoFile, T_NormalWord, T_OrIf, T_Pipeline, T_ProcSub, T_Redirecting, T_Script, T_SelectIn, T_SimpleCommand, T_SourceCommand, T_Subshell, T_UntilExpression, T_WhileExpression #-} instance Eq Token where OuterToken _ a == OuterToken _ b = a == b
src/ShellCheck/ASTLib.hs view
@@ -31,6 +31,7 @@ import Data.Functor.Identity import Data.List import Data.Maybe+import qualified Data.List.NonEmpty as NE import qualified Data.Map as Map import Numeric (showHex) @@ -157,9 +158,10 @@ _ -> False -- Is this token a flag where the - is unquoted?-isUnquotedFlag token = fromMaybe False $ do- str <- getLeadingUnquotedString token- return $ "-" `isPrefixOf` str+isUnquotedFlag token =+ case getLeadingUnquotedString token of+ Just ('-':_) -> True+ _ -> False -- getGnuOpts "erd:u:" will parse a list of arguments tokens like `read` -- -re -d : -u 3 bar@@ -444,6 +446,12 @@ -- Is this token a string literal? isLiteral t = isJust $ getLiteralString t +-- Is this token a string literal number?+isLiteralNumber t = fromMaybe False $ do+ s <- getLiteralString t+ guard $ all isDigit s+ return True+ -- Escape user data for messages. -- Messages generally avoid repeating user data, but sometimes it's helpful. e4m = escapeForMessage@@ -553,7 +561,7 @@ case t of T_DollarExpansion _ [c] -> extract c T_Backticked _ [c] -> extract c- T_DollarBraceCommandExpansion _ [c] -> extract c+ T_DollarBraceCommandExpansion _ _ [c] -> extract c _ -> Nothing where extract (T_Pipeline _ _ [cmd]) = getCommandName cmd@@ -608,7 +616,7 @@ T_Annotation _ _ t -> getCommandSequences t T_DollarExpansion _ cmds -> [cmds]- T_DollarBraceCommandExpansion _ cmds -> [cmds]+ T_DollarBraceCommandExpansion _ _ cmds -> [cmds] T_Backticked _ cmds -> [cmds] _ -> [] @@ -758,8 +766,8 @@ prop_executableFromShebang7 = executableFromShebang "/usr/bin/env --split-string bash -x" == "bash" prop_executableFromShebang8 = executableFromShebang "/usr/bin/env --split-string foo=bar bash -x" == "bash" prop_executableFromShebang9 = executableFromShebang "/usr/bin/env foo=bar dash" == "dash"-prop_executableFromShebang10 = executableFromShebang "/bin/busybox sh" == "ash"-prop_executableFromShebang11 = executableFromShebang "/bin/busybox ash" == "ash"+prop_executableFromShebang10 = executableFromShebang "/bin/busybox sh" == "busybox sh"+prop_executableFromShebang11 = executableFromShebang "/bin/busybox ash" == "busybox ash" -- Get the shell executable from a string like '/usr/bin/env bash' executableFromShebang :: String -> String@@ -776,7 +784,8 @@ [x] -> basename x (first:second:args) | basename first == "busybox" -> case basename second of- "sh" -> "ash" -- busybox sh is ash+ "sh" -> "busybox sh"+ "ash" -> "busybox ash" x -> x (first:args) | basename first == "env" -> fromEnvArgs args@@ -856,8 +865,7 @@ -- Get the variables from indices like ["x", "y"] in ${var[x+y+1]} prop_getIndexReferences1 = getIndexReferences "var[x+y+1]" == ["x", "y"] getIndexReferences s = fromMaybe [] $ do- match <- matchRegex re s- index <- match !!! 0+ index:_ <- matchRegex re s return $ matchAllStrings variableNameRegex index where re = mkRegex "(\\[.*\\])"@@ -868,8 +876,7 @@ prop_getOffsetReferences4 = getOffsetReferences "[foo]:bar:baz" == ["bar", "baz"] getOffsetReferences mods = fromMaybe [] $ do -- if mods start with [, then drop until ]- match <- matchRegex re mods- offsets <- match !!! 1+ _:offsets:_ <- matchRegex re mods return $ matchAllStrings variableNameRegex offsets where re = mkRegex "^(\\[.+\\])? *:([^-=?+].*)"@@ -886,11 +893,17 @@ in getBracedReference str == str _ -> False +-- Return the referenced variable if (and only if) it's an unmodified parameter expansion.+getUnmodifiedParameterExpansion t =+ case t of+ T_DollarBraced _ _ list -> do+ let str = concat $ oversimplify list+ guard $ getBracedReference str == str+ return str+ _ -> Nothing+ --- A list of the element and all its parents up to the root node.-getPath tree t = t :- case Map.lookup (getId t) tree of- Nothing -> []- Just parent -> getPath tree parent+getPath tree = NE.unfoldr $ \t -> (t, Map.lookup (getId t) tree) isClosingFileOp op = case op of@@ -902,6 +915,12 @@ case root of T_Annotation _ list _ -> [s | EnableComment s <- list] _ -> []++getExtendedAnalysisDirective :: Token -> Maybe Bool+getExtendedAnalysisDirective root =+ case root of+ T_Annotation _ list _ -> listToMaybe $ [s | ExtendedAnalysis s <- list]+ _ -> Nothing return [] runTests = $quickCheckAll
src/ShellCheck/Analytics.hs view
@@ -1,5 +1,5 @@ {-- Copyright 2012-2022 Vidar Holen+ Copyright 2012-2024 Vidar Holen This file is part of ShellCheck. https://www.shellcheck.net@@ -19,6 +19,7 @@ -} {-# LANGUAGE TemplateHaskell #-} {-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE PatternGuards #-} module ShellCheck.Analytics (checker, optionalChecks, ShellCheck.Analytics.runTests) where import ShellCheck.AST@@ -46,6 +47,7 @@ import Data.Ord import Data.Semigroup import Debug.Trace -- STRIP+import qualified Data.List.NonEmpty as NE import qualified Data.Map.Strict as Map import qualified Data.Set as S import Test.QuickCheck.All (forAllProperties)@@ -101,8 +103,7 @@ nodeChecks :: [Parameters -> Token -> Writer [TokenComment] ()] nodeChecks = [- checkUuoc- ,checkPipePitfalls+ checkPipePitfalls ,checkForInQuoted ,checkForInLs ,checkShorthandIf@@ -122,6 +123,7 @@ ,checkCaseAgainstGlob ,checkCommarrays ,checkOrNeq+ ,checkAndEq ,checkEchoWc ,checkConstantIfs ,checkPipedAssignment@@ -181,7 +183,6 @@ ,checkPipeToNowhere ,checkForLoopGlobVariables ,checkSubshelledTests- ,checkInvertedStringTest ,checkRedirectionToCommand ,checkDollarQuoteParen ,checkUselessBang@@ -201,6 +202,9 @@ ,checkOverwrittenExitCode ,checkUnnecessaryArithmeticExpansionIndex ,checkUnnecessaryParens+ ,checkPlusEqualsNumber+ ,checkExpansionWithRedirection+ ,checkUnaryTestA ] optionalChecks = map fst optionalTreeChecks@@ -229,6 +233,13 @@ }, nodeChecksToTreeCheck [checkNullaryExpansionTest]) ,(newCheckDescription {+ cdName = "avoid-negated-conditions",+ cdDescription = "Suggest removing unnecessary comparison negations",+ cdPositive = "[ ! \"$var\" -eq 1 ]",+ cdNegative = "[ \"$var\" -ne 1 ]"+ }, nodeChecksToTreeCheck [checkUnnecessarilyInvertedTest])++ ,(newCheckDescription { cdName = "add-default-case", cdDescription = "Suggest adding a default case in `case` statements", cdPositive = "case $? in 0) echo 'Success';; esac",@@ -269,6 +280,13 @@ cdPositive = "rm -r \"$(get_chroot_dir)/home\"", cdNegative = "set -e; dir=\"$(get_chroot_dir)\"; rm -r \"$dir/home\"" }, checkExtraMaskedReturns)++ ,(newCheckDescription {+ cdName = "useless-use-of-cat",+ cdDescription = "Check for Useless Use Of Cat (UUOC)",+ cdPositive = "cat foo | grep bar",+ cdNegative = "grep bar foo"+ }, nodeChecksToTreeCheck [checkUuoc]) ] optionalCheckMap :: Map.Map String (Parameters -> Token -> [TokenComment])@@ -343,13 +361,11 @@ hasFloatingPoint params = shellType params == Ksh -- Checks whether the current parent path is part of a condition-isCondition [] = False-isCondition [_] = False-isCondition (child:parent:rest) =- case child of- T_BatsTest {} -> True -- count anything in a @test as conditional- _ -> getId child `elem` map getId (getConditionChildren parent) || isCondition (parent:rest)+isCondition (x NE.:| xs) = foldr go (const False) xs x where+ go _ _ T_BatsTest{} = True -- count anything in a @test as conditional+ go parent go_rest child =+ getId child `elem` map getId (getConditionChildren parent) || go_rest parent getConditionChildren t = case t of T_AndIf _ left right -> [left]@@ -466,9 +482,8 @@ where isCommonCommand (Just s) = s `elem` commonCommands isCommonCommand _ = False- firstWordIsArg list = fromMaybe False $ do- head <- list !!! 0- return $ isGlob head || isUnquotedFlag head+ firstWordIsArg (head:_) = isGlob head || isUnquotedFlag head+ firstWordIsArg [] = False checkAssignAteCommand _ _ = return () @@ -489,18 +504,13 @@ checkWrongArithmeticAssignment params (T_SimpleCommand id [T_Assignment _ _ _ _ val] []) = sequence_ $ do str <- getNormalString val- match <- matchRegex regex str- var <- match !!! 0- op <- match !!! 1- Map.lookup var references+ var:op:_ <- matchRegex regex str+ guard $ S.member var references return . warn (getId val) 2100 $ "Use $((..)) for arithmetics, e.g. i=$((i " ++ op ++ " 2))" where regex = mkRegex "^([_a-zA-Z][_a-zA-Z0-9]*)([+*-]).+$"- references = foldl (flip ($)) Map.empty (map insertRef $ variableFlow params)- insertRef (Assignment (_, _, name, _)) =- Map.insert name ()- insertRef _ = Prelude.id+ references = S.fromList [name | Assignment (_, _, name, _) <- variableFlow params] getNormalString (T_NormalWord _ words) = do parts <- mapM getLiterals words@@ -562,7 +572,7 @@ hasParameter "print0", hasParameter "printf" ]) $ warn (getId find) 2038- "Use -print0/-0 or -exec + to allow for non-alphanumeric filenames."+ "Use 'find .. -print0 | xargs -0 ..' or 'find .. -exec .. +' to allow non-alphanumeric filenames." for ["ps", "grep"] $ \(ps:grep:_) ->@@ -645,10 +655,10 @@ prop_checkShebang10 = verifyNotTree checkShebang "#!foo\n# shellcheck shell=sh ignore=SC2239\ntrue" prop_checkShebang11 = verifyTree checkShebang "#!/bin/sh/\ntrue" prop_checkShebang12 = verifyTree checkShebang "#!/bin/sh/ -xe\ntrue"-prop_checkShebang13 = verifyTree checkShebang "#!/bin/busybox sh"+prop_checkShebang13 = verifyNotTree checkShebang "#!/bin/busybox sh" prop_checkShebang14 = verifyNotTree checkShebang "#!/bin/busybox sh\n# shellcheck shell=sh\n" prop_checkShebang15 = verifyNotTree checkShebang "#!/bin/busybox sh\n# shellcheck shell=dash\n"-prop_checkShebang16 = verifyTree checkShebang "#!/bin/busybox ash"+prop_checkShebang16 = verifyNotTree checkShebang "#!/bin/busybox ash" prop_checkShebang17 = verifyNotTree checkShebang "#!/bin/busybox ash\n# shellcheck shell=dash\n" prop_checkShebang18 = verifyNotTree checkShebang "#!/bin/busybox ash\n# shellcheck shell=sh\n" checkShebang params (T_Annotation _ list t) =@@ -796,7 +806,7 @@ where check t@(T_DollarExpansion _ c) = examine t c check t@(T_Backticked _ c) = examine t c- check t@(T_DollarBraceCommandExpansion _ c) = examine t c+ check t@(T_DollarBraceCommandExpansion _ _ c) = examine t c check _ = return () tree = parentMap params examine t contents =@@ -845,14 +855,14 @@ getRedirs _ = [] special x = "/dev/" `isPrefixOf` concat (oversimplify x) isInput t =- case drop 1 $ getPath (parentMap params) t of+ case NE.tail $ getPath (parentMap params) t of T_IoFile _ op _:_ -> case op of T_Less _ -> True _ -> False _ -> False isOutput t =- case drop 1 $ getPath (parentMap params) t of+ case NE.tail $ getPath (parentMap params) t of T_IoFile _ op _:_ -> case op of T_Greater _ -> True@@ -878,13 +888,16 @@ prop_checkShorthandIf6 = verifyNot checkShorthandIf "if foo && bar || baz; then true; fi" prop_checkShorthandIf7 = verifyNot checkShorthandIf "while foo && bar || baz; do true; done" prop_checkShorthandIf8 = verify checkShorthandIf "if true; then foo && bar || baz; fi"-checkShorthandIf params x@(T_OrIf _ (T_AndIf id _ _) (T_Pipeline _ _ t))- | not (isOk t || inCondition) =+prop_checkShorthandIf9 = verifyNot checkShorthandIf "foo && [ -x /file ] || bar"+prop_checkShorthandIf10 = verifyNot checkShorthandIf "foo && bar || true"+prop_checkShorthandIf11 = verify checkShorthandIf "foo && bar || false"+checkShorthandIf params x@(T_OrIf _ (T_AndIf id _ b) (T_Pipeline _ _ t))+ | not (isOk t || inCondition) && not (isTestCommand b) = info id 2015 "Note that A && B || C is not if-then-else. C may run when A is true." where isOk [t] = isAssignment t || fromMaybe False (do name <- getCommandBasename t- return $ name `elem` ["echo", "exit", "return", "printf"])+ return $ name `elem` ["echo", "exit", "return", "printf", "true", ":"]) isOk _ = False inCondition = isCondition $ getPath (parentMap params) x checkShorthandIf _ _ = return ()@@ -974,32 +987,32 @@ prop_checkArrayWithoutIndex10 = verifyTree checkArrayWithoutIndex "read -ra arr <<< 'foo bar'; echo \"$arr\"" prop_checkArrayWithoutIndex11 = verifyNotTree checkArrayWithoutIndex "read -rpfoobar r; r=42" checkArrayWithoutIndex params _ =- doVariableFlowAnalysis readF writeF defaultMap (variableFlow params)+ doVariableFlowAnalysis readF writeF defaultSet (variableFlow params) where- defaultMap = Map.fromList $ map (\x -> (x,())) arrayVariables+ defaultSet = S.fromList arrayVariables readF _ (T_DollarBraced id _ token) _ = do- map <- get+ s <- get return . maybeToList $ do name <- getLiteralString token- assigned <- Map.lookup name map+ guard $ S.member name s return $ makeComment WarningC id 2128 "Expanding an array without an index only gives the first element." readF _ _ _ = return [] writeF _ (T_Assignment id mode name [] _) _ (DataString _) = do- isArray <- gets (Map.member name)+ isArray <- gets (S.member name) return $ if not isArray then [] else case mode of Assign -> [makeComment WarningC id 2178 "Variable was used as an array but is now assigned a string."] Append -> [makeComment WarningC id 2179 "Use array+=(\"item\") to append items to an array."] writeF _ t name (DataArray _) = do- modify (Map.insert name ())+ modify (S.insert name) return [] writeF _ expr name _ = do if isIndexed expr- then modify (Map.insert name ())- else modify (Map.delete name)+ then modify (S.insert name)+ else modify (S.delete name) return [] isIndexed expr =@@ -1086,7 +1099,7 @@ return $ if name == "find" then getFindCommand cmd else if name == "git" then getGitCommand cmd else if name == "mumps" then getMumpsCommand cmd else name isProbablyOk =- any isOkAssignment (take 3 $ getPath parents t)+ any isOkAssignment (NE.take 3 $ getPath parents t) || commandName `elem` [ "trap" ,"sh"@@ -1098,8 +1111,11 @@ ,"xprop" ,"alias" ,"sudo" -- covering "sudo sh" and such+ ,"doas" -- same as sudo+ ,"run0" -- same as sudo ,"docker" -- like above ,"podman"+ ,"oc" ,"dpkg-query" ,"jq" -- could also check that user provides --arg ,"rename"@@ -1203,6 +1219,7 @@ case shellType params of Sh -> return () -- These are unsupported and will be caught by bashism checks. Dash -> err id 2073 $ "Escape \\" ++ op ++ " to prevent it redirecting."+ BusyboxSh -> err id 2073 $ "Escape \\" ++ op ++ " to prevent it redirecting." _ -> err id 2073 $ "Escape \\" ++ op ++ " to prevent it redirecting (or switch to [[ .. ]])." when (op `elem` arithmeticBinaryTestOps) $ do@@ -1263,7 +1280,8 @@ str = concat $ oversimplify c var = getBracedReference str in fromMaybe False $ do- state <- CF.getIncomingState (cfgAnalysis params) id+ cfga <- cfgAnalysis params+ state <- CF.getIncomingState cfga id value <- Map.lookup var $ CF.variablesInScope state return $ CF.numericalStatus (CF.variableValue value) >= CF.NumericalStatusMaybe _ ->@@ -1441,14 +1459,14 @@ prop_checkConstantNullary6 = verify checkConstantNullary "[ 1 ]" prop_checkConstantNullary7 = verify checkConstantNullary "[ false ]" checkConstantNullary _ (TC_Nullary _ _ t) | isConstant t =- case fromMaybe "" $ getLiteralString t of+ case onlyLiteralString t of "false" -> err (getId t) 2158 "[ false ] is true. Remove the brackets." "0" -> err (getId t) 2159 "[ 0 ] is true. Use 'false' instead." "true" -> style (getId t) 2160 "Instead of '[ true ]', just use 'true'." "1" -> style (getId t) 2161 "Instead of '[ 1 ]', use 'true'." _ -> err (getId t) 2078 "This expression is constant. Did you forget a $ somewhere?" where- string = fromMaybe "" $ getLiteralString t+ string = onlyLiteralString t checkConstantNullary _ _ = return () @@ -1457,9 +1475,8 @@ prop_checkForDecimals3 = verifyNot checkForDecimals "declare -A foo; foo[1.2]=bar" checkForDecimals params t@(TA_Expansion id _) = sequence_ $ do guard $ not (hasFloatingPoint params)- str <- getLiteralString t- first <- str !!! 0- guard $ isDigit first && '.' `elem` str+ first:rest <- getLiteralString t+ guard $ isDigit first && '.' `elem` rest return $ err id 2079 "(( )) doesn't support decimals. Use bc or awk." checkForDecimals _ _ = return () @@ -1493,7 +1510,7 @@ where isException [] = True isException s@(h:_) = any (`elem` "/.:#%?*@$-!+=^,") s || isDigit h- getWarning = fromMaybe noWarning . msum . map warningFor $ parents params t+ getWarning = fromMaybe noWarning . msum . NE.map warningFor $ parents params t warningFor t = case t of T_Arithmetic {} -> return normalWarning@@ -1524,6 +1541,7 @@ prop_checkComparisonAgainstGlob4 = verifyNot checkComparisonAgainstGlob "[ $cow = foo ]" prop_checkComparisonAgainstGlob5 = verify checkComparisonAgainstGlob "[[ $cow != $bar ]]" prop_checkComparisonAgainstGlob6 = verify checkComparisonAgainstGlob "[ $f != /* ]"+prop_checkComparisonAgainstGlob7 = verify checkComparisonAgainstGlob "#!/bin/busybox sh\n[[ $f == *foo* ]]" checkComparisonAgainstGlob _ (TC_Binary _ DoubleBracket op _ (T_NormalWord id [T_DollarBraced _ _ _])) | op `elem` ["=", "==", "!="] = warn id 2053 $ "Quote the right-hand side of " ++ op ++ " in [[ ]] to prevent glob matching."@@ -1531,10 +1549,14 @@ | op `elem` ["=", "==", "!="] && isGlob word = err (getId word) 2081 msg where- msg = if isBashLike params+ msg = if (shellType params) `elem` [Bash, Ksh] -- Busybox does not support glob matching then "[ .. ] can't match globs. Use [[ .. ]] or case statement." else "[ .. ] can't match globs. Use a case statement." +checkComparisonAgainstGlob params (TC_Binary _ DoubleBracket op _ word)+ | shellType params == BusyboxSh && op `elem` ["=", "==", "!="] && isGlob word =+ err (getId word) 2330 "BusyBox [[ .. ]] does not support glob matching. Use a case statement."+ checkComparisonAgainstGlob _ _ = return () prop_checkCaseAgainstGlob1 = verify checkCaseAgainstGlob "case foo in lol$n) foo;; esac"@@ -1621,6 +1643,64 @@ checkOrNeq _ _ = return () +prop_checkAndEq1 = verifyNot checkAndEq "cow=0; foo=0; if [[ $lol -eq cow && $lol -eq foo ]]; then echo foo; fi"+prop_checkAndEq2 = verifyNot checkAndEq "lol=0 foo=0; (( a==lol && a==foo ))"+prop_checkAndEq3 = verify checkAndEq "[ \"$a\" = lol && \"$a\" = foo ]"+prop_checkAndEq4 = verifyNot checkAndEq "[ a = $cow && b = $foo ]"+prop_checkAndEq5 = verifyNot checkAndEq "[[ $a = /home && $a = */public_html/* ]]"+prop_checkAndEq6 = verify checkAndEq "[ $a = a ] && [ $a = b ]"+prop_checkAndEq7 = verify checkAndEq "[ $a = a ] && [ $a = b ] || true"+prop_checkAndEq8 = verifyNot checkAndEq "[[ $a == x && $a == x ]]"+prop_checkAndEq9 = verifyNot checkAndEq "[ 0 -eq $FOO ] && [ 0 -eq $BAR ]"+prop_checkAndEq10 = verify checkAndEq "(( a == 1 && a == 2 ))"+prop_checkAndEq11 = verify checkAndEq "[ $x -eq 1 ] && [ $x -eq 2 ]"+prop_checkAndEq12 = verify checkAndEq "[ 1 -eq $x ] && [ $x -eq 2 ]"+prop_checkAndEq13 = verifyNot checkAndEq "[ 1 -eq $x ] && [ $x -eq 1 ]"+prop_checkAndEq14 = verifyNot checkAndEq "[ $a = $b ] && [ $a = $c ]"++checkAndEqOperands "-eq" rhs1 rhs2 = isLiteralNumber rhs1 && isLiteralNumber rhs2+checkAndEqOperands op rhs1 rhs2 | op == "=" || op == "==" = isLiteral rhs1 && isLiteral rhs2+checkAndEqOperands _ _ _ = False++-- For test-level "and": [ x = y -a x = z ]+checkAndEq _ (TC_And id typ op (TC_Binary _ _ op1 lhs1 rhs1 ) (TC_Binary _ _ op2 lhs2 rhs2))+ | op1 == op2 && lhs1 == lhs2 && rhs1 /= rhs2 && checkAndEqOperands op1 rhs1 rhs2 =+ warn id 2333 $ "You probably wanted " ++ (if typ == SingleBracket then "-o" else "||") ++ " here, otherwise it's always false."++-- For arithmetic context "and"+checkAndEq _ (TA_Binary id "&&" (TA_Binary _ "==" lhs1 rhs1) (TA_Binary _ "==" lhs2 rhs2))+ | lhs1 == lhs2 && isLiteralNumber rhs1 && isLiteralNumber rhs2 =+ warn id 2334 "You probably wanted || here, otherwise it's always false."++-- For command level "and": [ x = y ] && [ x = z ]+checkAndEq _ (T_AndIf id lhs rhs) = sequence_ $ do+ (lhs1, op1, rhs1) <- getExpr lhs+ (lhs2, op2, rhs2) <- getExpr rhs+ guard $ op1 == op2+ guard $ lhs1 == lhs2 && rhs1 /= rhs2+ guard $ checkAndEqOperands op1 rhs1 rhs2+ return $ warn id 2333 "You probably wanted || here, otherwise it's always false."+ where+ getExpr x =+ case x of+ T_AndIf _ lhs _ -> getExpr lhs -- Fetches x and y in `T_AndIf x (T_AndIf y z)`+ T_Pipeline _ _ [x] -> getExpr x+ T_Redirecting _ _ c -> getExpr c+ T_Condition _ _ c -> getExpr c+ TC_Binary _ _ op lhs rhs -> orient (lhs, op, rhs)+ _ -> Nothing++ -- Swap items so that the constant side is rhs (or Nothing if both/neither is constant)+ orient (lhs, op, rhs) =+ case (isConstant lhs, isConstant rhs) of+ (True, False) -> return (rhs, op, lhs)+ (False, True) -> return (lhs, op, rhs)+ _ -> Nothing+++checkAndEq _ _ = return ()++ prop_checkValidCondOps1 = verify checkValidCondOps "[[ a -xz b ]]" prop_checkValidCondOps2 = verify checkValidCondOps "[ -M a ]" prop_checkValidCondOps2a = verifyNot checkValidCondOps "[ 3 \\> 2 ]"@@ -1821,7 +1901,7 @@ T_Literal id s | not (quotesSingleThing a && quotesSingleThing b || s `elem` ["=", ":", "/"]- || isSpecial (getPath (parentMap params) trapped)+ || isSpecial (NE.toList $ getPath (parentMap params) trapped) ) -> warnAboutLiteral id _ -> return ()@@ -1886,7 +1966,9 @@ prop_checkSpuriousExec9 = verify checkSpuriousExec "for file in rc.d/*; do exec \"$file\"; done" prop_checkSpuriousExec10 = verifyNot checkSpuriousExec "exec file; r=$?; printf >&2 'failed\n'; return $r" prop_checkSpuriousExec11 = verifyNot checkSpuriousExec "exec file; :"-checkSpuriousExec _ = doLists+prop_checkSpuriousExec12 = verifyNot checkSpuriousExec "#!/bin/bash\nshopt -s execfail; exec foo; exec bar; echo 'Error'; exit 1;"+prop_checkSpuriousExec13 = verify checkSpuriousExec "#!/bin/dash\nshopt -s execfail; exec foo; exec bar; echo 'Error'; exit 1;"+checkSpuriousExec params t = when (not $ hasExecfail params) $ doLists t where doLists (T_Script _ _ cmds) = doList cmds False doLists (T_BraceGroup _ cmds) = doList cmds False@@ -2039,7 +2121,7 @@ -- from $foo=bar to foo=bar. This is not pretty but ok. quotesMayConflictWithSC2281 params t = case getPath (parentMap params) t of- _ : T_NormalWord parentId (me:T_Literal _ ('=':_):_) : T_SimpleCommand _ _ (cmd:_) : _ ->+ _ NE.:| T_NormalWord parentId (me:T_Literal _ ('=':_):_) : T_SimpleCommand _ _ (cmd:_) : _ -> (getId t) == (getId me) && (parentId == getId cmd) _ -> False @@ -2134,7 +2216,8 @@ addDoubleQuotesAround params token where- name = getBracedReference $ concat $ oversimplify list+ bracedString = concat $ oversimplify list+ name = getBracedReference bracedString parents = parentMap params needsQuoting = not (isArrayExpansion token) -- There's another warning for this@@ -2144,7 +2227,8 @@ && not (usedAsCommandName parents token) isClean = fromMaybe False $ do- state <- CF.getIncomingState (cfgAnalysis params) id+ cfga <- cfgAnalysis params+ state <- CF.getIncomingState cfga id value <- Map.lookup name $ CF.variablesInScope state return $ isCleanState value @@ -2153,14 +2237,10 @@ || CF.spaceStatus (CF.variableValue state) == CF.SpaceStatusClean isDefaultAssignment parents token =- let modifier = getBracedModifier $ bracedString token in+ let modifier = getBracedModifier bracedString in any (`isPrefixOf` modifier) ["=", ":="] && isParamTo parents ":" token - -- Given a T_DollarBraced, return a simplified version of the string contents.- bracedString (T_DollarBraced _ _ l) = concat $ oversimplify l- bracedString _ = error $ pleaseReport "bracedString on non-variable"- checkSpacefulnessCfg' _ _ _ = return () @@ -2258,7 +2338,7 @@ prop_checkFunctionsUsedExternally3 = verifyNotTree checkFunctionsUsedExternally "f() { :; }; echo f" prop_checkFunctionsUsedExternally4 =- verifyNotTree checkFunctionsUsedExternally "foo() { :; }; sudo \"foo\""+ verifyNotTree checkFunctionsUsedExternally "foo() { :; }; run0 \"foo\"" prop_checkFunctionsUsedExternally5 = verifyTree checkFunctionsUsedExternally "foo() { :; }; ssh host foo" prop_checkFunctionsUsedExternally6 =@@ -2268,7 +2348,7 @@ prop_checkFunctionsUsedExternally8 = verifyTree checkFunctionsUsedExternally "foo() { :; }; command sudo foo" prop_checkFunctionsUsedExternally9 =- verifyTree checkFunctionsUsedExternally "foo() { :; }; exec -c sudo foo"+ verifyTree checkFunctionsUsedExternally "foo() { :; }; exec -c doas foo" checkFunctionsUsedExternally params t = runNodeAnalysis checkCommand params t where@@ -2277,7 +2357,7 @@ (Just str, t) -> do let name = basename str let args = skipOver t argv- let argStrings = map (\x -> (fromMaybe "" $ getLiteralString x, x)) args+ let argStrings = map (\x -> (onlyLiteralString x, x)) args let candidates = getPotentialCommands name argStrings mapM_ (checkArg name (getId t)) candidates _ -> return ()@@ -2292,6 +2372,8 @@ "chroot" -> firstNonFlag "screen" -> firstNonFlag "sudo" -> firstNonFlag+ "doas" -> firstNonFlag+ "run0" -> firstNonFlag "xargs" -> firstNonFlag "tmux" -> firstNonFlag "ssh" -> take 1 $ drop 1 $ dropFlags argAndString@@ -2376,15 +2458,9 @@ checkUnusedAssignments params t = execWriter (mapM_ warnFor unused) where flow = variableFlow params- references = foldl (flip ($)) defaultMap (map insertRef flow)- insertRef (Reference (base, token, name)) =- Map.insert (stripSuffix name) ()- insertRef _ = id+ references = Map.union (Map.fromList [(stripSuffix name, ()) | Reference (base, token, name) <- flow]) defaultMap - assignments = foldl (flip ($)) Map.empty (map insertAssignment flow)- insertAssignment (Assignment (_, token, name, _)) | isVariableName name =- Map.insert name token- insertAssignment _ = id+ assignments = Map.fromList [(name, token) | Assignment (_, token, name, _) <- flow, isVariableName name] unused = Map.assocs $ Map.difference assignments references @@ -2448,6 +2524,7 @@ prop_checkUnassignedReferences50 = verifyNotTree checkUnassignedReferences "echo ${foo:+bar}" prop_checkUnassignedReferences51 = verifyNotTree checkUnassignedReferences "echo ${foo:+$foo}" prop_checkUnassignedReferences52 = verifyNotTree checkUnassignedReferences "wait -p pid; echo $pid"+prop_checkUnassignedReferences53 = verifyTree checkUnassignedReferences "x=($foo)" checkUnassignedReferences = checkUnassignedReferences' False checkUnassignedReferences' includeGlobals params t = warnings@@ -2503,14 +2580,12 @@ warnings = execWriter . sequence $ mapMaybe warningFor unassigned - -- Due to parsing, foo=( [bar]=baz ) parses 'bar' as a reference even for assoc arrays.- -- Similarly, ${foo[bar baz]} may not be referencing bar/baz. Just skip these.+ -- ${foo[bar baz]} may not be referencing bar/baz. Just skip these. -- We can also have ${foo:+$foo} should be treated like [[ -n $foo ]] && echo $foo isException var t = any shouldExclude $ getPath (parentMap params) t where shouldExclude t = case t of- T_Array {} -> True (T_DollarBraced _ _ l) -> let str = concat $ oversimplify l ref = getBracedReference str@@ -2653,7 +2728,7 @@ check path where name = getBracedReference $ concat $ oversimplify value- path = getPath (parentMap params) t+ path = NE.toList $ getPath (parentMap params) t idPath = map getId path check [] = return ()@@ -2702,7 +2777,7 @@ return $ isCommandMatch cmd (`elem` ["tr", "read"]) -- Check if this is a dereferencing context like [[ -v array[operandhere] ]]- isDereferenced = fromMaybe False . msum . map isDereferencingOp . getPath (parentMap p)+ isDereferenced = fromMaybe False . msum . NE.map isDereferencingOp . getPath (parentMap p) isDereferencingOp t = case t of TC_Binary _ DoubleBracket str _ _ -> return $ isDereferencingBinaryOp str@@ -2753,19 +2828,18 @@ prop_checkLoopKeywordScope6 = verify checkLoopKeywordScope "while true; do true | { break; }; done" prop_checkLoopKeywordScope7 = verifyNot checkLoopKeywordScope "#!/bin/ksh\nwhile true; do true | { break; }; done" checkLoopKeywordScope params t |- name `elem` map Just ["continue", "break"] =- if not $ any isLoop path- then if any isFunction $ take 1 path- -- breaking at a source/function invocation is an abomination. Let's ignore it.- then err (getId t) 2104 $ "In functions, use return instead of " ++ fromJust name ++ "."- else err (getId t) 2105 $ fromJust name ++ " is only valid in loops."- else case map subshellType $ filter (not . isFunction) path of+ Just name <- getCommandName t, name `elem` ["continue", "break"] =+ if any isLoop path+ then case map subshellType $ filter (not . isFunction) path of Just str:_ -> warn (getId t) 2106 $ "This only exits the subshell caused by the " ++ str ++ "." _ -> return ()+ else case path of+ -- breaking at a source/function invocation is an abomination. Let's ignore it.+ h:_ | isFunction h -> err (getId t) 2104 $ "In functions, use return instead of " ++ name ++ "."+ _ -> err (getId t) 2105 $ name ++ " is only valid in loops." where- name = getCommandName t- path = let p = getPath (parentMap params) t in filter relevant p+ path = let p = getPath (parentMap params) t in NE.filter relevant p subshellType t = case leadType params t of NoneScope -> Nothing SubshellScope str -> return str@@ -2784,6 +2858,7 @@ when (hasKeyword && hasParens) $ err id 2111 "ksh does not allow 'function' keyword and '()' at the same time." Dash -> forSh+ BusyboxSh -> forSh Sh -> forSh where@@ -2821,17 +2896,19 @@ prop_checkUnpassedInFunctions12 = verifyNotTree checkUnpassedInFunctions "foo() { echo ${!var*}; }; foo;" prop_checkUnpassedInFunctions13 = verifyNotTree checkUnpassedInFunctions "# shellcheck disable=SC2120\nfoo() { echo $1; }\nfoo\n" prop_checkUnpassedInFunctions14 = verifyTree checkUnpassedInFunctions "foo() { echo $#; }; foo"+prop_checkUnpassedInFunctions15 = verifyNotTree checkUnpassedInFunctions "foo() { echo ${1-x}; }; foo"+prop_checkUnpassedInFunctions16 = verifyNotTree checkUnpassedInFunctions "foo() { echo ${1:-x}; }; foo"+prop_checkUnpassedInFunctions17 = verifyNotTree checkUnpassedInFunctions "foo() { mycommand ${1+--verbose}; }; foo"+prop_checkUnpassedInFunctions18 = verifyNotTree checkUnpassedInFunctions "foo() { if mycheck; then foo ${1?Missing}; fi; }; foo" checkUnpassedInFunctions params root = execWriter $ mapM_ warnForGroup referenceGroups where functionMap :: Map.Map String Token- functionMap = Map.fromList $- map (\t@(T_Function _ _ _ name _) -> (name,t)) functions- functions = execWriter $ doAnalysis (tell . maybeToList . findFunction) root+ functionMap = Map.fromList $ execWriter $ doAnalysis (tell . maybeToList . findFunction) root findFunction t@(T_Function id _ _ name body) | any (isPositionalReference t) flow && not (any isPositionalAssignment flow)- = return t+ = return (name,t) where flow = getVariableFlow params body findFunction _ = Nothing @@ -2839,9 +2916,10 @@ case x of Assignment (_, _, str, _) -> isPositional str _ -> False+ isPositionalReference function x = case x of- Reference (_, t, str) -> isPositional str && t `isDirectChildOf` function+ Reference (_, t, str) -> isPositional str && t `isDirectChildOf` function && not (hasDefaultValue t) _ -> False isDirectChildOf child parent = fromMaybe False $ do@@ -2855,6 +2933,7 @@ referenceList :: [(String, Bool, Token)] referenceList = execWriter $ doAnalysis (sequence_ . checkCommand) root+ checkCommand :: Token -> Maybe (Writer [(String, Bool, Token)] ()) checkCommand t@(T_SimpleCommand _ _ (cmd:args)) = do str <- getLiteralString cmd@@ -2865,6 +2944,22 @@ isPositional str = str == "*" || str == "@" || str == "#" || (all isDigit str && str /= "0" && str /= "") + -- True if t is a variable that specifies a default value,+ -- such as ${1-x} or ${1:-x}.+ hasDefaultValue t =+ case t of+ T_DollarBraced _ True l ->+ let str = concat $ oversimplify l+ in isDefaultValueModifier $ getBracedModifier str+ _ -> False++ isDefaultValueModifier str =+ case str of+ ':':c:_ -> c `elem` handlesDefault+ c:_ -> c `elem` handlesDefault+ _ -> False+ where handlesDefault = "-+?"+ isArgumentless (_, b, _) = b referenceGroups = Map.elems $ foldr updateWith Map.empty referenceList updateWith x@(name, _, _) = Map.insertWith (++) name [x]@@ -2927,7 +3022,8 @@ prop_checkUnsupported3 = verify checkUnsupported "#!/bin/sh\ncase foo in bar) baz ;& esac" prop_checkUnsupported4 = verify checkUnsupported "#!/bin/ksh\ncase foo in bar) baz ;;& esac"-prop_checkUnsupported5 = verify checkUnsupported "#!/bin/bash\necho \"${ ls; }\""+prop_checkUnsupported5 = verifyNot checkUnsupported "#!/bin/bash\necho \"${ ls; }\""+prop_checkUnsupported6 = verify checkUnsupported "#!/bin/ash\necho \"${ ls; }\"" checkUnsupported params t = unless (null support || (shellType params `elem` support)) $ report name@@ -2941,7 +3037,7 @@ shellSupport t = case t of T_CaseExpression _ _ list -> forCase (map (\(a,_,_) -> a) list)- T_DollarBraceCommandExpansion {} -> ("${ ..; } command expansion", [Ksh])+ T_DollarBraceCommandExpansion {} -> ("${ ..; } command expansion", [Bash, Ksh]) _ -> ("", []) where forCase seps | CaseContinue `elem` seps = ("cases with ;;&", [Bash])@@ -3029,7 +3125,7 @@ T_DollarExpansion _ [x] -> getPipeline x T_Pipeline _ _ cmds -> return cmds _ -> fail "unknown"- isGrep = (`elem` ["grep", "egrep", "fgrep", "zgrep"])+ isGrep = (`elem` ["grep", "egrep", "fgrep", "bz3grep", "bzgrep", "xzgrep", "zgrep", "zipgrep", "zstdgrep"]) prop_checkTestArgumentSplitting1 = verify checkTestArgumentSplitting "[ -e *.mp3 ]" prop_checkTestArgumentSplitting2 = verifyNot checkTestArgumentSplitting "[[ $a == *b* ]]"@@ -3219,7 +3315,7 @@ return $ do warn (getId token) 2165 "This nested loop overrides the index variable of its parent." warn (getId next) 2167 "This parent loop has its index variable overridden."- path = drop 1 $ getPath (parentMap params) token+ path = NE.tail $ getPath (parentMap params) token loopVariable :: Token -> Maybe String loopVariable t = case t of@@ -3292,17 +3388,17 @@ -- We don't want to warn about composite expressions like -- [[ $? -eq 0 || $? -eq 4 ]] since these can be annoying to rewrite. isOnlyTestInCommand t =- case getPath (parentMap params) t of- _:(T_Condition {}):_ -> True- _:(T_Arithmetic {}):_ -> True- _:(TA_Sequence _ [_]):(T_Arithmetic {}):_ -> True+ case NE.tail $ getPath (parentMap params) t of+ (T_Condition {}):_ -> True+ (T_Arithmetic {}):_ -> True+ (TA_Sequence _ [_]):(T_Arithmetic {}):_ -> True -- Some negations and groupings are also fine- _:next@(TC_Unary _ _ "!" _):_ -> isOnlyTestInCommand next- _:next@(TA_Unary _ "!" _):_ -> isOnlyTestInCommand next- _:next@(TC_Group {}):_ -> isOnlyTestInCommand next- _:next@(TA_Sequence _ [_]):_ -> isOnlyTestInCommand next- _:next@(TA_Parentesis _ _):_ -> isOnlyTestInCommand next+ next@(TC_Unary _ _ "!" _):_ -> isOnlyTestInCommand next+ next@(TA_Unary _ "!" _):_ -> isOnlyTestInCommand next+ next@(TC_Group {}):_ -> isOnlyTestInCommand next+ next@(TA_Sequence _ [_]):_ -> isOnlyTestInCommand next+ next@(TA_Parenthesis _ _):_ -> isOnlyTestInCommand next _ -> False -- TODO: Do better $? tracking and filter on whether@@ -3322,7 +3418,7 @@ isFirstCommandInFunction = fromMaybe False $ do let path = getPath (parentMap params) token- func <- listToMaybe $ filter isFunction path+ func <- find isFunction path cmd <- getClosestCommand (parentMap params) token return $ getId cmd == getId (getFirstCommandInFunction func) @@ -3367,7 +3463,7 @@ _ -> return () where isInExpansion t =- case drop 1 $ getPath (parentMap params) t of+ case NE.tail $ getPath (parentMap params) t of T_DollarExpansion _ [_] : _ -> True T_Backticked _ [_] : _ -> True t@T_Annotation {} : _ -> isInExpansion t@@ -3521,7 +3617,7 @@ _ -> return () checkPart part = case part of T_DollarExpansion id _ -> forCommand id- T_DollarBraceCommandExpansion id _ -> forCommand id+ T_DollarBraceCommandExpansion id _ _ -> forCommand id T_Backticked id _ -> forCommand id T_DollarBraced id _ str | not (isCountingReference part)@@ -3597,6 +3693,8 @@ prop_checkPipeToNowhere18 = verifyNot checkPipeToNowhere "ls 1>&3 3>&1 3>&- | wc -l" prop_checkPipeToNowhere19 = verifyNot checkPipeToNowhere "find . -print0 | du --files0-from=/dev/stdin" prop_checkPipeToNowhere20 = verifyNot checkPipeToNowhere "find . | du --exclude-from=/dev/fd/0"+prop_checkPipeToNowhere21 = verifyNot checkPipeToNowhere "yes | cp -ri foo/* bar"+prop_checkPipeToNowhere22 = verifyNot checkPipeToNowhere "yes | rm --interactive *" data PipeType = StdoutPipe | StdoutStderrPipe | NoPipe deriving (Eq) checkPipeToNowhere :: Parameters -> Token -> WriterT [TokenComment] Identity ()@@ -3662,6 +3760,7 @@ commandSpecificException name cmd = case name of "du" -> any ((`elem` ["exclude-from", "files0-from"]) . snd) $ getAllFlags cmd+ _ | name `elem` interactiveFlagCmds -> hasInteractiveFlag cmd _ -> False warnAboutDupes (n, list@(_:_:_)) =@@ -3685,7 +3784,7 @@ name <- getCommandBasename cmd guard $ name `elem` nonReadingCommands guard . not $ hasAdditionalConsumers cmd- guard . not $ name `elem` ["cp", "mv", "rm"] && cmd `hasFlag` "i"+ guard . not $ name `elem` interactiveFlagCmds && hasInteractiveFlag cmd let suggestion = if name == "echo" then "Did you want 'cat' instead?"@@ -3700,6 +3799,9 @@ treeContains pred t = isNothing $ doAnalysis (guard . not . pred) t + interactiveFlagCmds = [ "cp", "mv", "rm" ]+ hasInteractiveFlag cmd = cmd `hasFlag` "i" || cmd `hasFlag` "interactive"+ mayConsume t = case t of T_ProcSub _ "<" _ -> True@@ -3766,32 +3868,32 @@ prop_checkUseBeforeDefinition2 = verifyNotTree checkUseBeforeDefinition "f() { true; }; f" prop_checkUseBeforeDefinition3 = verifyNotTree checkUseBeforeDefinition "if ! mycmd --version; then mycmd() { true; }; fi" prop_checkUseBeforeDefinition4 = verifyNotTree checkUseBeforeDefinition "mycmd || mycmd() { f; }"-checkUseBeforeDefinition _ t =- execWriter $ evalStateT (mapM_ examine $ revCommands) Map.empty+prop_checkUseBeforeDefinition5 = verifyTree checkUseBeforeDefinition "false || mycmd; mycmd() { f; }"+prop_checkUseBeforeDefinition6 = verifyNotTree checkUseBeforeDefinition "f() { one; }; f; f() { two; }; f"+checkUseBeforeDefinition :: Parameters -> Token -> [TokenComment]+checkUseBeforeDefinition params t = fromMaybe [] $ do+ cfga <- cfgAnalysis params+ let funcs = execState (doAnalysis findFunction t) Map.empty+ -- Green cut: no point enumerating commands if there are no functions.+ guard . not $ Map.null funcs+ return $ execWriter $ doAnalysis (findInvocation cfga funcs) t where- examine t = case t of- T_Pipeline _ _ [T_Redirecting _ _ (T_Function _ _ _ name _)] ->- modify $ Map.insert name t- T_Annotation _ _ w -> examine w- T_Pipeline _ _ cmds -> do- m <- get- unless (Map.null m) $- mapM_ (checkUsage m) $ concatMap recursiveSequences cmds- _ -> return ()-- checkUsage map cmd = sequence_ $ do- name <- getCommandName cmd- def <- Map.lookup name map- return $- err (getId cmd) 2218- "This function is only defined later. Move the definition up."+ findFunction t =+ case t of+ T_Function id _ _ name _ -> modify (Map.insertWith (++) name [id])+ _ -> return () - revCommands = reverse $ concat $ getCommandSequences t- recursiveSequences x =- let list = concat $ getCommandSequences x in- if null list- then [x]- else concatMap recursiveSequences list+ findInvocation cfga funcs t =+ case t of+ T_SimpleCommand id _ (cmd:_) -> sequence_ $ do+ name <- getLiteralString cmd+ invocations <- Map.lookup name funcs+ -- Is the function definitely being defined later?+ guard $ any (\c -> CF.doesPostDominate cfga c id) invocations+ -- Was one already defined, so it's actually a re-definition?+ guard . not $ any (\c -> CF.doesPostDominate cfga id c) invocations+ return $ err id 2218 "This function is only defined later. Move the definition up."+ _ -> return () prop_checkForLoopGlobVariables1 = verify checkForLoopGlobVariables "for i in $var/*.txt; do true; done" prop_checkForLoopGlobVariables2 = verifyNot checkForLoopGlobVariables "for i in \"$var\"/*.txt; do true; done"@@ -3841,7 +3943,7 @@ isFunctionBody path = case path of- (_:f:_) -> isFunction f+ (_ NE.:| f:_) -> isFunction f _ -> False isTestStructure t =@@ -3868,7 +3970,7 @@ -- This technically also triggers for `if true; then ( test ); fi` -- but it's still a valid suggestion. isCompoundCondition chain =- case dropWhile skippable (drop 1 chain) of+ case dropWhile skippable (NE.tail chain) of T_IfExpression {} : _ -> True T_WhileExpression {} : _ -> True T_UntilExpression {} : _ -> True@@ -3895,12 +3997,17 @@ T_Annotation {} -> True _ -> False -prop_checkInvertedStringTest1 = verify checkInvertedStringTest "[ ! -z $var ]"-prop_checkInvertedStringTest2 = verify checkInvertedStringTest "! [[ -n $var ]]"-prop_checkInvertedStringTest3 = verifyNot checkInvertedStringTest "! [ -x $var ]"-prop_checkInvertedStringTest4 = verifyNot checkInvertedStringTest "[[ ! -w $var ]]"-prop_checkInvertedStringTest5 = verifyNot checkInvertedStringTest "[ -z $var ]"-checkInvertedStringTest _ t =+prop_checkUnnecessarilyInvertedTest1 = verify checkUnnecessarilyInvertedTest "[ ! -z $var ]"+prop_checkUnnecessarilyInvertedTest2 = verify checkUnnecessarilyInvertedTest "! [[ -n $var ]]"+prop_checkUnnecessarilyInvertedTest3 = verifyNot checkUnnecessarilyInvertedTest "! [ -x $var ]"+prop_checkUnnecessarilyInvertedTest4 = verifyNot checkUnnecessarilyInvertedTest "[[ ! -w $var ]]"+prop_checkUnnecessarilyInvertedTest5 = verifyNot checkUnnecessarilyInvertedTest "[ -z $var ]"+prop_checkUnnecessarilyInvertedTest6 = verify checkUnnecessarilyInvertedTest "! [ $var != foo ]"+prop_checkUnnecessarilyInvertedTest7 = verify checkUnnecessarilyInvertedTest "[[ ! $var == foo ]]"+prop_checkUnnecessarilyInvertedTest8 = verifyNot checkUnnecessarilyInvertedTest "! [[ $var =~ .* ]]"+prop_checkUnnecessarilyInvertedTest9 = verify checkUnnecessarilyInvertedTest "[ ! $var -eq 0 ]"+prop_checkUnnecessarilyInvertedTest10 = verify checkUnnecessarilyInvertedTest "! [[ $var -gt 3 ]]"+checkUnnecessarilyInvertedTest _ t = case t of TC_Unary _ _ "!" (TC_Unary _ _ op _) -> case op of@@ -3913,8 +4020,35 @@ "-n" -> style (getId t) 2237 "Use [ -z .. ] instead of ! [ -n .. ]." "-z" -> style (getId t) 2237 "Use [ -n .. ] instead of ! [ -z .. ]." _ -> return ()+ TC_Unary _ _ "!" (TC_Binary _ bracketStyle op _ _) ->+ maybeSuggestRewrite True bracketStyle (getId t) op+ T_Banged _ (T_Pipeline _ _+ [T_Redirecting _ _ (T_Condition _ _ (TC_Binary _ bracketStyle op _ _))]) ->+ maybeSuggestRewrite False bracketStyle (getId t) op _ -> return ()+ where+ inversionMap = Map.fromList [+ ("=", "!="),+ ("==", "!="),+ ("!=", "="),+ ("-eq", "-ne"),+ ("-ne", "-eq"),+ ("-le", "-gt"),+ ("-gt", "-le"),+ ("-ge", "-lt"),+ ("-lt", "-ge")+ ]+ maybeSuggestRewrite bangInside bracketStyle id op = sequence_ $ do+ newOp <- Map.lookup op inversionMap+ let oldExpr = "a " ++ op ++ " b"+ let newExpr = "a " ++ newOp ++ " b"+ let bracket s = if bracketStyle == SingleBracket then "[ " ++ s ++ " ]" else "[[ " ++ s ++ " ]]"+ return $+ if bangInside+ then style id 2335 $ "Use " ++ newExpr ++ " instead of ! " ++ oldExpr ++ "."+ else style id 2335 $ "Use " ++ (bracket newExpr) ++ " instead of ! " ++ (bracket oldExpr) ++ "." + prop_checkRedirectionToCommand1 = verify checkRedirectionToCommand "ls > rm" prop_checkRedirectionToCommand2 = verifyNot checkRedirectionToCommand "ls > 'rm'" prop_checkRedirectionToCommand3 = verifyNot checkRedirectionToCommand "ls > myfile"@@ -3967,13 +4101,10 @@ prop_checkTranslatedStringVariable5 = verifyNot checkTranslatedStringVariable "foo=var; bar=val2; $\"foo bar\"" checkTranslatedStringVariable params (T_DollarDoubleQuoted id [T_Literal _ s]) | all isVariableChar s- && Map.member s assignments+ && S.member s assignments = warnWithFix id 2256 "This translated string is the name of a variable. Flip leading $ and \" if this should be a quoted substitution." (fix id) where- assignments = foldl (flip ($)) Map.empty (map insertAssignment $ variableFlow params)- insertAssignment (Assignment (_, token, name, _)) | isVariableName name =- Map.insert name token- insertAssignment _ = Prelude.id+ assignments = S.fromList [name | Assignment (_, _, name, _) <- variableFlow params, isVariableName name] fix id = fixWith [replaceStart id params 2 "\"$"] checkTranslatedStringVariable _ _ = return () @@ -4003,6 +4134,7 @@ prop_checkUselessBang7 = verifyNot checkUselessBang "set -e; x() { ! [ x ]; }" prop_checkUselessBang8 = verifyNot checkUselessBang "set -e; if { ! true; }; then true; fi" prop_checkUselessBang9 = verifyNot checkUselessBang "set -e; while ! true; do true; done"+prop_checkUselessBang10 = verify checkUselessBang "set -e\nshellcheck disable=SC0000\n! true\nrest" checkUselessBang params t = when (hasSetE params) $ mapM_ check (getNonReturningCommands t) where check t =@@ -4011,6 +4143,7 @@ addComment $ makeCommentWithFix InfoC id 2251 "This ! is not on a condition and skips errexit. Use `&& exit 1` instead, or make sure $? is checked." (fixWith [replaceStart id params 1 "", replaceEnd (getId cmd) params 0 " && exit 1"])+ T_Annotation _ _ t -> check t _ -> return () -- Get all the subcommands that aren't likely to be the return value@@ -4031,7 +4164,7 @@ isFunctionBody t = case getPath (parentMap params) t of- _:T_Function {}:_-> True+ _ NE.:| T_Function {}:_-> True _ -> False dropLast t =@@ -4046,7 +4179,8 @@ prop_checkModifiedArithmeticInRedirection4 = verify checkModifiedArithmeticInRedirection "cat <<< $((i++))" prop_checkModifiedArithmeticInRedirection5 = verify checkModifiedArithmeticInRedirection "cat << foo\n$((i++))\nfoo\n" prop_checkModifiedArithmeticInRedirection6 = verifyNot checkModifiedArithmeticInRedirection "#!/bin/dash\nls > $((i=i+1))"-checkModifiedArithmeticInRedirection params t = unless (shellType params == Dash) $+prop_checkModifiedArithmeticInRedirection7 = verifyNot checkModifiedArithmeticInRedirection "#!/bin/busybox sh\ncat << foo\n$((i++))\nfoo\n"+checkModifiedArithmeticInRedirection params t = unless (shellType params == Dash || shellType params == BusyboxSh) $ case t of T_Redirecting _ redirs (T_SimpleCommand _ _ (_:_)) -> mapM_ checkRedirs redirs _ -> return ()@@ -4200,7 +4334,7 @@ in mapM_ checkTest commandWithSeps checkTest (before, cmd, after) =- when (isTest cmd) $ do+ when (isTestCommand cmd) $ do checkPipe before checkPipe after @@ -4216,17 +4350,10 @@ T_AndIf _ _ rhs -> checkAnds id rhs T_OrIf _ _ rhs -> checkAnds id rhs T_Pipeline _ _ list | not (null list) -> checkAnds id (last list)- cmd -> when (isTest cmd) $+ cmd -> when (isTestCommand cmd) $ errWithFix id 2265 "Use && for logical AND. Single & will background and return true." $ (fixWith [replaceEnd id params 0 "&"]) - isTest t =- case t of- T_Condition {} -> True- T_SimpleCommand {} -> t `isCommand` "test"- T_Redirecting _ _ t -> isTest t- T_Annotation _ _ t -> isTest t- _ -> False prop_checkComparisonWithLeadingX1 = verify checkComparisonWithLeadingX "[ x$foo = xlol ]" prop_checkComparisonWithLeadingX2 = verify checkComparisonWithLeadingX "test x$foo = xlol"@@ -4234,14 +4361,16 @@ prop_checkComparisonWithLeadingX4 = verifyNot checkComparisonWithLeadingX "test $foo = xbar" prop_checkComparisonWithLeadingX5 = verify checkComparisonWithLeadingX "[ \"x$foo\" = 'xlol' ]" prop_checkComparisonWithLeadingX6 = verify checkComparisonWithLeadingX "[ x\"$foo\" = x'lol' ]"+prop_checkComparisonWithLeadingX7 = verify checkComparisonWithLeadingX "[ X$foo != Xbar ]" checkComparisonWithLeadingX params t = case t of- TC_Binary id typ op lhs rhs | op == "=" || op == "==" ->- check lhs rhs- T_SimpleCommand _ _ [cmd, lhs, op, rhs] |- getLiteralString cmd == Just "test" &&- getLiteralString op `elem` [Just "=", Just "=="] ->- check lhs rhs+ TC_Binary id typ op lhs rhs+ | op `elem` ["=", "==", "!="] ->+ check lhs rhs+ T_SimpleCommand _ _ [cmd, lhs, op, rhs]+ | getLiteralString cmd == Just "test" &&+ getLiteralString op `elem` [Just "=", Just "==", Just "!="] ->+ check lhs rhs _ -> return () where msg = "Avoid x-prefix in comparisons as it no longer serves a purpose."@@ -4251,19 +4380,20 @@ return $ styleWithFix (getId lhs) 2268 msg $ fixWith [l, r] fixLeadingX token =- case getWordParts token of- T_Literal id ('x':_):_ ->+ case getWordParts token of+ T_Literal id (c:_):_ | toLower c == 'x' -> case token of- -- The side is a single, unquoted x, so we have to quote- T_NormalWord _ [T_Literal id "x"] ->+ -- The side is a single, unquoted x or X, so we have to quote+ T_NormalWord _ [T_Literal id [c]] -> return $ replaceStart id params 1 "\"\"" -- Otherwise we can just delete it _ -> return $ replaceStart id params 1 ""- T_SingleQuoted id ('x':_):_ ->- -- Replace the single quote and x- return $ replaceStart id params 2 "'"+ T_SingleQuoted id (c:rest):_ | toLower c == 'x' ->+ -- Replace the single quote and the character x or X+ return $ replaceStart id params 2 "'" _ -> Nothing + prop_checkAssignToSelf1 = verify checkAssignToSelf "x=$x" prop_checkAssignToSelf2 = verify checkAssignToSelf "x=${x}" prop_checkAssignToSelf3 = verify checkAssignToSelf "x=\"$x\""@@ -4358,6 +4488,7 @@ Bash -> errWithFix id 2277 "Use BASH_ARGV0 to assign to $0 in bash (or use [ ] to compare)." bashfix Ksh -> err id 2278 "$0 can't be assigned in Ksh (but it does reflect the current function)." Dash -> err id 2279 "$0 can't be assigned in Dash. This becomes a command name."+ BusyboxSh -> err id 2279 "$0 can't be assigned in Busybox Ash. This becomes a command name." _ -> err id 2280 "$0 can't be assigned this way, and there is no portable alternative." leadingNumberMsg id = err id 2282 "Variable names can't start with numbers, so this is interpreted as a command."@@ -4380,9 +4511,9 @@ return $ isVariableName str isLeadingNumberVar s =- let lead = takeWhile (/= '=') s- in not (null lead) && isDigit (head lead)- && all isVariableChar lead && not (all isDigit lead)+ case takeWhile (/= '=') s of+ lead@(x:_) -> isDigit x && all isVariableChar lead && not (all isDigit lead)+ [] -> False msg cmd leading (T_Literal litId s) = do -- There are many different cases, and the order of the branches matter.@@ -4512,7 +4643,7 @@ checkCommandWithTrailingSymbol _ t = case t of T_SimpleCommand _ _ (cmd:_) ->- let str = fromJust $ getLiteralStringExt (\_ -> Just "x") cmd+ let str = getLiteralStringDef "x" cmd last = lastOrDefault 'x' str in case str of@@ -4541,13 +4672,13 @@ prop_checkRequireDoubleBracket3 = verifyNotTree checkRequireDoubleBracket "#!/bin/sh\n[ -x foo ]" prop_checkRequireDoubleBracket4 = verifyNotTree checkRequireDoubleBracket "[[ -x foo ]]" checkRequireDoubleBracket params =- if isBashLike params+ if (shellType params) `elem` [Bash, Ksh, BusyboxSh] then nodeChecksToTreeCheck [check] params else const [] where check _ t = case t of T_Condition id SingleBracket _ ->- styleWithFix id 2292 "Prefer [[ ]] over [ ] for tests in Bash/Ksh." (fixFor t)+ styleWithFix id 2292 "Prefer [[ ]] over [ ] for tests in Bash/Ksh/Busybox." (fixFor t) _ -> return () fixFor t = fixWith $@@ -4627,7 +4758,8 @@ -- Is this one of the 'for' arrays? (loopWord, _) <- find ((==arrayName) . snd) arrays -- Are we still in this loop?- guard $ getId loop `elem` map getId (getPath parents t)+ let loopId = getId loop+ guard $ any (\t -> loopId == getId t) (getPath parents t) return [ makeComment WarningC (getId loopWord) 2302 "This loops over values. To loop over keys, use \"${!array[@]}\".", makeComment WarningC (getId arrayRef) 2303 $ (e4m name) ++ " is an array value, not a key. Use directly or loop over keys instead."@@ -4709,7 +4841,7 @@ literalArg <- getUnquotedLiteral cmd Map.lookup literalArg functions_ - checkCmd cmd = go $ getPath (parentMap params) cmd+ checkCmd cmd = go $ NE.toList $ getPath (parentMap params) cmd where go (child:parent:rest) = do case parent of@@ -4823,15 +4955,15 @@ ++ "separately to avoid masking its return value (or use '|| true' " ++ "to ignore).") - isMaskDeliberate t = hasParent isOrIf t+ isMaskDeliberate t = any isOrIf $ NE.init $ parents params t where- isOrIf _ (T_OrIf _ _ (T_Pipeline _ _ [T_Redirecting _ _ cmd]))+ isOrIf (T_OrIf _ _ (T_Pipeline _ _ [T_Redirecting _ _ cmd])) = getCommandBasename cmd `elem` [Just "true", Just ":"]- isOrIf _ _ = False+ isOrIf _ = False - isCheckedElsewhere t = hasParent isDeclaringCommand t+ isCheckedElsewhere t = any isDeclaringCommand $ NE.tail $ parents params t where- isDeclaringCommand t _ = fromMaybe False $ do+ isDeclaringCommand t = fromMaybe False $ do cmd <- getCommand t basename <- getCommandBasename cmd return $@@ -4851,16 +4983,7 @@ ,"shopt" ] - isTransparentCommand t = fromMaybe False $ do- basename <- getCommandBasename t- return $ basename == "time"-- parentChildPairs t = go $ parents params t- where- go (child:parent:rest) = (parent, child):go (parent:rest)- go _ = []-- hasParent pred t = any (uncurry pred) (parentChildPairs t)+ isTransparentCommand t = getCommandBasename t == Just "time" -- hard error on negated command that is not last@@ -4906,15 +5029,33 @@ prop_checkCommandIsUnreachable1 = verify checkCommandIsUnreachable "foo; bar; exit; baz" prop_checkCommandIsUnreachable2 = verify checkCommandIsUnreachable "die() { exit; }; foo; bar; die; baz" prop_checkCommandIsUnreachable3 = verifyNot checkCommandIsUnreachable "foo; bar || exit; baz"+prop_checkCommandIsUnreachable4 = verifyNot checkCommandIsUnreachable "f() { foo; }; # Maybe sourced"+prop_checkCommandIsUnreachable5 = verify checkCommandIsUnreachable "f() { foo; }; exit # Not sourced" checkCommandIsUnreachable params t = case t of T_Pipeline {} -> sequence_ $ do- state <- CF.getIncomingState (cfgAnalysis params) id+ cfga <- cfgAnalysis params+ state <- CF.getIncomingState cfga (getId t) guard . not $ CF.stateIsReachable state guard . not $ isSourced params t- return $ info id 2317 "Command appears to be unreachable. Check usage (or ignore if invoked indirectly)."+ guard . not $ any (\t -> isUnreachable t || isUnreachableFunction t) $ NE.drop 1 $ getPath (parentMap params) t+ return $ info (getId t) 2317 "Command appears to be unreachable. Check usage (or ignore if invoked indirectly)."+ T_Function id _ _ _ _ ->+ when (isUnreachableFunction t+ && (not . any isUnreachableFunction . NE.drop 1 $ getPath (parentMap params) t)+ && (not $ isSourced params t)) $+ info id 2329 "This function is never invoked. Check usage (or ignored if invoked indirectly)." _ -> return ()- where id = getId t+ where+ isUnreachableFunction :: Token -> Bool+ isUnreachableFunction f =+ case f of+ T_Function id _ _ _ t -> isUnreachable t+ _ -> False+ isUnreachable t = fromMaybe False $ do+ cfga <- cfgAnalysis params+ state <- CF.getIncomingState cfga (getId t)+ return . not $ CF.stateIsReachable state prop_checkOverwrittenExitCode1 = verify checkOverwrittenExitCode "x; [ $? -eq 1 ] || [ $? -eq 2 ]"@@ -4931,14 +5072,15 @@ _ -> return () where check id = sequence_ $ do- state <- CF.getIncomingState (cfgAnalysis params) id+ cfga <- cfgAnalysis params+ state <- CF.getIncomingState cfga id let exitCodeIds = CF.exitCodes state guard . not $ S.null exitCodeIds let idToToken = idMap params- exitCodeTokens <- sequence $ map (\k -> Map.lookup k idToToken) $ S.toList exitCodeIds+ exitCodeTokens <- traverse (\k -> Map.lookup k idToToken) $ S.toList exitCodeIds return $ do- when (all isCondition exitCodeTokens && not (usedUnconditionally t exitCodeIds)) $+ when (all isCondition exitCodeTokens && not (usedUnconditionally cfga t exitCodeIds)) $ warn id 2319 "This $? refers to a condition, not a command. Assign to a variable to avoid it being overwritten." when (all isPrinting exitCodeTokens) $ warn id 2320 "This $? refers to echo/printf, not a previous command. Assign to variable to avoid it being overwritten."@@ -4951,8 +5093,8 @@ -- If we don't do anything based on the condition, assume we wanted the condition itself -- This helps differentiate `x; [ $? -gt 0 ] && exit $?` vs `[ cond ]; exit $?`- usedUnconditionally t testIds =- all (\c -> CF.doesPostDominate (cfgAnalysis params) (getId t) c) testIds+ usedUnconditionally cfga t testIds =+ all (\c -> CF.doesPostDominate cfga (getId t) c) testIds isPrinting t = case getCommandBasename t of@@ -4993,14 +5135,14 @@ T_ForArithmetic _ x y z _ -> mapM_ (checkLeading "for (((x); (y); (z))) is the same as for ((x; y; z))") [x,y,z] T_Assignment _ _ _ [t] _ -> checkLeading "a[(x)] is the same as a[x]" t T_Arithmetic _ t -> checkLeading "(( (x) )) is the same as (( x ))" t- TA_Parentesis _ (TA_Sequence _ [ TA_Parentesis id _ ]) ->+ TA_Parenthesis _ (TA_Sequence _ [ TA_Parenthesis id _ ]) -> styleWithFix id 2322 "In arithmetic contexts, ((x)) is the same as (x). Prefer only one layer of parentheses." $ fix id _ -> return () where checkLeading str t = case t of- TA_Sequence _ [TA_Parentesis id _ ] -> styleWithFix id 2323 (str ++ ". Prefer not wrapping in additional parentheses.") $ fix id+ TA_Sequence _ [TA_Parenthesis id _ ] -> styleWithFix id 2323 (str ++ ". Prefer not wrapping in additional parentheses.") $ fix id _ -> return () fix id =@@ -5009,6 +5151,92 @@ replaceEnd id params 1 "" -- Remove ")" ] ++prop_checkPlusEqualsNumber1 = verify checkPlusEqualsNumber "x+=1"+prop_checkPlusEqualsNumber2 = verify checkPlusEqualsNumber "x+=42"+prop_checkPlusEqualsNumber3 = verifyNot checkPlusEqualsNumber "(( x += 1 ))"+prop_checkPlusEqualsNumber4 = verifyNot checkPlusEqualsNumber "declare -i x=0; x+=1"+prop_checkPlusEqualsNumber5 = verifyNot checkPlusEqualsNumber "x+='1'"+prop_checkPlusEqualsNumber6 = verifyNot checkPlusEqualsNumber "n=foo; x+=n"+prop_checkPlusEqualsNumber7 = verify checkPlusEqualsNumber "n=4; x+=n"+prop_checkPlusEqualsNumber8 = verify checkPlusEqualsNumber "n=4; x+=$n"+prop_checkPlusEqualsNumber9 = verifyNot checkPlusEqualsNumber "declare -ia var; var[x]+=1"+checkPlusEqualsNumber params t =+ case t of+ T_Assignment id Append var _ word -> sequence_ $ do+ cfga <- cfgAnalysis params+ state <- CF.getIncomingState cfga id+ guard $ isNumber state word+ guard . not $ fromMaybe False $ CF.variableMayBeDeclaredInteger state var+ -- Recommend "typeset" because ksh does not have "declare".+ return $ warn id 2324 "var+=1 will append, not increment. Use (( var += 1 )), typeset -i var, or quote number to silence."+ _ -> return ()++ where+ isNumber state word =+ let+ unquotedLiteral = getUnquotedLiteral word+ isEmpty = unquotedLiteral == Just ""+ isUnquotedNumber = not isEmpty && maybe False (all isDigit) unquotedLiteral+ isNumericalVariableName = fromMaybe False $ do+ str <- unquotedLiteral+ CF.variableMayBeAssignedInteger state str+ isNumericalVariableExpansion =+ case word of+ T_NormalWord _ [part] -> fromMaybe False $ do+ str <- getUnmodifiedParameterExpansion part+ CF.variableMayBeAssignedInteger state str+ _ -> False+ in+ isUnquotedNumber || isNumericalVariableName || isNumericalVariableExpansion++++prop_checkExpansionWithRedirection1 = verify checkExpansionWithRedirection "var=$(foo > bar)"+prop_checkExpansionWithRedirection2 = verify checkExpansionWithRedirection "var=`foo 1> bar`"+prop_checkExpansionWithRedirection3 = verify checkExpansionWithRedirection "var=${ foo >> bar; }"+prop_checkExpansionWithRedirection4 = verify checkExpansionWithRedirection "var=$(foo | bar > baz)"+prop_checkExpansionWithRedirection5 = verifyNot checkExpansionWithRedirection "stderr=$(foo 2>&1 > /dev/null)"+prop_checkExpansionWithRedirection6 = verifyNot checkExpansionWithRedirection "var=$(foo; bar > baz)"+prop_checkExpansionWithRedirection7 = verifyNot checkExpansionWithRedirection "var=$(foo > bar; baz)"+prop_checkExpansionWithRedirection8 = verifyNot checkExpansionWithRedirection "var=$(cat <&3)"+checkExpansionWithRedirection params t =+ case t of+ T_DollarExpansion id [cmd] -> check id cmd+ T_Backticked id [cmd] -> check id cmd+ T_DollarBraceCommandExpansion id _ [cmd] -> check id cmd+ _ -> return ()+ where+ check id pipe =+ case pipe of+ (T_Pipeline _ _ t@(_:_)) -> checkCmd id (last t)+ _ -> return ()++ checkCmd captureId (T_Redirecting _ redirs _) = foldr (walk captureId) (return ()) redirs++ walk captureId t acc =+ case t of+ T_FdRedirect _ _ (T_IoDuplicate _ _ "1") -> return ()+ T_FdRedirect id "1" (T_IoDuplicate _ _ _) -> return ()+ T_FdRedirect id "" (T_IoDuplicate _ op _) | op `elem` [T_GREATAND (Id 0), T_Greater (Id 0)] -> emit id captureId True+ T_FdRedirect id str (T_IoFile _ op file) | str `elem` ["", "1"] && op `elem` [ T_DGREAT (Id 0), T_Greater (Id 0) ] ->+ emit id captureId $ getLiteralString file /= Just "/dev/null"+ _ -> acc++ emit redirectId captureId suggestTee = do+ warn captureId 2327 "This command substitution will be empty because the command's output gets redirected away."+ err redirectId 2328 $ "This redirection takes output away from the command substitution" ++ if suggestTee then " (use tee to duplicate)." else "."+++prop_checkUnaryTestA1 = verify checkUnaryTestA "[ -a foo ]"+prop_checkUnaryTestA2 = verify checkUnaryTestA "[ ! -a foo ]"+prop_checkUnaryTestA3 = verifyNot checkUnaryTestA "[ foo -a bar ]"+checkUnaryTestA params t =+ case t of+ TC_Unary id _ "-a" _ ->+ styleWithFix id 2331 "For file existence, prefer standard -e over legacy -a." $+ fixWith [replaceStart id params 2 "-e"]+ _ -> return () return [] runTests = $( [| $(forAllProperties) (quickCheckWithResult (stdArgs { maxSuccess = 1 }) ) |])
src/ShellCheck/AnalyzerLib.hs view
@@ -41,6 +41,7 @@ import Data.List import Data.Maybe import Data.Semigroup+import qualified Data.List.NonEmpty as NE import qualified Data.Map as Map import Test.QuickCheck.All (forAllProperties)@@ -88,6 +89,8 @@ hasSetE :: Bool, -- Whether this script has 'set -o pipefail' anywhere. hasPipefail :: Bool,+ -- Whether this script has 'shopt -s execfail' anywhere.+ hasExecfail :: Bool, -- A linear (bad) analysis of data flow variableFlow :: [StackData], -- A map from Id to Token@@ -103,7 +106,7 @@ -- map from token id to start and end position tokenPositions :: Map.Map Id (Position, Position), -- Result from Control Flow Graph analysis (including data flow analysis)- cfgAnalysis :: CF.CFGAnalysis+ cfgAnalysis :: Maybe CF.CFGAnalysis } deriving (Show) -- TODO: Cache results of common AST ops here@@ -196,8 +199,10 @@ } in force withFix +-- makeParameters :: CheckSpec -> Parameters makeParameters spec = params where+ extendedAnalysis = fromMaybe True $ msum [asExtendedAnalysis spec, getExtendedAnalysisDirective root] params = Parameters { rootNode = root, shellType = fromMaybe (determineShell (asFallbackShell spec) root) $ asShellType spec,@@ -206,26 +211,35 @@ case shellType params of Bash -> isOptionSet "lastpipe" root Dash -> False+ BusyboxSh -> False Sh -> False Ksh -> True, hasInheritErrexit = case shellType params of Bash -> isOptionSet "inherit_errexit" root Dash -> True+ BusyboxSh -> True Sh -> True Ksh -> False, hasPipefail = case shellType params of Bash -> isOptionSet "pipefail" root Dash -> True+ BusyboxSh -> isOptionSet "pipefail" root Sh -> True Ksh -> isOptionSet "pipefail" root,+ hasExecfail =+ case shellType params of+ Bash -> isOptionSet "execfail" root+ _ -> False, shellTypeSpecified = isJust (asShellType spec) || isJust (asFallbackShell spec), idMap = getTokenMap root, parentMap = getParentTree root, variableFlow = getVariableFlow params root, tokenPositions = asTokenPositions spec,- cfgAnalysis = CF.analyzeControlFlow cfParams root+ cfgAnalysis = do+ guard extendedAnalysis+ return $ CF.analyzeControlFlow cfParams root } cfParams = CF.CFGParameters { CF.cfLastpipe = hasLastpipe params,@@ -284,8 +298,8 @@ prop_determineShell8 = determineShellTest' (Just Ksh) "#!/bin/sh" == Sh prop_determineShell9 = determineShellTest "#!/bin/env -S dash -x" == Dash prop_determineShell10 = determineShellTest "#!/bin/env --split-string= dash -x" == Dash-prop_determineShell11 = determineShellTest "#!/bin/busybox sh" == Dash -- busybox sh is a specific shell, not posix sh-prop_determineShell12 = determineShellTest "#!/bin/busybox ash" == Dash+prop_determineShell11 = determineShellTest "#!/bin/busybox sh" == BusyboxSh -- busybox sh is a specific shell, not posix sh+prop_determineShell12 = determineShellTest "#!/bin/busybox ash" == BusyboxSh determineShellTest = determineShellTest' Nothing determineShellTest' fallbackShell = determineShell fallbackShell . fromJust . prRoot . pScript@@ -333,14 +347,14 @@ isQuoteFreeNode strict shell tree t = isQuoteFreeElement t ||- (fromMaybe False $ msum $ map isQuoteFreeContext $ drop 1 $ getPath tree t)+ (fromMaybe False $ msum $ map isQuoteFreeContext $ NE.tail $ getPath tree t) where -- Is this node self-quoting in itself? isQuoteFreeElement t = case t of- T_Assignment {} -> assignmentIsQuoting t- T_FdRedirect {} -> True- _ -> False+ T_Assignment id _ _ _ _ -> assignmentIsQuoting id+ T_FdRedirect {} -> True+ _ -> False -- Are any subnodes inherently self-quoting? isQuoteFreeContext t =@@ -350,7 +364,7 @@ TC_Binary _ DoubleBracket _ _ _ -> return True TA_Sequence {} -> return True T_Arithmetic {} -> return True- T_Assignment {} -> return $ assignmentIsQuoting t+ T_Assignment id _ _ _ _ -> return $ assignmentIsQuoting id T_Redirecting {} -> return False T_DoubleQuoted _ _ -> return True T_DollarDoubleQuoted _ _ -> return True@@ -365,11 +379,11 @@ -- Check whether this assignment is self-quoting due to being a recognized -- assignment passed to a Declaration Utility. This will soon be required -- by POSIX: https://austingroupbugs.net/view.php?id=351- assignmentIsQuoting t = shellParsesParamsAsAssignments || not (isAssignmentParamToCommand t)+ assignmentIsQuoting id = shellParsesParamsAsAssignments || not (isAssignmentParamToCommand id) shellParsesParamsAsAssignments = shell /= Sh -- Is this assignment a parameter to a command like export/typeset/etc?- isAssignmentParamToCommand (T_Assignment id _ _ _ _) =+ isAssignmentParamToCommand id = case Map.lookup id tree of Just (T_SimpleCommand _ _ (_:args)) -> id `elem` (map getId args) _ -> False@@ -395,7 +409,7 @@ -- Get the parent command (T_Redirecting) of a Token, if any. getClosestCommand :: Map.Map Id Token -> Token -> Maybe Token getClosestCommand tree t =- findFirst findCommand $ getPath tree t+ findFirst findCommand $ NE.toList $ getPath tree t where findCommand t = case t of@@ -409,7 +423,7 @@ return $ getClosestCommand (parentMap params) t -- Is the token used as a command name (the first word in a T_SimpleCommand)?-usedAsCommandName tree token = go (getId token) (tail $ getPath tree token)+usedAsCommandName tree token = go (getId token) (NE.tail $ getPath tree token) where go currentId (T_NormalWord id [word]:rest) | currentId == getId word = go id rest@@ -426,7 +440,9 @@ return $ getPath (parentMap params) t isParentOf tree parent child =- elem (getId parent) . map getId $ getPath tree child+ any (\t -> parentId == getId t) (getPath tree child)+ where+ parentId = getId parent parents params = getPath (parentMap params) @@ -525,7 +541,9 @@ T_BatsTest {} -> [ (t, t, "lines", DataArray SourceExternal), (t, t, "status", DataString SourceInteger),- (t, t, "output", DataString SourceExternal)+ (t, t, "output", DataString SourceExternal),+ (t, t, "stderr", DataString SourceExternal),+ (t, t, "stderr_lines", DataArray SourceExternal) ] -- Count [[ -v foo ]] as an "assignment".@@ -547,9 +565,13 @@ T_FdRedirect _ ('{':var) op -> -- {foo}>&2 modifies foo [(t, t, takeWhile (/= '}') var, DataString SourceInteger) | not $ isClosingFileOp op] - T_CoProc _ name _ ->- [(t, t, fromMaybe "COPROC" name, DataArray SourceInteger)]+ T_CoProc _ Nothing _ ->+ [(t, t, "COPROC", DataArray SourceInteger)] + T_CoProc _ (Just token) _ -> do+ name <- maybeToList $ getLiteralString token+ [(t, t, name, DataArray SourceInteger)]+ --Points to 'for' rather than variable T_ForIn id str [] _ -> [(t, t, str, DataString SourceExternal)] T_ForIn id str words _ -> [(t, t, str, DataString $ SourceFrom words)]@@ -810,7 +832,7 @@ return (context, token, getBracedReference str) isArithmeticAssignment t = case getPath parents t of- this: TA_Assignment _ "=" lhs _ :_ -> lhs == t+ this NE.:| TA_Assignment _ "=" lhs _ :_ -> lhs == t _ -> False isDereferencingBinaryOp = (`elem` ["-eq", "-ne", "-lt", "-le", "-gt", "-ge"])@@ -892,15 +914,6 @@ supportsArrays Ksh = True supportsArrays _ = False --- Returns true if the shell is Bash or Ksh (sorry for the name, Ksh)-isBashLike :: Parameters -> Bool-isBashLike params =- case shellType params of- Bash -> True- Ksh -> True- Dash -> False- Sh -> False- isTrueAssignmentSource c = case c of DataString SourceChecked -> False@@ -918,6 +931,14 @@ Assignment (_, _, n, source) -> isTrueAssignmentSource source && n == name _ -> False +isTestCommand t =+ case t of+ T_Condition {} -> True+ T_SimpleCommand {} -> t `isCommand` "test"+ T_Redirecting _ _ t -> isTestCommand t+ T_Annotation _ _ t -> isTestCommand t+ T_Pipeline _ _ [t] -> isTestCommand t+ _ -> False return [] runTests = $( [| $(forAllProperties) (quickCheckWithResult (stdArgs { maxSuccess = 1 }) ) |])
src/ShellCheck/CFG.hs view
@@ -51,6 +51,7 @@ import Data.Array.Unboxed import Data.Array.ST import Data.List hiding (map)+import qualified Data.List.NonEmpty as NE import Data.Maybe import qualified Data.Map as M import qualified Data.Set as S@@ -111,8 +112,8 @@ -- Actions we track data CFEffect =- CFSetProps Scope String (S.Set CFVariableProp)- | CFUnsetProps Scope String (S.Set CFVariableProp)+ CFSetProps (Maybe Scope) String (S.Set CFVariableProp)+ | CFUnsetProps (Maybe Scope) String (S.Set CFVariableProp) | CFReadVariable String | CFWriteVariable String CFValue | CFWriteGlobal String CFValue@@ -192,7 +193,7 @@ base idToRange = M.fromList mapping- isRealEdge (from, to, edge) = case edge of CFEFlow -> True; _ -> False+ isRealEdge (from, to, edge) = case edge of CFEFlow -> True; CFEExit -> True; _ -> False onlyRealEdges = filter isRealEdge edges (_, mainExit) = fromJust $ M.lookup (getId root) idToRange @@ -294,19 +295,19 @@ regularEdges = filter isRegularEdge edges inDegree = counter $ map (\(from,to,_) -> from) regularEdges outDegree = counter $ map (\(from,to,_) -> to) regularEdges- structuralNodes = S.fromList $ map fst $ filter isStructural nodes+ structuralNodes = S.fromList [node | (node, CFStructuralNode) <- nodes] candidateNodes = S.filter isLinear structuralNodes edgesToCollapse = S.fromList $ filter filterEdges regularEdges remapping :: M.Map Node Node- remapping = foldl' (\m (new, old) -> M.insert old new m) M.empty $ map orderEdge $ S.toList edgesToCollapse- recursiveRemapping = M.fromList $ map (\c -> (c, recursiveLookup remapping c)) $ M.keys remapping+ remapping = M.fromList $ map orderEdge $ S.toList edgesToCollapse+ recursiveRemapping = M.mapWithKey (\c _ -> recursiveLookup remapping c) remapping filterEdges (a,b,_) = a `S.member` candidateNodes && b `S.member` candidateNodes - orderEdge (a,b,_) = if a < b then (a,b) else (b,a)- counter = foldl' (\map key -> M.insertWith (+) key 1 map) M.empty+ orderEdge (a,b,_) = if a < b then (b,a) else (a,b)+ counter = M.fromListWith (+) . map (\key -> (key, 1)) isRegularEdge (_, _, CFEFlow) = True isRegularEdge _ = False @@ -316,11 +317,6 @@ Nothing -> node Just x -> recursiveLookup map x - isStructural (node, label) =- case label of- CFStructuralNode -> True- _ -> False- isLinear node = M.findWithDefault 0 node inDegree == 1 && M.findWithDefault 0 node outDegree == 1@@ -494,7 +490,7 @@ TA_Binary _ _ a b -> sequentially [a,b] TA_Expansion _ list -> sequentially list TA_Sequence _ list -> sequentially list- TA_Parentesis _ t -> build t+ TA_Parenthesis _ t -> build t TA_Trinary _ cond a b -> do condition <- build cond@@ -578,7 +574,7 @@ T_Array _ list -> sequentially list - T_Assignment {} -> buildAssignment DefaultScope t+ T_Assignment {} -> buildAssignment Nothing t T_Backgrounded id body -> do start <- newStructuralNode@@ -614,15 +610,15 @@ T_CaseExpression id t [] -> build t - T_CaseExpression id t list -> do+ T_CaseExpression id t list@(hd:tl) -> do start <- newStructuralNode token <- build t- branches <- mapM buildBranch list+ branches <- mapM buildBranch (hd NE.:| tl) end <- newStructuralNode - let neighbors = zip branches $ tail branches- let (_, firstCond, _) = head branches- let (_, lastCond, lastBody) = last branches+ let neighbors = zip (NE.toList branches) $ NE.tail branches+ let (_, firstCond, _) = NE.head branches+ let (_, lastCond, lastBody) = NE.last branches linkRange start token linkRange token firstCond@@ -672,10 +668,18 @@ status <- newNodeRange $ CFSetExitCode id linkRange cond status - T_CoProc id maybeName t -> do- let name = fromMaybe "COPROC" maybeName+ T_CoProc id maybeNameToken t -> do+ -- If unspecified, "COPROC". If not a constant string, Nothing.+ let maybeName = case maybeNameToken of+ Just x -> getLiteralString x+ Nothing -> Just "COPROC"++ let parentNode = case maybeName of+ Just str -> applySingle $ IdTagged id $ CFWriteVariable str CFValueArray+ Nothing -> CFStructuralNode+ start <- newStructuralNode- parent <- newNodeRange $ applySingle $ IdTagged id $ CFWriteVariable name CFValueArray+ parent <- newNodeRange parentNode child <- subshell id "coproc" $ build t end <- newNodeRange $ CFSetExitCode id @@ -712,6 +716,9 @@ linkRange totalRead result else return totalRead + T_DollarBraceCommandExpansion id _ body ->+ sequentially body+ T_DoubleQuoted _ list -> sequentially list T_DollarExpansion id body ->@@ -857,8 +864,8 @@ status <- newNodeRange (CFSetExitCode id) linkRange assignments status - T_SimpleCommand id vars list@(cmd:_) ->- handleCommand t vars list $ getUnquotedLiteral cmd+ T_SimpleCommand id vars (cmd:args) ->+ handleCommand t vars (cmd NE.:| args) $ getUnquotedLiteral cmd T_SingleQuoted _ _ -> none @@ -887,7 +894,9 @@ T_Less _ -> none T_ParamSubSpecialChar _ _ -> none - x -> error ("Unimplemented: " ++ show x)+ x -> do+ error ("Unimplemented: " ++ show x) -- STRIP+ none -- Still in `where` clause forInHelper id name words body = do@@ -923,8 +932,8 @@ -- TODO: Handle assignments in declaring commands case literalCmd of- Just "exit" -> regularExpansion vars args $ handleExit- Just "return" -> regularExpansion vars args $ handleReturn+ Just "exit" -> regularExpansion vars (NE.toList args) $ handleExit+ Just "return" -> regularExpansion vars (NE.toList args) $ handleReturn Just "unset" -> regularExpansionWithStatus vars args $ handleUnset args Just "declare" -> handleDeclare args@@ -947,14 +956,14 @@ -- This will mostly behave like 'command' but ok Just "builtin" -> case args of- [_] -> regular- (_:newargs@(newcmd:_)) ->- handleCommand newcmd vars newargs $ getLiteralString newcmd+ _ NE.:| [] -> regular+ (_ NE.:| newcmd:newargs) ->+ handleCommand newcmd vars (newcmd NE.:| newargs) $ getLiteralString newcmd Just "command" -> case args of- [_] -> regular- (_:newargs@(newcmd:_)) ->- handleOthers (getId newcmd) vars newargs $ getLiteralString newcmd+ _ NE.:| [] -> regular+ (_ NE.:| newcmd:newargs) ->+ handleOthers (getId newcmd) vars (newcmd NE.:| newargs) $ getLiteralString newcmd _ -> regular where@@ -982,7 +991,7 @@ unreachable <- newNode CFUnreachable return $ Range ret unreachable - handleUnset (cmd:args) = do+ handleUnset (cmd NE.:| args) = do case () of _ | "n" `elem` flagNames -> unsetWith CFUndefineNameref _ | "v" `elem` flagNames -> unsetWith CFUndefineVariable@@ -994,14 +1003,14 @@ (names, flags) = partition (null . fst) pairs flagNames = map fst flags literalNames :: [(Token, String)] -- Literal names to unset, e.g. [(myfuncToken, "myfunc")]- literalNames = mapMaybe (\(_, t) -> getLiteralString t >>= (return . (,) t)) names+ literalNames = mapMaybe (\(_, t) -> (,) t <$> getLiteralString t) names -- Apply a constructor like CFUndefineVariable to each literalName, and tag with its id unsetWith c = newNodeRange $ CFApplyEffects $ map (\(token, name) -> IdTagged (getId token) $ c name) literalNames variableAssignRegex = mkRegex "^([_a-zA-Z][_a-zA-Z0-9]*)=" - handleDeclare (cmd:args) = do+ handleDeclare (cmd NE.:| args) = do isFunc <- asks cfIsFunction -- This is a bit of a kludge: we don't have great support for things like -- 'declare -i x=$x' so do one round with declare x=$x, followed by declare -i x@@ -1028,9 +1037,9 @@ scope isFunc = case () of- _ | global -> GlobalScope- _ | isFunc -> LocalScope- _ -> DefaultScope+ _ | global -> Just GlobalScope+ _ | isFunc -> Just LocalScope+ _ -> Nothing addedProps = S.fromList $ concat $ [ [ CFVPArray | array ],@@ -1058,7 +1067,7 @@ let id = getId t pre = [t]- literal = fromJust $ getLiteralStringExt (const $ Just "\0") t+ literal = getLiteralStringDef "\0" t isKnown = '\0' `notElem` literal match = fmap head $ variableAssignRegex `matchRegex` literal name = fromMaybe literal match@@ -1090,7 +1099,7 @@ in concatMap (drop 1) plusses - handlePrintf (cmd:args) =+ handlePrintf (cmd NE.:| args) = newNodeRange $ CFApplyEffects $ maybeToList findVar where findVar = do@@ -1099,7 +1108,7 @@ name <- getLiteralString arg return $ IdTagged (getId arg) $ CFWriteVariable name CFValueString - handleWait (cmd:args) =+ handleWait (cmd NE.:| args) = newNodeRange $ CFApplyEffects $ maybeToList findVar where findVar = do@@ -1108,7 +1117,7 @@ name <- getLiteralString arg return $ IdTagged (getId arg) $ CFWriteVariable name CFValueInteger - handleMapfile (cmd:args) =+ handleMapfile (cmd NE.:| args) = newNodeRange $ CFApplyEffects [findVar] where findVar =@@ -1128,7 +1137,7 @@ guard $ isVariableName name return (getId c, name) - handleRead (cmd:args) = newNodeRange $ CFApplyEffects main+ handleRead (cmd NE.:| args) = newNodeRange $ CFApplyEffects main where main = fromMaybe fallback $ do flags <- getGnuOpts flagsForRead args@@ -1158,7 +1167,7 @@ in map (\(id, name) -> IdTagged id $ CFWriteVariable name value) namesOrDefault - handleDEFINE (cmd:args) =+ handleDEFINE (cmd NE.:| args) = newNodeRange $ CFApplyEffects $ maybeToList findVar where findVar = do@@ -1168,14 +1177,14 @@ return $ IdTagged (getId name) $ CFWriteVariable str CFValueString handleOthers id vars args cmd =- regularExpansion vars args $ do+ regularExpansion vars (NE.toList args) $ do exe <- newNodeRange $ CFExecuteCommand cmd status <- newNodeRange $ CFSetExitCode id linkRange exe status regularExpansion vars args p = do args <- sequentially args- assignments <- mapM (buildAssignment PrefixScope) vars+ assignments <- mapM (buildAssignment (Just PrefixScope)) vars exe <- p dropAssignments <- if null vars@@ -1187,15 +1196,15 @@ linkRanges $ [args] ++ assignments ++ [exe] ++ dropAssignments - regularExpansionWithStatus vars args@(cmd:_) p = do- initial <- regularExpansion vars args p+ regularExpansionWithStatus vars args@(cmd NE.:| _) p = do+ initial <- regularExpansion vars (NE.toList args) p status <- newNodeRange $ CFSetExitCode (getId cmd) linkRange initial status none = newStructuralNode -data Scope = DefaultScope | GlobalScope | LocalScope | PrefixScope+data Scope = GlobalScope | LocalScope | PrefixScope deriving (Eq, Ord, Show, Generic, NFData) buildAssignment scope t = do@@ -1209,10 +1218,10 @@ let valueType = if null indices then f id value else CFValueArray let scoper = case scope of- PrefixScope -> CFWritePrefix- LocalScope -> CFWriteLocal- GlobalScope -> CFWriteGlobal- DefaultScope -> CFWriteVariable+ Just PrefixScope -> CFWritePrefix+ Just LocalScope -> CFWriteLocal+ Just GlobalScope -> CFWriteGlobal+ Nothing -> CFWriteVariable write <- newNodeRange $ applySingle $ IdTagged id $ scoper var valueType linkRanges [expand, index, read, write] where@@ -1301,7 +1310,10 @@ reversed = grev withExitEdges postDoms = dom reversed mainexit (_, maxNode) = nodeRange graph- asArray = array (0, maxNode) postDoms+ -- Holes in the array cause "Exception: (Array.!): undefined array element" while+ -- inspecting/debugging, so fill the array first and then update.+ initializedArray = listArray (0, maxNode) $ repeat []+ asArray = initializedArray // postDoms return [] runTests = $( [| $(forAllProperties) (quickCheckWithResult (stdArgs { maxSuccess = 1 }) ) |])
src/ShellCheck/CFGAnalysis.hs view
@@ -59,6 +59,8 @@ ,getIncomingState ,getOutgoingState ,doesPostDominate+ ,variableMayBeDeclaredInteger+ ,variableMayBeAssignedInteger ,ShellCheck.CFGAnalysis.runTests -- STRIP ) where @@ -131,7 +133,7 @@ literalValue = Nothing } }- flatVars = M.unionsWith (\_ last -> last) $ map mapStorage [sGlobalValues s, sLocalValues s, sPrefixValues s]+ flatVars = M.unions $ map mapStorage [sPrefixValues s, sLocalValues s, sGlobalValues s] -- Conveniently get the state before a token id getIncomingState :: CFGAnalysis -> Id -> Maybe ProgramState@@ -153,6 +155,20 @@ (targetStart, _) <- M.lookup target $ tokenToRange analysis return $ targetStart `elem` (postDominators analysis ! baseEnd) +-- See if any execution path results in the variable containing a state+variableMayHaveState :: ProgramState -> String -> CFVariableProp -> Maybe Bool+variableMayHaveState state var property = do+ value <- M.lookup var $ variablesInScope state+ return $ any (S.member property) $ variableProperties value++-- See if any execution path declares the variable an integer (declare -i).+variableMayBeDeclaredInteger state var = variableMayHaveState state var CFVPInteger++-- See if any execution path suggests the variable may contain an integer value+variableMayBeAssignedInteger state var = do+ value <- M.lookup var $ variablesInScope state+ return $ (numericalStatus $ variableValue value) >= NumericalStatusMaybe+ getDataForNode analysis node = M.lookup node $ nodeToData analysis -- The current state of data flow at a point in the program, potentially as a diff@@ -283,7 +299,6 @@ PrefixScope -> (sPrefixValues, insertPrefix) LocalScope -> (sLocalValues, insertLocal) GlobalScope -> (sGlobalValues, insertGlobal)- DefaultScope -> error $ pleaseReport "Unresolved scope in dependency" alreadyExists = isJust $ vmLookup name $ mapToCheck state in@@ -657,7 +672,7 @@ _ | vmIsQuickEqual base diff -> diff _ -> VersionedMap { mapVersion = -1,- mapStorage = M.unionWith (flip const) (mapStorage base) (mapStorage diff)+ mapStorage = M.union (mapStorage diff) (mapStorage base) } -- Set a variable. This includes properties. Applies it to the appropriate scope.@@ -814,7 +829,7 @@ f (s:rest) = do -- Go up the stack until we find the value, and add -- a dependency on each state (including where it was found)- res <- fromMaybe (f rest) (return <$> get (stackState s) key)+ res <- maybe (f rest) return (get (stackState s) key) modifySTRef (dependencies s) $ S.insert $ dep key res return res @@ -1104,34 +1119,34 @@ CFSetProps scope name props -> case scope of- DefaultScope -> do+ Nothing -> do state <- readVariable ctx name writeVariable ctx name $ addProperties props state- GlobalScope -> do+ Just GlobalScope -> do state <- readGlobal ctx name writeGlobal ctx name $ addProperties props state- LocalScope -> do+ Just LocalScope -> do out <- readSTRef (cOutput ctx) state <- readLocal ctx name writeLocal ctx name $ addProperties props state- PrefixScope -> do+ Just PrefixScope -> do -- Prefix values become local state <- readLocal ctx name writeLocal ctx name $ addProperties props state CFUnsetProps scope name props -> case scope of- DefaultScope -> do+ Nothing -> do state <- readVariable ctx name writeVariable ctx name $ removeProperties props state- GlobalScope -> do+ Just GlobalScope -> do state <- readGlobal ctx name writeGlobal ctx name $ removeProperties props state- LocalScope -> do+ Just LocalScope -> do out <- readSTRef (cOutput ctx) state <- readLocal ctx name writeLocal ctx name $ removeProperties props state- PrefixScope -> do+ Just PrefixScope -> do -- Prefix values become local state <- readLocal ctx name writeLocal ctx name $ removeProperties props state@@ -1271,7 +1286,7 @@ else do let (next, rest) = S.deleteFindMin ps nexts <- process states next- writeSTRef pending $ foldl (flip S.insert) rest nexts+ writeSTRef pending $ S.union (S.fromList nexts) rest f (n-1) pending states process states node = do@@ -1335,7 +1350,7 @@ -- All nodes we've touched invocations <- readSTRef $ cInvocations ctx- let invokedNodes = M.fromDistinctAscList $ map (\c -> (c, ())) $ S.toList $ M.keysSet $ groupByNode $ M.map snd invocations+ let invokedNodes = M.fromSet (const ()) $ S.unions $ map (M.keysSet . snd) $ M.elems invocations -- Invoke all functions that were declared but not invoked -- This is so that we still get warnings for dead code@@ -1358,7 +1373,7 @@ -- Fill in the map with unreachable states for anything we didn't get to let baseStates = M.fromDistinctAscList $ map (\c -> (c, (unreachableState, unreachableState))) $ uncurry enumFromTo $ nodeRange $ cfGraph cfg- let allStates = M.unionWith (flip const) baseStates invokedStates+ let allStates = M.union invokedStates baseStates -- Convert to external states let nodeToData = M.map (\(a,b) -> (internalToExternal a, internalToExternal b)) allStates
src/ShellCheck/Checker.hs view
@@ -25,6 +25,7 @@ import ShellCheck.Interface import ShellCheck.Parser +import Debug.Trace -- DO NOT SUBMIT import Data.Either import Data.Functor import Data.List@@ -86,6 +87,7 @@ asCheckSourced = csCheckSourced spec, asExecutionMode = Executed, asTokenPositions = tokenPositions,+ asExtendedAnalysis = csExtendedAnalysis spec, asOptionalChecks = getEnableDirectives root ++ csOptionalChecks spec } where as = newAnalysisSpec root let analysisMessages =@@ -219,6 +221,9 @@ prop_worksWhenSourcingWithDashDash = null $ checkWithIncludes [("lib", "bar=1")] "source -- lib; echo \"$bar\"" +prop_worksWhenSourcingWithDashP =+ null $ checkWithIncludes [("lib", "bar=1")] "source -p \"$MYPATH\" lib; echo \"$bar\""+ prop_worksWhenDotting = null $ checkWithIncludes [("lib", "bar=1")] ". lib; echo \"$bar\"" @@ -506,6 +511,56 @@ where result = checkWithRc "disable=1104" emptyCheckSpec { csScript = "!/bin/bash\necho 'hello world'"+ }++prop_sourceWithHereDocWorks = null result+ where+ result = checkWithIncludes [("bar", "true\n")] "source bar << eof\nlol\neof"++prop_hereDocsAreParsedWithoutTrailingLinefeed = 1044 `elem` result+ where+ result = check "cat << eof"++prop_hereDocsWillHaveParsedIndices = null result+ where+ result = check "#!/bin/bash\nmy_array=(a b)\ncat <<EOF >> ./test\n $(( 1 + my_array[1] ))\nEOF"++prop_rcCanSuppressDfa = null result+ where+ result = checkWithRc "extended-analysis=false" emptyCheckSpec {+ csScript = "#!/bin/sh\nexit; foo;"+ }++prop_fileCanSuppressDfa = null $ traceShowId result+ where+ result = checkWithRc "" emptyCheckSpec {+ csScript = "#!/bin/sh\n# shellcheck extended-analysis=false\nexit; foo;"+ }++prop_fileWinsWhenSuppressingDfa1 = null result+ where+ result = checkWithRc "extended-analysis=true" emptyCheckSpec {+ csScript = "#!/bin/sh\n# shellcheck extended-analysis=false\nexit; foo;"+ }++prop_fileWinsWhenSuppressingDfa2 = result == [2317]+ where+ result = checkWithRc "extended-analysis=false" emptyCheckSpec {+ csScript = "#!/bin/sh\n# shellcheck extended-analysis=true\nexit; foo;"+ }++prop_flagWinsWhenSuppressingDfa1 = result == [2317]+ where+ result = checkWithRc "extended-analysis=false" emptyCheckSpec {+ csScript = "#!/bin/sh\n# shellcheck extended-analysis=false\nexit; foo;",+ csExtendedAnalysis = Just True+ }++prop_flagWinsWhenSuppressingDfa2 = null result+ where+ result = checkWithRc "extended-analysis=true" emptyCheckSpec {+ csScript = "#!/bin/sh\n# shellcheck extended-analysis=true\nexit; foo;",+ csExtendedAnalysis = Just False } return []
src/ShellCheck/Checks/Commands.hs view
@@ -20,6 +20,7 @@ {-# LANGUAGE TemplateHaskell #-} {-# LANGUAGE FlexibleContexts #-} {-# LANGUAGE MultiWayIf #-}+{-# LANGUAGE PatternGuards #-} -- This module contains checks that examine specific commands by name. module ShellCheck.Checks.Commands (checker, optionalChecks, ShellCheck.Checks.Commands.runTests) where@@ -42,6 +43,7 @@ import qualified Data.Graph.Inductive.Graph as G import Data.List import Data.Maybe+import qualified Data.List.NonEmpty as NE import qualified Data.Map.Strict as M import qualified Data.Set as S import Test.QuickCheck.All (forAllProperties)@@ -181,16 +183,15 @@ checkCommand map t@(T_SimpleCommand id cmdPrefix (cmd:rest)) = sequence_ $ do name <- getLiteralString cmd return $- if '/' `elem` name- then- M.findWithDefault nullCheck (Basename $ basename name) map t- else if name == "builtin" && not (null rest) then- let t' = T_SimpleCommand id cmdPrefix rest- selectedBuiltin = fromMaybe "" $ getLiteralString . head $ rest- in M.findWithDefault nullCheck (Exactly selectedBuiltin) map t'- else do- M.findWithDefault nullCheck (Exactly name) map t- M.findWithDefault nullCheck (Basename name) map t+ if | '/' `elem` name ->+ M.findWithDefault nullCheck (Basename $ basename name) map t+ | name == "builtin", (h:_) <- rest ->+ let t' = T_SimpleCommand id cmdPrefix rest+ selectedBuiltin = onlyLiteralString h+ in M.findWithDefault nullCheck (Exactly selectedBuiltin) map t'+ | otherwise -> do+ M.findWithDefault nullCheck (Exactly name) map t+ M.findWithDefault nullCheck (Basename name) map t where basename = reverse . takeWhile (/= '/') . reverse@@ -299,7 +300,7 @@ "'expr' expects 3+ arguments but sees 1. Make sure each operator/operand is a separate argument, and escape <>&|." [first, second] |- (fromMaybe "" $ getLiteralString first) /= "length"+ onlyLiteralString first /= "length" && not (willSplit first || willSplit second) -> do checkOp first warn (getId t) 2307@@ -724,6 +725,9 @@ prop_checkGetPrintfFormats5 = getPrintfFormats "%bPassed: %d, %bFailed: %d%b, Skipped: %d, %bErrored: %d%b\\n" == "bdbdbdbdb" prop_checkGetPrintfFormats6 = getPrintfFormats "%s%s" == "ss" prop_checkGetPrintfFormats7 = getPrintfFormats "%s\n%s" == "ss"+prop_checkGetPrintfFormats8 = getPrintfFormats "%ld" == "d"+prop_checkGetPrintfFormats9 = getPrintfFormats "%lld" == "d"+prop_checkGetPrintfFormats10 = getPrintfFormats "%Q" == "Q" getPrintfFormats = getFormats where -- Get the arguments in the string as a string of type characters,@@ -742,17 +746,17 @@ regexBasedGetFormats rest = case matchRegex re rest of- Just [width, precision, typ, rest, _] ->+ Just [width, precision, len, typ, rest, _] -> (if width == "*" then "*" else "") ++ (if precision == "*" then "*" else "") ++ typ ++ getFormats rest Nothing -> take 1 rest ++ getFormats rest where -- constructed based on specifications in "man printf"- re = mkRegex "#?-?\\+? ?0?(\\*|\\d*)\\.?(\\d*|\\*)([diouxXfFeEgGaAcsbq])((\n|.)*)"- -- \____ _____/\___ ____/ \____ ____/\_________ _________/ \______ /- -- V V V V V- -- flags field width precision format character rest+ re = mkRegex "^#?-?\\+? ?0?(\\*|\\d*)\\.?(\\d*|\\*)(hh|h|l|ll|q|L|j|z|Z|t)?([diouxXfFeEgGaAcsbqQSC])((\n|.)*)"+ -- \____ _____/\___ ____/ \____ ____/\__________ ___________/\___________ ___________/\___ ___/+ -- V V V V V V+ -- flags field width precision length modifier format character rest -- field width and precision can be specified with an '*' instead of a digit, -- in which case printf will accept one more argument for each '*' used @@ -930,7 +934,7 @@ prop_checkTimedCommand3 = verifyNot checkTimedCommand "#!/bin/sh\ntime sleep 1" checkTimedCommand = CommandCheck (Exactly "time") f where f (T_SimpleCommand _ _ (c:args@(_:_))) =- whenShell [Sh, Dash] $ do+ whenShell [Sh, Dash, BusyboxSh] $ do let cmd = last args -- "time" is parsed with a command as argument when (isPiped cmd) $ warn (getId c) 2176 "'time' is undefined for pipelines. time single stage or bash -c instead."@@ -954,7 +958,7 @@ prop_checkLocalScope1 = verify checkLocalScope "local foo=3" prop_checkLocalScope2 = verifyNot checkLocalScope "f() { local foo=3; }" checkLocalScope = CommandCheck (Exactly "local") $ \t ->- whenShell [Bash, Dash] $ do -- Ksh allows it, Sh doesn't support local+ whenShell [Bash, Dash, BusyboxSh] $ do -- Ksh allows it, Sh doesn't support local path <- getPathM t unless (any isFunctionLike path) $ err (getId $ getCommandTokenOrThis t) 2168 "'local' is only valid in functions."@@ -1005,8 +1009,8 @@ sequence_ $ do options <- getLiteralString arg1 getoptsVar <- getLiteralString name- (T_WhileExpression _ _ body) <- findFirst whileLoop path- caseCmd@(T_CaseExpression _ var _) <- mapMaybe findCase body !!! 0+ (T_WhileExpression _ _ body) <- findFirst whileLoop (NE.toList path)+ T_CaseExpression id var list <- mapMaybe findCase body !!! 0 -- Make sure getopts name and case variable matches [T_DollarBraced _ _ bracedWord] <- return $ getWordParts var@@ -1016,11 +1020,11 @@ -- Make sure the variable isn't modified guard . not $ modifiesVariable params (T_BraceGroup (Id 0) body) getoptsVar - return $ check (getId arg1) (map (:[]) $ filter (/= ':') options) caseCmd+ return $ check (getId arg1) (map (:[]) $ filter (/= ':') options) id list f _ = return () - check :: Id -> [String] -> Token -> Analysis- check optId opts (T_CaseExpression id _ list) = do+ check :: Id -> [String] -> Id -> [(CaseType, [Token], [Token])] -> Analysis+ check optId opts id list = do unless (Nothing `M.member` handledMap) $ do mapM_ (warnUnhandled optId id) $ catMaybes $ M.keys notHandled @@ -1236,12 +1240,11 @@ where f t = sequence_ $ do opts <- parseOpts $ arguments t- let nonFlags = [x | ("",(x, _)) <- opts]- commandArg <- nonFlags !!! 0+ (_,(commandArg, _)) <- find (null . fst) opts command <- getLiteralString commandArg guard $ command `elem` builtins return $ warn (getId t) 2232 $ "Can't use sudo with builtins like " ++ command ++ ". Did you want sudo sh -c .. instead?"- builtins = [ "cd", "eval", "export", "history", "read", "source", "wait" ]+ builtins = [ "cd", "command", "declare", "eval", "exec", "exit", "export", "hash", "history", "local", "popd", "pushd", "read", "readonly", "return", "set", "source", "trap", "type", "typeset", "ulimit", "umask", "unset", "wait" ] -- This mess is why ShellCheck prefers not to know. parseOpts = getBsdOpts "vAknSbEHPa:g:h:p:u:c:T:r:" @@ -1430,26 +1433,27 @@ prop_checkBackreferencingDeclaration7 = verify (checkBackreferencingDeclaration "declare") "declare x=var $k=$x" checkBackreferencingDeclaration cmd = CommandCheck (Exactly cmd) check where- check t = foldM_ perArg M.empty $ arguments t+ check t = do+ maybeCfga <- asks cfgAnalysis+ mapM_ (\cfga -> foldM_ (perArg cfga) M.empty $ arguments t) maybeCfga - perArg leftArgs t =+ perArg cfga leftArgs t = case t of T_Assignment id _ name idx t -> do- warnIfBackreferencing leftArgs $ t:idx+ warnIfBackreferencing cfga leftArgs $ t:idx return $ M.insert name id leftArgs t -> do- warnIfBackreferencing leftArgs [t]+ warnIfBackreferencing cfga leftArgs [t] return leftArgs - warnIfBackreferencing backrefs l = do- references <- findReferences l+ warnIfBackreferencing cfga backrefs l = do+ references <- findReferences cfga l let reused = M.intersection backrefs references mapM msg $ M.toList reused msg (name, id) = warn id 2318 $ "This assignment is used again in this '" ++ cmd ++ "', but won't have taken effect. Use two '" ++ cmd ++ "'s." - findReferences list = do- cfga <- asks cfgAnalysis+ findReferences cfga list = do let graph = CF.graph cfga let nodesMap = CF.tokenToNodes cfga let nodes = S.unions $ map (\id -> M.findWithDefault S.empty id nodesMap) $ map getId $ list
src/ShellCheck/Checks/ControlFlow.hs view
@@ -78,7 +78,7 @@ runNodeChecks :: [ControlFlowNodeCheck] -> ControlFlowCheck runNodeChecks perNode = do cfg <- asks cfgAnalysis- runOnAll cfg+ mapM_ runOnAll cfg where getData datas n@(node, label) = do (pre, post) <- M.lookup node datas
src/ShellCheck/Checks/Custom.hs view
@@ -1,7 +1,7 @@ {- This empty file is provided for ease of patching in site specific checks. However, there are no guarantees regarding compatibility between versions.--} +-} {-# LANGUAGE TemplateHaskell #-} module ShellCheck.Checks.Custom (checker, ShellCheck.Checks.Custom.runTests) where
src/ShellCheck/Checks/ShellSupport.hs view
@@ -19,6 +19,7 @@ -} {-# LANGUAGE TemplateHaskell #-} {-# LANGUAGE FlexibleContexts #-}+{-# LANGUAGE ViewPatterns #-} module ShellCheck.Checks.ShellSupport (checker , ShellCheck.Checks.ShellSupport.runTests) where import ShellCheck.AST@@ -60,6 +61,9 @@ ,checkBraceExpansionVars ,checkMultiDimensionalArrays ,checkPS1Assignments+ ,checkMultipleBangs+ ,checkBangAfterPipe+ ,checkNegatedUnaryOps ] testChecker (ForShell _ t) =@@ -73,22 +77,24 @@ prop_checkForDecimals1 = verify checkForDecimals "((3.14*c))" prop_checkForDecimals2 = verify checkForDecimals "foo[1.2]=bar" prop_checkForDecimals3 = verifyNot checkForDecimals "declare -A foo; foo[1.2]=bar"-checkForDecimals = ForShell [Sh, Dash, Bash] f+checkForDecimals = ForShell [Sh, Dash, BusyboxSh, Bash] f where f t@(TA_Expansion id _) = sequence_ $ do- str <- getLiteralString t- first <- str !!! 0- guard $ isDigit first && '.' `elem` str+ first:rest <- getLiteralString t+ guard $ isDigit first && '.' `elem` rest return $ err id 2079 "(( )) doesn't support decimals. Use bc or awk." f _ = return () prop_checkBashisms = verify checkBashisms "while read a; do :; done < <(a)"-prop_checkBashisms2 = verify checkBashisms "[ foo -nt bar ]"+prop_checkBashisms2 = verifyNot checkBashisms "[ foo -nt bar ]" prop_checkBashisms3 = verify checkBashisms "echo $((i++))" prop_checkBashisms4 = verify checkBashisms "rm !(*.hs)" prop_checkBashisms5 = verify checkBashisms "source file" prop_checkBashisms6 = verify checkBashisms "[ \"$a\" == 42 ]"+prop_checkBashisms6b = verify checkBashisms "test \"$a\" == 42"+prop_checkBashisms6c = verify checkBashisms "[ foo =~ bar ]"+prop_checkBashisms6d = verify checkBashisms "test foo =~ bar" prop_checkBashisms7 = verify checkBashisms "echo ${var[1]}" prop_checkBashisms8 = verify checkBashisms "echo ${!var[@]}" prop_checkBashisms9 = verify checkBashisms "echo ${!var*}"@@ -104,6 +110,7 @@ prop_checkBashisms19 = verify checkBashisms "foo > file*.txt" prop_checkBashisms20 = verify checkBashisms "read -ra foo" prop_checkBashisms21 = verify checkBashisms "[ -a foo ]"+prop_checkBashisms21b = verify checkBashisms "test -a foo" prop_checkBashisms22 = verifyNot checkBashisms "[ foo -a bar ]" prop_checkBashisms23 = verify checkBashisms "trap mything ERR INT" prop_checkBashisms24 = verifyNot checkBashisms "trap mything INT TERM"@@ -184,49 +191,82 @@ prop_checkBashisms97 = verify checkBashisms "#!/bin/sh\necho ${var,}" prop_checkBashisms98 = verify checkBashisms "#!/bin/sh\necho ${var^^}" prop_checkBashisms99 = verify checkBashisms "#!/bin/dash\necho [^f]oo"-checkBashisms = ForShell [Sh, Dash] $ \t -> do+prop_checkBashisms100 = verify checkBashisms "read -r"+prop_checkBashisms101 = verify checkBashisms "read"+prop_checkBashisms102 = verifyNot checkBashisms "read -r foo"+prop_checkBashisms103 = verifyNot checkBashisms "read foo"+prop_checkBashisms104 = verifyNot checkBashisms "read ''"+prop_checkBashisms105 = verifyNot checkBashisms "#!/bin/busybox sh\nset -o pipefail"+prop_checkBashisms106 = verifyNot checkBashisms "#!/bin/busybox sh\nx=x\n[[ \"$x\" = \"$x\" ]]"+prop_checkBashisms107 = verifyNot checkBashisms "#!/bin/busybox sh\nx=x\n[ \"$x\" == \"$x\" ]"+prop_checkBashisms108 = verifyNot checkBashisms "#!/bin/busybox sh\necho magic &> /dev/null"+prop_checkBashisms109 = verifyNot checkBashisms "#!/bin/busybox sh\ntrap stop EXIT SIGTERM"+prop_checkBashisms110 = verifyNot checkBashisms "#!/bin/busybox sh\nsource /dev/null"+prop_checkBashisms111 = verify checkBashisms "#!/bin/dash\nx='test'\n${x:0:3}" -- SC3057+prop_checkBashisms112 = verifyNot checkBashisms "#!/bin/busybox sh\nx='test'\n${x:0:3}" -- SC3057+prop_checkBashisms113 = verify checkBashisms "#!/bin/dash\nx='test'\n${x/st/xt}" -- SC3060+prop_checkBashisms114 = verifyNot checkBashisms "#!/bin/busybox sh\nx='test'\n${x/st/xt}" -- SC3060+prop_checkBashisms115 = verify checkBashisms "#!/bin/busybox sh\nx='test'\n${!x}" -- SC3053+prop_checkBashisms116 = verify checkBashisms "#!/bin/busybox sh\nx='test'\n${x[1]}" -- SC3054+prop_checkBashisms117 = verify checkBashisms "#!/bin/busybox sh\nx='test'\n${!x[@]}" -- SC3055+prop_checkBashisms118 = verify checkBashisms "#!/bin/busybox sh\nxyz=1\n${!x*}" -- SC3056+prop_checkBashisms119 = verify checkBashisms "#!/bin/busybox sh\nx='test'\n${x^^[t]}" -- SC3059+prop_checkBashisms120 = verify checkBashisms "#!/bin/sh\n[ x == y ]"+prop_checkBashisms121 = verifyNot checkBashisms "#!/bin/sh\n# shellcheck shell=busybox\n[ x == y ]"+prop_checkBashisms122 = verify checkBashisms "#!/bin/dash\n$'a'"+prop_checkBashisms123 = verifyNot checkBashisms "#!/bin/busybox sh\n$'a'"+prop_checkBashisms124 = verify checkBashisms "#!/bin/dash\ntype -p test"+prop_checkBashisms125 = verifyNot checkBashisms "#!/bin/busybox sh\ntype -p test"+prop_checkBashisms126 = verifyNot checkBashisms "#!/bin/busybox sh\nread -p foo -r bar"+prop_checkBashisms127 = verifyNot checkBashisms "#!/bin/busybox sh\necho -ne foo"+prop_checkBashisms128 = verify checkBashisms "#!/bin/dash\ntype -p test"+prop_checkBashisms129 = verify checkBashisms "#!/bin/sh\n[ -k /tmp ]"+prop_checkBashisms130 = verifyNot checkBashisms "#!/bin/dash\ntest -k /tmp"+prop_checkBashisms131 = verify checkBashisms "#!/bin/sh\n[ -o errexit ]"+checkBashisms = ForShell [Sh, Dash, BusyboxSh] $ \t -> do params <- ask kludge params t where -- This code was copy-pasted from Analytics where params was a variable kludge params = bashism where- isDash = shellType params == Dash+ isBusyboxSh = shellType params == BusyboxSh+ isDash = shellType params == Dash || isBusyboxSh warnMsg id code s = if isDash then err id code $ "In dash, " ++ s ++ " not supported." else warn id code $ "In POSIX sh, " ++ s ++ " undefined."+ asStr = getLiteralString bashism (T_ProcSub id _ _) = warnMsg id 3001 "process substitution is" bashism (T_Extglob id _ _) = warnMsg id 3002 "extglob is"- bashism (T_DollarSingleQuoted id _) = warnMsg id 3003 "$'..' is"+ bashism (T_DollarSingleQuoted id _) =+ unless isBusyboxSh $ warnMsg id 3003 "$'..' is" bashism (T_DollarDoubleQuoted id _) = warnMsg id 3004 "$\"..\" is" bashism (T_ForArithmetic id _ _ _ _) = warnMsg id 3005 "arithmetic for loops are" bashism (T_Arithmetic id _) = warnMsg id 3006 "standalone ((..)) is" bashism (T_DollarBracket id _) = warnMsg id 3007 "$[..] in place of $((..)) is" bashism (T_SelectIn id _ _ _) = warnMsg id 3008 "select loops are" bashism (T_BraceExpansion id _) = warnMsg id 3009 "brace expansion is"- bashism (T_Condition id DoubleBracket _) = warnMsg id 3010 "[[ ]] is"+ bashism (T_Condition id DoubleBracket _) =+ unless isBusyboxSh $ warnMsg id 3010 "[[ ]] is" bashism (T_HereString id _) = warnMsg id 3011 "here-strings are"- bashism (TC_Binary id SingleBracket op _ _)- | op `elem` [ "<", ">", "\\<", "\\>", "<=", ">=", "\\<=", "\\>="] =- unless isDash $ warnMsg id 3012 $ "lexicographical " ++ op ++ " is"- bashism (TC_Binary id SingleBracket op _ _)- | op `elem` [ "-ot", "-nt", "-ef" ] =- unless isDash $ warnMsg id 3013 $ op ++ " is"- bashism (TC_Binary id SingleBracket "==" _ _) =- warnMsg id 3014 "== in place of = is"- bashism (TC_Binary id SingleBracket "=~" _ _) =- warnMsg id 3015 "=~ regex matching is"- bashism (TC_Unary id SingleBracket "-v" _) =- warnMsg id 3016 "unary -v (in place of [ -n \"${var+x}\" ]) is"- bashism (TC_Unary id _ "-a" _) =- warnMsg id 3017 "unary -a in place of -e is"++ bashism (TC_Binary id _ op _ _) =+ checkTestOp bashismBinaryTestFlags op id+ bashism (T_SimpleCommand id _ [asStr -> Just "test", lhs, asStr -> Just op, rhs]) =+ checkTestOp bashismBinaryTestFlags op id+ bashism (TC_Unary id _ op _) =+ checkTestOp bashismUnaryTestFlags op id+ bashism (T_SimpleCommand id _ [asStr -> Just "test", asStr -> Just op, _]) =+ checkTestOp bashismUnaryTestFlags op id+ bashism (TA_Unary id op _) | op `elem` [ "|++", "|--", "++|", "--|"] = warnMsg id 3018 $ filter (/= '|') op ++ " is" bashism (TA_Binary id "**" _ _) = warnMsg id 3019 "exponentials are"- bashism (T_FdRedirect id "&" (T_IoFile _ (T_Greater _) _)) = warnMsg id 3020 "&> is"+ bashism (T_FdRedirect id "&" (T_IoFile _ (T_Greater _) _)) =+ unless isBusyboxSh $ warnMsg id 3020 "&> is" bashism (T_FdRedirect id "" (T_IoFile _ (T_GREATAND _) file)) = unless (all isDigit $ onlyLiteralString file) $ warnMsg id 3021 ">& filename (as opposed to >& fd) is" bashism (T_FdRedirect id ('{':_) _) = warnMsg id 3022 "named file descriptors are"@@ -246,7 +286,8 @@ warnMsg id 3028 $ str ++ " is" bashism t@(T_DollarBraced id _ token) = do- mapM_ check expansion+ unless isBusyboxSh $ mapM_ check simpleExpansions+ mapM_ check advancedExpansions when (isBashVariable var) $ warnMsg id 3028 $ var ++ " is" where@@ -274,8 +315,12 @@ bashism t@(T_SimpleCommand _ _ (cmd:arg:_)) | t `isCommand` "echo" && argString `matches` flagRegex =- if isDash+ if isBusyboxSh then+ unless (argString `matches` busyboxFlagRegex) $+ warnMsg (getId arg) 3036 "echo flags besides -n and -e"+ else if isDash+ then when (argString /= "-n") $ warnMsg (getId arg) 3036 "echo flags besides -n" else@@ -283,6 +328,7 @@ where argString = concat $ oversimplify arg flagRegex = mkRegex "^-[eEsn]+$"+ busyboxFlagRegex = mkRegex "^-[en]+$" bashism t@(T_SimpleCommand _ _ (cmd:arg:_)) | getLiteralString cmd == Just "exec" && "-" `isPrefixOf` concat (oversimplify arg) =@@ -356,7 +402,8 @@ (\x -> (not . null . snd $ x) && snd x `notElem` allowed) flags return . warnMsg (getId word) 3045 $ name ++ " -" ++ flag ++ " is" - when (name == "source") $ warnMsg id 3046 "'source' in place of '.' is"+ when (name == "source" && not isBusyboxSh) $+ warnMsg id 3046 "'source' in place of '.' is" when (name == "trap") $ let check token = sequence_ $ do@@ -365,7 +412,7 @@ return $ do when (upper `elem` ["ERR", "DEBUG", "RETURN"]) $ warnMsg (getId token) 3047 $ "trapping " ++ str ++ " is"- when ("SIG" `isPrefixOf` upper) $+ when (not isBusyboxSh && "SIG" `isPrefixOf` upper) $ warnMsg (getId token) 3048 "prefixing signal names with 'SIG' is" when (not isDash && upper /= str) $@@ -379,6 +426,9 @@ let literal = onlyLiteralString format guard $ "%q" `isInfixOf` literal return $ warnMsg (getId format) 3050 "printf %q is"++ when (name == "read" && all isFlag rest) $+ warnMsg (getId cmd) 3061 "read without a variable is" where unsupportedCommands = [ "let", "caller", "builtin", "complete", "compgen", "declare", "dirs", "disown",@@ -392,17 +442,19 @@ ("hash", Just $ if isDash then ["r", "v"] else ["r"]), ("jobs", Just ["l", "p"]), ("printf", Just []),- ("read", Just $ if isDash then ["r", "p"] else ["r"]),+ ("read", Just $ if isDash || isBusyboxSh then ["r", "p"] else ["r"]), ("readonly", Just ["p"]), ("trap", Just []),- ("type", Just []),+ ("type", Just $ if isBusyboxSh then ["p"] else []), ("ulimit", if isDash then Nothing else Just ["f"]), ("umask", Just ["S"]), ("unset", Just ["f", "v"]), ("wait", Just []) ] bashism t@(T_SourceCommand id src _)- | getCommandName src == Just "source" = warnMsg id 3051 "'source' in place of '.' is"+ | getCommandName src == Just "source" =+ unless isBusyboxSh $+ warnMsg id 3051 "'source' in place of '.' is" bashism (TA_Expansion _ (T_Literal id str : _)) | str `matches` radix = warnMsg id 3052 "arithmetic base conversion is" where@@ -410,14 +462,16 @@ bashism _ = return () varChars="_0-9a-zA-Z"- expansion = let re = mkRegex in [+ advancedExpansions = let re = mkRegex in [ (re $ "^![" ++ varChars ++ "]", 3053, "indirect expansion is"), (re $ "^[" ++ varChars ++ "]+\\[.*\\]$", 3054, "array references are"), (re $ "^![" ++ varChars ++ "]+\\[[*@]]$", 3055, "array key expansion is"), (re $ "^![" ++ varChars ++ "]+[*@]$", 3056, "name matching prefixes are"),+ (re $ "^[" ++ varChars ++ "*@]+(\\[.*\\])?[,^]", 3059, "case modification is")+ ]+ simpleExpansions = let re = mkRegex in [ (re $ "^[" ++ varChars ++ "*@]+:[^-=?+]", 3057, "string indexing is"), (re $ "^([*@][%#]|#[@*])", 3058, "string operations on $@/$* are"),- (re $ "^[" ++ varChars ++ "*@]+(\\[.*\\])?[,^]", 3059, "case modification is"), (re $ "^[" ++ varChars ++ "*@]+(\\[.*\\])?/", 3060, "string replacement is") ] bashVars = [@@ -443,6 +497,50 @@ Assignment (_, _, name, _) -> name == var _ -> False + checkTestOp table op id = sequence_ $ do+ (code, shells, msg) <- Map.lookup op table+ guard . not $ shellType params `elem` shells+ return $ warnMsg id code (msg op)+++buildTestFlagMap list = Map.fromList $ concatMap (\(x,y) -> map (\c -> (c,y)) x) list+bashismBinaryTestFlags = buildTestFlagMap [+ -- ([list of applicable flags],+ -- (error code, exempt shells, message builder :: String -> String)),+ --+ -- Distinct error codes allow the wiki to give more helpful, targeted+ -- information.+ (["<", ">", "\\<", "\\>", "<=", ">=", "\\<=", "\\>="],+ (3012, [Dash, BusyboxSh], \op -> "lexicographical " ++ op ++ " is")),+ (["=="],+ (3014, [BusyboxSh], \op -> op ++ " in place of = is")),+ (["=~"],+ (3015, [], \op -> op ++ " regex matching is")),++ ([], (0,[],const ""))+ ]+bashismUnaryTestFlags = buildTestFlagMap [+ (["-v"],+ (3016, [], \op -> "test " ++ op ++ " (in place of [ -n \"${var+x}\" ]) is")),+ (["-a"],+ (3017, [], \op -> "unary " ++ op ++ " in place of -e is")),+ (["-o"],+ (3062, [], \op -> "test " ++ op ++ " to check options is")),+ (["-R"],+ (3063, [], \op -> "test " ++ op ++ " and namerefs in general are")),+ (["-N"],+ (3064, [], \op -> "test " ++ op ++ " is")),+ (["-k"],+ (3065, [Dash, BusyboxSh], \op -> "test " ++ op ++ " is")),+ (["-G"],+ (3066, [Dash, BusyboxSh], \op -> "test " ++ op ++ " is")),+ (["-O"],+ (3067, [Dash, BusyboxSh], \op -> "test " ++ op ++ " is")),++ ([], (0,[],const ""))+ ]++ prop_checkEchoSed1 = verify checkEchoSed "FOO=$(echo \"$cow\" | sed 's/foo/bar/g')" prop_checkEchoSed1b = verify checkEchoSed "FOO=$(sed 's/foo/bar/g' <<< \"$cow\")" prop_checkEchoSed2 = verify checkEchoSed "rm $(echo $cow | sed -e 's,foo,bar,')"@@ -557,6 +655,47 @@ enclosedRegex = mkRegex "\\\\\\[.*\\\\\\]" -- FIXME: shouldn't be eager escapeRegex = mkRegex "\\\\x1[Bb]|\\\\e|\x1B|\\\\033" ++prop_checkMultipleBangs1 = verify checkMultipleBangs "! ! true"+prop_checkMultipleBangs2 = verifyNot checkMultipleBangs "! true"+checkMultipleBangs = ForShell [Dash, BusyboxSh, Sh] f+ where+ f token = case token of+ T_Banged id (T_Banged _ _) ->+ err id 2325 "Multiple ! in front of pipelines are a bash/ksh extension. Use only 0 or 1."+ _ -> return ()+++prop_checkBangAfterPipe1 = verify checkBangAfterPipe "true | ! true"+prop_checkBangAfterPipe2 = verifyNot checkBangAfterPipe "true | ( ! true )"+prop_checkBangAfterPipe3 = verifyNot checkBangAfterPipe "! ! true | true"+checkBangAfterPipe = ForShell [Dash, BusyboxSh, Sh, Bash] f+ where+ f token = case token of+ T_Pipeline _ _ cmds -> mapM_ check cmds+ _ -> return ()++ check token = case token of+ T_Banged id _ ->+ err id 2326 "! is not allowed in the middle of pipelines. Use command group as in cmd | { ! cmd; } if necessary."+ _ -> return ()+++prop_checkNegatedUnaryOps1 = verify checkNegatedUnaryOps "[ ! -o braceexpand ]"+prop_checkNegatedUnaryOps2 = verifyNot checkNegatedUnaryOps "[ -o braceexpand ]"+prop_checkNegatedUnaryOps3 = verifyNot checkNegatedUnaryOps "[[ ! -o braceexpand ]]"+prop_checkNegatedUnaryOps4 = verifyNot checkNegatedUnaryOps "! [ -o braceexpand ]"+prop_checkNegatedUnaryOps5 = verify checkNegatedUnaryOps "[ ! -a file ]"+checkNegatedUnaryOps = ForShell [Bash] f+ where+ f token = case token of+ TC_Unary id SingleBracket "!" (TC_Unary _ _ op _) | op `elem` ["-a", "-o"] ->+ err id 2332 $ msg op+ _ -> return ()++ msg "-o" = "[ ! -o opt ] is always true because -o becomes logical OR. Use [[ ]] or ! [ -o opt ]."+ msg "-a" = "[ ! -a file ] is always true because -a becomes logical AND. Use -e instead."+ msg _ = pleaseReport "unhandled negated unary message" return [] runTests = $( [| $(forAllProperties) (quickCheckWithResult (stdArgs { maxSuccess = 1 }) ) |])
src/ShellCheck/Data.hs view
@@ -49,6 +49,7 @@ "LINES", "MAIL", "MAILCHECK", "MAILPATH", "OPTERR", "PATH", "POSIXLY_CORRECT", "PROMPT_COMMAND", "PROMPT_DIRTRIM", "PS0", "PS1", "PS2", "PS3", "PS4", "SHELL", "TIMEFORMAT", "TMOUT", "TMPDIR",+ "BASH_MONOSECONDS", "BASH_TRAPSIG", "GLOBSORT", "auto_resume", "histchars", -- Other@@ -62,6 +63,9 @@ , "FLAGS_ARGC", "FLAGS_ARGV", "FLAGS_ERROR", "FLAGS_FALSE", "FLAGS_HELP", "FLAGS_PARENT", "FLAGS_RESERVED", "FLAGS_TRUE", "FLAGS_VERSION", "flags_error", "flags_return"++ -- Bats+ ,"stderr", "stderr_lines" ] specialIntegerVariables = [@@ -75,7 +79,7 @@ "EPOCHREALTIME", "EPOCHSECONDS", "LINENO", "OPTIND", "PPID", "RANDOM", "READLINE_ARGUMENT", "READLINE_MARK", "READLINE_POINT", "SECONDS", "SHELLOPTS", "SHLVL", "SRANDOM", "UID", "COLUMNS", "HISTFILESIZE",- "HISTSIZE", "LINES"+ "HISTSIZE", "LINES", "BASH_MONOSECONDS", "BASH_TRAPSIG" -- shflags , "FLAGS_ERROR", "FLAGS_FALSE", "FLAGS_TRUE"@@ -156,11 +160,15 @@ "sh" -> return Sh "bash" -> return Bash "bats" -> return Bash+ "busybox" -> return BusyboxSh -- Used for directives and --shell=busybox+ "busybox sh" -> return BusyboxSh+ "busybox ash" -> return BusyboxSh "dash" -> return Dash "ash" -> return Dash -- There's also a warning for this. "ksh" -> return Ksh "ksh88" -> return Ksh "ksh93" -> return Ksh+ "oksh" -> return Ksh _ -> Nothing flagsForRead = "sreu:n:N:i:p:a:t:"
src/ShellCheck/Formatter/CheckStyle.hs view
@@ -24,8 +24,8 @@ import Data.Char import Data.List-import GHC.Exts import System.IO+import qualified Data.List.NonEmpty as NE format :: IO Formatter format = return Formatter {@@ -45,12 +45,12 @@ else mapM_ outputGroup fileGroups where comments = crComments cr- fileGroups = groupWith sourceFile comments+ fileGroups = NE.groupWith sourceFile comments outputGroup group = do- let filename = sourceFile (head group)+ let filename = sourceFile (NE.head group) result <- siReadFile sys (Just True) filename let contents = either (const "") id result- outputFile filename contents group+ outputFile filename contents (NE.toList group) outputFile filename contents warnings = do let comments = makeNonVirtual warnings contents
src/ShellCheck/Formatter/Diff.hs view
@@ -191,11 +191,17 @@ let (last, rest) = splitAt 1 $ reverse x in (reverse rest, last) +-- git patch does not like `\` on Windows+normalizePath path =+ case path of+ c:rest -> (if c == pathSeparator then '/' else c) : normalizePath rest+ [] -> []+ formatDoc color (DiffDoc name lf regions) = let (most, last) = splitLast regions in- (color bold $ "--- " ++ ("a" </> name)) ++ "\n" ++- (color bold $ "+++ " ++ ("b" </> name)) ++ "\n" +++ (color bold $ "--- " ++ (normalizePath $ "a" </> name)) ++ "\n" +++ (color bold $ "+++ " ++ (normalizePath $ "b" </> name)) ++ "\n" ++ concatMap (formatRegion color LinefeedOk) most ++ concatMap (formatRegion color lf) last
src/ShellCheck/Formatter/GCC.hs view
@@ -23,8 +23,8 @@ import ShellCheck.Formatter.Format import Data.List-import GHC.Exts import System.IO+import qualified Data.List.NonEmpty as NE format :: IO Formatter format = return Formatter {@@ -39,13 +39,13 @@ outputAll cr sys = mapM_ f groups where comments = crComments cr- groups = groupWith sourceFile comments- f :: [PositionedComment] -> IO ()+ groups = NE.groupWith sourceFile comments+ f :: NE.NonEmpty PositionedComment -> IO () f group = do- let filename = sourceFile (head group)+ let filename = sourceFile (NE.head group) result <- siReadFile sys (Just True) filename let contents = either (const "") id result- outputResult filename contents group+ outputResult filename contents (NE.toList group) outputResult filename contents warnings = do let comments = makeNonVirtual warnings contents
src/ShellCheck/Formatter/JSON1.hs view
@@ -27,9 +27,9 @@ import Data.Aeson import Data.IORef import Data.Monoid-import GHC.Exts import System.IO import qualified Data.ByteString.Lazy.Char8 as BL+import qualified Data.List.NonEmpty as NE format :: IO Formatter format = do@@ -114,10 +114,10 @@ collectResult ref cr sys = mapM_ f groups where comments = crComments cr- groups = groupWith sourceFile comments- f :: [PositionedComment] -> IO ()+ groups = NE.groupWith sourceFile comments+ f :: NE.NonEmpty PositionedComment -> IO () f group = do- let filename = sourceFile (head group)+ let filename = sourceFile (NE.head group) result <- siReadFile sys (Just True) filename let contents = either (const "") id result let comments' = makeNonVirtual comments contents
src/ShellCheck/Formatter/TTY.hs view
@@ -31,9 +31,9 @@ import Data.IORef import Data.List import Data.Maybe-import GHC.Exts import System.IO import System.Info+import qualified Data.List.NonEmpty as NE wikiLink = "https://www.shellcheck.net/wiki/" @@ -117,19 +117,19 @@ color <- getColorFunc $ foColorOption options let comments = crComments result appendComments ref comments (fromIntegral $ foWikiLinkCount options)- let fileGroups = groupWith sourceFile comments+ let fileGroups = NE.groupWith sourceFile comments mapM_ (outputForFile color sys) fileGroups outputForFile color sys comments = do- let fileName = sourceFile (head comments)+ let fileName = sourceFile (NE.head comments) result <- siReadFile sys (Just True) fileName let contents = either (const "") id result let fileLinesList = lines contents let lineCount = length fileLinesList let fileLines = listArray (1, lineCount) fileLinesList- let groups = groupWith lineNo comments+ let groups = NE.groupWith lineNo comments forM_ groups $ \commentsForLine -> do- let lineNum = fromIntegral $ lineNo (head commentsForLine)+ let lineNum = fromIntegral $ lineNo (NE.head commentsForLine) let line = if lineNum < 1 || lineNum > lineCount then "" else fileLines ! fromIntegral lineNum@@ -139,7 +139,7 @@ putStrLn (color "source" line) forM_ commentsForLine $ \c -> putStrLn $ color (severityText c) $ cuteIndent c putStrLn ""- showFixedString color commentsForLine (fromIntegral lineNum) fileLines+ showFixedString color (toList commentsForLine) (fromIntegral lineNum) fileLines -- Pick out only the lines necessary to show a fix in action sliceFile :: Fix -> Array Int String -> (Fix, Array Int String)@@ -169,7 +169,7 @@ -- and/or other unrelated lines. let (excerptFix, excerpt) = sliceFile mergedFix fileLines -- in the spirit of error prone- putStrLn $ color "message" "Did you mean: "+ putStrLn $ color "message" "Did you mean:" putStrLn $ unlines $ applyFix excerptFix excerpt cuteIndent :: PositionedComment -> String
src/ShellCheck/Interface.hs view
@@ -1,5 +1,5 @@ {-- Copyright 2012-2019 Vidar Holen+ Copyright 2012-2024 Vidar Holen This file is part of ShellCheck. https://www.shellcheck.net@@ -21,14 +21,14 @@ module ShellCheck.Interface ( SystemInterface(..)- , CheckSpec(csFilename, csScript, csCheckSourced, csIncludedWarnings, csExcludedWarnings, csShellTypeOverride, csMinSeverity, csIgnoreRC, csOptionalChecks)+ , CheckSpec(csFilename, csScript, csCheckSourced, csIncludedWarnings, csExcludedWarnings, csShellTypeOverride, csMinSeverity, csIgnoreRC, csExtendedAnalysis, csOptionalChecks) , CheckResult(crFilename, crComments) , ParseSpec(psFilename, psScript, psCheckSourced, psIgnoreRC, psShellTypeOverride) , ParseResult(prComments, prTokenPositions, prRoot)- , AnalysisSpec(asScript, asShellType, asFallbackShell, asExecutionMode, asCheckSourced, asTokenPositions, asOptionalChecks)+ , AnalysisSpec(asScript, asShellType, asFallbackShell, asExecutionMode, asCheckSourced, asTokenPositions, asExtendedAnalysis, asOptionalChecks) , AnalysisResult(arComments) , FormatterOptions(foColorOption, foWikiLinkCount)- , Shell(Ksh, Sh, Bash, Dash)+ , Shell(Ksh, Sh, Bash, Dash, BusyboxSh) , ExecutionMode(Executed, Sourced) , ErrorMessage , Code@@ -39,11 +39,12 @@ , ColorOption(ColorAuto, ColorAlways, ColorNever) , TokenComment(tcId, tcComment, tcFix) , emptyCheckResult- , newParseResult- , newAnalysisSpec , newAnalysisResult+ , newAnalysisSpec , newFormatterOptions+ , newParseResult , newPosition+ , newSystemInterface , newTokenComment , mockedSystemInterface , mockRcFile@@ -99,6 +100,7 @@ csIncludedWarnings :: Maybe [Integer], csShellTypeOverride :: Maybe Shell, csMinSeverity :: Severity,+ csExtendedAnalysis :: Maybe Bool, csOptionalChecks :: [String] } deriving (Show, Eq) @@ -123,6 +125,7 @@ csIncludedWarnings = Nothing, csShellTypeOverride = Nothing, csMinSeverity = StyleC,+ csExtendedAnalysis = Nothing, csOptionalChecks = [] } @@ -135,6 +138,14 @@ psShellTypeOverride = Nothing } +newSystemInterface :: Monad m => SystemInterface m+newSystemInterface =+ SystemInterface {+ siReadFile = \_ _ -> return $ Left "Not implemented",+ siFindSource = \_ _ _ name -> return name,+ siGetConfig = \_ -> return Nothing+ }+ -- Parser input and output data ParseSpec = ParseSpec { psFilename :: String,@@ -165,6 +176,7 @@ asExecutionMode :: ExecutionMode, asCheckSourced :: Bool, asOptionalChecks :: [String],+ asExtendedAnalysis :: Maybe Bool, asTokenPositions :: Map.Map Id (Position, Position) } @@ -175,6 +187,7 @@ asExecutionMode = Executed, asCheckSourced = False, asOptionalChecks = [],+ asExtendedAnalysis = Nothing, asTokenPositions = Map.empty } @@ -212,7 +225,7 @@ } -- Supporting data types-data Shell = Ksh | Sh | Bash | Dash deriving (Show, Eq)+data Shell = Ksh | Sh | Bash | Dash | BusyboxSh deriving (Show, Eq) data ExecutionMode = Executed | Sourced deriving (Show, Eq) type ErrorMessage = String@@ -311,7 +324,7 @@ -- For testing mockedSystemInterface :: [(String, String)] -> SystemInterface Identity-mockedSystemInterface files = SystemInterface {+mockedSystemInterface files = (newSystemInterface :: SystemInterface Identity) { siReadFile = rf, siFindSource = fs, siGetConfig = const $ return Nothing@@ -326,4 +339,3 @@ mockRcFile rcfile mock = mock { siGetConfig = const . return $ Just (".shellcheckrc", rcfile) }-
src/ShellCheck/Parser.hs view
@@ -46,7 +46,9 @@ import Text.Parsec.Pos import qualified Control.Monad.Reader as Mr import qualified Control.Monad.State as Ms+import qualified Data.List.NonEmpty as NE import qualified Data.Map.Strict as Map+import Debug.Trace import Test.QuickCheck.All (quickCheckAll) @@ -65,10 +67,14 @@ doubleQuote = char '"' variableStart = upper <|> lower <|> oneOf "_" variableChars = upper <|> lower <|> digit <|> oneOf "_"--- Chars to allow in function names-functionChars = variableChars <|> oneOf ":+?-./^@,"+-- Chars to allow function names to start with+functionStartChars = variableChars <|> oneOf ":+?-./^@,"+-- Chars to allow inside function names+functionChars = variableChars <|> oneOf "#:+?-./^@,"+-- Chars to allow function names to start with, using the 'function' keyword+extendedFunctionStartChars = functionStartChars <|> oneOf "[]*=!" -- Chars to allow in functions using the 'function' keyword-extendedFunctionChars = functionChars <|> oneOf "[]*=!"+extendedFunctionChars = extendedFunctionStartChars <|> oneOf "[]*=!" specialVariable = oneOf (concat specialVariables) paramSubSpecialChars = oneOf "/:+-=%" quotableChars = "|&;<>()\\ '\t\n\r\xA0" ++ doubleQuotableChars@@ -140,15 +146,9 @@ parseProblemAt pos ErrorC 1017 "Literal carriage return. Run script through tr -d '\\r' ." return '\r' -almostSpace =- choice [- check '\xA0' "unicode non-breaking space",- check '\x200B' "unicode zerowidth space"- ]- where- check c name = do- parseNote ErrorC 1018 $ "This is a " ++ name ++ ". Delete and retype it."- char c+almostSpace = do+ parseNote ErrorC 1018 $ "This is a unicode space. Delete and retype it."+ oneOf "\xA0\x2002\x2003\x2004\x2005\x2006\x2007\x2008\x2009\x200B\x202F" return ' ' --------- Message/position annotation on top of user state@@ -160,7 +160,7 @@ deriving (Show) data HereDocContext =- HereDocPending Token [Context] -- on linefeed, read this T_HereDoc+ HereDocPending Id Dashed Quoted String [Context] -- on linefeed, read this T_HereDoc deriving (Show) data UserState = UserState {@@ -238,12 +238,12 @@ hereDocMap = Map.insert id list map } -addPendingHereDoc t = do+addPendingHereDoc id d q str = do state <- getState context <- getCurrentContexts let docs = pendingHereDocs state putState $ state {- pendingHereDocs = HereDocPending t context : docs+ pendingHereDocs = HereDocPending id d q str context : docs } popPendingHereDocs = do@@ -826,7 +826,7 @@ char ')' id <- endSpan start spacing- return $ TA_Parentesis id s+ return $ TA_Parenthesis id s readArithTerm = readGroup <|> readVariable <|> readExpansion @@ -1057,6 +1057,16 @@ "This shell type is unknown. Use e.g. sh or bash." return [ShellOverride shell] + "extended-analysis" -> do+ pos <- getPosition+ value <- plainOrQuoted $ many1 letter+ case value of+ "true" -> return [ExtendedAnalysis True]+ "false" -> return [ExtendedAnalysis False]+ _ -> do+ parseNoteAt pos ErrorC 1146 "Unknown extended-analysis value. Expected true/false."+ return []+ "external-sources" -> do pos <- getPosition value <- plainOrQuoted $ many1 letter@@ -1194,7 +1204,7 @@ readDollarBracedLiteral = do start <- startSpan- vars <- (readBraceEscaped <|> (anyChar >>= \x -> return [x])) `reluctantlyTill1` bracedQuotable+ vars <- (readBraceEscaped <|> ((\x -> [x]) <$> anyChar)) `reluctantlyTill1` bracedQuotable id <- endSpan start return $ T_Literal id $ concat vars @@ -1556,7 +1566,7 @@ return $ concat strings readGenericLiteral1 endExp = do- strings <- (readGenericEscaped <|> (anyChar >>= \x -> return [x])) `reluctantlyTill1` endExp+ strings <- (readGenericEscaped <|> ((\x -> [x]) <$> anyChar)) `reluctantlyTill1` endExp return $ concat strings readGenericEscaped = do@@ -1690,16 +1700,17 @@ prop_readDollarBraceCommandExpansion1 = isOk readDollarBraceCommandExpansion "${ ls; }" prop_readDollarBraceCommandExpansion2 = isOk readDollarBraceCommandExpansion "${\nls\n}"-readDollarBraceCommandExpansion = called "ksh ${ ..; } command expansion" $ do+prop_readDollarBraceCommandExpansion3 = isOk readDollarBraceCommandExpansion "${| REPLY=42; }"+readDollarBraceCommandExpansion = called "ksh-style ${ ..; } command expansion" $ do start <- startSpan- try $ do- string "${"- whitespace+ c <- try $ do+ string "${"+ char '|' <|> whitespace allspacing term <- readTerm- char '}' <|> fail "Expected } to end the ksh ${ ..; } command expansion"+ char '}' <|> fail "Expected } to end the ksh-style ${ ..; } command expansion" id <- endSpan start- return $ T_DollarBraceCommandExpansion id term+ return $ T_DollarBraceCommandExpansion id (if c == '|' then Piped else Unpiped) term prop_readDollarBraced1 = isOk readDollarBraced "${foo//bar/baz}" prop_readDollarBraced2 = isOk readDollarBraced "${foo/'{cow}'}"@@ -1835,7 +1846,7 @@ -- add empty tokens for now, read the rest in readPendingHereDocs let doc = T_HereDoc hid dashed quoted endToken []- addPendingHereDoc doc+ addPendingHereDoc hid dashed quoted endToken return doc where unquote :: String -> (Quoted, String)@@ -1856,7 +1867,7 @@ docs <- popPendingHereDocs mapM_ readDoc docs where- readDoc (HereDocPending (T_HereDoc id dashed quoted endToken _) ctx) =+ readDoc (HereDocPending id dashed quoted endToken ctx) = swapContext ctx $ do docStartPos <- getPosition@@ -2200,17 +2211,18 @@ readSource :: Monad m => Token -> SCParser m Token-readSource t@(T_Redirecting _ _ (T_SimpleCommand cmdId _ (cmd:file':rest'))) = do- let file = getFile file' rest'+readSource t@(T_Redirecting _ _ (T_SimpleCommand cmdId _ (cmd:args'))) = do+ let file = getFile args' override <- getSourceOverride let literalFile = do- name <- override `mplus` getLiteralString file `mplus` stripDynamicPrefix file+ name <- override `mplus` (getLiteralString =<< file) `mplus` (stripDynamicPrefix =<< file) -- Hack to avoid 'source ~/foo' trying to read from literal tilde guard . not $ "~/" `isPrefixOf` name return name+ let fileId = fromMaybe (getId cmd) (getId <$> file) case literalFile of Nothing -> do- parseNoteAtId (getId file) WarningC 1090+ parseNoteAtId fileId WarningC 1090 "ShellCheck can't follow non-constant source. Use a directive to specify location." return t Just filename -> do@@ -2218,7 +2230,7 @@ if not proceed then do -- FIXME: This actually gets squashed without -a- parseNoteAtId (getId file) InfoC 1093+ parseNoteAtId fileId InfoC 1093 "This file appears to be recursively sourced. Ignoring." return t else do@@ -2236,7 +2248,7 @@ return (contents, resolved) case input of Left err -> do- parseNoteAtId (getId file) InfoC 1091 $+ parseNoteAtId fileId InfoC 1091 $ "Not following: " ++ err return t Right script -> do@@ -2248,18 +2260,19 @@ return $ T_SourceCommand id1 t (T_Include id2 src) let failed = do- parseNoteAtId (getId file) WarningC 1094+ parseNoteAtId fileId WarningC 1094 "Parsing of sourced file failed. Ignoring it." return t included <|> failed where- getFile :: Token -> [Token] -> Token- getFile file (next:rest) =- case getLiteralString file of- Just "--" -> next- x -> file- getFile file _ = file+ getFile :: [Token] -> Maybe Token+ getFile (first:rest) =+ case getLiteralString first of+ Just "--" -> rest !!! 0+ Just "-p" -> rest !!! 1+ _ -> return first+ getFile _ = Nothing getSourcePath t = case t of@@ -2283,23 +2296,32 @@ subRead name script = withContext (ContextSource name) $- inSeparateContext $- subParse (initialPos name) (readScriptFile True) script+ inSeparateContext $ do+ oldState <- getState+ setState $ oldState { pendingHereDocs = [] }+ result <- subParse (initialPos name) (readScriptFile True) script+ newState <- getState+ setState $ newState { pendingHereDocs = pendingHereDocs oldState }+ return result readSource t = return t prop_readPipeline = isOk readPipeline "! cat /etc/issue | grep -i ubuntu" prop_readPipeline2 = isWarning readPipeline "!cat /etc/issue | grep -i ubuntu" prop_readPipeline3 = isOk readPipeline "for f; do :; done|cat"+prop_readPipeline4 = isOk readPipeline "! ! true"+prop_readPipeline5 = isOk readPipeline "true | ! true" readPipeline = do unexpecting "keyword/token" readKeyword- do- (T_Bang id) <- g_Bang- pipe <- readPipeSequence- return $ T_Banged id pipe- <|>- readPipeSequence+ readBanged readPipeSequence +readBanged parser = do+ pos <- getPosition+ (T_Bang id) <- g_Bang+ next <- readBanged parser+ return $ T_Banged id next+ <|> parser+ prop_readAndOr = isOk readAndOr "grep -i lol foo || exit 1" prop_readAndOr1 = isOk readAndOr "# shellcheck disable=1\nfoo" prop_readAndOr2 = isOk readAndOr "# shellcheck disable=1\n# lol\n# shellcheck disable=3\nfoo"@@ -2354,14 +2376,14 @@ readPipeSequence = do start <- startSpan- (cmds, pipes) <- sepBy1WithSeparators readCommand+ (cmds, pipes) <- sepBy1WithSeparators (readBanged readCommand) (readPipe `thenSkip` (spacing >> readLineBreak)) id <- endSpan start spacing return $ T_Pipeline id pipes cmds where sepBy1WithSeparators p s = do- let elems = p >>= \x -> return ([x], [])+ let elems = (\x -> ([x], [])) <$> p let seps = do separator <- s return $ \(a,b) (c,d) -> (a++c, b ++ d ++ [separator])@@ -2384,6 +2406,10 @@ ] readCmdName = do+ -- If the command name is `!` then+ optional . lookAhead . try $ do+ char '!'+ whitespace -- Ignore alias suppression optional . try $ do char '\\'@@ -2733,6 +2759,8 @@ prop_readFunctionDefinition11 = isWarning readFunctionDefinition "function foo{\ntrue\n}" prop_readFunctionDefinition12 = isOk readFunctionDefinition "function []!() { true; }" prop_readFunctionDefinition13 = isOk readFunctionDefinition "@require(){ true; }"+prop_readFunctionDefinition14 = isOk readFunctionDefinition "foo#bar(){ :; }"+prop_readFunctionDefinition15 = isNotOk readFunctionDefinition "#bar(){ :; }" readFunctionDefinition = called "function" $ do start <- startSpan functionSignature <- try readFunctionSignature@@ -2750,7 +2778,7 @@ string "function" whitespace spacing- name <- many1 extendedFunctionChars+ name <- (:) <$> extendedFunctionStartChars <*> many extendedFunctionChars spaces <- spacing hasParens <- wasIncluded readParens when (not hasParens && null spaces) $@@ -2759,7 +2787,7 @@ return $ \id -> T_Function id (FunctionKeyword True) (FunctionParentheses hasParens) name readWithoutFunction = try $ do- name <- many1 functionChars+ name <- (:) <$> functionStartChars <*> many functionChars guard $ name /= "time" -- Interferes with time ( foo ) spacing readParens@@ -2777,17 +2805,29 @@ prop_readCoProc1 = isOk readCoProc "coproc foo { echo bar; }" prop_readCoProc2 = isOk readCoProc "coproc { echo bar; }" prop_readCoProc3 = isOk readCoProc "coproc echo bar"+prop_readCoProc4 = isOk readCoProc "coproc a=b echo bar"+prop_readCoProc5 = isOk readCoProc "coproc 'foo' { echo bar; }"+prop_readCoProc6 = isOk readCoProc "coproc \"foo$$\" { echo bar; }"+prop_readCoProc7 = isOk readCoProc "coproc 'foo' ( echo bar )"+prop_readCoProc8 = isOk readCoProc "coproc \"foo$$\" while true; do true; done" readCoProc = called "coproc" $ do start <- startSpan try $ do string "coproc"- whitespace+ spacing1 choice [ try $ readCompoundCoProc start, readSimpleCoProc start ] where readCompoundCoProc start = do- var <- optionMaybe $- readVariableName `thenSkip` whitespace- body <- readBody readCompoundCommand+ notFollowedBy2 readAssignmentWord+ (var, body) <- choice [+ try $ do+ body <- readBody readCompoundCommand+ return (Nothing, body),+ try $ do+ var <- readNormalWord `thenSkip` spacing+ body <- readBody readCompoundCommand+ return (Just var, body)+ ] id <- endSpan start return $ T_CoProc id var body readSimpleCoProc start = do@@ -2891,8 +2931,8 @@ kludgeAwayQuotes :: String -> SourcePos -> (String, SourcePos) kludgeAwayQuotes s p = case s of- first:rest@(_:_) ->- let (last:backwards) = reverse rest+ first:second:rest ->+ let (last NE.:| backwards) = NE.reverse (second NE.:| rest) middle = reverse backwards in if first `elem` "'\"" && first == last@@ -3322,10 +3362,12 @@ then do commands <- readCompoundListOrEmpty id <- endSpan start+ readPendingHereDocs verifyEof let script = T_Annotation annotationId annotations $ T_Script id shebang commands- reparseIndices script+ userstate <- getState+ reparseIndices $ reattachHereDocs script (hereDocMap userstate) else do many anyChar id <- endSpan start@@ -3335,8 +3377,8 @@ verifyShebang pos s = do case isValidShell s of Just True -> return ()- Just False -> parseProblemAt pos ErrorC 1071 "ShellCheck only supports sh/bash/dash/ksh scripts. Sorry!"- Nothing -> parseProblemAt pos ErrorC 1008 "This shebang was unrecognized. ShellCheck only supports sh/bash/dash/ksh. Add a 'shell' directive to specify."+ Just False -> parseProblemAt pos ErrorC 1071 "ShellCheck only supports sh/bash/dash/ksh/'busybox sh' scripts. Sorry!"+ Nothing -> parseProblemAt pos ErrorC 1008 "This shebang was unrecognized. ShellCheck only supports sh/bash/dash/ksh/'busybox sh'. Add a 'shell' directive to specify." isValidShell s = let good = null s || any (`isPrefixOf` s) goodShells@@ -3352,16 +3394,20 @@ "sh", "ash", "dash",+ "busybox sh", "bash", "bats",- "ksh"+ "ksh",+ "oksh" ] badShells = [ "awk", "csh", "expect",+ "fish", "perl", "python",+ "python3", "ruby", "tcsh", "zsh"@@ -3414,14 +3460,23 @@ isWarning p s = parsesCleanly p s == Just False -- The string parses with warnings isNotOk p s = parsesCleanly p s == Nothing -- The string does not parse -parsesCleanly parser string = runIdentity $ do- (res, sys) <- runParser testEnvironment- (parser >> eof >> getState) "-" string- case (res, sys) of- (Right userState, systemState) ->- return $ Just . null $ parseNotes userState ++ parseProblems systemState- (Left _, _) -> return Nothing+-- If the parser matches the string, return Right [ParseNotes+ParseProblems]+-- If it does not match the string, return Left [ParseProblems]+getParseOutput parser string = runIdentity $ do+ (res, systemState) <- runParser testEnvironment+ (parser >> eof >> getState) "-" string+ return $ case res of+ Right userState ->+ Right $ parseNotes userState ++ parseProblems systemState+ Left _ -> Left $ parseProblems systemState +-- If the parser matches the string, return Just whether it was clean (without emitting suggestions)+-- Otherwise, Nothing+parsesCleanly parser string =+ case getParseOutput parser string of+ Right list -> Just $ null list+ Left _ -> Nothing+ parseWithNotes parser = do item <- parser state <- getState@@ -3438,9 +3493,8 @@ pos = errorPos parsecError getStringFromParsec errors =- case map f errors of- r -> unwords (take 1 $ catMaybes $ reverse r) ++- " Fix any mentioned problems and try again."+ headOrDefault "" (mapMaybe f $ reverse errors) +++ " Fix any mentioned problems and try again." where f err = case err of@@ -3471,8 +3525,7 @@ return newParseResult { prComments = map toPositionedComment $ nub $ parseNotes userstate ++ parseProblems state, prTokenPositions = Map.map startEndPosToPos (positionMap userstate),- prRoot = Just $- reattachHereDocs script (hereDocMap userstate)+ prRoot = Just script } Left err -> do let context = contextStack state@@ -3490,13 +3543,11 @@ -- A final pass for ignoring parse errors after failed parsing isIgnored stack note = any (contextItemDisablesCode False (codeForParseNote note)) stack -notesForContext list = zipWith ($) [first, second] $ filter isName list+notesForContext list = zipWith ($) [first, second] [(pos, str) | ContextName pos str <- list] where- isName (ContextName _ _) = True- isName _ = False- first (ContextName pos str) = ParseNote pos pos ErrorC 1073 $+ first (pos, str) = ParseNote pos pos ErrorC 1073 $ "Couldn't parse this " ++ str ++ ". Fix to allow more checks."- second (ContextName pos str) = ParseNote pos pos InfoC 1009 $+ second (pos, str) = ParseNote pos pos InfoC 1009 $ "The mentioned syntax error was in this " ++ str ++ "." -- Go over all T_UnparsedIndex and reparse them as either arithmetic or text
test/shellcheck.hs view
@@ -18,21 +18,24 @@ main = do putStrLn "Running ShellCheck tests..."- results <- sequence [- ShellCheck.Analytics.runTests- ,ShellCheck.AnalyzerLib.runTests- ,ShellCheck.ASTLib.runTests- ,ShellCheck.CFG.runTests- ,ShellCheck.CFGAnalysis.runTests- ,ShellCheck.Checker.runTests- ,ShellCheck.Checks.Commands.runTests- ,ShellCheck.Checks.ControlFlow.runTests- ,ShellCheck.Checks.Custom.runTests- ,ShellCheck.Checks.ShellSupport.runTests- ,ShellCheck.Fixer.runTests- ,ShellCheck.Formatter.Diff.runTests- ,ShellCheck.Parser.runTests+ failures <- filter (not . snd) <$> mapM sequenceA tests+ if null failures then exitSuccess else do+ putStrLn "Tests failed for the following module(s):"+ mapM (putStrLn . ("- ShellCheck." ++) . fst) failures+ exitFailure+ where+ tests =+ [ ("Analytics" , ShellCheck.Analytics.runTests)+ , ("AnalyzerLib" , ShellCheck.AnalyzerLib.runTests)+ , ("ASTLib" , ShellCheck.ASTLib.runTests)+ , ("CFG" , ShellCheck.CFG.runTests)+ , ("CFGAnalysis" , ShellCheck.CFGAnalysis.runTests)+ , ("Checker" , ShellCheck.Checker.runTests)+ , ("Checks.Commands" , ShellCheck.Checks.Commands.runTests)+ , ("Checks.ControlFlow" , ShellCheck.Checks.ControlFlow.runTests)+ , ("Checks.Custom" , ShellCheck.Checks.Custom.runTests)+ , ("Checks.ShellSupport", ShellCheck.Checks.ShellSupport.runTests)+ , ("Fixer" , ShellCheck.Fixer.runTests)+ , ("Formatter.Diff" , ShellCheck.Formatter.Diff.runTests)+ , ("Parser" , ShellCheck.Parser.runTests) ]- if and results- then exitSuccess- else exitFailure