diff --git a/AUTHORS b/AUTHORS new file mode 100644 --- /dev/null +++ b/AUTHORS @@ -0,0 +1,1 @@ +Andrew Robbins diff --git a/README b/README new file mode 100644 --- /dev/null +++ b/README diff --git a/only.1 b/only.1 new file mode 100644 --- /dev/null +++ b/only.1 @@ -0,0 +1,204 @@ +.\" Process this file with +.\" groff -man -Tascii foo.1 +.\" +.TH ONLY 1 "" Haskell "" +.SH NAME +only \- an advanced filter for words, lines, and more +.SH SYNOPSIS +.B only [\-[bcwlf] +.I EXPR +.B ] ... +.I file +.B ... +.SH DESCRIPTION +.B Only +is an advanced filtering tool, like +.B grep, +but instead of filtering only on lines, +it can also filter on characters or words, called +.I tokens +in general. +When tokens +.I match, +there are two options that allow for greater control than +.B grep. +They can appear before and/or after a +.I regex +and are called +.I absolute indices +and +.I relative indices, +respectively. +.I Absolute indices +refer to matches, whereas +.I relative indices +refer to tokens, with the match being token zero. +For example, +.B -l +.I N/regex/M +will show the M-th line after the N-th occurance of +.I regex. +.P +For a more detailed description, see below. +.SH OPTIONS +.IP (-b|--bytes)=EXPR +Byte mode +.IP (-c|--chars)=EXPR +Character mode +.IP (-w|--words)=EXPR +Word mode +.IP (-l|--lines)=EXPR +Line mode +.IP (-f|--files)=EXPR +File mode +.SH EXTENDED DESCRIPTION +The original goal of +.B 'only' +was to combine the features of +.B head, +.B tail, +.B grep, +and +.B cut +into a single utility that was capable of all of their features, +but with the power to do so much more. For example, +.B head +and +.B tail +are good for selecting the first n-lines or last n-lines of a file, +but what if you want lines 10-30? Neither utility would be very +good alone, and combining them to accomplish your goal would be +a nightmare. Granted, one could probably construct a one liner in +.B awk +or +.B perl +to achieve the desired effect, but at the expense of clarity. +.P +To overview the features of +.B only, +there are two major kinds of inputs: +.I files +and +.I modes. +A file can +either be a filename or +.B '\-' +which means standard input. +The modes currently supported are: +.B bytes, +.B characters, +.B words, +.B lines, +and +.B files. +The difference between each mode is what the pattern /^.*$/ will match. +When no pattern is given, and a number is given instead, then +it will refer to the appropriate token type, for example, the first word, +the second line, etc. +.P +In byte mode, the input is broken up into 8-bit octets, +so the patterns must only match a single byte. In character mode, the input +is broken up according to the specified encoding (or UTF-8 if unspecified), +where each character may be multiple bytes. In word mode, the separators +can be any white-space, so it tries to remember what +separator was there in the beginning, +and puts it back before displaying. +In line mode, +.B only +behaves very similar to +.B grep +but with a few extra features. +In file mode, the filenames are not shown (unless -F is used) but the +entire file is shown if it matches the pattern. +.SH \ \ \ Syntax +.I Matching expressions +are expressions written in a small language +that forms a super-set of regular expressions. +The +.B syntax +of matching expressions +are the same regardless of what the current mode is. +This is true even of byte mode, where you must write "\\xFF" if you want a +non-printable character. Matching expressions can be as simple as a number +or a word. First, +.B only +tries to parse an expression as a number, then as an expression of the form +.I M/regex/N +and if that fails, then it treats the entire expression as a regex. Each M +and N may be a numeric expression. +.I Numeric expressions +have the syntax (in pseudo-Parsec): + + num = [+\-][0-9]+ + numeric = sepBy numbers ',' + numbers = num ';' num ':' num # from A to C step (A-B) + | num ':' num ';' num # from A to B step C + | num ':' num # from A to B + | num + +which means you can specify just a single number (3) or something as complicated +as multiple ranges (such as 3:5,100:109). These numeric expressions can occur on +either side of the regex, or both sides with a combined effect. The +syntax of the entire matching expression is: + + expr = do optional numeric + c <- punct ; regex ; c + (try c ; regex ; c ; optional num + | optional numeric) + | numeric + | regex + +where +.I punct +is any ASCII punctuation character except ".,:;", +and +.I regex +is a POSIX extended regular expression. +.\" This serves to discribe the syntax of matching expressions. +.SH \ \ \ Semantics +The +.B semantics +of matching expressions are a little harder to describe. However, a generalization +of the example given above should hold true: + + "N/regex/M" means the M-th +.I tokens +relative to the N-th +.I matches + +The default for +.I N +is +.B 1:-1 +and the default for +.I M +is +.B 0. The +.I N +are known as +.I absolute indices, +and the +.I M +are known as +.I relative indices. +Absolute indices will take the list of matches (the list of tokens that were matched by the regular expression), and apply use the numbers in M as the indices of this list. This gives you the ability to select the first match (1) or the last match (-1). If you use negative numbers, then it will count from the end of file going backwards, so (-2) would be the second to last match. Relative indices will take the list of matches, the original list of tokens, and for each match, it forms a virtual list where 0 refers to the match's index in the list of tokens. This allows one to emulate +.B grep's +\-A (after) and \-B (before) options. +.P +Here are some equivalent command-lines for "after" a match: + + grep -A3 expr file.txt + only -l/expr/0:3 file.txt + +Here are some equivalent command-lines for "before" a match: + + grep -B3 expr file.txt + only -l/expr/-3:0 file.txt + +Normally, these would be used to select +line numbers, like if you got a compiler error in a file with 10 million lines, +and you just wanted to see the surrounding text. +.SH FILES +.I ~/.onlyrc +.RS +A user configuration file. [Not implemented yet] diff --git a/only.cabal b/only.cabal --- a/only.cabal +++ b/only.cabal @@ -1,5 +1,5 @@ Cabal-version: >= 1.2.0.0 -version: 0.0.4.0 +version: 0.0.5.0 name: only license: GPL @@ -15,6 +15,24 @@ on word patterns or line patterns like never before! Not only can you search with 'only -l patt' but you can select the n-th match with '-l n\/patt\/' and the next 3 lines with '-l \/patt\/0:3'. +extra-source-files: + AUTHORS + LICENSE + README + only.1 + tests/e-both.txt + tests/i-all.sh + tests/Makefile + tests/README + tests/t-eq-abs.sh + tests/t-eq-nil.sh + tests/t-eq-regex-char.sh + tests/t-eq-regex-zero.sh + tests/t-eq-regex.sh + tests/t-grep-after.sh + tests/t-grep-before.sh + tests/t-grep-dot.sh + tests/t-grep-to.sh executable only main-is: only.hs diff --git a/only.hs b/only.hs --- a/only.hs +++ b/only.hs @@ -20,7 +20,7 @@ progName = "only" -- usage stuff -version = "only (GNU Only) 0.0.4.0\n" +version = "only (GNU Only) 0.0.5.0\n" header = unlines [ version, @@ -36,8 +36,9 @@ "\tNUMBER Token (word/line) selection", "\t/REGEX/NUMBER Token (word/line) relative to match (0-based, +/-)", "\tNUMBER/REGEX/ Select which matches to print (1-based, +)", - "\tNUMBER/REGEX/NUMBER Both", "", - " where [NUMBER] is one of:", + "\tNUMBER/REGEX/NUMBER Both", + "\t/REGEX/:/REGEX/ Between two expressions", + "", " where [NUMBER] is one of:", "\tAT One", "\tFROM:TO Range", "\tFROM:TO;STEP Range with step", @@ -77,7 +78,8 @@ -- data stuff type Match = (Int, String) -data OnlyExpr = OnlyExpr { +data OnlyExpr = OnlyRange [Int] String String Int + | OnlyExpr { oeAbs :: [Int], oePat :: String, oeRel :: [Int]} deriving (Read, Show, Eq) @@ -102,64 +104,79 @@ print ctx -- parser stuff -parseDigits :: CharParser () [Int] -parseDigits = do +parseNum :: CharParser () Int +parseNum = do + -- this is where to implement '--negative' behaviour + sign <- option "" (char '-' >> return "-") + num <- many1 digit + return (read $ sign ++ num) + +parseNums :: CharParser () [Int] +parseNums = do lists <- sepBy ((try nexts) <|> (try steps) <|> (try range) <|> number) (char ',' >> return []) return (concat lists) - where digits :: CharParser () Int - digits = do - -- this is where to implement '--negative' behaviour - sign <- option "" (char '-' >> return "-") - num <- many1 digit - return (read $ sign ++ num) - number :: CharParser () [Int] + where number :: CharParser () [Int] number = do - num <- digits + num <- parseNum return [num] range :: CharParser () [Int] range = do - start <- digits + start <- parseNum char ':' - end <- digits + end <- parseNum return [start .. end] steps :: CharParser () [Int] steps = do - start <- digits + start <- parseNum char ':' - end <- digits + end <- parseNum char ';' - step <- digits + step <- parseNum return [start, (start + step) .. end] nexts :: CharParser () [Int] nexts = do - start <- digits + start <- parseNum char ';' - next <- digits + next <- parseNum char ':' - end <- digits + end <- parseNum return [start, next .. end] parseRegex :: CharParser () OnlyExpr -parseRegex = (try regex) <|> (try number) <|> word +parseRegex = (try pattRange) <|> (try pattIndex) <|> (try number) <|> word where word = do str <- many anyChar return (OnlyExpr [] str []) number = do - abs <- parseDigits + abs <- parseNums if abs == [] then word else return (OnlyExpr abs "" []) - regex = do - abs <- parseDigits + regexChar = do ch <- noneOf $ ".,:;" ++ ['A'..'Z'] ++ ['a'..'z'] + return ch + regex = do + ch <- regexChar pat <- many $ noneOf [ch] char ch - rel <- parseDigits + return pat + pattIndex = do + abs <- parseNums + pat <- regex + rel <- parseNums return (OnlyExpr abs pat rel) + pattRange = do + abs <- parseNums + pat1 <- regex + char ':' + pat2 <- regex + rel <- parseNum -- only one number allowed! + return (OnlyRange abs pat1 pat2 rel) + readRegex :: FilePath -> String -> IO OnlyExpr readRegex path expr = if expr == "" @@ -257,25 +274,52 @@ doSep :: (String -> [String]) -> String -> OnlyCtx -> OnlyMode -> IO OnlyCtx doSep sep expr ctx@(OnlyCtx orig _) mode = do + -- initial state + let full = map (\n -> (n, getIndex seps n "")) [1..length seps] + seps = sep orig + + -- normalize expression path <- fileGet - oe@(OnlyExpr abs pat rel) <- readRegex path expr + oexpr <- readRegex path expr + parts <- case oexpr of + oe@(OnlyExpr abs pat rel) -> return [doParts full oe] + oe@(OnlyRange _ _ _ _) -> do + let oes = doRange full oe + return (map (doParts full) oes) + -- final processing + return (ctx {ocParts = concat parts}) - let abs2ls :: [Match] -> Int -> Match - abs2ls ms n = getIndex ms n (-1, "") - rel2ls :: [Match] -> [Match] -> Match -> Int -> Match - rel2ls fs as (na, _) nr = tupleLookup n fs where n = na + nr - tupleLookup n xs = (n, (fromMaybe "")$lookup n xs) +doAbs :: [a] -> [Int] -> [Int] +doAbs some abs = if abs == [] then [1..length some] else abs - let seps = sep orig - some = doMatch oe full - full = map (\n -> (n, getIndex seps n "")) [1..length seps] - abss = [abs2ls some a | - a <- (if abs == [] then [1..length some] else abs)] - rels = [rel2ls full abss a r | - r <- (if rel == [] then [0] else rel), a <- abss] - parts = map (initCtx . snd) rels +doRel :: [Int] -> [Int] +doRel rel = if rel == [] then [0] else rel - return (ctx {ocParts = parts}) +doParts :: [Match] -> OnlyExpr -> [OnlyCtx] +doParts full oe@(OnlyExpr abs pat rel) = parts where + abs2ls :: [Match] -> Int -> Match + abs2ls ms n = getIndex ms n (-1, "") + rel2ls :: [Match] -> [Match] -> Match -> Int -> Match + rel2ls fs as (na, _) nr = tupleLookup n fs where n = na + nr + tupleLookup n xs = (n, (fromMaybe "")$lookup n xs) + abss = [abs2ls some a | a <- doAbs some abs] + rels = [rel2ls full abss a r | r <- doRel rel, a <- abss] + parts = map (initCtx . snd) rels + some = doMatch oe full + +doRange :: [Match] -> OnlyExpr -> [OnlyExpr] +doRange full (OnlyRange abs pat1 pat2 rel) = oes where + oes = map (m2exp ls) (doAbs ls abs) + ls = filter (/=[]) $ map (toRange match2) match1 + match1 = doMatch (OnlyExpr [] pat1 []) full + match2 = doMatch (OnlyExpr [] pat2 []) full + m2exp ls k = OnlyExpr [k] pat1 (getIndex ls k []) + nextAfter :: Int -> [Match] -> [Match] + nextAfter i ms = dropWhile (\(j,_) -> i >= j) ms + toRange :: [Match] -> Match -> [Int] + toRange ms m@(i,_) = case nextAfter i ms of + (j,_):_ -> [0 .. j - i + rel] + [] -> [] doMatch :: OnlyExpr -> [Match] -> [Match] doMatch oe = if pat == "" then id diff --git a/tests/Makefile b/tests/Makefile new file mode 100644 --- /dev/null +++ b/tests/Makefile @@ -0,0 +1,8 @@ +all: test + +test: + @echo + @for TEST in $$(ls t-*) ; do bash $$TEST ; done + +clean: + @if [ x"$$(echo *~)" != x'*~' ] ; then rm -f *~ ; fi diff --git a/tests/README b/tests/README new file mode 100644 --- /dev/null +++ b/tests/README @@ -0,0 +1,6 @@ +The files in this directory (t/) +have the following naming convention: + + e-*.txt = example files + i-*.sh = include files + t-*.sh = test files diff --git a/tests/e-both.txt b/tests/e-both.txt new file mode 100644 --- /dev/null +++ b/tests/e-both.txt @@ -0,0 +1,6 @@ +1 Alfred - This is going to be the longest comment section of them all... +2 Barbra - She has nothing to say. +3 Cheese - Mice love it! +4 Dorsey - The only one who seems to know anything. +5 Edward - Not just a syllable anymore! +6 Finley - Someone who you might not want to meet. diff --git a/tests/i-all.sh b/tests/i-all.sh new file mode 100644 --- /dev/null +++ b/tests/i-all.sh @@ -0,0 +1,15 @@ +NAME="$0" +BASE="${NAME%.sh}" +ONLY="../dist/build/only/only" +GREP="grep" + +cd "$(dirname $NAME)" + +function test-eq () { + if [ x"$1" = x"$2" ] ; then + echo -n PASS + else + echo -n FAIL + fi + echo " : $BASE" +} diff --git a/tests/t-eq-abs.sh b/tests/t-eq-abs.sh new file mode 100644 --- /dev/null +++ b/tests/t-eq-abs.sh @@ -0,0 +1,6 @@ +#!/bin/sh +. i-all.sh + +test-eq \ + "$($ONLY e-both.txt -l3)" \ + "$($ONLY e-both.txt -l3//)" diff --git a/tests/t-eq-nil.sh b/tests/t-eq-nil.sh new file mode 100644 --- /dev/null +++ b/tests/t-eq-nil.sh @@ -0,0 +1,6 @@ +#!/bin/sh +. i-all.sh + +test-eq \ + "$($ONLY e-both.txt -l '')" \ + "$($ONLY e-both.txt -l //)" diff --git a/tests/t-eq-regex-char.sh b/tests/t-eq-regex-char.sh new file mode 100644 --- /dev/null +++ b/tests/t-eq-regex-char.sh @@ -0,0 +1,6 @@ +#!/bin/sh +. i-all.sh + +test-eq \ + "$($ONLY e-both.txt -l @to@)" \ + "$($ONLY e-both.txt -l /to/)" diff --git a/tests/t-eq-regex-zero.sh b/tests/t-eq-regex-zero.sh new file mode 100644 --- /dev/null +++ b/tests/t-eq-regex-zero.sh @@ -0,0 +1,6 @@ +#!/bin/sh +. i-all.sh + +test-eq \ + "$($ONLY e-both.txt -l3//)" \ + "$($ONLY e-both.txt -l3//0)" diff --git a/tests/t-eq-regex.sh b/tests/t-eq-regex.sh new file mode 100644 --- /dev/null +++ b/tests/t-eq-regex.sh @@ -0,0 +1,6 @@ +#!/bin/sh +. i-all.sh + +test-eq \ + "$($ONLY e-both.txt -lto)" \ + "$($ONLY e-both.txt -l/to/)" diff --git a/tests/t-grep-after.sh b/tests/t-grep-after.sh new file mode 100644 --- /dev/null +++ b/tests/t-grep-after.sh @@ -0,0 +1,6 @@ +#!/bin/sh +. i-all.sh + +test-eq \ + "$($GREP -A2 She e-both.txt)" \ + "$($ONLY -l/She/0:2 e-both.txt)" diff --git a/tests/t-grep-before.sh b/tests/t-grep-before.sh new file mode 100644 --- /dev/null +++ b/tests/t-grep-before.sh @@ -0,0 +1,6 @@ +#!/bin/sh +. i-all.sh + +test-eq \ + "$($GREP -B1 She e-both.txt)" \ + "$($ONLY -l/She/-1:0 e-both.txt)" diff --git a/tests/t-grep-dot.sh b/tests/t-grep-dot.sh new file mode 100644 --- /dev/null +++ b/tests/t-grep-dot.sh @@ -0,0 +1,6 @@ +#!/bin/sh +. i-all.sh + +test-eq \ + "$($GREP '\.' e-both.txt)" \ + "$($ONLY -l'\.' e-both.txt)" diff --git a/tests/t-grep-to.sh b/tests/t-grep-to.sh new file mode 100644 --- /dev/null +++ b/tests/t-grep-to.sh @@ -0,0 +1,6 @@ +#!/bin/sh +. i-all.sh + +test-eq \ + "$($GREP to e-both.txt)" \ + "$($ONLY -lto e-both.txt)"