diff --git a/README b/README --- a/README +++ b/README @@ -5,124 +5,125 @@ arx tmpx * (//+ (//+ *)?)? DESCRIPTION - The arx tool automates a common task in the world of operations automa‐ - tion: packing code, sending it to a remote machine, unpacking in a tem‐ - porary directory, running a task therein and then removing the tempo‐ - rary directory. One might do this when setting up a moderately compli‐ - cated back-up script, installing a new version of nginx or even just to - run jobs across ones infrastructure. + A UNIX executable is a simple thing -- a file the kernel can execute, + one way or another, via an interpreter or directly as object code. + Every executable induces a family of executions -- instances of execu- + tion with different command line arguments, with different files in the + working directory and with different environment variables present. - The arx tool has no in-built notion of remote connections or server - clusters; all automation is captured as Bourne compatible scripts that - use a small number of UNIX utilities in a broadly portable way. At - present, the utilities used are sed, tr, date, head, and tar. The calls - to tar sometimes use -j and -z; these calls to tar may result in calls - to bzip2 and gzip. Scripts have been tested with dash and the GNU tools - as well as the sh and tools that are part of busybox. + The arx tool captures the parameters of an execution and encodes them + as an executable, making for easy, consistent transfer and repetition + of a particular run. The generated executable ensures that each run + occurs in a freshly allocated temporary directory, with only the + desired files in scope; it uses traps to ensure the cleanup of this + directory; and its format is a simple POSIX shell script, relying on + just a few shell tools. - The tmpx subcommand of arx offers a variety of options for bundling - code and a task to run. The shdat subcommand exposes the lower-level - functionality of encoding binary data in a shell script that outputs - that binary data, using HERE documents and some odd replacement rules +DEPENDENCIES + The arx tool relies on the presence of sed, tr, date, head, tar, hex- + dump and sh. When unpacking tar archives, it may use the -j or -z + (bzip2 and gzip, respectively) options of tar. Scripts have been tested + with dash and the GNU tools as well as the sh implementation and user- + land tools that are part of busybox. + +APPLICATION + The tmpx subcommand of arx offers a variety of options for bundling + code and a task to run. The shdat subcommand exposes the lower-level + functionality of encoding binary data in a shell script that outputs + that binary data, using HERE documents and some odd replacement rules for nulls. - Scripts generated by tmpx and shdat may be fed to sh over STDIN to exe‐ - cute them. This can be helpful when using ssh and sudo to set up an + Scripts generated by tmpx and shdat may be fed to sh over STDIN to exe- + cute them. This can be helpful when using ssh and sudo to set up an execution context; for example: arx tmpx ... | ssh user@host.com sudo sh - For all subcommands, when options overlap in their effect -- for exam‐ - ple, setting the output with -o -- the rightmost option takes prece‐ - dence. Whenever -h, -? or --help is present on the command line, help + For all subcommands, when options overlap in their effect -- for exam- + ple, setting the output with -o -- the rightmost option takes prece- + dence. Whenever -h, -? or --help is present on the command line, help is displayed and the program exits. - When paths are specified on an arx command line, they must be quali‐ - fied, starting with /, ./ or ../. This simplifies the command line syn‐ + When paths are specified on an arx command line, they must be quali- + fied, starting with /, ./ or ../. This simplifies the command line syn- tax, overall, without introducing troublesome ambiguities. TMPX The tmpx subcommand bundles together archives, environment settings and - an executable or shell command in to a Bourne-compatible script that - runs the command or executable in a temporary directory, after having + an executable or shell command in to a Bourne-compatible script that + runs the command or executable in a temporary directory, after having unpacked the archives and set the environment. - Any number of file path arguments may be specified; they will be inter‐ - preted as tar archives to include in bundled script. If - is given, - then STDIN will be included as an archive stream. If no arguments are - given, it is assumed that no archives are desired and only the command + Any number of file path arguments may be specified; they will be inter- + preted as tar archives to include in bundled script. If - is given, + then STDIN will be included as an archive stream. If no arguments are + given, it is assumed that no archives are desired and only the command and environment are bundled. - The temporary directory created by the script is different for each - invocation, with a name of the form /tmp/tmpx... The - timestamp used is a UTC, ISO 8601 format timestamp. One happy conse‐ - quence of this is that earlier jobs sort ASCIIbetically before later - jobs. After execution, the temporary directory is removed (or not, + The temporary directory created by the script is different for each + invocation, with a name of the form /tmp/tmpx... The + timestamp used is a UTC, ISO 8601 format timestamp. One happy conse- + quence of this is that earlier jobs sort ASCIIbetically before later + jobs. After execution, the temporary directory is removed (or not, depending on the -rm[10!_] family of options). -rm0, -rm1, -rm_, -rm! - - By default, the temporary directory created by the script + By default, the temporary directory created by the script will be deleted no matter the exit status status of the task. These options cause a script to be generated that deletes the - temporary directory only on success, only on failure, always + temporary directory only on success, only on failure, always (the default) or never. -b - - Please see the documentation for this option, shared with + Please see the documentation for this option, shared with shdat, below. -o - - By default, the generated script is sent to STDOUT. With -o, + By default, the generated script is sent to STDOUT. With -o, output is redirected to the given path. -e - - Causes the file specified to be packaged as the task to be - run. A binary executable, a Ruby script or a longish shell + Causes the file specified to be packaged as the task to be + run. A binary executable, a Ruby script or a longish shell script all fit here. - In addition to these options, arguments of the form VAR=VALUE are rec‐ - ognized as environment mappings and stored away in the script, to be + In addition to these options, arguments of the form VAR=VALUE are rec- + ognized as environment mappings and stored away in the script, to be sourced on execution. - Without -e, the tmpx subcommand tries to find the task to be run as a - sequence of arguments delimited by a run of slashes. The following + Without -e, the tmpx subcommand tries to find the task to be run as a + sequence of arguments delimited by a run of slashes. The following forms are all recognized: arx tmpx ...some args... // ...command... arx tmpx ...some args... // ...command... // ...more args... arx tmpx // ...command... // ...some args... - The slash runs must have the same number of slashes and must be the - longest continuous runs of slashes on the command line. The command + The slash runs must have the same number of slashes and must be the + longest continuous runs of slashes on the command line. The command will be included as is in a Bourne shell script. SHDAT - The shdat subcommand translates binary data in to a shell script which - outputs the binary data. The data is encoded in HERE documents in such - a way that data without NULs is not changed and that data with NULs is - minimally expanded: about 1% for randomish data like compressed tar‐ + The shdat subcommand translates binary data in to a shell script which + outputs the binary data. The data is encoded in HERE documents in such + a way that data without NULs is not changed and that data with NULs is + minimally expanded: about 1% for randomish data like compressed tar- balls and about 10% in pathological cases. - The shdat subcommand can be given any number of paths, which will be + The shdat subcommand can be given any number of paths, which will be concatenated in the order given. If no path is given, or if - is given, then STDIN will be read. -b - - The size of data chunks to place in each HERE document. The - argument is a positive integer followed by suffixes like B, - K, KiB, M and MiB, in the manner of dd, head and many other - tools. The default is 4MiB. This is unlikely to make a dif‐ + The size of data chunks to place in each HERE document. The + argument is a positive integer followed by suffixes like B, + K, KiB, M and MiB, in the manner of dd, head and many other + tools. The default is 4MiB. This is unlikely to make a dif- ference for you unless the generated script is intended to be run on a memory-constrained system. -o - - By default, the generated script is sent to STDOUT. With -o, + By default, the generated script is sent to STDOUT. With -o, output is redirected to the given path. EXAMPLES @@ -145,11 +146,11 @@ -rm0 ./dist/arx-0.0.0.tar.gz | sh BUGS - The command line parser offers no hints or help of any kind; it fails - with the simple message "argument error". The two most common mistakes + The command line parser offers no hints or help of any kind; it fails + with the simple message "argument error". The two most common mistakes I make are: - · Not qualifying paths with /, ./ or ../. + o Not qualifying paths with /, ./ or ../. - · Not specifying a subcommand (tmpx or shdat). + o Not specifying a subcommand (tmpx or shdat). diff --git a/System/Posix/ARX/Programs.hs b/System/Posix/ARX/Programs.hs --- a/System/Posix/ARX/Programs.hs +++ b/System/Posix/ARX/Programs.hs @@ -56,7 +56,6 @@ or not to delete successful (exit code zero) runs; the second determines whether or not to delete failed (exit code non-zero) runs. -} - data TMPX = TMPX SHDAT LazyB.ByteString -- Code of task to run. [(Sh.Var, Sh.Val)] -- Environment mapping. Bool -- Destroy tmp if task runs successfully. diff --git a/arx.cabal b/arx.cabal --- a/arx.cabal +++ b/arx.cabal @@ -1,5 +1,5 @@ name : arx -version : 0.1.0 +version : 0.1.1 category : Text license : BSD3 license-file : LICENSE diff --git a/docs/blessed/arx.man b/docs/blessed/arx.man --- a/docs/blessed/arx.man +++ b/docs/blessed/arx.man @@ -1,4 +1,4 @@ -.TH "ARX" "1" "2011-12-20" "0.1.0" "arx" +.TH "ARX" "1" "2012-02-24" "0.1.0" "arx" .SH NAME arx \- archived execution . @@ -42,21 +42,27 @@ .fi .SH DESCRIPTION .sp -The \fIarx\fP tool automates a common task in the world of operations automation: -packing code, sending it to a remote machine, unpacking in a temporary -directory, running a task therein and then removing the temporary directory. -One might do this when setting up a moderately complicated back\-up script, -installing a new version of nginx or even just to run jobs across ones -infrastructure. +A UNIX executable is a simple thing \-\- a file the kernel can execute, one way +or another, via an interpreter or directly as object code. Every executable +induces a family of executions \-\- instances of execution with different +command line arguments, with different files in the working directory and with +different environment variables present. .sp -The \fIarx\fP tool has no in\-built notion of remote connections or server -clusters; all automation is captured as Bourne compatible scripts that use a -small number of UNIX utilities in a broadly portable way. At present, the -utilities used are \fIsed\fP, \fItr\fP, \fIdate\fP, \fIhead\fP, and \fItar\fP. The calls to \fItar\fP -sometimes use \fI\-j\fP and \fI\-z\fP; these calls to \fItar\fP may result in calls to -\fIbzip2\fP and \fIgzip\fP. Scripts have been tested with \fIdash\fP and the GNU tools as -well as the \fIsh\fP and tools that are part of \fIbusybox\fP. +The \fIarx\fP tool captures the parameters of an execution and encodes them as an +executable, making for easy, consistent transfer and repetition of a +particular run. The generated executable ensures that each run occurs in a +freshly allocated temporary directory, with only the desired files in scope; +it uses traps to ensure the cleanup of this directory; and its format is a +simple POSIX shell script, relying on just a few shell tools. +.SH DEPENDENCIES .sp +The \fIarx\fP tool relies on the presence of \fIsed\fP, \fItr\fP, \fIdate\fP, \fIhead\fP, \fItar\fP, +\fIhexdump\fP and \fIsh\fP. When unpacking tar archives, it may use the \fI\-j\fP or \fI\-z\fP +(\fIbzip2\fP and \fIgzip\fP, respectively) options of \fItar\fP. Scripts have been tested +with \fIdash\fP and the GNU tools as well as the \fIsh\fP implementation and userland +tools that are part of \fIbusybox\fP. +.SH APPLICATION +.sp The \fItmpx\fP subcommand of \fIarx\fP offers a variety of options for bundling code and a task to run. The \fIshdat\fP subcommand exposes the lower\-level functionality of encoding binary data in a shell script that outputs that @@ -104,23 +110,19 @@ .INDENT 0.0 .TP .B \fB\-rm0\fP, \fB\-rm1\fP, \fB\-rm_\fP, \fB\-rm!\fP -.sp By default, the temporary directory created by the script will be deleted no matter the exit status status of the task. These options cause a script to be generated that deletes the temporary directory only on success, only on failure, always (the default) or never. .TP .B \fB\-b \fP -.sp Please see the documentation for this option, shared with \fIshdat\fP, below. .TP .B \fB\-o \fP -.sp By default, the generated script is sent to STDOUT. With \fB\-o\fP, output is redirected to the given path. .TP .B \fB\-e \fP -.sp Causes the file specified to be packaged as the task to be run. A binary executable, a Ruby script or a longish shell script all fit here. .UNINDENT @@ -162,7 +164,6 @@ .INDENT 0.0 .TP .B \fB\-b \fP -.sp The size of data chunks to place in each HERE document. The argument is a positive integer followed by suffixes like \fBB\fP, \fBK\fP, \fBKiB\fP, \fBM\fP and \fBMiB\fP, in the manner of \fBdd\fP, \fBhead\fP and many other tools. The @@ -170,7 +171,6 @@ generated script is intended to be run on a memory\-constrained system. .TP .B \fB\-o \fP -.sp By default, the generated script is sent to STDOUT. With \fB\-o\fP, output is redirected to the given path. .UNINDENT @@ -205,10 +205,8 @@ simple message "argument error". The two most common mistakes I make are: .INDENT 0.0 .IP \(bu 2 -. Not qualifying paths with \fB/\fP, \fB./\fP or \fB../\fP. .IP \(bu 2 -. Not specifying a subcommand (\fItmpx\fP or \fIshdat\fP). .UNINDENT .SH AUTHOR diff --git a/docs/blessed/arx.txt b/docs/blessed/arx.txt --- a/docs/blessed/arx.txt +++ b/docs/blessed/arx.txt @@ -5,124 +5,125 @@ arx tmpx * (//+ (//+ *)?)? DESCRIPTION - The arx tool automates a common task in the world of operations automa‐ - tion: packing code, sending it to a remote machine, unpacking in a tem‐ - porary directory, running a task therein and then removing the tempo‐ - rary directory. One might do this when setting up a moderately compli‐ - cated back-up script, installing a new version of nginx or even just to - run jobs across ones infrastructure. + A UNIX executable is a simple thing -- a file the kernel can execute, + one way or another, via an interpreter or directly as object code. + Every executable induces a family of executions -- instances of execu- + tion with different command line arguments, with different files in the + working directory and with different environment variables present. - The arx tool has no in-built notion of remote connections or server - clusters; all automation is captured as Bourne compatible scripts that - use a small number of UNIX utilities in a broadly portable way. At - present, the utilities used are sed, tr, date, head, and tar. The calls - to tar sometimes use -j and -z; these calls to tar may result in calls - to bzip2 and gzip. Scripts have been tested with dash and the GNU tools - as well as the sh and tools that are part of busybox. + The arx tool captures the parameters of an execution and encodes them + as an executable, making for easy, consistent transfer and repetition + of a particular run. The generated executable ensures that each run + occurs in a freshly allocated temporary directory, with only the + desired files in scope; it uses traps to ensure the cleanup of this + directory; and its format is a simple POSIX shell script, relying on + just a few shell tools. - The tmpx subcommand of arx offers a variety of options for bundling - code and a task to run. The shdat subcommand exposes the lower-level - functionality of encoding binary data in a shell script that outputs - that binary data, using HERE documents and some odd replacement rules +DEPENDENCIES + The arx tool relies on the presence of sed, tr, date, head, tar, hex- + dump and sh. When unpacking tar archives, it may use the -j or -z + (bzip2 and gzip, respectively) options of tar. Scripts have been tested + with dash and the GNU tools as well as the sh implementation and user- + land tools that are part of busybox. + +APPLICATION + The tmpx subcommand of arx offers a variety of options for bundling + code and a task to run. The shdat subcommand exposes the lower-level + functionality of encoding binary data in a shell script that outputs + that binary data, using HERE documents and some odd replacement rules for nulls. - Scripts generated by tmpx and shdat may be fed to sh over STDIN to exe‐ - cute them. This can be helpful when using ssh and sudo to set up an + Scripts generated by tmpx and shdat may be fed to sh over STDIN to exe- + cute them. This can be helpful when using ssh and sudo to set up an execution context; for example: arx tmpx ... | ssh user@host.com sudo sh - For all subcommands, when options overlap in their effect -- for exam‐ - ple, setting the output with -o -- the rightmost option takes prece‐ - dence. Whenever -h, -? or --help is present on the command line, help + For all subcommands, when options overlap in their effect -- for exam- + ple, setting the output with -o -- the rightmost option takes prece- + dence. Whenever -h, -? or --help is present on the command line, help is displayed and the program exits. - When paths are specified on an arx command line, they must be quali‐ - fied, starting with /, ./ or ../. This simplifies the command line syn‐ + When paths are specified on an arx command line, they must be quali- + fied, starting with /, ./ or ../. This simplifies the command line syn- tax, overall, without introducing troublesome ambiguities. TMPX The tmpx subcommand bundles together archives, environment settings and - an executable or shell command in to a Bourne-compatible script that - runs the command or executable in a temporary directory, after having + an executable or shell command in to a Bourne-compatible script that + runs the command or executable in a temporary directory, after having unpacked the archives and set the environment. - Any number of file path arguments may be specified; they will be inter‐ - preted as tar archives to include in bundled script. If - is given, - then STDIN will be included as an archive stream. If no arguments are - given, it is assumed that no archives are desired and only the command + Any number of file path arguments may be specified; they will be inter- + preted as tar archives to include in bundled script. If - is given, + then STDIN will be included as an archive stream. If no arguments are + given, it is assumed that no archives are desired and only the command and environment are bundled. - The temporary directory created by the script is different for each - invocation, with a name of the form /tmp/tmpx... The - timestamp used is a UTC, ISO 8601 format timestamp. One happy conse‐ - quence of this is that earlier jobs sort ASCIIbetically before later - jobs. After execution, the temporary directory is removed (or not, + The temporary directory created by the script is different for each + invocation, with a name of the form /tmp/tmpx... The + timestamp used is a UTC, ISO 8601 format timestamp. One happy conse- + quence of this is that earlier jobs sort ASCIIbetically before later + jobs. After execution, the temporary directory is removed (or not, depending on the -rm[10!_] family of options). -rm0, -rm1, -rm_, -rm! - - By default, the temporary directory created by the script + By default, the temporary directory created by the script will be deleted no matter the exit status status of the task. These options cause a script to be generated that deletes the - temporary directory only on success, only on failure, always + temporary directory only on success, only on failure, always (the default) or never. -b - - Please see the documentation for this option, shared with + Please see the documentation for this option, shared with shdat, below. -o - - By default, the generated script is sent to STDOUT. With -o, + By default, the generated script is sent to STDOUT. With -o, output is redirected to the given path. -e - - Causes the file specified to be packaged as the task to be - run. A binary executable, a Ruby script or a longish shell + Causes the file specified to be packaged as the task to be + run. A binary executable, a Ruby script or a longish shell script all fit here. - In addition to these options, arguments of the form VAR=VALUE are rec‐ - ognized as environment mappings and stored away in the script, to be + In addition to these options, arguments of the form VAR=VALUE are rec- + ognized as environment mappings and stored away in the script, to be sourced on execution. - Without -e, the tmpx subcommand tries to find the task to be run as a - sequence of arguments delimited by a run of slashes. The following + Without -e, the tmpx subcommand tries to find the task to be run as a + sequence of arguments delimited by a run of slashes. The following forms are all recognized: arx tmpx ...some args... // ...command... arx tmpx ...some args... // ...command... // ...more args... arx tmpx // ...command... // ...some args... - The slash runs must have the same number of slashes and must be the - longest continuous runs of slashes on the command line. The command + The slash runs must have the same number of slashes and must be the + longest continuous runs of slashes on the command line. The command will be included as is in a Bourne shell script. SHDAT - The shdat subcommand translates binary data in to a shell script which - outputs the binary data. The data is encoded in HERE documents in such - a way that data without NULs is not changed and that data with NULs is - minimally expanded: about 1% for randomish data like compressed tar‐ + The shdat subcommand translates binary data in to a shell script which + outputs the binary data. The data is encoded in HERE documents in such + a way that data without NULs is not changed and that data with NULs is + minimally expanded: about 1% for randomish data like compressed tar- balls and about 10% in pathological cases. - The shdat subcommand can be given any number of paths, which will be + The shdat subcommand can be given any number of paths, which will be concatenated in the order given. If no path is given, or if - is given, then STDIN will be read. -b - - The size of data chunks to place in each HERE document. The - argument is a positive integer followed by suffixes like B, - K, KiB, M and MiB, in the manner of dd, head and many other - tools. The default is 4MiB. This is unlikely to make a dif‐ + The size of data chunks to place in each HERE document. The + argument is a positive integer followed by suffixes like B, + K, KiB, M and MiB, in the manner of dd, head and many other + tools. The default is 4MiB. This is unlikely to make a dif- ference for you unless the generated script is intended to be run on a memory-constrained system. -o - - By default, the generated script is sent to STDOUT. With -o, + By default, the generated script is sent to STDOUT. With -o, output is redirected to the given path. EXAMPLES @@ -145,11 +146,11 @@ -rm0 ./dist/arx-0.0.0.tar.gz | sh BUGS - The command line parser offers no hints or help of any kind; it fails - with the simple message "argument error". The two most common mistakes + The command line parser offers no hints or help of any kind; it fails + with the simple message "argument error". The two most common mistakes I make are: - · Not qualifying paths with /, ./ or ../. + o Not qualifying paths with /, ./ or ../. - · Not specifying a subcommand (tmpx or shdat). + o Not specifying a subcommand (tmpx or shdat). diff --git a/model-scripts/tmpx.sh b/model-scripts/tmpx.sh --- a/model-scripts/tmpx.sh +++ b/model-scripts/tmpx.sh @@ -13,19 +13,18 @@ done if $tmp then - dir=/tmp/tmpx.`date -u +%FT%TZ`.$$ - rm -rf $dir + dir=/tmp/tmpx-`hexdump -n8 -e '"%08x-%08x"' < /dev/urandom` : ${rm_:=true} if $rm_ then - trap "case \$?/$rm0/$rm1 in - 0/true/*) rm -rf $dir ;; - [1-9]*/*/true) rm -rf $dir ;; - esac" EXIT - trap "exit 2" HUP INT QUIT BUS SEGV PIPE TERM + trap 'case $?/$rm0/$rm1 in + 0/true/*) rm -rf "$dir" ;; + [1-9]*/*/true) rm -rf "$dir" ;; + esac' EXIT + trap 'exit 2' HUP INT QUIT BUS SEGV PIPE TERM fi - mkdir $dir - cd $dir + mkdir "$dir" + cd "$dir" fi go () { unpack_env > ./env