diff --git a/docs/hacking.rst b/docs/hacking.rst
--- a/docs/hacking.rst
+++ b/docs/hacking.rst
@@ -36,10 +36,16 @@
     Guide.
 
   * You may wish to set the environment variable
-    ``FUTHARK_COMPILER_DEBUGGING=1``.  Currently this only has the
-    effect of making the frontend print internal names, but it may
-    control more things in the future.
+    ``FUTHARK_COMPILER_DEBUGGING=1``.  This has the following
+    effects:
 
+    * The frontend prints internal names.  (This may affect code
+      generation in some cases, so turn it off when actually
+      generating code.)
+
+    * Tools that talk to server-mode executables will print the
+      messages sent back and worth on the standard error stream.
+
 .. _`stack`: https://docs.haskellstack.org/en/stable/README/
 .. _`Profiling`: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/profiling.html
 
@@ -93,6 +99,43 @@
 allows you to tailor your own compilation pipeline using command line
 options.  It is also useful for seeing what the AST looks like after
 specific passes.
+
+Running compiler pipelines
+--------------------------
+
+You can run the various compiler passes in whatever order you wish.
+There are also various shorthands for running entire standard
+pipelines:
+
+* ``--gpu``: pipeline used for GPU backends (stopping just
+  before adding memory information).
+
+* ``--gpu-mem``: pipeline used for GPU backends, with memory
+  information.  This will show the IR that is passed to ImpGen.
+
+* ``--seq``: pipeline used for sequential backends (stopping just
+  before adding memory information).
+
+* ``--seq-mem``: pipeline used for sequential backends, with memory
+  information.  This will show the IR that is passed to ImpGen.
+
+* ``--mc``: pipeline used for multicore backends (stopping just
+  before adding memory information).
+
+* ``--mc-mem``: pipeline used for multicore backends, with memory
+  information.  This will show the IR that is passed to ImpGen.
+
+By default, ``futhark dev`` will print the resulting IR.  You can
+switch to a different *action* with one of the following options:
+
+* ``--compile-imperative``: generate sequential ImpCode and print it.
+
+* ``--compile-imperative-kernels``: generate GPU ImpCode and print it.
+
+* ``--compile-imperative-multicore``: generate multicore ImpCode and print it.
+
+You must use the appropriate pipeline as well (e.g. ``--gpu-mem`` for
+``--compile-imperative-kernels``).
 
 When you are about to have a bad day
 ------------------------------------
diff --git a/docs/language-reference.rst b/docs/language-reference.rst
--- a/docs/language-reference.rst
+++ b/docs/language-reference.rst
@@ -110,8 +110,8 @@
 Futhark type system is entirely structural, and type abbreviations are
 merely shorthands (with one exception, see
 :ref:`sizes-in-abbreviations`).  The only exception is abstract types
-whose definition has been hidden via the module system (see `Module
-System`_).
+whose definition has been hidden via the module system (see
+:ref:`module-system`).
 
 .. productionlist::
    tuple_type: "(" ")" | "(" `type` ("," `type`)+ ")"
@@ -216,9 +216,9 @@
 Declarations
 ------------
 
-A Futhark file or module consists of a sequence of declarations.  Each
-declaration is processed in order, and a declaration can only refer to
-names bound by preceding declarations.
+A Futhark module consists of a sequence of declarations.  Files are
+also modules.  Each declaration is processed in order, and a
+declaration can only refer to names bound by preceding declarations.
 
 .. productionlist::
    dec:   `val_bind` | `type_bind` | `mod_bind` | `mod_type_bind`
@@ -227,20 +227,26 @@
       : | "local" `dec`
       : | "#[" attr "]" dec
 
-The ``open`` declaration brings names defined in another module into
-scope (see also `Module System`_).  For the meaning of ``import``, see
-`Referring to Other Files`_.  If a declaration is prefixed with
-``local``, whatever names it defines will *not* be visible outside the
-current module.  In particular ``local open`` is used to bring names
-from another module into scope, without making those names available
-to users of the module being defined.  In most cases, using module
-type ascription is a better idea.
+Any names defined by a declaration inside a module are by default
+visible to users of that module (see :ref:`module-system`).
 
+* ``open mod_exp`` brings names bound in ``mod_exp`` into the current scope.
+  These names will also be visible to users of the module.
+
+* ``local dec`` has the meaning of ``dec``, but any names bound by
+  ``dec`` will not be visible outside the module.
+
+* ``import "foo"`` is a shorthand for ``local open import "foo"``,
+  where the ``import`` is interpreted as a module expression (see
+  :ref:`module-system`).
+
+* ``#[attr] dec`` adds an attribute to a declaration (see :ref:`attributes`).
+
 Declaring Functions and Values
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. productionlist::
-   val_bind:   ("def" | "entry" | "let") (`id` | "(" `binop` ")") `type_param`* `pat`+ [":" `type`] "=" `exp`
+   val_bind:   ("def" | "entry" | "let") (`id` | "(" `binop` ")") `type_param`* `pat`* [":" `type`] "=" `exp`
            : | ("def" | "entry" | "let") `pat` `binop` `pat` [":" `type`] "=" `exp`
 
 **Note:** using ``let`` to define top-level bindings is deprecated.
@@ -364,7 +370,7 @@
 ~~~~~~~~~~~~~~~~~~
 
 .. productionlist::
-   type_bind: "type" ["^" | "~"] `id` `type_param`* "=" `type`
+   type_bind: ("type" | "type^" | "type~") `id` `type_param`* "=" `type`
    type_param: "[" `id` "]" | "'" `id` | "'~" `id` | "'^" `id`
 
 Type abbreviations function as shorthands for the purpose of
@@ -1452,8 +1458,8 @@
 
 .. _module-system:
 
-Module System
--------------
+Modules
+-------
 
 .. productionlist::
    mod_bind: "module" `id` `mod_param`* "=" [":" mod_type_exp] "=" `mod_exp`
@@ -1469,101 +1475,32 @@
 respectively.  Module names exist in the same name space as values,
 but module types are their own name space.
 
-Named modules are declared as::
-
-  module name = ...
-
-A named module type is defined as::
-
-  module type name = ...
-
-Where a module expression can be the name of another module, an
-application of a parametric module, or a sequence of declarations
-enclosed in curly braces::
-
-  module Vec3 = {
-    type t = ( f32 , f32 , f32 )
-    def add(a: t) (b: t): t =
-      let (a1, a2, a3) = a in
-      let (b1, b2, b3) = b in
-      (a1 + b1, a2 + b2 , a3 + b3)
-  }
-
-  module AlsoVec3 = Vec3
-
-Functions and types within modules can be accessed using dot
-notation::
-
-    type vector = Vec3.t
-    def double(v: vector): vector = Vec3.add v v
-
-We can also use ``open Vec3`` to bring the names defined by ``Vec3``
-into the current scope.  Multiple modules can be opened simultaneously
-by separating their names with spaces.  In case several modules define
-the same names, the ones mentioned last take precedence.  The first
-argument to ``open`` may be a full module expression.
-
-Named module types are defined as::
-
-  module type ModuleTypeName = ...
-
-A module type expression can be the name of another module type, or a
-sequence of *specifications*, or *specs*, enclosed in curly braces.  A
-spec can be a *value spec*, indicating the presence of a function or
-value, an *abstract type spec*, or a *type abbreviation spec*.  For
-example::
-
-  module type Addable = {
-    type t                 -- abstract type spec
-    type two_ts = (t,t)    -- type abbreviation spec
-    val add: t -> t -> t   -- value spec
-  }
-
-This module type specifies the presence of an *abstract type* ``t``,
-as well as a function operating on values of type ``t``.  We can use
-*module type ascription* to restrict a module to what is exposed by
-some module type::
-
-  module AbstractVec = Vec3 : Addable
-
-The definition of ``AbstractVec.t`` is now hidden.  In fact, with this
-module type, we can neither construct values of type ``AbstractVec.T``
-or convert them to anything else, making this a rather useless use of
-abstraction.  As a derived form, we can write ``module M: S = e`` to
-mean ``module M = e : S``.
-
-In a value spec, sizes in types on the left-hand side of a function
-arrow must not be anonymous.  For example, this is forbidden::
-
-  val sum: []t -> t
-
-Instead write::
+Module bindings
+~~~~~~~~~~~~~~~
 
-  val sum [n]: [n]t -> t
+``module m = mod_exp``
+......................
 
-But this is allowed, because the empty size is not to the left of a
-function arrow::
+Binds *m* to the module produced by the module expression ``mod_exp``.
+Any name x in the module produced by ``mod_exp`` can then be accessed
+with ``m.x``.
 
-  val evens [n]: [n]i32 -> []i32
+``module m : mod_type_exp = mod_exp``
+.....................................
 
-Parametric modules allow us to write definitions that abstract over
-modules.  For example::
+Shorthand for ``module m = mod_exp : mod_type_exp``.
 
-  module Times = \(M: Addable) -> {
-    def times (x: M.t) (k: i32): M.t =
-      loop x' = x for i < k do
-        M.add x' x
-  }
+``module m mod_params... = mod_exp``
+....................................
 
-We can instantiate ``Times`` with any module that fulfils the module
-type ``Addable`` and get back a module that defines a function
-``times``::
+Shorthand for ``module m = \mod_params... -> mod_exp``.  This produces
+a parametric module.
 
-  module Vec3Times = Times Vec3
+``module type mt = mod_type_exp``
+.................................
 
-Now ``Vec3Times.times`` is a function of type ``Vec3.t -> int ->
-Vec3.t``.  As a derived form, we can write ``module M p = e`` to mean
-``module M = \p -> e``.
+Binds *mt* to the module type produced by the module type expression
+``mod_type_exp``.
 
 Module Expressions
 ~~~~~~~~~~~~~~~~~~
@@ -1624,7 +1561,7 @@
 ................
 
 Returns a module that contains the definitions of the file ``"foo"``
-relative to the current file.  See :ref:`other-files`.
+relative to the current file.
 
 Module Type Expressions
 ~~~~~~~~~~~~~~~~~~~~~~~
@@ -1639,14 +1576,13 @@
 
 
 .. productionlist::
-   spec:   "val" `id` `type_param`* ":" `spec_type`
-       : | "val" `binop` `type_param`* ":" `spec_type`
-       : | "type" ["^"] `id` `type_param`* "=" `type`
-       : | "type" ["^"] `id` `type_param`*
+   spec:   "val" `id` `type_param`* ":" `type`
+       : | "val" `binop` `type_param`* ":" `type`
+       : | ("type" | "type^" | "type~") `id` `type_param`* "=" `type`
+       : | ("type" | "type^" | "type~") `id` `type_param`*
        : | "module" `id` ":" `mod_type_exp`
        : | "include" `mod_type_exp`
        : | "#[" attr "]" spec
-   spec_type: `type` | `type` "->" `spec_type`
 
 Module types classify modules, with the only (unimportant) difference
 in expressivity being that modules can contain module types, but
@@ -1654,10 +1590,29 @@
 module type. They can specify of course that a module contains a
 *submodule* of a specific module type.
 
+A module type expression can be the name of another module type, or a
+sequence of *specifications*, or *specs*, enclosed in curly braces.  A
+spec can be a *value spec*, indicating the presence of a function or
+value, an *abstract type spec*, or a *type abbreviation spec*.
+
+In a value spec, sizes in types on the left-hand side of a function
+arrow must not be anonymous.  For example, this is forbidden::
+
+  val sum: []t -> t
+
+Instead write::
+
+  val sum [n]: [n]t -> t
+
+But this is allowed, because the empty size is not to the left of a
+function arrow::
+
+  val evens [n]: [n]i32 -> []i32
+
 .. _other-files:
 
-Referring to Other Files
-------------------------
+Referencing Other Files
+-----------------------
 
 You can refer to external files in a Futhark file like this::
 
@@ -1682,6 +1637,10 @@
 In fact, a plain ``import "file"`` is equivalent to::
 
   local open import "file"
+
+To re-export names from another file in the current module, use::
+
+  open import "file"
 
 .. _attributes:
 
diff --git a/docs/man/futhark.rst b/docs/man/futhark.rst
--- a/docs/man/futhark.rst
+++ b/docs/man/futhark.rst
@@ -30,6 +30,11 @@
 Check whether a Futhark program type checks.  With ``-w``, no warnings
 are printed.
 
+futhark check-syntax PROGRAM
+----------------------------
+
+Check whether a Futhark program is syntactically correct.
+
 futhark datacmp FILE_A FILE_B
 -----------------------------
 
diff --git a/futhark.cabal b/futhark.cabal
--- a/futhark.cabal
+++ b/futhark.cabal
@@ -1,6 +1,6 @@
 cabal-version: 2.4
 name:           futhark
-version:        0.21.6
+version:        0.21.7
 synopsis:       An optimising compiler for a functional, array-oriented language.
 
 description:    Futhark is a small programming language designed to be compiled to
@@ -18,6 +18,7 @@
                 Regarding the internal design of the compiler, the following modules make
                 good starting points:
                 .
+                * "Futhark" contains a basic architectural overview of the compiler.
                 * "Futhark.IR.Syntax" explains the
                   basic design of the intermediate representation (IR).
                 * "Futhark.Construct" explains how to write code that
@@ -52,6 +53,7 @@
 
 library
   exposed-modules:
+      Futhark
       Futhark.Actions
       Futhark.Analysis.Alias
       Futhark.Analysis.CallGraph
@@ -282,6 +284,7 @@
       Language.Futhark.Core
       Language.Futhark.Interpreter
       Language.Futhark.Parser
+      Language.Futhark.Parser.Monad
       Language.Futhark.Prelude
       Language.Futhark.Pretty
       Language.Futhark.Prop
@@ -308,7 +311,7 @@
       Paths_futhark
   hs-source-dirs:
       src
-  ghc-options: -Wall -Wcompat -Wno-incomplete-uni-patterns -Wredundant-constraints -Wincomplete-record-updates -Wmissing-export-lists
+  ghc-options: -Wall -Wcompat -Wno-incomplete-uni-patterns -Wredundant-constraints -Wincomplete-record-updates -Wmissing-export-lists -Wunused-packages
   build-tool-depends:
       alex:alex
     , happy:happy
@@ -356,17 +359,16 @@
     , time >=1.6.0.1
     , transformers >=0.3
     , vector >=0.12
-    , vector-binary-instances >=0.2.2.0
     , versions >=5.0.0
     , zip-archive >=0.3.1.1
     , zlib >=0.6.1.2
   default-language: Haskell2010
 
 executable futhark
-  main-is: src/futhark.hs
+  main-is: src/main.hs
   other-modules:
       Paths_futhark
-  ghc-options: -Wall -Wcompat -Wno-incomplete-uni-patterns -Wredundant-constraints -Wincomplete-record-updates -Wmissing-export-lists -threaded -rtsopts "-with-rtsopts=-N -qg1 -A16M"
+  ghc-options: -Wall -Wcompat -Wno-incomplete-uni-patterns -Wredundant-constraints -Wincomplete-record-updates -Wmissing-export-lists -Wunused-packages -threaded -rtsopts "-with-rtsopts=-N -qg1 -A16M"
   build-depends:
       base
     , futhark
diff --git a/prelude/math.fut b/prelude/math.fut
--- a/prelude/math.fut
+++ b/prelude/math.fut
@@ -240,9 +240,9 @@
   def u32 (x: u32) = intrinsics.itob_i32_bool (intrinsics.sign_i32 x)
   def u64 (x: u64) = intrinsics.itob_i64_bool (intrinsics.sign_i64 x)
 
-  def f16 (x: f16) = x != 0f16
-  def f32 (x: f32) = x != 0f32
-  def f64 (x: f64) = x != 0f64
+  def f16 (x: f16) = intrinsics.ftob_f16_bool x
+  def f32 (x: f32) = intrinsics.ftob_f32_bool x
+  def f64 (x: f64) = intrinsics.ftob_f64_bool x
 
   def bool (x: bool) = x
 }
@@ -877,7 +877,7 @@
   def f32 (x: f32) = intrinsics.fpconv_f32_f64 x
   def f64 (x: f64) = intrinsics.fpconv_f64_f64 x
 
-  def bool (x: bool) = if x then 1f64 else 0f64
+  def bool (x: bool) = intrinsics.btof_bool_f64 x
 
   def from_fraction (x: i64) (y: i64) = i64 x / i64 y
   def to_i64 (x: f64) = intrinsics.fptosi_f64_i64 x
@@ -986,7 +986,7 @@
   def f32 (x: f32) = intrinsics.fpconv_f32_f32 x
   def f64 (x: f64) = intrinsics.fpconv_f64_f32 x
 
-  def bool (x: bool) = if x then 1f32 else 0f32
+  def bool (x: bool) = intrinsics.btof_bool_f32 x
 
   def from_fraction (x: i64) (y: i64) = i64 x / i64 y
   def to_i64 (x: f32) = intrinsics.fptosi_f32_i64 x
@@ -1099,7 +1099,7 @@
   def f32 (x: f32) = intrinsics.fpconv_f32_f16 x
   def f64 (x: f64) = intrinsics.fpconv_f64_f16 x
 
-  def bool (x: bool) = if x then 1f16 else 0f16
+  def bool (x: bool) = intrinsics.btof_bool_f16 x
 
   def from_fraction (x: i64) (y: i64) = i64 x / i64 y
   def to_i64 (x: f16) = intrinsics.fptosi_f16_i64 x
diff --git a/prelude/prelude.fut b/prelude/prelude.fut
--- a/prelude/prelude.fut
+++ b/prelude/prelude.fut
@@ -17,7 +17,7 @@
 def t64 (x: f64): i32 = i32.f64 x
 
 -- | Negate a boolean.  `not x` is the same as `!x`.  This function is
--- mostly useful for passing to `map
+-- mostly useful for passing to `map`.
 def not (x: bool): bool = !x
 
 -- | Semantically just identity, but serves as an optimisation
diff --git a/rts/c/scalar.h b/rts/c/scalar.h
--- a/rts/c/scalar.h
+++ b/rts/c/scalar.h
@@ -1155,6 +1155,14 @@
   }
 }
 
+static inline bool ftob_f32_bool(float x) {
+  return x != 0;
+}
+
+static inline float btof_bool_f32(bool x) {
+  return x ? 1 : 0;
+}
+
 #ifdef __OPENCL_VERSION__
 static inline float futrts_log32(float x) {
   return log(x);
@@ -1651,6 +1659,14 @@
   } else {
     return (uint64_t) (int64_t) x;
   }
+}
+
+static inline bool ftob_f64_bool(double x) {
+  return x != 0;
+}
+
+static inline double btof_bool_f64(bool x) {
+  return x ? 1 : 0;
 }
 
 static inline int64_t futrts_to_bits64(double x) {
diff --git a/rts/c/scalar_f16.h b/rts/c/scalar_f16.h
--- a/rts/c/scalar_f16.h
+++ b/rts/c/scalar_f16.h
@@ -120,6 +120,14 @@
   return (uint64_t) x;
 }
 
+static inline bool ftob_f16_bool(f16 x) {
+  return x != (f16)0;
+}
+
+static inline f16 btof_bool_f16(bool x) {
+  return x ? 1 : 0;
+}
+
 #ifndef EMULATE_F16
 
 #ifdef __OPENCL_VERSION__
diff --git a/rts/python/scalar.py b/rts/python/scalar.py
--- a/rts/python/scalar.py
+++ b/rts/python/scalar.py
@@ -158,14 +158,26 @@
   return np.int8(x)
 
 def btoi_bool_i16(x):
-  return np.int8(x)
+  return np.int16(x)
 
 def btoi_bool_i32(x):
-  return np.int8(x)
+  return np.int32(x)
 
 def btoi_bool_i64(x):
-  return np.int8(x)
+  return np.int64(x)
 
+def ftob_T_bool(x):
+  return bool(x)
+
+def btof_bool_f16(x):
+  return np.float16(x)
+
+def btof_bool_f32(x):
+  return np.float32(x)
+
+def btof_bool_f64(x):
+  return np.float64(x)
+
 def zext_i8_i8(x):
   return np.int8(np.uint8(x))
 
@@ -250,6 +262,7 @@
 sext_i8_i32 = sext_i16_i32 = sext_i32_i32 = sext_i64_i32 = sext_T_i32
 sext_i8_i64 = sext_i16_i64 = sext_i32_i64 = sext_i64_i64 = sext_T_i64
 itob_i8_bool = itob_i16_bool = itob_i32_bool = itob_i64_bool = itob_T_bool
+ftob_f16_bool = ftob_f32_bool = ftob_f64_bool = ftob_T_bool
 
 def clz_T(x):
   n = np.int32(0)
diff --git a/src/Futhark.hs b/src/Futhark.hs
new file mode 100644
--- /dev/null
+++ b/src/Futhark.hs
@@ -0,0 +1,145 @@
+-- |
+--
+-- This module contains no code.  Its purpose is to explain the
+-- overall architecture of the code base, with links to the relevant
+-- modules and definitions.  It is intended for people who intend to
+-- hack on the compiler itself, not for those who just want to use its
+-- public API as a library.  Most of what's here discusses the
+-- compiler itself, but we'll also provide pointers for where to find
+-- the code for the various other tools and subcommands.
+--
+-- Much of the documentation here is by referencing other modules that
+-- contain more information.  Make sure to follow the links.  Also,
+-- make sure to click the *source* links to study the actual code.
+-- Many of the links are provided not because the exposed module API
+-- is particularly interesting, but because their implementation is.
+--
+-- = Compiler
+--
+-- At a very high level, the Futhark compiler has a fairly
+-- conventional architecture.  After reading and type-checking a
+-- program, it is gradually transformed, passing through a variety of
+-- different /intermediate representations/ (IRs), where the specifics
+-- depend on which code generator is intended.  The final result is
+-- then written to one or more files.  We can divide the compiler into
+-- roughly three parts: frontend, middle-end [sic], and backend.
+--
+-- == Frontend
+--
+-- The frontend of the compiler is concerned with parsing the Futhark
+-- source language, type-checking it, and transforming it to the core
+-- IR used by the middle-end of the compiler.  The following modules
+-- are of particular interest:
+--
+-- * "Language.Futhark.Syntax" contains the source language AST definition.
+--
+-- * "Language.Futhark.Parser" contains parsers for the source
+--   language and various fragments.
+--
+-- * "Futhark.Compiler" and "Futhark.Compiler.Program" contains
+--   functions for conveniently reading and type-checking an entire
+--   Futhark source program, chasing down imports, and so on.
+--
+-- * "Language.Futhark.TypeChecker" is the type checker itself.
+--
+-- * /Internalisation/ of the source language to the core IR is in
+--   "Futhark.Internalise".
+--
+-- == Middle-end
+--
+-- The compiler middle-end is based around /passes/ that are
+-- essentially pure functions that accept a program as input and
+-- produce a program as output.  A composition of such passes is
+-- called a /pipeline/.  The various compiler backends (@futhark
+-- opencl@, @futhark multicore@, etc) use different pipelines.  See
+-- "Futhark.Pass" and (less importantly) "Futhark.Pipeline".  The
+-- actual pipelines we use are in "Futhark.Passes".
+--
+-- The compiler front-end produces a core IR program that is then
+-- processed by a pipeline in the middle-end.  The fundamental
+-- structure of this IR is defined in "Futhark.IR.Syntax".  As
+-- mentioned in that module, the IR supports multiple
+-- /representations/.  The middle-end always starts with the program
+-- in the SOACS representation ("Futhark.IR.SOACS"), which closely
+-- resembles the kind of free-form nested parallelism that the source
+-- language permits.  Depending on the specific compilation pipeline,
+-- the program will eventually be transformed to various other
+-- representations.  The output of the middle-end is always a core IR
+-- program in some representation (probably not the same one that it
+-- started with).
+--
+-- Many passes will involve constructing core IR AST fragments.  See
+-- "Futhark.Construct" for advice and guidance on how to do that.
+--
+-- === Main representations
+--
+-- * "Futhark.IR.SOACS": the initial core representation of any
+--   Futhark program.  This is the representation on which we perform
+--   optimisations such as fusion, inlining, and a host of other
+--   cleanup.
+--
+-- * "Futhark.IR.GPU": a representation where parallelism is expressed
+--   with flat /segmented operations/, and a few other GPU-specific
+--   operations are also supported.  The pass
+--   "Futhark.Pass.ExtractKernels" transforms a
+--   'Futhark.IR.SOACS.SOACS' program to a 'Futhark.IR.GPU.GPU'
+--   program.
+--
+-- * "Futhark.IR.MC": a representation where parallelism is expressed
+--   with flat /segmented operations/, and a few other multicore-specific
+--   operations are also supported.  The pass
+--   "Futhark.Pass.ExtractMulticore" transforms a
+--   'Futhark.IR.SOACS.SOACS' program to a 'Futhark.IR.MC.MC'
+--   program.
+--
+-- * "Futhark.IR.GPUMem": like 'Futhark.IR.GPU.GPU', but with memory
+--   information.  See "Futhark.IR.Mem" for information on how we
+--   represent memory in the Futhark compiler.  This representation is
+--   produced by "Futhark.Pass.ExplicitAllocations.GPU"
+--
+-- * "Futhark.IR.MCMem": the multicore counterpart to
+--   'Futhark.IR.GPUMem.GPUMem'.  Produced by
+--   "Futhark.Pass.ExplicitAllocations.MC".
+--
+-- == Backend
+--
+-- The backend accepts a program in some core IR representation.
+-- Currently this must always be a representation with memory
+-- information (e.g. 'Futhark.IR.GPUMem.GPUMem').  It then translates
+-- the program to the /imperative/ IR, which we call ImpCode.
+--
+-- * "Futhark.CodeGen.ImpCode": The main definition of ImpCode, which
+--   is an extensible representation like the core IR.
+--
+-- * "Futhark.CodeGen.ImpCode.GPU": an example of ImpCode
+--   extended/specialised to handle GPU operations (but at a high
+--   level, before generating actual CUDA/OpenCL kernel code).
+--
+-- * "Futhark.CodeGen.ImpGen": a translator from core IR to ImpCode.
+--   Heavily parameterised so that it can handle the various dialects
+--   of both core IR and ImpCode that it must be expected to work
+--   with.  In practice, when adding a new backend (or modifying an
+--   existing one), you'll be working on more specialised modules such
+--   as "Futhark.CodeGen.ImpGen.GPU".
+--
+-- Ultimately, ImpCode must then be translated to some /real/
+-- executable language, which in our case means C or Python.
+--
+-- * "Futhark.CodeGen.Backends.GenericC": framework for translating
+--   ImpCode to C.  Rather convoluted because it is used in a large
+--   number of settings.  The module
+--   "Futhark.CodeGen.Backends.SequentialC" shows perhaps the simplest
+--   use of it.
+--
+-- * "Futhark.CodeGen.Backends.GenericPython": the generic Python code
+--   generator (see "Futhark.CodeGen.Backends.SequentialPython" for a
+--   usage example).
+--
+-- = Command line interface
+--
+-- The @futhark@ program dispatches to a Haskell-level main function
+-- based on its first argument (the subcommand).  Some of these
+-- subcommands are implemented as their own modules under the
+-- @Futhark.CLI@ hierarchy.  Others, particularly the simplest ones,
+-- are bundled together in "Futhark.CLI.Misc".
+module Futhark () where
diff --git a/src/Futhark/Analysis/CallGraph.hs b/src/Futhark/Analysis/CallGraph.hs
--- a/src/Futhark/Analysis/CallGraph.hs
+++ b/src/Futhark/Analysis/CallGraph.hs
@@ -118,10 +118,10 @@
 buildFGStms :: Stms SOACS -> FunCalls
 buildFGStms = mconcat . map buildFGstm . stmsToList
 
-buildFGBody :: Body -> FunCalls
+buildFGBody :: Body SOACS -> FunCalls
 buildFGBody = buildFGStms . bodyStms
 
-buildFGstm :: Stm -> FunCalls
+buildFGstm :: Stm SOACS -> FunCalls
 buildFGstm (Let (Pat (p : _)) aux (Apply fname _ _ _)) =
   FunCalls (M.singleton (patElemName p) (stmAuxAttrs aux, fname)) (S.singleton fname)
 buildFGstm (Let _ _ (Op op)) = execWriter $ mapSOACM folder op
@@ -148,7 +148,7 @@
   foldMap noinlineDef (progFuns prog)
     <> foldMap onStm (foldMap (bodyStms . funDefBody) (progFuns prog) <> progConsts prog)
   where
-    onStm :: Stm -> S.Set Name
+    onStm :: Stm SOACS -> S.Set Name
     onStm (Let _ aux (Apply fname _ _ _))
       | "noinline" `inAttrs` stmAuxAttrs aux =
         S.singleton fname
diff --git a/src/Futhark/Analysis/LastUse.hs b/src/Futhark/Analysis/LastUse.hs
--- a/src/Futhark/Analysis/LastUse.hs
+++ b/src/Futhark/Analysis/LastUse.hs
@@ -5,7 +5,7 @@
 {-# LANGUAGE TypeFamilies #-}
 
 -- | Provides last-use analysis for Futhark programs.
-module Futhark.Analysis.LastUse (LastUseMap, analyseGPUMem, analyseSeqMem) where
+module Futhark.Analysis.LastUse (LastUseMap, Used, analyseGPUMem, analyseSeqMem) where
 
 import Control.Monad.Reader
 import Data.Bifunctor (first)
diff --git a/src/Futhark/Analysis/PrimExp.hs b/src/Futhark/Analysis/PrimExp.hs
--- a/src/Futhark/Analysis/PrimExp.hs
+++ b/src/Futhark/Analysis/PrimExp.hs
@@ -23,6 +23,7 @@
     leafExpTypes,
     true,
     false,
+    fromBool,
     constFoldPrimExp,
 
     -- * Construction
@@ -586,6 +587,10 @@
 true, false :: TPrimExp Bool v
 true = TPrimExp $ ValueExp $ BoolValue True
 false = TPrimExp $ ValueExp $ BoolValue False
+
+-- | Conversion from Bool to 'TPrimExp'
+fromBool :: Bool -> TPrimExp Bool v
+fromBool b = if b then true else false
 
 -- | Boolean negation smart constructor.
 bNot :: TPrimExp Bool v -> TPrimExp Bool v
diff --git a/src/Futhark/Analysis/PrimExp/Parse.hs b/src/Futhark/Analysis/PrimExp/Parse.hs
--- a/src/Futhark/Analysis/PrimExp/Parse.hs
+++ b/src/Futhark/Analysis/PrimExp/Parse.hs
@@ -19,33 +19,31 @@
 import Futhark.Util.Pretty (prettyText)
 import Text.Megaparsec
 
-type Parser = Parsec Void T.Text
-
-pBinOp :: Parser BinOp
+pBinOp :: Parsec Void T.Text BinOp
 pBinOp = choice $ map p allBinOps
   where
     p op = keyword (prettyText op) $> op
 
-pCmpOp :: Parser CmpOp
+pCmpOp :: Parsec Void T.Text CmpOp
 pCmpOp = choice $ map p allCmpOps
   where
     p op = keyword (prettyText op) $> op
 
-pUnOp :: Parser UnOp
+pUnOp :: Parsec Void T.Text UnOp
 pUnOp = choice $ map p allUnOps
   where
     p op = keyword (prettyText op) $> op
 
-pConvOp :: Parser ConvOp
+pConvOp :: Parsec Void T.Text ConvOp
 pConvOp = choice $ map p allConvOps
   where
     p op = keyword (prettyText op) $> op
 
-parens :: Parser a -> Parser a
+parens :: Parsec Void T.Text a -> Parsec Void T.Text a
 parens = between (lexeme "(") (lexeme ")")
 
 -- | Parse a 'PrimExp' given a leaf parser.
-pPrimExp :: PrimType -> Parser v -> Parser (PrimExp v)
+pPrimExp :: PrimType -> Parsec Void T.Text v -> Parsec Void T.Text (PrimExp v)
 pPrimExp t pLeaf =
   choice
     [ flip LeafExp t <$> pLeaf,
diff --git a/src/Futhark/Analysis/Rephrase.hs b/src/Futhark/Analysis/Rephrase.hs
--- a/src/Futhark/Analysis/Rephrase.hs
+++ b/src/Futhark/Analysis/Rephrase.hs
@@ -64,12 +64,12 @@
 rephrasePat ::
   Monad m =>
   (from -> m to) ->
-  PatT from ->
-  m (PatT to)
+  Pat from ->
+  m (Pat to)
 rephrasePat = traverse
 
 -- | Rephrase a pattern element.
-rephrasePatElem :: Monad m => (from -> m to) -> PatElemT from -> m (PatElemT to)
+rephrasePatElem :: Monad m => (from -> m to) -> PatElem from -> m (PatElem to)
 rephrasePatElem rephraser (PatElem ident from) =
   PatElem ident <$> rephraser from
 
diff --git a/src/Futhark/Analysis/SymbolTable.hs b/src/Futhark/Analysis/SymbolTable.hs
--- a/src/Futhark/Analysis/SymbolTable.hs
+++ b/src/Futhark/Analysis/SymbolTable.hs
@@ -366,7 +366,7 @@
 defBndEntry ::
   (ASTRep rep, IndexOp (Op rep)) =>
   SymbolTable rep ->
-  PatElem rep ->
+  PatElem (LetDec rep) ->
   Names ->
   Stm rep ->
   LetBoundEntry rep
diff --git a/src/Futhark/Analysis/UsageTable.hs b/src/Futhark/Analysis/UsageTable.hs
--- a/src/Futhark/Analysis/UsageTable.hs
+++ b/src/Futhark/Analysis/UsageTable.hs
@@ -129,6 +129,7 @@
 instance Monoid Usages where
   mempty = Usages 0
 
+-- | A kind of usage.
 consumedU, inResultU, presentU, sizeU :: Usages
 consumedU = Usages 1
 inResultU = Usages 2
diff --git a/src/Futhark/Builder.hs b/src/Futhark/Builder.hs
--- a/src/Futhark/Builder.hs
+++ b/src/Futhark/Builder.hs
@@ -46,7 +46,7 @@
 class ASTRep rep => BuilderOps rep where
   mkExpDecB ::
     (MonadBuilder m, Rep m ~ rep) =>
-    Pat rep ->
+    Pat (LetDec rep) ->
     Exp rep ->
     m (ExpDec rep)
   mkBodyB ::
@@ -62,7 +62,7 @@
 
   default mkExpDecB ::
     (MonadBuilder m, Buildable rep) =>
-    Pat rep ->
+    Pat (LetDec rep) ->
     Exp rep ->
     m (ExpDec rep)
   mkExpDecB pat e = return $ mkExpDec pat e
diff --git a/src/Futhark/Builder/Class.hs b/src/Futhark/Builder/Class.hs
--- a/src/Futhark/Builder/Class.hs
+++ b/src/Futhark/Builder/Class.hs
@@ -42,8 +42,8 @@
   ) =>
   Buildable rep
   where
-  mkExpPat :: [Ident] -> Exp rep -> Pat rep
-  mkExpDec :: Pat rep -> Exp rep -> ExpDec rep
+  mkExpPat :: [Ident] -> Exp rep -> Pat (LetDec rep)
+  mkExpDec :: Pat (LetDec rep) -> Exp rep -> ExpDec rep
   mkBody :: Stms rep -> Result -> Body rep
   mkLetNames ::
     (MonadFreshNames m, HasScope rep m) =>
@@ -72,7 +72,7 @@
   MonadBuilder m
   where
   type Rep m :: Data.Kind.Type
-  mkExpDecM :: Pat (Rep m) -> Exp (Rep m) -> m (ExpDec (Rep m))
+  mkExpDecM :: Pat (LetDec (Rep m)) -> Exp (Rep m) -> m (ExpDec (Rep m))
   mkBodyM :: Stms (Rep m) -> Result -> m (Body (Rep m))
   mkLetNamesM :: [VName] -> Exp (Rep m) -> m (Stm (Rep m))
 
@@ -127,7 +127,7 @@
 -- | Add a statement with the given pattern and expression.
 letBind ::
   MonadBuilder m =>
-  Pat (Rep m) ->
+  Pat (LetDec (Rep m)) ->
   Exp (Rep m) ->
   m ()
 letBind pat e =
diff --git a/src/Futhark/CLI/Defs.hs b/src/Futhark/CLI/Defs.hs
--- a/src/Futhark/CLI/Defs.hs
+++ b/src/Futhark/CLI/Defs.hs
@@ -13,7 +13,7 @@
 import Language.Futhark
 
 isBuiltin :: String -> Bool
-isBuiltin = ("prelude/" `isPrefixOf`)
+isBuiltin = ("/prelude/" `isPrefixOf`)
 
 data DefKind = Value | Module | ModuleType | Type
 
diff --git a/src/Futhark/CLI/Misc.hs b/src/Futhark/CLI/Misc.hs
--- a/src/Futhark/CLI/Misc.hs
+++ b/src/Futhark/CLI/Misc.hs
@@ -5,6 +5,7 @@
   ( mainImports,
     mainHash,
     mainDataget,
+    mainCheckSyntax,
   )
 where
 
@@ -24,7 +25,7 @@
 import System.IO
 
 isBuiltin :: String -> Bool
-isBuiltin = ("prelude/" `isPrefixOf`)
+isBuiltin = ("/prelude/" `isPrefixOf`)
 
 -- | @futhark imports@
 mainImports :: String -> [String] -> IO ()
@@ -80,3 +81,10 @@
     testSpecRuns = testActionRuns . testAction
     testActionRuns CompileTimeFailure {} = []
     testActionRuns (RunCases ios _ _) = concatMap iosTestRuns ios
+
+-- | @futhark check-syntax@
+mainCheckSyntax :: String -> [String] -> IO ()
+mainCheckSyntax = mainWithOptions () [] "program" $ \args () ->
+  case args of
+    [file] -> Just $ void $ readUntypedProgramOrDie file
+    _ -> Nothing
diff --git a/src/Futhark/CLI/REPL.hs b/src/Futhark/CLI/REPL.hs
--- a/src/Futhark/CLI/REPL.hs
+++ b/src/Futhark/CLI/REPL.hs
@@ -15,19 +15,19 @@
 import Data.Char
 import Data.List (intercalate, intersperse)
 import qualified Data.List.NonEmpty as NE
+import qualified Data.Map as M
 import Data.Maybe
 import qualified Data.Text as T
 import qualified Data.Text.IO as T
 import Data.Version
 import Futhark.Compiler
 import Futhark.MonadFreshNames
-import Futhark.Pipeline
-import Futhark.Util (fancyTerminal, toPOSIX)
+import Futhark.Util (fancyTerminal)
 import Futhark.Util.Options
 import Futhark.Version
 import Language.Futhark
 import qualified Language.Futhark.Interpreter as I
-import Language.Futhark.Parser hiding (EOF)
+import Language.Futhark.Parser
 import qualified Language.Futhark.Semantic as T
 import qualified Language.Futhark.TypeChecker as T
 import NeatInterpolation (text)
@@ -81,30 +81,30 @@
           Left (Load file) -> do
             liftIO $ T.putStrLn $ "Loading " <> T.pack file
             maybe_new_state <-
-              liftIO $ newFutharkiState (futharkiCount s) $ Just file
+              liftIO $ newFutharkiState (futharkiCount s) (futharkiProg s) $ Just file
             case maybe_new_state of
               Right new_state -> toploop new_state
               Left err -> do
                 liftIO $ putStrLn err
                 toploop s'
-          Right _ -> return ()
+          Right _ -> pure ()
 
       finish s = do
         quit <- if fancyTerminal then confirmQuit else pure True
-        if quit then return () else toploop s
+        if quit then pure () else toploop s
 
-  maybe_init_state <- liftIO $ newFutharkiState 0 maybe_prog
+  maybe_init_state <- liftIO $ newFutharkiState 0 noLoadedProg maybe_prog
   s <- case maybe_init_state of
     Left prog_err -> do
-      noprog_init_state <- liftIO $ newFutharkiState 0 Nothing
+      noprog_init_state <- liftIO $ newFutharkiState 0 noLoadedProg Nothing
       case noprog_init_state of
         Left err ->
           error $ "Failed to initialise interpreter state: " ++ err
         Right s -> do
           liftIO $ putStrLn prog_err
-          return s {futharkiLoaded = maybe_prog}
+          pure s {futharkiLoaded = maybe_prog}
     Right s ->
-      return s
+      pure s
   Haskeline.runInputT Haskeline.defaultSettings $ toploop s
 
   putStrLn "Leaving 'futhark repl'."
@@ -113,9 +113,9 @@
 confirmQuit = do
   c <- Haskeline.getInputChar "Quit REPL? (y/n) "
   case c of
-    Nothing -> return True -- EOF
-    Just 'y' -> return True
-    Just 'n' -> return False
+    Nothing -> pure True -- EOF
+    Just 'y' -> pure True
+    Just 'n' -> pure False
     _ -> confirmQuit
 
 -- | Representation of breaking at a breakpoint, to allow for
@@ -128,8 +128,7 @@
   }
 
 data FutharkiState = FutharkiState
-  { futharkiImports :: Imports,
-    futharkiNameSource :: VNameSource,
+  { futharkiProg :: LoadedProg,
     futharkiCount :: Int,
     futharkiEnv :: (T.Env, I.Ctx),
     -- | Are we currently stopped at a breakpoint?
@@ -141,55 +140,49 @@
     futharkiLoaded :: Maybe FilePath
   }
 
-newFutharkiState :: Int -> Maybe FilePath -> IO (Either String FutharkiState)
-newFutharkiState count maybe_file = runExceptT $ do
-  (imports, src, tenv, ienv) <- case maybe_file of
+extendEnvs :: LoadedProg -> (T.Env, I.Ctx) -> [String] -> (T.Env, I.Ctx)
+extendEnvs prog (tenv, ictx) opens = (tenv', ictx')
+  where
+    tenv' = T.envWithImports t_imports tenv
+    ictx' = I.ctxWithImports i_envs ictx
+    t_imports = filter ((`elem` opens) . fst) $ lpImports prog
+    i_envs = map snd $ filter ((`elem` opens) . fst) $ M.toList $ I.ctxImports ictx
+
+newFutharkiState :: Int -> LoadedProg -> Maybe FilePath -> IO (Either String FutharkiState)
+newFutharkiState count prev_prog maybe_file = runExceptT $ do
+  (prog, tenv, ienv) <- case maybe_file of
     Nothing -> do
       -- Load the builtins through the type checker.
-      (_, imports, src) <- badOnLeft show =<< runExceptT (readLibrary [] [])
+      (_, prog) <- badOnLeft show =<< runExceptT (reloadProg prev_prog [])
       -- Then into the interpreter.
       ienv <-
         foldM
           (\ctx -> badOnLeft show <=< runInterpreter' . I.interpretImport ctx)
           I.initialCtx
-          $ map (fmap fileProg) imports
+          $ map (fmap fileProg) (lpImports prog)
 
-      -- Then make the prelude available in the type checker.
-      (tenv, d, src') <-
-        badOnLeft pretty . snd $
-          T.checkDec imports src T.initialEnv (T.mkInitialImport ".") $
-            mkOpen "/prelude/prelude"
-      -- Then in the interpreter.
-      ienv' <- badOnLeft show =<< runInterpreter' (I.interpretDec ienv d)
-      return (imports, src', tenv, ienv')
+      let (tenv, ienv') =
+            extendEnvs prog (T.initialEnv, ienv) ["/prelude/prelude"]
+
+      pure (prog, tenv, ienv')
     Just file -> do
-      (ws, imports, src) <-
-        badOnLeft show
-          =<< liftIO
-            ( runExceptT (readProgram [] file)
-                `catch` \(err :: IOException) ->
-                  return (externalErrorS (show err))
-            )
+      (ws, prog) <- badOnLeft show =<< runExceptT (reloadProg prev_prog [file])
       liftIO $ putStrLn $ pretty ws
 
-      let imp = T.mkInitialImport "."
-      ienv1 <-
-        foldM (\ctx -> badOnLeft show <=< runInterpreter' . I.interpretImport ctx) I.initialCtx $
-          map (fmap fileProg) imports
-      (tenv1, d1, src') <-
-        badOnLeft pretty . snd . T.checkDec imports src T.initialEnv imp $
-          mkOpen "/prelude/prelude"
-      (tenv2, d2, src'') <-
-        badOnLeft pretty . snd . T.checkDec imports src' tenv1 imp $
-          mkOpen $ toPOSIX $ dropExtension file
-      ienv2 <- badOnLeft show =<< runInterpreter' (I.interpretDec ienv1 d1)
-      ienv3 <- badOnLeft show =<< runInterpreter' (I.interpretDec ienv2 d2)
-      return (imports, src'', tenv2, ienv3)
+      ienv <-
+        foldM
+          (\ctx -> badOnLeft show <=< runInterpreter' . I.interpretImport ctx)
+          I.initialCtx
+          $ map (fmap fileProg) (lpImports prog)
 
+      let (tenv, ienv') =
+            extendEnvs prog (T.initialEnv, ienv) ["/prelude/prelude", dropExtension file]
+
+      pure (prog, tenv, ienv')
+
   return
     FutharkiState
-      { futharkiImports = imports,
-        futharkiNameSource = src,
+      { futharkiProg = prog,
         futharkiCount = count,
         futharkiEnv = (tenv, ienv),
         futharkiBreaking = Nothing,
@@ -199,16 +192,13 @@
       }
   where
     badOnLeft :: (err -> String) -> Either err a -> ExceptT String IO a
-    badOnLeft _ (Right x) = return x
+    badOnLeft _ (Right x) = pure x
     badOnLeft p (Left err) = throwError $ p err
 
 getPrompt :: FutharkiM String
 getPrompt = do
   i <- gets futharkiCount
-  return $ "[" ++ show i ++ "]> "
-
-mkOpen :: FilePath -> UncheckedDec
-mkOpen f = OpenDec (ModImport f NoInfo mempty) mempty
+  pure $ "[" ++ show i ++ "]> "
 
 -- The ExceptT part is more of a continuation, really.
 newtype FutharkiM a = FutharkiM {runFutharkiM :: ExceptT StopReason (StateT FutharkiState (Haskeline.InputT IO)) a}
@@ -229,7 +219,7 @@
   case T.uncons line of
     Nothing
       | isJust breaking -> throwError Stop
-      | otherwise -> return ()
+      | otherwise -> pure ()
     Just (':', command) -> do
       let (cmdname, rest) = T.break isSpace command
           arg = T.dropWhileEnd isSpace $ T.dropWhile isSpace rest
@@ -253,36 +243,38 @@
     inputLine prompt = do
       inp <- FutharkiM $ lift $ lift $ Haskeline.getInputLine prompt
       case inp of
-        Just s -> return $ T.pack s
+        Just s -> pure $ T.pack s
         Nothing -> throwError EOF
 
 getIt :: FutharkiM (Imports, VNameSource, T.Env, I.Ctx)
 getIt = do
-  imports <- gets futharkiImports
-  src <- gets futharkiNameSource
+  imports <- gets $ lpImports . futharkiProg
+  src <- gets $ lpNameSource . futharkiProg
   (tenv, ienv) <- gets futharkiEnv
-  return (imports, src, tenv, ienv)
+  pure (imports, src, tenv, ienv)
 
 onDec :: UncheckedDec -> FutharkiM ()
 onDec d = do
-  (imports, src, tenv, ienv) <- getIt
+  old_imports <- gets $ lpImports . futharkiProg
   cur_import <- gets $ T.mkInitialImport . fromMaybe "." . futharkiLoaded
+  let mkImport = uncurry $ T.mkImportFrom cur_import
+      files = map (T.includeToFilePath . mkImport) $ decImports d
 
-  -- Most of the complexity here concerns the dealing with the fact
-  -- that 'import "foo"' is a declaration.  We have to involve a lot
-  -- of machinery to load this external code before executing the
-  -- declaration itself.
-  let basis = Basis imports src ["/prelude/prelude"]
-      mkImport = uncurry $ T.mkImportFrom cur_import
-  imp_r <- runExceptT $ readImports basis (map mkImport $ decImports d)
+  imp_r <- runExceptT $ do
+    prog <- lift $ gets futharkiProg
+    extendProg prog files
 
   case imp_r of
     Left e -> liftIO $ print e
-    Right (_, imports', src') ->
-      case T.checkDec imports' src' tenv cur_import d of
+    Right (_ws, prog) -> do
+      env <- gets futharkiEnv
+      let (tenv, ienv) = extendEnvs prog env (map fst $ decImports d)
+          imports = lpImports prog
+          src = lpNameSource prog
+      case T.checkDec imports src tenv cur_import d of
         (_, Left e) -> liftIO $ putStrLn $ pretty e
-        (_, Right (tenv', d', src'')) -> do
-          let new_imports = filter ((`notElem` map fst imports) . fst) imports'
+        (_, Right (tenv', d', src')) -> do
+          let new_imports = filter ((`notElem` map fst old_imports) . fst) imports
           int_r <- runInterpreter $ do
             let onImport ienv' (s, imp) =
                   I.interpretImport ienv' (s, T.fileProg imp)
@@ -293,8 +285,7 @@
             Right ienv' -> modify $ \s ->
               s
                 { futharkiEnv = (tenv', ienv'),
-                  futharkiImports = imports',
-                  futharkiNameSource = src''
+                  futharkiProg = prog {lpNameSource = src'}
                 }
 
 onExp :: UncheckedExp -> FutharkiM ()
@@ -332,7 +323,7 @@
 runInterpreter m = runF m (return . Right) intOp
   where
     intOp (I.ExtOpError err) =
-      return $ Left err
+      pure $ Left err
     intOp (I.ExtOpTrace w v c) = do
       liftIO $ putStrLn $ w ++ ": " ++ v
       c
@@ -384,7 +375,7 @@
 runInterpreter' :: MonadIO m => F I.ExtOp a -> m (Either I.InterpreterError a)
 runInterpreter' m = runF m (return . Right) intOp
   where
-    intOp (I.ExtOpError err) = return $ Left err
+    intOp (I.ExtOpError err) = pure $ Left err
     intOp (I.ExtOpTrace w v c) = do
       liftIO $ putStrLn $ w ++ ": " ++ v
       c
@@ -411,9 +402,7 @@
   case f prompt e of
     Left err -> liftIO $ print err
     Right e' -> do
-      imports <- gets futharkiImports
-      src <- gets futharkiNameSource
-      (tenv, _) <- gets futharkiEnv
+      (imports, src, tenv, _) <- getIt
       case snd $ g imports src tenv e' of
         Left err -> liftIO $ putStrLn $ pretty err
         Right x -> liftIO $ putStrLn $ h x
diff --git a/src/Futhark/CLI/Run.hs b/src/Futhark/CLI/Run.hs
--- a/src/Futhark/CLI/Run.hs
+++ b/src/Futhark/CLI/Run.hs
@@ -18,7 +18,7 @@
 import Futhark.Util.Options
 import Language.Futhark
 import qualified Language.Futhark.Interpreter as I
-import Language.Futhark.Parser hiding (EOF)
+import Language.Futhark.Parser
 import qualified Language.Futhark.Semantic as T
 import qualified Language.Futhark.TypeChecker as T
 import System.Exit
diff --git a/src/Futhark/CodeGen/Backends/CCUDA/Boilerplate.hs b/src/Futhark/CodeGen/Backends/CCUDA/Boilerplate.hs
--- a/src/Futhark/CodeGen/Backends/CCUDA/Boilerplate.hs
+++ b/src/Futhark/CodeGen/Backends/CCUDA/Boilerplate.hs
@@ -391,7 +391,7 @@
                  ctx->error = cuda_setup(&ctx->cuda, cuda_program, cfg->nvrtc_opts);
 
                  if (ctx->error != NULL) {
-                   return NULL;
+                   futhark_panic(1, "%s\n", ctx->error);
                  }
 
                  typename int32_t no_error = -1;
diff --git a/src/Futhark/CodeGen/Backends/GenericWASM.hs b/src/Futhark/CodeGen/Backends/GenericWASM.hs
--- a/src/Futhark/CodeGen/Backends/GenericWASM.hs
+++ b/src/Futhark/CodeGen/Backends/GenericWASM.hs
@@ -7,6 +7,7 @@
     GC.asLibrary,
     GC.asExecutable,
     GC.asServer,
+    EntryPointType,
     JSEntryPoint (..),
     emccExportNames,
     javascriptWrapper,
@@ -325,6 +326,8 @@
     (arraynd_p, arraynd_flat_p, arraynd_dims_p) = (T.pack arraynd, T.pack arraynd_flat, T.pack arraynd_dims)
     args = T.pack $ intercalate ", " ["'" ++ ftype ++ "'", show d, heap, fshape, fvalues, ffree]
 
+-- | Javascript code that can be appended to the generated module to
+-- run a Futhark server instance on startup.
 runServer :: T.Text
 runServer =
   [text|
@@ -334,5 +337,6 @@
      server.run();
    }|]
 
+-- | The names exported by the generated module.
 libraryExports :: T.Text
 libraryExports = "export {newFutharkContext, FutharkContext, FutharkArray, FutharkOpaque};"
diff --git a/src/Futhark/CodeGen/Backends/MulticoreC.hs b/src/Futhark/CodeGen/Backends/MulticoreC.hs
--- a/src/Futhark/CodeGen/Backends/MulticoreC.hs
+++ b/src/Futhark/CodeGen/Backends/MulticoreC.hs
@@ -490,7 +490,7 @@
   C.ToIdent a =>
   M.Map VName Space ->
   String ->
-  Code ->
+  MCCode ->
   a ->
   [(VName, (C.Type, ValueType))] ->
   [(VName, (C.Type, ValueType))] ->
diff --git a/src/Futhark/CodeGen/Backends/MulticoreWASM.hs b/src/Futhark/CodeGen/Backends/MulticoreWASM.hs
--- a/src/Futhark/CodeGen/Backends/MulticoreWASM.hs
+++ b/src/Futhark/CodeGen/Backends/MulticoreWASM.hs
@@ -25,6 +25,18 @@
 import Futhark.IR.MCMem
 import Futhark.MonadFreshNames
 
+-- | Compile Futhark program to wasm-multicore program (some assembly
+-- required).
+--
+-- The triple that is returned consists of
+--
+-- * Generated C code (to be passed to Emscripten).
+--
+-- * JavaScript wrapper code that presents a nicer interface to the
+--   Emscripten-produced code (this should be put in a @.class.js@
+--   file by itself).
+--
+-- * Options that should be passed to @emcc@.
 compileProg ::
   MonadFreshNames m =>
   T.Text ->
diff --git a/src/Futhark/CodeGen/Backends/SequentialC/Boilerplate.hs b/src/Futhark/CodeGen/Backends/SequentialC/Boilerplate.hs
--- a/src/Futhark/CodeGen/Backends/SequentialC/Boilerplate.hs
+++ b/src/Futhark/CodeGen/Backends/SequentialC/Boilerplate.hs
@@ -1,10 +1,12 @@
 {-# LANGUAGE QuasiQuotes #-}
 
+-- | Boilerplate for sequential C code.
 module Futhark.CodeGen.Backends.SequentialC.Boilerplate (generateBoilerplate) where
 
 import qualified Futhark.CodeGen.Backends.GenericC as GC
 import qualified Language.C.Quote.OpenCL as C
 
+-- | Generate the necessary boilerplate.
 generateBoilerplate :: GC.CompilerM op s ()
 generateBoilerplate = do
   cfg <- GC.publicDef "context_config" GC.InitDecl $ \s ->
diff --git a/src/Futhark/CodeGen/Backends/SequentialWASM.hs b/src/Futhark/CodeGen/Backends/SequentialWASM.hs
--- a/src/Futhark/CodeGen/Backends/SequentialWASM.hs
+++ b/src/Futhark/CodeGen/Backends/SequentialWASM.hs
@@ -25,7 +25,18 @@
 import Futhark.IR.SeqMem
 import Futhark.MonadFreshNames
 
--- | Compile the program to sequential C with a JavaScript wrapper.
+-- | Compile Futhark program to wasm program (some assembly
+-- required).
+--
+-- The triple that is returned consists of
+--
+-- * Generated C code (to be passed to Emscripten).
+--
+-- * JavaScript wrapper code that presents a nicer interface to the
+--   Emscripten-produced code (this should be put in a @.class.js@
+--   file by itself).
+--
+-- * Options that should be passed to @emcc@.
 compileProg :: MonadFreshNames m => T.Text -> Prog SeqMem -> m (ImpGen.Warnings, (GC.CParts, T.Text, [String]))
 compileProg version prog = do
   (ws, prog') <- ImpGen.compileProg prog
diff --git a/src/Futhark/CodeGen/Backends/SimpleRep.hs b/src/Futhark/CodeGen/Backends/SimpleRep.hs
--- a/src/Futhark/CodeGen/Backends/SimpleRep.hs
+++ b/src/Futhark/CodeGen/Backends/SimpleRep.hs
@@ -230,6 +230,9 @@
 cScalarDefs :: T.Text
 cScalarDefs = scalarH <> scalarF16H
 
+-- | @storageSize pt rank shape@ produces an expression giving size
+-- taken when storing this value in the binary value format.  It is
+-- assumed that the @shape@ is an array with @rank@ dimensions.
 storageSize :: PrimType -> Int -> C.Exp -> C.Exp
 storageSize pt rank shape =
   [C.cexp|$int:header_size +
@@ -258,6 +261,8 @@
     (TypeUnsigned, IntType Int32) -> " u32"
     (TypeUnsigned, IntType Int64) -> " u64"
 
+-- | Produce code for storing the header (everything besides the
+-- actual payload) for a value of this type.
 storeValueHeader :: Signedness -> PrimType -> Int -> C.Exp -> C.Exp -> [C.Stm]
 storeValueHeader sign pt rank shape dest =
   [C.cstms|
@@ -276,6 +281,8 @@
                 memcpy($exp:dest, $exp:shape, $int:rank*sizeof(typename int64_t));
                 $exp:dest += $int:rank*sizeof(typename int64_t);|]
 
+-- | Produce code for loading the header (everything besides the
+-- actual payload) for a value of this type.
 loadValueHeader :: Signedness -> PrimType -> Int -> C.Exp -> C.Exp -> [C.Stm]
 loadValueHeader sign pt rank shape src =
   [C.cstms|
diff --git a/src/Futhark/CodeGen/ImpCode.hs b/src/Futhark/CodeGen/ImpCode.hs
--- a/src/Futhark/CodeGen/ImpCode.hs
+++ b/src/Futhark/CodeGen/ImpCode.hs
@@ -2,13 +2,56 @@
 {-# LANGUAGE Strict #-}
 {-# LANGUAGE TupleSections #-}
 
--- | Imperative intermediate language used as a stepping stone in code generation.
+-- | ImpCode is an imperative intermediate language used as a stepping
+-- stone in code generation.  The functional core IR
+-- ("Futhark.IR.Syntax") gets translated into ImpCode by
+-- "Futhark.CodeGen.ImpGen".  Later we then translate ImpCode to, for
+-- example, C.
 --
--- This is a generic representation parametrised on an extensible
--- arbitrary operation.
+-- == Basic design
 --
--- Originally inspired by the paper "Defunctionalizing Push Arrays"
--- (FHPC '14).
+-- ImpCode distinguishes between /statements/ ('Code'), which may have
+-- side effects, and /expressions/ ('Exp') which do not.  Expressions
+-- involve only scalars and have a type.  The actual expression
+-- definition is in "Futhark.Analysis.PrimExp", specifically
+-- 'Futhark.Analysis.PrimExp.PrimExp' and its phantom-typed variant
+-- 'Futhark.Analysis.PrimExp.TPrimExp'.
+--
+-- 'Code' is a generic representation parametrised on an extensible
+-- arbitrary operation, represented by the 'Op' constructor.  Specific
+-- instantiations of ImpCode, such as
+-- "Futhark.CodeGen.ImpCode.Multicore", will pass in a specific kind
+-- of operation to express backend-specific functionality (in the case
+-- of multicore, this is
+-- 'Futhark.CodeGen.ImpCode.Multicore.Multicore').
+--
+-- == Arrays and memory
+--
+-- ImpCode does not have arrays. 'DeclareArray' is for declaring
+-- constant array literals, not arrays in general.  Instead, ImpCode
+-- deals only with memory.  Array operations present in core IR
+-- programs are turned into 'Write', v'Read', and 'Copy' operations
+-- that use flat indexes and offsets based on the index function of
+-- the original array.
+--
+-- == Scoping
+--
+-- ImpCode is much simpler than the functional core IR; partly because
+-- we hope to do less work on it.  We don't have real optimisation
+-- passes on ImpCode.  One result of this simplicity is that ImpCode
+-- has a fairly naive view of scoping.  The /only/ things that can
+-- bring new names into scope are 'DeclareMem', 'DeclareScalar',
+-- 'DeclareArray', 'For', and function parameters.  In particular,
+-- 'Op's /cannot/ bind parameters.  The standard workaround is to
+-- define 'Op's that retrieve the value of an implicit parameter and
+-- assign it to a variable declared with the normal
+-- mechanisms. 'Futhark.CodeGen.ImpCode.Multicore.GetLoopBounds' is an
+-- example of this pattern.
+--
+-- == Inspiration
+--
+-- ImpCode was originally inspired by the paper "Defunctionalizing
+-- Push Arrays" (FHPC '14).
 module Futhark.CodeGen.ImpCode
   ( Definitions (..),
     Functions (..),
@@ -241,8 +284,7 @@
   | -- | @Write mem i t space vol v@ writes the value @v@ to
     -- @mem@ offset by @i@ elements of type @t@.  The
     -- 'Space' argument is the memory space of @mem@
-    -- (technically redundant, but convenient).  Note that
-    -- /reading/ is done with an 'Exp' ('Read').
+    -- (technically redundant, but convenient).
     Write VName (Count Elements (TExp Int64)) PrimType Space Volatility Exp
   | -- | Set a scalar variable.
     SetScalar VName Exp
diff --git a/src/Futhark/CodeGen/ImpCode/GPU.hs b/src/Futhark/CodeGen/ImpCode/GPU.hs
--- a/src/Futhark/CodeGen/ImpCode/GPU.hs
+++ b/src/Futhark/CodeGen/ImpCode/GPU.hs
@@ -5,9 +5,7 @@
 -- of a kernel invocation.
 module Futhark.CodeGen.ImpCode.GPU
   ( Program,
-    Function,
-    FunctionT (Function),
-    Code,
+    HostCode,
     KernelCode,
     KernelConst (..),
     KernelConstExp,
@@ -22,23 +20,19 @@
   )
 where
 
-import Futhark.CodeGen.ImpCode hiding (Code, Function)
-import qualified Futhark.CodeGen.ImpCode as Imp
+import Futhark.CodeGen.ImpCode
 import Futhark.IR.GPU.Sizes
 import Futhark.IR.Pretty ()
 import Futhark.Util.Pretty
 
 -- | A program that calls kernels.
-type Program = Imp.Definitions HostOp
-
--- | A function that calls kernels.
-type Function = Imp.Function HostOp
+type Program = Definitions HostOp
 
 -- | Host-level code that can call kernels.
-type Code = Imp.Code HostOp
+type HostCode = Code HostOp
 
 -- | Code inside a kernel.
-type KernelCode = Imp.Code KernelOp
+type KernelCode = Code KernelOp
 
 -- | A run-time constant related to kernels.
 newtype KernelConst = SizeConst Name
@@ -51,17 +45,17 @@
 data HostOp
   = CallKernel Kernel
   | GetSize VName Name SizeClass
-  | CmpSizeLe VName Name SizeClass Imp.Exp
+  | CmpSizeLe VName Name SizeClass Exp
   | GetSizeMax VName SizeClass
   deriving (Show)
 
 -- | A generic kernel containing arbitrary kernel code.
 data Kernel = Kernel
-  { kernelBody :: Imp.Code KernelOp,
+  { kernelBody :: Code KernelOp,
     -- | The host variables referenced by the kernel.
     kernelUses :: [KernelUse],
-    kernelNumGroups :: [Imp.Exp],
-    kernelGroupSize :: [Imp.Exp],
+    kernelNumGroups :: [Exp],
+    kernelGroupSize :: [Exp],
     -- | A short descriptive and _unique_ name - should be
     -- alphanumeric and without spaces.
     kernelName :: Name,
@@ -156,7 +150,7 @@
   | Atomic Space AtomicOp
   | Barrier Fence
   | MemFence Fence
-  | LocalAlloc VName (Count Bytes (Imp.TExp Int64))
+  | LocalAlloc VName (Count Bytes (TExp Int64))
   | -- | Perform a barrier and also check whether any
     -- threads have failed an assertion.  Make sure all
     -- threads would reach all 'ErrorSync's if any of them
@@ -170,17 +164,17 @@
 -- This old value is stored in the first 'VName'.  The second 'VName'
 -- is the memory block to update.  The 'Exp' is the new value.
 data AtomicOp
-  = AtomicAdd IntType VName VName (Count Elements (Imp.TExp Int64)) Exp
-  | AtomicFAdd FloatType VName VName (Count Elements (Imp.TExp Int64)) Exp
-  | AtomicSMax IntType VName VName (Count Elements (Imp.TExp Int64)) Exp
-  | AtomicSMin IntType VName VName (Count Elements (Imp.TExp Int64)) Exp
-  | AtomicUMax IntType VName VName (Count Elements (Imp.TExp Int64)) Exp
-  | AtomicUMin IntType VName VName (Count Elements (Imp.TExp Int64)) Exp
-  | AtomicAnd IntType VName VName (Count Elements (Imp.TExp Int64)) Exp
-  | AtomicOr IntType VName VName (Count Elements (Imp.TExp Int64)) Exp
-  | AtomicXor IntType VName VName (Count Elements (Imp.TExp Int64)) Exp
-  | AtomicCmpXchg PrimType VName VName (Count Elements (Imp.TExp Int64)) Exp Exp
-  | AtomicXchg PrimType VName VName (Count Elements (Imp.TExp Int64)) Exp
+  = AtomicAdd IntType VName VName (Count Elements (TExp Int64)) Exp
+  | AtomicFAdd FloatType VName VName (Count Elements (TExp Int64)) Exp
+  | AtomicSMax IntType VName VName (Count Elements (TExp Int64)) Exp
+  | AtomicSMin IntType VName VName (Count Elements (TExp Int64)) Exp
+  | AtomicUMax IntType VName VName (Count Elements (TExp Int64)) Exp
+  | AtomicUMin IntType VName VName (Count Elements (TExp Int64)) Exp
+  | AtomicAnd IntType VName VName (Count Elements (TExp Int64)) Exp
+  | AtomicOr IntType VName VName (Count Elements (TExp Int64)) Exp
+  | AtomicXor IntType VName VName (Count Elements (TExp Int64)) Exp
+  | AtomicCmpXchg PrimType VName VName (Count Elements (TExp Int64)) Exp Exp
+  | AtomicXchg PrimType VName VName (Count Elements (TExp Int64)) Exp
   deriving (Show)
 
 instance FreeIn AtomicOp where
diff --git a/src/Futhark/CodeGen/ImpCode/Multicore.hs b/src/Futhark/CodeGen/ImpCode/Multicore.hs
--- a/src/Futhark/CodeGen/ImpCode/Multicore.hs
+++ b/src/Futhark/CodeGen/ImpCode/Multicore.hs
@@ -3,10 +3,8 @@
 -- | Multicore imperative code.
 module Futhark.CodeGen.ImpCode.Multicore
   ( Program,
-    Function,
-    FunctionT (Function),
-    Code,
     Multicore (..),
+    MCCode,
     Scheduling (..),
     SchedulerInfo (..),
     AtomicOp (..),
@@ -15,23 +13,16 @@
   )
 where
 
-import Futhark.CodeGen.ImpCode hiding (Code, Function)
-import qualified Futhark.CodeGen.ImpCode as Imp
+import Futhark.CodeGen.ImpCode
 import Futhark.Util.Pretty
 
--- | An imperative program.
-type Program = Imp.Functions Multicore
-
--- | An imperative function.
-type Function = Imp.Function Multicore
-
--- | A piece of imperative code, with multicore operations inside.
-type Code = Imp.Code Multicore
+-- | An imperative multicore program.
+type Program = Functions Multicore
 
 -- | A multicore operation.
 data Multicore
   = SegOp String [Param] ParallelTask (Maybe ParallelTask) [Param] SchedulerInfo
-  | ParLoop String Code [Param]
+  | ParLoop String (Code Multicore) [Param]
   | -- | Retrieve inclusive start and exclusive end indexes of the
     -- chunk we are supposed to be executing.  Only valid immediately
     -- inside a 'ParLoop' construct!
@@ -44,17 +35,20 @@
     GetNumTasks VName
   | Atomic AtomicOp
 
+-- | Multicore code.
+type MCCode = Code Multicore
+
 -- | Atomic operations return the value stored before the update.
 -- This old value is stored in the first 'VName'.  The second 'VName'
 -- is the memory block to update.  The 'Exp' is the new value.
 data AtomicOp
-  = AtomicAdd IntType VName VName (Count Elements (Imp.TExp Int32)) Exp
-  | AtomicSub IntType VName VName (Count Elements (Imp.TExp Int32)) Exp
-  | AtomicAnd IntType VName VName (Count Elements (Imp.TExp Int32)) Exp
-  | AtomicOr IntType VName VName (Count Elements (Imp.TExp Int32)) Exp
-  | AtomicXor IntType VName VName (Count Elements (Imp.TExp Int32)) Exp
-  | AtomicXchg PrimType VName VName (Count Elements (Imp.TExp Int32)) Exp
-  | AtomicCmpXchg PrimType VName VName (Count Elements (Imp.TExp Int32)) VName Exp
+  = AtomicAdd IntType VName VName (Count Elements (TExp Int32)) Exp
+  | AtomicSub IntType VName VName (Count Elements (TExp Int32)) Exp
+  | AtomicAnd IntType VName VName (Count Elements (TExp Int32)) Exp
+  | AtomicOr IntType VName VName (Count Elements (TExp Int32)) Exp
+  | AtomicXor IntType VName VName (Count Elements (TExp Int32)) Exp
+  | AtomicXchg PrimType VName VName (Count Elements (TExp Int32)) Exp
+  | AtomicCmpXchg PrimType VName VName (Count Elements (TExp Int32)) VName Exp
   deriving (Show)
 
 instance FreeIn AtomicOp where
@@ -66,12 +60,17 @@
   freeIn' (AtomicCmpXchg _ _ arr i retval x) = freeIn' arr <> freeIn' i <> freeIn' x <> freeIn' retval
   freeIn' (AtomicXchg _ _ arr i x) = freeIn' arr <> freeIn' i <> freeIn' x
 
+-- | Information about parallel work that is do be done.  This is
+-- passed to the scheduler to help it make scheduling decisions.
 data SchedulerInfo = SchedulerInfo
-  { iterations :: Imp.Exp, -- The number of total iterations for a task
-    scheduling :: Scheduling -- The type scheduling for the task
+  { -- | The number of total iterations for a task.
+    iterations :: Exp,
+    -- | The type scheduling for the task.
+    scheduling :: Scheduling
   }
 
-newtype ParallelTask = ParallelTask Code
+-- | A task for a v'SegOp'.
+newtype ParallelTask = ParallelTask (Code Multicore)
 
 -- | Whether the Scheduler should schedule the tasks as Dynamic
 -- or it is restainted to Static
diff --git a/src/Futhark/CodeGen/ImpCode/OpenCL.hs b/src/Futhark/CodeGen/ImpCode/OpenCL.hs
--- a/src/Futhark/CodeGen/ImpCode/OpenCL.hs
+++ b/src/Futhark/CodeGen/ImpCode/OpenCL.hs
@@ -8,11 +8,9 @@
 -- operation that allows one to execute an OpenCL kernel.
 module Futhark.CodeGen.ImpCode.OpenCL
   ( Program (..),
-    Function,
-    FunctionT (Function),
-    Code,
     KernelName,
     KernelArg (..),
+    CLCode,
     OpenCL (..),
     KernelSafety (..),
     numFailureParams,
@@ -25,8 +23,7 @@
 
 import qualified Data.Map as M
 import qualified Data.Text as T
-import Futhark.CodeGen.ImpCode hiding (Code, Function)
-import qualified Futhark.CodeGen.ImpCode as Imp
+import Futhark.CodeGen.ImpCode
 import Futhark.IR.GPU.Sizes
 import Futhark.Util.Pretty
 
@@ -52,11 +49,8 @@
     failureBacktrace :: String
   }
 
--- | A function calling OpenCL kernels.
-type Function = Imp.Function OpenCL
-
 -- | A piece of code calling OpenCL.
-type Code = Imp.Code OpenCL
+type CLCode = Code OpenCL
 
 -- | The name of a kernel.
 type KernelName = Name
diff --git a/src/Futhark/CodeGen/ImpCode/Sequential.hs b/src/Futhark/CodeGen/ImpCode/Sequential.hs
--- a/src/Futhark/CodeGen/ImpCode/Sequential.hs
+++ b/src/Futhark/CodeGen/ImpCode/Sequential.hs
@@ -1,26 +1,16 @@
 -- | Sequential imperative code.
 module Futhark.CodeGen.ImpCode.Sequential
   ( Program,
-    Function,
-    FunctionT (Function),
-    Code,
     Sequential,
     module Futhark.CodeGen.ImpCode,
   )
 where
 
-import Futhark.CodeGen.ImpCode hiding (Code, Function)
-import qualified Futhark.CodeGen.ImpCode as Imp
+import Futhark.CodeGen.ImpCode
 import Futhark.Util.Pretty
 
 -- | An imperative program.
-type Program = Imp.Definitions Sequential
-
--- | An imperative function.
-type Function = Imp.Function Sequential
-
--- | A piece of imperative code.
-type Code = Imp.Code Sequential
+type Program = Definitions Sequential
 
 -- | Phantom type for identifying sequential imperative code.
 data Sequential
diff --git a/src/Futhark/CodeGen/ImpGen.hs b/src/Futhark/CodeGen/ImpGen.hs
--- a/src/Futhark/CodeGen/ImpGen.hs
+++ b/src/Futhark/CodeGen/ImpGen.hs
@@ -130,6 +130,7 @@
 import qualified Data.DList as DL
 import Data.Either
 import Data.List (find)
+import Data.List.NonEmpty (NonEmpty (..))
 import qualified Data.Map.Strict as M
 import Data.Maybe
 import qualified Data.Set as S
@@ -155,13 +156,13 @@
 import Prelude hiding (quot)
 
 -- | How to compile an t'Op'.
-type OpCompiler rep r op = Pat rep -> Op rep -> ImpM rep r op ()
+type OpCompiler rep r op = Pat (LetDec rep) -> Op rep -> ImpM rep r op ()
 
 -- | How to compile some 'Stms'.
 type StmsCompiler rep r op = Names -> Stms rep -> ImpM rep r op () -> ImpM rep r op ()
 
 -- | How to compile an 'Exp'.
-type ExpCompiler rep r op = Pat rep -> Exp rep -> ImpM rep r op ()
+type ExpCompiler rep r op = Pat (LetDec rep) -> Exp rep -> ImpM rep r op ()
 
 type CopyCompiler rep r op =
   PrimType ->
@@ -377,7 +378,7 @@
 
   putNameSource $ stateNameSource s''
   warnings $ stateWarnings s''
-  return (x, stateCode s'')
+  pure (x, stateCode s'')
 
 -- | Execute a code generation action, returning the code that was
 -- emitted.
@@ -391,7 +392,7 @@
   x <- m
   new_code <- gets stateCode
   modify $ \s -> s {stateCode = prev_code}
-  return (x, new_code)
+  pure (x, new_code)
 
 -- | Execute a code generation action, wrapping the generated code
 -- within a 'Imp.Comment' with the given description.
@@ -561,7 +562,7 @@
           ++ mkExts epts fparams
       mkExts _ _ = []
 
-  return (inparams, arrayds, mkExts eparams val_params)
+  pure (inparams, arrayds, mkExts eparams val_params)
   where
     isArrayDecl x (ArrayDecl y _ _) = x == y
 
@@ -636,7 +637,7 @@
   evs <- case maybe_orig_epts of
     Just orig_epts -> compileExternalValues orig_rts orig_epts maybe_params
     Nothing -> pure []
-  return (evs, catMaybes maybe_params, dests)
+  pure (evs, catMaybes maybe_params, dests)
 
 compileFunDef ::
   Mem rep inner =>
@@ -664,9 +665,9 @@
       compileStms (freeIn ses) stms $
         forM_ (zip dests ses) $ \(d, SubExpRes _ se) -> copyDWIMDest d [] se []
 
-      return (outparams, inparams, results, args)
+      pure (outparams, inparams, results, args)
 
-compileBody :: Pat rep -> Body rep -> ImpM rep r op ()
+compileBody :: Pat (LetDec rep) -> Body rep -> ImpM rep r op ()
 compileBody pat (Body _ stms ses) = do
   dests <- destinationFromPat pat
   compileStms (freeIn ses) stms $
@@ -692,12 +693,12 @@
         Prim pt -> do
           emit $ Imp.DeclareScalar tmp Imp.Nonvolatile pt
           emit $ Imp.SetScalar tmp $ toExp' pt se
-          return $ emit $ Imp.SetScalar (paramName p) $ Imp.var tmp pt
+          pure $ emit $ Imp.SetScalar (paramName p) $ Imp.var tmp pt
         Mem space | Var v <- se -> do
           emit $ Imp.DeclareMem tmp space
           emit $ Imp.SetMem tmp v space
-          return $ emit $ Imp.SetMem (paramName p) tmp space
-        _ -> return $ return ()
+          pure $ emit $ Imp.SetMem (paramName p) tmp space
+        _ -> pure $ pure ()
     sequence_ copy_to_merge_params
 
 compileStms :: Names -> Stms rep -> ImpM rep r op () -> ImpM rep r op ()
@@ -734,25 +735,25 @@
       mapM_ (emit . uncurry Imp.Free) to_free
       emit bs_code
 
-      return $ freeIn e_code <> live_after
+      pure $ freeIn e_code <> live_after
     compileStms' _ [] = do
       code <- collect m
       emit code
-      return $ freeIn code <> alive_after_stms
+      pure $ freeIn code <> alive_after_stms
 
     patternAllocs = S.fromList . mapMaybe isMemPatElem . patElems
     isMemPatElem pe = case patElemType pe of
       Mem space -> Just (patElemName pe, space)
       _ -> Nothing
 
-compileExp :: Pat rep -> Exp rep -> ImpM rep r op ()
+compileExp :: Pat (LetDec rep) -> Exp rep -> ImpM rep r op ()
 compileExp pat e = do
   ec <- asks envExpCompiler
   ec pat e
 
 defCompileExp ::
   (Mem rep inner) =>
-  Pat rep ->
+  Pat (LetDec rep) ->
   Exp rep ->
   ImpM rep r op ()
 defCompileExp pat (If cond tbranch fbranch _) =
@@ -766,9 +767,9 @@
     compileArg (se, _) = do
       t <- subExpType se
       case (se, t) of
-        (_, Prim pt) -> return $ Just $ Imp.ExpArg $ toExp' pt se
-        (Var v, Mem {}) -> return $ Just $ Imp.MemArg v
-        _ -> return Nothing
+        (_, Prim pt) -> pure $ Just $ Imp.ExpArg $ toExp' pt se
+        (Var v, Mem {}) -> pure $ Just $ Imp.MemArg v
+        _ -> pure Nothing
 defCompileExp pat (BasicOp op) = defCompileBasicOp pat op
 defCompileExp pat (DoLoop merge form body) = do
   attrs <- askAttrs
@@ -787,7 +788,7 @@
             | Prim _ <- paramType p =
               copyDWIM (paramName p) [] (Var a) [DimFix $ Imp.le64 i]
             | otherwise =
-              return ()
+              pure ()
 
       bound' <- toExp bound
 
@@ -834,7 +835,7 @@
 
 defCompileBasicOp ::
   Mem rep inner =>
-  Pat rep ->
+  Pat (LetDec rep) ->
   BasicOp ->
   ImpM rep r op ()
 defCompileBasicOp (Pat [pe]) (SubExp se) =
@@ -877,7 +878,7 @@
   | Just idxs <- sliceIndices slice =
     copyDWIM (patElemName pe) [] (Var src) $ map (DimFix . toInt64Exp) idxs
 defCompileBasicOp _ Index {} =
-  return ()
+  pure ()
 defCompileBasicOp (Pat [pe]) (Update safety _ slice se) =
   case safety of
     Unsafe -> write
@@ -902,7 +903,7 @@
     copy_elem <- collect $ copyDWIM (patElemName pe) (map (DimFix . Imp.le64) is) se []
     emit $ foldl (.) id (zipWith Imp.For is ds') copy_elem
 defCompileBasicOp _ Scratch {} =
-  return ()
+  pure ()
 defCompileBasicOp (Pat [pe]) (Iota n e s it) = do
   e' <- toExp e
   s' <- toExp s
@@ -917,7 +918,7 @@
   copyDWIM (patElemName pe) [] (Var src) []
 defCompileBasicOp (Pat [pe]) (Manifest _ src) =
   copyDWIM (patElemName pe) [] (Var src) []
-defCompileBasicOp (Pat [pe]) (Concat i x ys _) = do
+defCompileBasicOp (Pat [pe]) (Concat i (x :| ys) _) = do
   offs_glb <- dPrimV "tmp_offs" 0
 
   forM_ (x : ys) $ \y -> do
@@ -951,11 +952,11 @@
     isLiteral (Constant v) = Just v
     isLiteral _ = Nothing
 defCompileBasicOp _ Rearrange {} =
-  return ()
+  pure ()
 defCompileBasicOp _ Rotate {} =
-  return ()
+  pure ()
 defCompileBasicOp _ Reshape {} =
-  return ()
+  pure ()
 defCompileBasicOp _ (UpdateAcc acc is vs) = sComment "UpdateAcc" $ do
   -- We are abusing the comment mechanism to wrap the operator in
   -- braces when we end up generating code.  This is necessary because
@@ -1025,7 +1026,7 @@
 dVars ::
   Mem rep inner =>
   Maybe (Exp rep) ->
-  [PatElem rep] ->
+  [PatElem (LetDec rep)] ->
   ImpM rep r op ()
 dVars e = mapM_ dVar
   where
@@ -1043,7 +1044,7 @@
   emit $ Imp.DeclareScalar name' Imp.Volatile t
   addVar name' $ ScalarVar Nothing $ ScalarEntry t
   name' <~~ untyped e
-  return $ TV name' t
+  pure $ TV name' t
 
 dPrim_ :: VName -> PrimType -> ImpM rep r op ()
 dPrim_ name t = do
@@ -1057,7 +1058,7 @@
 dPrim name t = do
   name' <- newVName name
   dPrim_ name' t
-  return $ TV name' t
+  pure $ TV name' t
 
 dPrimV_ :: VName -> Imp.TExp t -> ImpM rep r op ()
 dPrimV_ name e = do
@@ -1070,13 +1071,13 @@
 dPrimV name e = do
   name' <- dPrim name $ primExpType $ untyped e
   name' <-- e
-  return name'
+  pure name'
 
 dPrimVE :: String -> Imp.TExp t -> ImpM rep r op (Imp.TExp t)
 dPrimVE name e = do
   name' <- dPrim name $ primExpType $ untyped e
   name' <-- e
-  return $ tvExp name'
+  pure $ tvExp name'
 
 memBoundToVarEntry ::
   Maybe (Exp rep) ->
@@ -1120,9 +1121,9 @@
     ScalarVar _ entry' ->
       emit $ Imp.DeclareScalar name Imp.Nonvolatile $ entryScalarType entry'
     ArrayVar _ _ ->
-      return ()
+      pure ()
     AccVar {} ->
-      return ()
+      pure ()
   addVar name entry
 
 dScope ::
@@ -1148,11 +1149,11 @@
   concat <$> mapM funcallTarget dests
   where
     funcallTarget (ScalarDestination name) =
-      return [name]
+      pure [name]
     funcallTarget (ArrayDestination _) =
-      return []
+      pure []
     funcallTarget (MemoryDestination name) =
-      return [name]
+      pure [name]
 
 -- | A typed variable, which we can turn into a typed expression, or
 -- use as the target for an assignment.  This is used to aid in type
@@ -1196,11 +1197,11 @@
 
 instance ToExp SubExp where
   toExp (Constant v) =
-    return $ Imp.ValueExp v
+    pure $ Imp.ValueExp v
   toExp (Var v) =
     lookupVar v >>= \case
       ScalarVar _ (ScalarEntry pt) ->
-        return $ Imp.var v pt
+        pure $ Imp.var v pt
       _ -> error $ "toExp SubExp: SubExp is not a primitive type: " ++ pretty v
 
   toExp' _ (Constant v) = Imp.ValueExp v
@@ -1230,7 +1231,7 @@
 nameForFun :: String -> ImpM rep r op Name
 nameForFun s = do
   fname <- askFunction
-  return $ maybe "" (<> ".") fname <> nameFromString s
+  pure $ maybe "" (<> ".") fname <> nameFromString s
 
 askEnv :: ImpM rep r op r
 askEnv = asks envEnv
@@ -1272,27 +1273,27 @@
   putVTable $ f old_vtable
   a <- m
   putVTable old_vtable
-  return a
+  pure a
 
 lookupVar :: VName -> ImpM rep r op (VarEntry rep)
 lookupVar name = do
   res <- gets $ M.lookup name . stateVTable
   case res of
-    Just entry -> return entry
+    Just entry -> pure entry
     _ -> error $ "Unknown variable: " ++ pretty name
 
 lookupArray :: VName -> ImpM rep r op ArrayEntry
 lookupArray name = do
   res <- lookupVar name
   case res of
-    ArrayVar _ entry -> return entry
+    ArrayVar _ entry -> pure entry
     _ -> error $ "ImpGen.lookupArray: not an array: " ++ pretty name
 
 lookupMemory :: VName -> ImpM rep r op MemEntry
 lookupMemory name = do
   res <- lookupVar name
   case res of
-    MemVar _ entry -> return entry
+    MemVar _ entry -> pure entry
     _ -> error $ "Unknown memory block: " ++ pretty name
 
 lookupArraySpace :: VName -> ImpM rep r op Space
@@ -1327,12 +1328,12 @@
             )
         Just (arrs@(arr : _), Nothing) -> do
           space <- lookupArraySpace arr
-          return (acc, space, arrs, map toInt64Exp (shapeDims ispace), Nothing)
+          pure (acc, space, arrs, map toInt64Exp (shapeDims ispace), Nothing)
         Nothing ->
           error $ "ImpGen.lookupAcc: unlisted accumulator: " ++ pretty name
     _ -> error $ "ImpGen.lookupAcc: not an accumulator: " ++ pretty name
 
-destinationFromPat :: Pat rep -> ImpM rep r op [ValueDestination]
+destinationFromPat :: Pat (LetDec rep) -> ImpM rep r op [ValueDestination]
 destinationFromPat = mapM inspect . patElems
   where
     inspect pe = do
@@ -1340,13 +1341,13 @@
       entry <- lookupVar name
       case entry of
         ArrayVar _ (ArrayEntry MemLoc {} _) ->
-          return $ ArrayDestination Nothing
+          pure $ ArrayDestination Nothing
         MemVar {} ->
-          return $ MemoryDestination name
+          pure $ MemoryDestination name
         ScalarVar {} ->
-          return $ ScalarDestination name
+          pure $ ScalarDestination name
         AccVar {} ->
-          return $ ArrayDestination Nothing
+          pure $ ArrayDestination Nothing
 
 fullyIndexArray ::
   VName ->
@@ -1371,14 +1372,23 @@
 -- More complicated read/write operations that use index functions.
 
 copy :: CopyCompiler rep r op
-copy bt dest src =
-  unless
-    ( memLocName dest == memLocName src
-        && memLocIxFun dest `IxFun.equivalent` memLocIxFun src
-    )
-    $ do
-      cc <- asks envCopyCompiler
-      cc bt dest src
+copy
+  bt
+  dst@(MemLoc dst_name _ dst_ixfn@(IxFun.IxFun dst_lmads@(dst_lmad :| _) _ _))
+  src@(MemLoc src_name _ src_ixfn@(IxFun.IxFun src_lmads@(src_lmad :| _) _ _)) = do
+    -- If we can statically determine that the two index-functions
+    -- are equivalent, don't do anything
+    unless (dst_name == src_name && dst_ixfn `IxFun.equivalent` src_ixfn) $
+      -- It's also possible that we can dynamically determine that the two
+      -- index-functions are equivalent.
+      sUnless
+        ( fromBool (dst_name == src_name && length dst_lmads == 1 && length src_lmads == 1)
+            .&&. IxFun.dynamicEqualsLMAD dst_lmad src_lmad
+        )
+        $ do
+          -- If none of the above is true, actually do the copy
+          cc <- asks envCopyCompiler
+          cc bt dst src
 
 -- | Is this copy really a mapping with transpose?
 isMapTransposeCopy ::
@@ -1434,7 +1444,7 @@
   exists <- hasFunction fname
   unless exists $ emitFunction fname $ mapTransposeFunction fname bt
 
-  return fname
+  pure fname
 
 -- | Use an 'Imp.Copy' if possible, otherwise 'copyElementWise'.
 defaultCopy :: CopyCompiler rep r op
@@ -1646,10 +1656,10 @@
               pretty dest_slice
             ]
     (ArrayDestination Nothing, _) ->
-      return () -- Nothing to do; something else set some memory
+      pure () -- Nothing to do; something else set some memory
       -- somewhere.
     (_, AccVar {}) ->
-      return () -- Nothing to do; accumulators are phantoms.
+      pure () -- Nothing to do; accumulators are phantoms.
 
 -- | Copy from here to there; both destination and source be
 -- indexeded.  If so, they better be arrays of enough dimensions.
@@ -1686,11 +1696,11 @@
 copyDWIMFix dest dest_is src src_is =
   copyDWIM dest (map DimFix dest_is) src (map DimFix src_is)
 
--- | @compileAlloc pat size space@ allocates @n@ bytes of memory in @space@,
--- writing the result to @dest@, which must be a single
--- 'MemoryDestination',
+-- | @compileAlloc pat size space@ allocates @n@ bytes of memory in
+-- @space@, writing the result to @pat@, which must contain a single
+-- memory-typed element.
 compileAlloc ::
-  Mem rep inner => Pat rep -> SubExp -> Space -> ImpM rep r op ()
+  Mem rep inner => Pat (LetDec rep) -> SubExp -> Space -> ImpM rep r op ()
 compileAlloc (Pat [mem]) e space = do
   let e' = Imp.bytes $ toInt64Exp e
   allocator <- asks $ M.lookup space . envAllocCompilers
@@ -1771,7 +1781,7 @@
   name' <- newVName name
   emit $ Imp.DeclareMem name' space
   addVar name' $ MemVar Nothing $ MemEntry space
-  return name'
+  pure name'
 
 sAlloc_ :: VName -> Count Bytes (Imp.TExp Int64) -> Space -> ImpM rep r op ()
 sAlloc_ name' size' space = do
@@ -1784,13 +1794,13 @@
 sAlloc name size space = do
   name' <- sDeclareMem name space
   sAlloc_ name' size space
-  return name'
+  pure name'
 
 sArray :: String -> PrimType -> ShapeBase SubExp -> VName -> IxFun -> ImpM rep r op VName
 sArray name bt shape mem ixfun = do
   name' <- newVName name
   dArray name' bt shape mem ixfun
-  return name'
+  pure name'
 
 -- | Declare an array in row-major order in the given memory block.
 sArrayInMem :: String -> PrimType -> ShapeBase SubExp -> VName -> ImpM rep r op VName
diff --git a/src/Futhark/CodeGen/ImpGen/GPU.hs b/src/Futhark/CodeGen/ImpGen/GPU.hs
--- a/src/Futhark/CodeGen/ImpGen/GPU.hs
+++ b/src/Futhark/CodeGen/ImpGen/GPU.hs
@@ -106,7 +106,7 @@
 compileProgCUDA = compileProg $ HostEnv cudaAtomics CUDA mempty
 
 opCompiler ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   Op GPUMem ->
   CallKernelGen ()
 opCompiler dest (Alloc e space) =
@@ -158,7 +158,7 @@
 sizeClassWithEntryPoint _ size_class = size_class
 
 segOpCompiler ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   SegOp SegLevel GPUMem ->
   CallKernelGen ()
 segOpCompiler pat (SegMap lvl space _ kbody) =
@@ -178,7 +178,7 @@
 -- otherwise protected by their own multi-versioning branches deeper
 -- down.  Currently the compiler will not generate multi-versioning
 -- that makes this a problem, but it might in the future.
-checkLocalMemoryReqs :: Imp.Code -> CallKernelGen (Maybe (Imp.TExp Bool))
+checkLocalMemoryReqs :: Imp.HostCode -> CallKernelGen (Maybe (Imp.TExp Bool))
 checkLocalMemoryReqs code = do
   scope <- askScope
   let alloc_sizes = map (sum . map alignedSize . localAllocSizes . Imp.kernelBody) $ getGPU code
@@ -211,7 +211,7 @@
     alignedSize x = x + ((8 - (x `rem` 8)) `rem` 8)
 
 withAcc ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   [(Shape, [VName], Maybe (Lambda GPUMem, [SubExp]))] ->
   Lambda GPUMem ->
   CallKernelGen ()
@@ -313,7 +313,7 @@
 mapTransposeName :: PrimType -> String
 mapTransposeName bt = "gpu_map_transpose_" ++ pretty bt
 
-mapTransposeFunction :: PrimType -> Imp.Function
+mapTransposeFunction :: PrimType -> Imp.Function Imp.HostOp
 mapTransposeFunction bt =
   Imp.Function Nothing [] params transpose_code [] []
   where
diff --git a/src/Futhark/CodeGen/ImpGen/GPU/Base.hs b/src/Futhark/CodeGen/ImpGen/GPU/Base.hs
--- a/src/Futhark/CodeGen/ImpGen/GPU/Base.hs
+++ b/src/Futhark/CodeGen/ImpGen/GPU/Base.hs
@@ -107,7 +107,7 @@
 -- | The sizes of nested iteration spaces in the kernel.
 type SegOpSizes = S.Set [SubExp]
 
--- | Find the sizes of nested parallelism in a 'SegOp' body.
+-- | Find the sizes of nested parallelism in a t'SegOp' body.
 segOpSizes :: Stms GPUMem -> SegOpSizes
 segOpSizes = onStms
   where
@@ -172,7 +172,7 @@
   sOp $ Imp.LocalAlloc mem size
 
 kernelAlloc ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   SubExp ->
   Space ->
   InKernelGen ()
@@ -189,7 +189,7 @@
 
 splitSpace ::
   (ToExp w, ToExp i, ToExp elems_per_thread) =>
-  Pat GPUMem ->
+  Pat LetDecMem ->
   SplitOrdering ->
   w ->
   i ->
@@ -246,8 +246,8 @@
 
 -- | Assign iterations of a for-loop to all threads in the kernel.
 -- The passed-in function is invoked with the (symbolic) iteration.
--- 'threadOperations' will be in effect in the body.  For
--- multidimensional loops, use 'groupCoverSpace'.
+-- The body must contain thread-level code.  For multidimensional
+-- loops, use 'groupCoverSpace'.
 kernelLoop ::
   IntExp t =>
   Imp.TExp t ->
@@ -506,7 +506,7 @@
 --
 -- 2. Executes the body of @lam@.
 --
--- 3. Binds the 'SubExp's that are the 'Result' of @lam@ to the
+-- 3. Binds the t'SubExp's that are the 'Result' of @lam@ to the
 -- provided @dest@s, again interpreted as the destination for a
 -- 'copyDWIM'.
 applyLambda ::
@@ -1942,7 +1942,7 @@
 
 compileGroupResult ::
   SegSpace ->
-  PatElem GPUMem ->
+  PatElem LetDecMem ->
   KernelResult ->
   InKernelGen ()
 compileGroupResult _ pe (TileReturns _ [(w, per_group_elems)] what) = do
@@ -2035,7 +2035,7 @@
 
 compileThreadResult ::
   SegSpace ->
-  PatElem GPUMem ->
+  PatElem LetDecMem ->
   KernelResult ->
   InKernelGen ()
 compileThreadResult _ _ RegTileReturns {} =
diff --git a/src/Futhark/CodeGen/ImpGen/GPU/SegHist.hs b/src/Futhark/CodeGen/ImpGen/GPU/SegHist.hs
--- a/src/Futhark/CodeGen/ImpGen/GPU/SegHist.hs
+++ b/src/Futhark/CodeGen/ImpGen/GPU/SegHist.hs
@@ -378,7 +378,7 @@
       return (l', do_op')
 
 histKernelGlobalPass ::
-  [PatElem GPUMem] ->
+  [PatElem LetDecMem] ->
   Count NumGroups (Imp.TExp Int64) ->
   Count GroupSize (Imp.TExp Int64) ->
   SegSpace ->
@@ -469,7 +469,7 @@
                       do_op (bucket_is ++ is)
 
 histKernelGlobal ::
-  [PatElem GPUMem] ->
+  [PatElem LetDecMem] ->
   Count NumGroups SubExp ->
   Count GroupSize SubExp ->
   SegSpace ->
@@ -577,7 +577,7 @@
 histKernelLocalPass ::
   TV Int32 ->
   Count NumGroups (Imp.TExp Int64) ->
-  [PatElem GPUMem] ->
+  [PatElem LetDecMem] ->
   Count NumGroups (Imp.TExp Int64) ->
   Count GroupSize (Imp.TExp Int64) ->
   SegSpace ->
@@ -825,7 +825,7 @@
 histKernelLocal ::
   TV Int32 ->
   Count NumGroups (Imp.TExp Int64) ->
-  [PatElem GPUMem] ->
+  [PatElem LetDecMem] ->
   Count NumGroups SubExp ->
   Count GroupSize SubExp ->
   SegSpace ->
@@ -869,7 +869,7 @@
     AtomicLocking _ -> 6
 
 localMemoryCase ::
-  [PatElem GPUMem] ->
+  [PatElem LetDecMem] ->
   Imp.TExp Int32 ->
   SegSpace ->
   Imp.TExp Int64 ->
@@ -1024,7 +1024,7 @@
 
 -- | Generate code for a segmented histogram called from the host.
 compileSegHist ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   Count NumGroups SubExp ->
   Count GroupSize SubExp ->
   SegSpace ->
diff --git a/src/Futhark/CodeGen/ImpGen/GPU/SegMap.hs b/src/Futhark/CodeGen/ImpGen/GPU/SegMap.hs
--- a/src/Futhark/CodeGen/ImpGen/GPU/SegMap.hs
+++ b/src/Futhark/CodeGen/ImpGen/GPU/SegMap.hs
@@ -17,7 +17,7 @@
 
 -- | Compile 'SegMap' instance code.
 compileSegMap ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   SegLevel ->
   SegSpace ->
   KernelBody GPUMem ->
diff --git a/src/Futhark/CodeGen/ImpGen/GPU/SegRed.hs b/src/Futhark/CodeGen/ImpGen/GPU/SegRed.hs
--- a/src/Futhark/CodeGen/ImpGen/GPU/SegRed.hs
+++ b/src/Futhark/CodeGen/ImpGen/GPU/SegRed.hs
@@ -77,7 +77,7 @@
 -- | Compile 'SegRed' instance to host-level code with calls to
 -- various kernels.
 compileSegRed ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   SegLevel ->
   SegSpace ->
   [SegBinOp GPUMem] ->
@@ -96,7 +96,7 @@
 
 -- | Like 'compileSegRed', but where the body is a monadic action.
 compileSegRed' ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   SegLevel ->
   SegSpace ->
   [SegBinOp GPUMem] ->
@@ -171,7 +171,7 @@
       sAllocArrayPerm "segred_tmp" pt full_shape (Space "device") perm
 
 nonsegmentedReduction ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   Count NumGroups SubExp ->
   Count GroupSize SubExp ->
   SegSpace ->
@@ -255,7 +255,7 @@
   emit $ Imp.DebugPrint "" Nothing
 
 smallSegmentsReduction ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   Count NumGroups SubExp ->
   Count GroupSize SubExp ->
   SegSpace ->
@@ -364,7 +364,7 @@
   emit $ Imp.DebugPrint "" Nothing
 
 largeSegmentsReduction ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   Count NumGroups SubExp ->
   Count GroupSize SubExp ->
   SegSpace ->
@@ -702,7 +702,7 @@
 
 reductionStageTwo ::
   KernelConstants ->
-  [PatElem GPUMem] ->
+  [PatElem LetDecMem] ->
   Imp.TExp Int32 ->
   Imp.TExp Int32 ->
   [Imp.TExp Int64] ->
diff --git a/src/Futhark/CodeGen/ImpGen/GPU/SegScan.hs b/src/Futhark/CodeGen/ImpGen/GPU/SegScan.hs
--- a/src/Futhark/CodeGen/ImpGen/GPU/SegScan.hs
+++ b/src/Futhark/CodeGen/ImpGen/GPU/SegScan.hs
@@ -49,7 +49,7 @@
 -- | Compile 'SegScan' instance to host-level code with calls to
 -- various kernels.
 compileSegScan ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   SegLevel ->
   SegSpace ->
   [SegBinOp GPUMem] ->
diff --git a/src/Futhark/CodeGen/ImpGen/GPU/SegScan/SinglePass.hs b/src/Futhark/CodeGen/ImpGen/GPU/SegScan/SinglePass.hs
--- a/src/Futhark/CodeGen/ImpGen/GPU/SegScan/SinglePass.hs
+++ b/src/Futhark/CodeGen/ImpGen/GPU/SegScan/SinglePass.hs
@@ -209,7 +209,7 @@
 -- | Compile 'SegScan' instance to host-level code with calls to a
 -- single-pass kernel.
 compileSegScan ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   SegLevel ->
   SegSpace ->
   SegBinOp GPUMem ->
diff --git a/src/Futhark/CodeGen/ImpGen/GPU/SegScan/TwoPass.hs b/src/Futhark/CodeGen/ImpGen/GPU/SegScan/TwoPass.hs
--- a/src/Futhark/CodeGen/ImpGen/GPU/SegScan/TwoPass.hs
+++ b/src/Futhark/CodeGen/ImpGen/GPU/SegScan/TwoPass.hs
@@ -92,7 +92,7 @@
 
 writeToScanValues ::
   [VName] ->
-  ([PatElem GPUMem], SegBinOp GPUMem, [KernelResult]) ->
+  ([PatElem LetDecMem], SegBinOp GPUMem, [KernelResult]) ->
   InKernelGen ()
 writeToScanValues gtids (pes, scan, scan_res)
   | shapeRank (segBinOpShape scan) > 0 =
@@ -108,7 +108,7 @@
 
 readToScanValues ::
   [Imp.TExp Int64] ->
-  [PatElem GPUMem] ->
+  [PatElem LetDecMem] ->
   SegBinOp GPUMem ->
   InKernelGen ()
 readToScanValues is pes scan
@@ -123,7 +123,7 @@
   Imp.TExp Int64 ->
   [Imp.TExp Int64] ->
   [Imp.TExp Int64] ->
-  [PatElem GPUMem] ->
+  [PatElem LetDecMem] ->
   SegBinOp GPUMem ->
   InKernelGen ()
 readCarries chunk_id chunk_offset dims' vec_is pes scan
@@ -146,7 +146,7 @@
 
 -- | Produce partially scanned intervals; one per workgroup.
 scanStage1 ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   Count NumGroups SubExp ->
   Count GroupSize SubExp ->
   SegSpace ->
@@ -317,7 +317,7 @@
   return (num_threads, elems_per_group, crossesSegment)
 
 scanStage2 ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   TV Int32 ->
   Imp.TExp Int64 ->
   Count NumGroups SubExp ->
@@ -394,7 +394,7 @@
                   [localArrayIndex constants t]
 
 scanStage3 ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   Count NumGroups SubExp ->
   Count GroupSize SubExp ->
   Imp.TExp Int64 ->
@@ -481,7 +481,7 @@
 -- | Compile 'SegScan' instance to host-level code with calls to
 -- various kernels.
 compileSegScan ::
-  Pat GPUMem ->
+  Pat LetDecMem ->
   SegLevel ->
   SegSpace ->
   [SegBinOp GPUMem] ->
diff --git a/src/Futhark/CodeGen/ImpGen/GPU/ToOpenCL.hs b/src/Futhark/CodeGen/ImpGen/GPU/ToOpenCL.hs
--- a/src/Futhark/CodeGen/ImpGen/GPU/ToOpenCL.hs
+++ b/src/Futhark/CodeGen/ImpGen/GPU/ToOpenCL.hs
@@ -3,7 +3,7 @@
 {-# LANGUAGE TupleSections #-}
 
 -- | This module defines a translation from imperative code with
--- kernels to imperative code with OpenCL calls.
+-- kernels to imperative code with OpenCL or CUDA calls.
 module Futhark.CodeGen.ImpGen.GPU.ToOpenCL
   ( kernelsToOpenCL,
     kernelsToCUDA,
@@ -33,8 +33,12 @@
 import qualified Language.C.Syntax as C
 import NeatInterpolation (untrimming)
 
-kernelsToCUDA, kernelsToOpenCL :: ImpGPU.Program -> ImpOpenCL.Program
+-- | Generate CUDA host and device code.
+kernelsToCUDA :: ImpGPU.Program -> ImpOpenCL.Program
 kernelsToCUDA = translateGPU TargetCUDA
+
+-- | Generate OpenCL host and device code.
+kernelsToOpenCL :: ImpGPU.Program -> ImpOpenCL.Program
 kernelsToOpenCL = translateGPU TargetOpenCL
 
 -- | Translate a kernels-program to an OpenCL-program.
@@ -134,7 +138,7 @@
 
 type AllFunctions = ImpGPU.Functions ImpGPU.HostOp
 
-lookupFunction :: Name -> AllFunctions -> Maybe ImpGPU.Function
+lookupFunction :: Name -> AllFunctions -> Maybe (ImpGPU.Function HostOp)
 lookupFunction fname (ImpGPU.Functions fs) = lookup fname fs
 
 type OnKernelM = ReaderT AllFunctions (State ToOpenCL)
@@ -168,14 +172,9 @@
 
 -- Compilation of a device function that is not not invoked from the
 -- host, but is invoked by (perhaps multiple) kernels.
-generateDeviceFun :: Name -> ImpGPU.Function -> OnKernelM ()
-generateDeviceFun fname host_func = do
-  -- Functions are a priori always considered host-level, so we have
-  -- to convert them to device code.  This is where most of our
-  -- limitations on device-side functions (no arrays, no parallelism)
-  -- comes from.
-  let device_func = fmap toDevice host_func
-  when (any memParam $ functionInput host_func) bad
+generateDeviceFun :: Name -> ImpGPU.Function ImpGPU.KernelOp -> OnKernelM ()
+generateDeviceFun fname device_func = do
+  when (any memParam $ functionInput device_func) bad
 
   failures <- gets clFailures
 
@@ -199,9 +198,6 @@
   -- right clFailures.
   void $ ensureDeviceFuns $ functionBody device_func
   where
-    toDevice :: HostOp -> KernelOp
-    toDevice _ = bad
-
     memParam MemParam {} = True
     memParam ScalarParam {} = False
 
@@ -209,7 +205,7 @@
 
 -- Ensure that this device function is available, but don't regenerate
 -- it if it already exists.
-ensureDeviceFun :: Name -> ImpGPU.Function -> OnKernelM ()
+ensureDeviceFun :: Name -> ImpGPU.Function ImpGPU.KernelOp -> OnKernelM ()
 ensureDeviceFun fname host_func = do
   exists <- gets $ M.member fname . clDevFuns
   unless exists $ generateDeviceFun fname host_func
@@ -221,10 +217,19 @@
     forM (S.toList called) $ \fname -> do
       def <- asks $ lookupFunction fname
       case def of
-        Just func -> do
-          ensureDeviceFun fname func
+        Just host_func -> do
+          -- Functions are a priori always considered host-level, so we have
+          -- to convert them to device code.  This is where most of our
+          -- limitations on device-side functions (no arrays, no parallelism)
+          -- comes from.
+          let device_func = fmap toDevice host_func
+          ensureDeviceFun fname device_func
           return $ Just fname
         Nothing -> return Nothing
+  where
+    bad = compilerLimitationS "Cannot generate GPU functions that contain parallelism."
+    toDevice :: HostOp -> KernelOp
+    toDevice _ = bad
 
 onKernel :: KernelTarget -> Kernel -> OnKernelM OpenCL
 onKernel target kernel = do
diff --git a/src/Futhark/CodeGen/ImpGen/Multicore.hs b/src/Futhark/CodeGen/ImpGen/Multicore.hs
--- a/src/Futhark/CodeGen/ImpGen/Multicore.hs
+++ b/src/Futhark/CodeGen/ImpGen/Multicore.hs
@@ -83,7 +83,7 @@
                 error $ "Missing locks for " ++ pretty acc
 
 withAcc ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   [(Shape, [VName], Maybe (Lambda MCMem, [SubExp]))] ->
   Lambda MCMem ->
   MulticoreGen ()
@@ -116,7 +116,7 @@
   defCompileExp dest e
 
 compileMCOp ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   MCOp MCMem () ->
   ImpM MCMem HostEnv Imp.Multicore ()
 compileMCOp _ (OtherOp ()) = pure ()
@@ -151,10 +151,10 @@
       scheduling_info (decideScheduling' op seq_code)
 
 compileSegOp ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegOp () MCMem ->
   TV Int32 ->
-  ImpM MCMem HostEnv Imp.Multicore Imp.Code
+  ImpM MCMem HostEnv Imp.Multicore Imp.MCCode
 compileSegOp pat (SegHist _ space histops _ kbody) ntasks =
   compileSegHist pat space histops kbody ntasks
 compileSegOp pat (SegScan _ space scans _ kbody) ntasks =
diff --git a/src/Futhark/CodeGen/ImpGen/Multicore/Base.hs b/src/Futhark/CodeGen/ImpGen/Multicore/Base.hs
--- a/src/Futhark/CodeGen/ImpGen/Multicore/Base.hs
+++ b/src/Futhark/CodeGen/ImpGen/Multicore/Base.hs
@@ -13,6 +13,7 @@
     renameHistOpLambda,
     atomicUpdateLocking,
     AtomicUpdate (..),
+    DoAtomicUpdate,
     Locking (..),
     getSpace,
     getLoopBounds,
@@ -103,7 +104,7 @@
 
 -- When the SegRed's return value is a scalar
 -- we perform a call by value-result in the segop function
-getReturnParams :: Pat MCMem -> SegOp () MCMem -> MulticoreGen [Imp.Param]
+getReturnParams :: Pat LetDecMem -> SegOp () MCMem -> MulticoreGen [Imp.Param]
 getReturnParams pat SegRed {} =
   -- It's a good idea to make sure any prim values are initialised, as
   -- we will load them (redundantly) in the task code, and
@@ -123,7 +124,7 @@
 
 compileThreadResult ::
   SegSpace ->
-  PatElem MCMem ->
+  PatElem LetDecMem ->
   KernelResult ->
   MulticoreGen ()
 compileThreadResult space pe (Returns _ _ what) = do
@@ -156,7 +157,7 @@
       let full_shape = Shape [num_threads] <> shape <> arrayShape t
       sAllocArray s (elemType t) full_shape DefaultSpace
 
-isLoadBalanced :: Imp.Code -> Bool
+isLoadBalanced :: Imp.MCCode -> Bool
 isLoadBalanced (a Imp.:>>: b) = isLoadBalanced a && isLoadBalanced b
 isLoadBalanced (Imp.For _ _ a) = isLoadBalanced a
 isLoadBalanced (Imp.If _ a b) = isLoadBalanced a && isLoadBalanced b
@@ -168,7 +169,7 @@
 segBinOpComm' :: [SegBinOp rep] -> Commutativity
 segBinOpComm' = mconcat . map segBinOpComm
 
-decideScheduling' :: SegOp () rep -> Imp.Code -> Imp.Scheduling
+decideScheduling' :: SegOp () rep -> Imp.MCCode -> Imp.Scheduling
 decideScheduling' SegHist {} _ = Imp.Static
 decideScheduling' SegScan {} _ = Imp.Static
 decideScheduling' (SegRed _ _ reds _ _) code =
@@ -177,16 +178,16 @@
     Noncommutative -> Imp.Static
 decideScheduling' SegMap {} code = decideScheduling code
 
-decideScheduling :: Imp.Code -> Imp.Scheduling
+decideScheduling :: Imp.MCCode -> Imp.Scheduling
 decideScheduling code =
   if isLoadBalanced code
     then Imp.Static
     else Imp.Dynamic
 
 -- | Try to extract invariant allocations.  If we assume that the
--- given 'Imp.Code' is the body of a 'SegOp', then it is always safe
+-- given 'Imp.MCCode' is the body of a 'SegOp', then it is always safe
 -- to move the immediate allocations to the prebody.
-extractAllocations :: Imp.Code -> (Imp.Code, Imp.Code)
+extractAllocations :: Imp.MCCode -> (Imp.MCCode, Imp.MCCode)
 extractAllocations segop_code = f segop_code
   where
     declared = Imp.declaredIn segop_code
diff --git a/src/Futhark/CodeGen/ImpGen/Multicore/SegHist.hs b/src/Futhark/CodeGen/ImpGen/Multicore/SegHist.hs
--- a/src/Futhark/CodeGen/ImpGen/Multicore/SegHist.hs
+++ b/src/Futhark/CodeGen/ImpGen/Multicore/SegHist.hs
@@ -16,12 +16,12 @@
 import Prelude hiding (quot, rem)
 
 compileSegHist ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [HistOp MCMem] ->
   KernelBody MCMem ->
   TV Int32 ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 compileSegHist pat space histops kbody nsubtasks
   | [_] <- unSegSpace space =
     nonsegmentedHist pat space histops kbody nsubtasks
@@ -37,12 +37,12 @@
 histSize = product . map toInt64Exp . shapeDims . histShape
 
 nonsegmentedHist ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [HistOp MCMem] ->
   KernelBody MCMem ->
   TV Int32 ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 nonsegmentedHist pat space histops kbody num_histos = do
   let ns = map snd $ unSegSpace space
       ns_64 = map toInt64Exp ns
@@ -91,7 +91,7 @@
       return $ f l'
 
 atomicHistogram ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [HistOp MCMem] ->
   KernelBody MCMem ->
@@ -161,7 +161,7 @@
 -- across the histogram indicies.
 -- This is expected to be fast if len(histDest) is small
 subHistogram ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [HistOp MCMem] ->
   TV Int32 ->
@@ -287,11 +287,11 @@
 -- parallelize over the segments,
 -- where each segment is updated sequentially.
 segmentedHist ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [HistOp MCMem] ->
   KernelBody MCMem ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 segmentedHist pat space histops kbody = do
   emit $ Imp.DebugPrint "Segmented segHist" Nothing
   collect $ do
@@ -300,11 +300,11 @@
     emit $ Imp.Op $ Imp.ParLoop "segmented_hist" body free_params
 
 compileSegHistBody ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [HistOp MCMem] ->
   KernelBody MCMem ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 compileSegHistBody pat space histops kbody = collect $ do
   let (is, ns) = unzip $ unSegSpace space
       ns_64 = map toInt64Exp ns
diff --git a/src/Futhark/CodeGen/ImpGen/Multicore/SegMap.hs b/src/Futhark/CodeGen/ImpGen/Multicore/SegMap.hs
--- a/src/Futhark/CodeGen/ImpGen/Multicore/SegMap.hs
+++ b/src/Futhark/CodeGen/ImpGen/Multicore/SegMap.hs
@@ -13,7 +13,7 @@
 
 writeResult ::
   [VName] ->
-  PatElemT dec ->
+  PatElem dec ->
   KernelResult ->
   MulticoreGen ()
 writeResult is pe (Returns _ _ se) =
@@ -29,10 +29,10 @@
   error $ "writeResult: cannot handle " ++ pretty res
 
 compileSegMapBody ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   KernelBody MCMem ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 compileSegMapBody pat space (KernelBody _ kstms kres) = collect $ do
   let (is, ns) = unzip $ unSegSpace space
       ns' = map toInt64Exp ns
@@ -45,10 +45,10 @@
       zipWithM_ (writeResult is) (patElems pat) kres
 
 compileSegMap ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   KernelBody MCMem ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 compileSegMap pat space kbody = collect $ do
   body <- compileSegMapBody pat space kbody
   free_params <- freeParams body
diff --git a/src/Futhark/CodeGen/ImpGen/Multicore/SegRed.hs b/src/Futhark/CodeGen/ImpGen/Multicore/SegRed.hs
--- a/src/Futhark/CodeGen/ImpGen/Multicore/SegRed.hs
+++ b/src/Futhark/CodeGen/ImpGen/Multicore/SegRed.hs
@@ -1,6 +1,7 @@
 module Futhark.CodeGen.ImpGen.Multicore.SegRed
   ( compileSegRed,
     compileSegRed',
+    DoSegBody,
   )
 where
 
@@ -16,12 +17,12 @@
 
 -- | Generate code for a SegRed construct
 compileSegRed ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [SegBinOp MCMem] ->
   KernelBody MCMem ->
   TV Int32 ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 compileSegRed pat space reds kbody nsubtasks =
   compileSegRed' pat space reds nsubtasks $ \red_cont ->
     compileStms mempty (kernelBodyStms kbody) $ do
@@ -35,12 +36,12 @@
 
 -- | Like 'compileSegRed', but where the body is a monadic action.
 compileSegRed' ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [SegBinOp MCMem] ->
   TV Int32 ->
   DoSegBody ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 compileSegRed' pat space reds nsubtasks kbody
   | [_] <- unSegSpace space =
     nonsegmentedReduction pat space reds nsubtasks kbody
@@ -72,12 +73,12 @@
 nextParams slug = drop (length (slugNeutral slug)) $ slugParams slug
 
 nonsegmentedReduction ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [SegBinOp MCMem] ->
   TV Int32 ->
   DoSegBody ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 nonsegmentedReduction pat space reds nsubtasks kbody = collect $ do
   thread_res_arrs <- groupResultArrays "reduce_stage_1_tid_res_arr" (tvSize nsubtasks) reds
   let slugs1 = zipWith SegBinOpSlug reds thread_res_arrs
@@ -163,7 +164,7 @@
   emit $ Imp.Op $ Imp.ParLoop "segred_stage_1" fbody free_params
 
 reductionStage2 ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   Imp.TExp Int32 ->
   [SegBinOpSlug] ->
@@ -200,11 +201,11 @@
 -- Maybe we should select the work of the inner loop
 -- based on n_segments and dimensions etc.
 segmentedReduction ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [SegBinOp MCMem] ->
   DoSegBody ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 segmentedReduction pat space reds kbody =
   collect $ do
     body <- compileSegRedBody pat space reds kbody
@@ -212,11 +213,11 @@
     emit $ Imp.Op $ Imp.ParLoop "segmented_segred" body free_params
 
 compileSegRedBody ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [SegBinOp MCMem] ->
   DoSegBody ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 compileSegRedBody pat space reds kbody = do
   let (is, ns) = unzip $ unSegSpace space
       ns_64 = map toInt64Exp ns
diff --git a/src/Futhark/CodeGen/ImpGen/Multicore/SegScan.hs b/src/Futhark/CodeGen/ImpGen/Multicore/SegScan.hs
--- a/src/Futhark/CodeGen/ImpGen/Multicore/SegScan.hs
+++ b/src/Futhark/CodeGen/ImpGen/Multicore/SegScan.hs
@@ -14,12 +14,12 @@
 
 -- Compile a SegScan construct
 compileSegScan ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [SegBinOp MCMem] ->
   KernelBody MCMem ->
   TV Int32 ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 compileSegScan pat space reds kbody nsubtasks
   | [_] <- unSegSpace space =
     nonsegmentedScan pat space reds kbody nsubtasks
@@ -45,12 +45,12 @@
       sAllocArray s pt full_shape DefaultSpace
 
 nonsegmentedScan ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [SegBinOp MCMem] ->
   KernelBody MCMem ->
   TV Int32 ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 nonsegmentedScan pat space scan_ops kbody nsubtasks = do
   emit $ Imp.DebugPrint "nonsegmented segScan" Nothing
   collect $ do
@@ -64,7 +64,7 @@
       scanStage3 pat space scan_ops3 kbody
 
 scanStage1 ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [SegBinOp MCMem] ->
   KernelBody MCMem ->
@@ -129,7 +129,7 @@
   emit $ Imp.Op $ Imp.ParLoop "scan_stage_1" body free_params
 
 scanStage2 ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   TV Int32 ->
   SegSpace ->
   [SegBinOp MCMem] ->
@@ -187,7 +187,7 @@
 -- Stage 3 : Finally each thread partially scans a chunk of the input
 --           reading its corresponding carry-in
 scanStage3 ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [SegBinOp MCMem] ->
   KernelBody MCMem ->
@@ -252,11 +252,11 @@
 -- parallelize over the segments and each segment is
 -- scanned sequentially.
 segmentedScan ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [SegBinOp MCMem] ->
   KernelBody MCMem ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 segmentedScan pat space scan_ops kbody = do
   emit $ Imp.DebugPrint "segmented segScan" Nothing
   collect $ do
@@ -265,11 +265,11 @@
     emit $ Imp.Op $ Imp.ParLoop "seg_scan" body free_params
 
 compileSegScanBody ::
-  Pat MCMem ->
+  Pat LetDecMem ->
   SegSpace ->
   [SegBinOp MCMem] ->
   KernelBody MCMem ->
-  MulticoreGen Imp.Code
+  MulticoreGen Imp.MCCode
 compileSegScanBody pat space scan_ops kbody = collect $ do
   let (is, ns) = unzip $ unSegSpace space
       ns_64 = map toInt64Exp ns
diff --git a/src/Futhark/Compiler/Config.hs b/src/Futhark/Compiler/Config.hs
--- a/src/Futhark/Compiler/Config.hs
+++ b/src/Futhark/Compiler/Config.hs
@@ -1,3 +1,4 @@
+-- | Configuration of compiler behaviour that is universal to all backends.
 module Futhark.Compiler.Config
   ( FutharkConfig (..),
     newFutharkConfig,
diff --git a/src/Futhark/Compiler/Program.hs b/src/Futhark/Compiler/Program.hs
--- a/src/Futhark/Compiler/Program.hs
+++ b/src/Futhark/Compiler/Program.hs
@@ -1,4 +1,5 @@
 {-# LANGUAGE FlexibleContexts #-}
+{-# LANGUAGE LambdaCase #-}
 {-# LANGUAGE OverloadedStrings #-}
 
 -- | Low-level compilation parts.  Look at "Futhark.Compiler" for a
@@ -6,142 +7,227 @@
 module Futhark.Compiler.Program
   ( readLibrary,
     readUntypedLibrary,
-    readImports,
     Imports,
     FileModule (..),
     E.Warnings,
-    Basis (..),
-    emptyBasis,
+    LoadedProg (lpNameSource),
+    noLoadedProg,
+    lpImports,
+    reloadProg,
+    extendProg,
   )
 where
 
+import Control.Concurrent (forkIO)
+import Control.Concurrent.MVar
+  ( MVar,
+    modifyMVar,
+    newEmptyMVar,
+    newMVar,
+    putMVar,
+    readMVar,
+  )
 import Control.Monad
 import Control.Monad.Except
-import Control.Monad.State
-import Data.List (intercalate, isPrefixOf)
-import Data.Maybe
+import Control.Monad.State (execStateT, gets, modify)
+import Data.Bifunctor (first)
+import Data.List (intercalate, isPrefixOf, sort)
+import qualified Data.Map as M
+import Data.Maybe (mapMaybe)
 import qualified Data.Text as T
+import qualified Data.Text.IO as T
+import Data.Time.Clock (UTCTime)
 import Futhark.Error
 import Futhark.FreshNames
-import Futhark.Util (readFileSafely)
-import Futhark.Util.Pretty (line, ppr, (</>))
+import Futhark.Util (interactWithFileSafely, nubOrd, startupTime)
+import Futhark.Util.Pretty (line, ppr, text, (</>))
 import qualified Language.Futhark as E
 import Language.Futhark.Parser
 import Language.Futhark.Prelude
 import Language.Futhark.Semantic
 import qualified Language.Futhark.TypeChecker as E
 import Language.Futhark.Warnings
+import System.Directory (getModificationTime)
 import System.FilePath (normalise)
 import qualified System.FilePath.Posix as Posix
 
-newtype ReaderState = ReaderState
-  {alreadyRead :: [(ImportName, E.UncheckedProg)]}
+data LoadedFile fm = LoadedFile
+  { lfPath :: FilePath,
+    lfImportName :: ImportName,
+    lfMod :: fm,
+    -- | Modification time of the underlying file.
+    lfModTime :: UTCTime
+  }
+  deriving (Eq, Ord, Show)
 
--- | A little monad for parsing a Futhark program.
-type ReaderM m = StateT ReaderState m
+newtype UncheckedImport = UncheckedImport
+  { unChecked ::
+      Either CompilerError (LoadedFile E.UncheckedProg, [(ImportName, MVar UncheckedImport)])
+  }
 
-runReaderM ::
-  (MonadError CompilerError m) =>
-  ReaderM m a ->
-  m [(ImportName, E.UncheckedProg)]
-runReaderM m = reverse . alreadyRead <$> execStateT m (ReaderState mempty)
+-- | If mapped to Nothing, treat it as present.  This is used when
+-- reloading programs.
+type ReaderState = MVar (M.Map ImportName (Maybe (MVar UncheckedImport)))
 
-readImportFile ::
+newState :: [ImportName] -> IO ReaderState
+newState known = newMVar $ M.fromList $ zip known $ repeat Nothing
+
+orderedImports ::
   (MonadError CompilerError m, MonadIO m) =>
-  ImportName ->
-  ReaderM m (T.Text, FilePath)
+  [(ImportName, MVar UncheckedImport)] ->
+  m [(ImportName, LoadedFile E.UncheckedProg)]
+orderedImports = fmap reverse . flip execStateT [] . mapM_ (spelunk [])
+  where
+    spelunk steps (include, mvar)
+      | include `elem` steps =
+        externalErrorS $
+          "Import cycle: "
+            ++ intercalate
+              " -> "
+              (map includeToString $ reverse $ include : steps)
+      | otherwise = do
+        prev <- gets $ lookup include
+        case prev of
+          Just _ -> pure ()
+          Nothing -> do
+            (file, more_imports) <-
+              either throwError pure . unChecked =<< liftIO (readMVar mvar)
+            mapM_ (spelunk (include : steps)) more_imports
+            modify ((include, file) :)
+
+newImportMVar :: IO UncheckedImport -> IO (Maybe (MVar UncheckedImport))
+newImportMVar m = do
+  mvar <- newEmptyMVar
+  void $ forkIO $ putMVar mvar =<< m
+  pure $ Just mvar
+
+contentsAndModTime :: FilePath -> IO (Maybe (Either String (T.Text, UTCTime)))
+contentsAndModTime filepath =
+  interactWithFileSafely $
+    (,) <$> T.readFile filepath <*> getModificationTime filepath
+
+readImportFile :: ImportName -> IO (Either CompilerError (LoadedFile T.Text))
 readImportFile include = do
   -- First we try to find a file of the given name in the search path,
   -- then we look at the builtin library if we have to.  For the
   -- builtins, we don't use the search path.
   let filepath = includeToFilePath include
-  r <- liftIO $ readFileSafely filepath
+  r <- contentsAndModTime filepath
   case (r, lookup prelude_str prelude) of
-    (Just (Right s), _) -> return (s, filepath)
-    (Just (Left e), _) -> externalErrorS e
-    (Nothing, Just t) -> return (t, prelude_str)
-    (Nothing, Nothing) -> externalErrorS not_found
+    (Just (Right (s, mod_time)), _) ->
+      pure $ Right $ loaded filepath s mod_time
+    (Just (Left e), _) -> pure $ Left $ ExternalError $ text e
+    (Nothing, Just s) ->
+      pure $ Right $ loaded prelude_str s startupTime
+    (Nothing, Nothing) -> pure $ Left $ ExternalError $ text not_found
   where
     prelude_str = "/" Posix.</> includeToString include Posix.<.> "fut"
 
+    loaded path s mod_time =
+      LoadedFile
+        { lfImportName = include,
+          lfPath = path,
+          lfMod = s,
+          lfModTime = mod_time
+        }
+
     not_found =
       "Error at " ++ E.locStr (E.srclocOf include)
         ++ ": could not find import '"
         ++ includeToString include
         ++ "'."
 
-readImport ::
-  (MonadError CompilerError m, MonadIO m) =>
-  [ImportName] ->
-  ImportName ->
-  ReaderM m ()
-readImport steps include
-  | include `elem` steps =
-    externalErrorS $
-      "Import cycle: "
-        ++ intercalate
-          " -> "
-          (map includeToString $ reverse $ include : steps)
-  | otherwise = do
-    already_done <- gets $ isJust . lookup include . alreadyRead
+handleFile ::
+  ReaderState -> LoadedFile T.Text -> IO UncheckedImport
+handleFile state_mvar (LoadedFile file_name import_name file_contents mod_time) = do
+  case parseFuthark file_name file_contents of
+    Left err -> pure $ UncheckedImport $ Left $ ExternalError $ text $ show err
+    Right prog -> do
+      let imports = map (uncurry (mkImportFrom import_name)) $ E.progImports prog
+      mvars <-
+        mapMaybe sequenceA . zip imports
+          <$> mapM (readImport state_mvar) imports
+      let file =
+            LoadedFile
+              { lfPath = file_name,
+                lfImportName = import_name,
+                lfModTime = mod_time,
+                lfMod = prog
+              }
+      pure $ UncheckedImport $ Right (file, mvars)
 
-    unless already_done $
-      uncurry (handleFile steps include) =<< readImportFile include
+readImport :: ReaderState -> ImportName -> IO (Maybe (MVar UncheckedImport))
+readImport state_mvar include =
+  modifyMVar state_mvar $ \state ->
+    case M.lookup include state of
+      Just x -> pure (state, x)
+      Nothing -> do
+        prog_mvar <- newImportMVar $ do
+          readImportFile include >>= \case
+            Left e -> pure $ UncheckedImport $ Left e
+            Right file -> handleFile state_mvar file
+        pure (M.insert include prog_mvar state, prog_mvar)
 
-handleFile ::
+readUntypedLibraryExceptKnown ::
   (MonadIO m, MonadError CompilerError m) =>
   [ImportName] ->
-  ImportName ->
-  T.Text ->
-  FilePath ->
-  ReaderM m ()
-handleFile steps import_name file_contents file_name = do
-  prog <- case parseFuthark file_name file_contents of
-    Left err -> externalErrorS $ show err
-    Right prog -> return prog
-
-  let steps' = import_name : steps
-  mapM_ (readImport steps' . uncurry (mkImportFrom import_name)) $
-    E.progImports prog
-
-  modify $ \s ->
-    s {alreadyRead = (import_name, prog) : alreadyRead s}
-
--- | Pre-typechecked imports, including a starting point for the name source.
-data Basis = Basis
-  { basisImports :: Imports,
-    basisNameSource :: VNameSource,
-    -- | Files that should be implicitly opened.
-    basisRoots :: [String]
-  }
+  [FilePath] ->
+  m [LoadedFile E.UncheckedProg]
+readUntypedLibraryExceptKnown known fps = do
+  state_mvar <- liftIO $ newState known
+  let prelude_import = mkInitialImport "/prelude/prelude"
+  prelude_mvar <- liftIO $ readImport state_mvar prelude_import
+  fps_mvars <- liftIO (mapM (onFile state_mvar) fps)
+  let unknown_mvars = onlyUnknown ((prelude_import, prelude_mvar) : fps_mvars)
+  map snd <$> orderedImports unknown_mvars
+  where
+    onlyUnknown = mapMaybe sequenceA
+    onFile state_mvar fp =
+      modifyMVar state_mvar $ \state -> do
+        case M.lookup include state of
+          Just prog_mvar -> pure (state, (include, prog_mvar))
+          Nothing -> do
+            prog_mvar <- newImportMVar $ do
+              r <- contentsAndModTime fp
+              case r of
+                Just (Right (fs, mod_time)) -> do
+                  handleFile state_mvar $
+                    LoadedFile
+                      { lfImportName = include,
+                        lfMod = fs,
+                        lfModTime = mod_time,
+                        lfPath = fp
+                      }
+                Just (Left e) ->
+                  pure $ UncheckedImport $ Left $ ExternalError $ text $ show e
+                Nothing ->
+                  pure $ UncheckedImport $ Left $ ExternalError $ text $ fp <> ": file not found."
+            pure (M.insert include prog_mvar state, (include, prog_mvar))
+      where
+        include = mkInitialImport fp_name
+        (fp_name, _) = Posix.splitExtension fp
 
--- | A basis that contains no imports, and has a properly initialised
--- name source.
-emptyBasis :: Basis
-emptyBasis =
-  Basis
-    { basisImports = mempty,
-      basisNameSource = src,
-      basisRoots = mempty
-    }
+asImports :: [LoadedFile (VNameSource, FileModule)] -> Imports
+asImports = map f
   where
-    src = newNameSource $ E.maxIntrinsicTag + 1
+    f lf = (includeToString (lfImportName lf), snd $ lfMod lf)
 
-typeCheckProgram ::
+typeCheckProg ::
   MonadError CompilerError m =>
-  Basis ->
-  [(ImportName, E.UncheckedProg)] ->
-  m (E.Warnings, Imports, VNameSource)
-typeCheckProgram basis =
-  foldM f (mempty, basisImports basis, basisNameSource basis)
+  [LoadedFile (VNameSource, FileModule)] ->
+  VNameSource ->
+  [LoadedFile E.UncheckedProg] ->
+  m (E.Warnings, [LoadedFile (VNameSource, FileModule)], VNameSource)
+typeCheckProg orig_imports orig_src =
+  foldM f (mempty, orig_imports, orig_src)
   where
     roots = ["/prelude/prelude"]
 
-    f (ws, imports, src) (import_name, prog) = do
+    f (ws, imports, src) (LoadedFile path import_name prog mod_time) = do
       let prog'
             | "/prelude" `isPrefixOf` includeToFilePath import_name = prog
             | otherwise = prependRoots roots prog
-      case E.checkProg imports src import_name prog' of
+      case E.checkProg (asImports imports) src import_name prog' of
         (prog_ws, Left err) -> do
           let ws' = ws <> prog_ws
           externalError $
@@ -151,49 +237,123 @@
         (prog_ws, Right (m, src')) ->
           pure
             ( ws <> prog_ws,
-              imports ++ [(includeToString import_name, m)],
+              imports ++ [LoadedFile path import_name (src, m) mod_time],
               src'
             )
 
 setEntryPoints ::
   [E.Name] ->
   [FilePath] ->
-  [(ImportName, E.UncheckedProg)] ->
-  [(ImportName, E.UncheckedProg)]
-setEntryPoints extra_eps fps = map onProg
+  [LoadedFile E.UncheckedProg] ->
+  [LoadedFile E.UncheckedProg]
+setEntryPoints extra_eps fps = map onFile
   where
     fps' = map normalise fps
-    onProg (name, prog)
-      | includeToFilePath name `elem` fps' =
-        (name, prog {E.progDecs = map onDec (E.progDecs prog)})
+    onFile lf
+      | includeToFilePath (lfImportName lf) `elem` fps' =
+        lf {lfMod = prog {E.progDecs = map onDec (E.progDecs prog)}}
       | otherwise =
-        (name, prog)
+        lf
+      where
+        prog = lfMod lf
 
     onDec (E.ValDec vb)
       | E.valBindName vb `elem` extra_eps =
         E.ValDec vb {E.valBindEntryPoint = Just E.NoInfo}
     onDec dec = dec
 
--- | Read (and parse) all source files (including the builtin prelude)
--- corresponding to a set of root files.
-readUntypedLibrary ::
-  (MonadIO m, MonadError CompilerError m) =>
-  [FilePath] ->
-  m [(ImportName, E.UncheckedProg)]
-readUntypedLibrary fps = runReaderM $ do
-  readImport [] (mkInitialImport "/prelude/prelude")
-  mapM_ onFile fps
+prependRoots :: [FilePath] -> E.UncheckedProg -> E.UncheckedProg
+prependRoots roots (E.Prog doc ds) =
+  E.Prog doc $ map mkImport roots ++ ds
   where
-    onFile fp = do
-      r <- liftIO $ readFileSafely fp
-      case r of
-        Just (Right fs) ->
-          handleFile [] (mkInitialImport fp_name) fs fp
-        Just (Left e) -> externalErrorS e
-        Nothing -> externalErrorS $ fp ++ ": file not found."
-      where
-        (fp_name, _) = Posix.splitExtension fp
+    mkImport fp =
+      -- We do not use ImportDec here, because we do not want the
+      -- type checker to issue a warning about a redundant import.
+      E.LocalDec (E.OpenDec (E.ModImport fp E.NoInfo mempty) mempty) mempty
 
+-- | A loaded, type-checked program.  This can be used to extract
+-- information about the program, but also to speed up subsequent
+-- reloads.
+data LoadedProg = LoadedProg
+  { lpRoots :: [FilePath],
+    -- | The 'VNameSource' is the name source just *before* the module
+    -- was type checked.
+    lpFiles :: [LoadedFile (VNameSource, FileModule)],
+    -- | Final name source.
+    lpNameSource :: VNameSource
+  }
+
+-- | The 'Imports' of a 'LoadedProg', as expected by e.g. type
+-- checking functions.
+lpImports :: LoadedProg -> Imports
+lpImports = map f . lpFiles
+  where
+    f lf = (includeToString (lfImportName lf), snd $ lfMod lf)
+
+unchangedImports ::
+  MonadIO m =>
+  VNameSource ->
+  [LoadedFile (VNameSource, FileModule)] ->
+  m ([LoadedFile (VNameSource, FileModule)], VNameSource)
+unchangedImports src [] = pure ([], src)
+unchangedImports src (f : fs)
+  | "/prelude" `isPrefixOf` includeToFilePath (lfImportName f) =
+    first (f :) <$> unchangedImports src fs
+  | otherwise = do
+    changed <-
+      maybe True (either (const True) (> lfModTime f))
+        <$> liftIO (interactWithFileSafely (getModificationTime $ lfPath f))
+    if changed
+      then pure ([], fst $ lfMod f)
+      else first (f :) <$> unchangedImports src fs
+
+-- | A "loaded program" containing no actual files.  Use this as a
+-- starting point for 'reloadProg'
+noLoadedProg :: LoadedProg
+noLoadedProg =
+  LoadedProg
+    { lpRoots = [],
+      lpFiles = mempty,
+      lpNameSource = newNameSource $ E.maxIntrinsicTag + 1
+    }
+
+-- | Find out how many of the old imports can be used.  Here we are
+-- forced to be overly conservative, because our type checker
+-- enforces a linear ordering.
+usableLoadedProg :: MonadIO m => LoadedProg -> [FilePath] -> m LoadedProg
+usableLoadedProg (LoadedProg roots imports src) new_roots
+  | sort roots == sort new_roots = do
+    (imports', src') <- unchangedImports src imports
+    pure $ LoadedProg [] imports' src'
+  | otherwise =
+    pure noLoadedProg
+
+-- | Extend a loaded program with (possibly new) files.
+extendProg ::
+  (MonadError CompilerError m, MonadIO m) =>
+  LoadedProg ->
+  [FilePath] ->
+  m (E.Warnings, LoadedProg)
+extendProg lp new_roots = do
+  new_imports_untyped <-
+    readUntypedLibraryExceptKnown (map lfImportName $ lpFiles lp) new_roots
+  (ws, imports, src') <-
+    typeCheckProg (lpFiles lp) (lpNameSource lp) new_imports_untyped
+  pure (ws, LoadedProg (nubOrd (lpRoots lp ++ new_roots)) imports src')
+
+-- | Load some new files, reusing as much of the previously loaded
+-- program as possible.  This does not *extend* the currently loaded
+-- program the way 'extendProg' does it, so it is always correct (if
+-- less efficient) to pass 'noLoadedProg'.
+reloadProg ::
+  (MonadError CompilerError m, MonadIO m) =>
+  LoadedProg ->
+  [FilePath] ->
+  m (E.Warnings, LoadedProg)
+reloadProg lp new_roots = do
+  lp' <- usableLoadedProg lp new_roots
+  extendProg lp' new_roots
+
 -- | Read and type-check some Futhark files.
 readLibrary ::
   (MonadError CompilerError m, MonadIO m) =>
@@ -204,30 +364,19 @@
   [FilePath] ->
   m (E.Warnings, Imports, VNameSource)
 readLibrary extra_eps fps =
-  typeCheckProgram emptyBasis . setEntryPoints (E.defaultEntryPoint : extra_eps) fps
-    =<< readUntypedLibrary fps
-
--- | Read and type-check Futhark imports (no @.fut@ extension; may
--- refer to baked-in prelude).  This is an exotic operation that
--- probably only makes sense in an interactive environment.
-readImports ::
-  (MonadError CompilerError m, MonadIO m) =>
-  Basis ->
-  [ImportName] ->
-  m
-    ( E.Warnings,
-      Imports,
-      VNameSource
-    )
-readImports basis imps = do
-  files <- runReaderM $ mapM (readImport []) imps
-  typeCheckProgram basis files
+  fmap frob
+    . typeCheckProg mempty (lpNameSource noLoadedProg)
+    . setEntryPoints (E.defaultEntryPoint : extra_eps) fps
+    =<< readUntypedLibraryExceptKnown [] fps
+  where
+    frob (x, y, z) = (x, asImports y, z)
 
-prependRoots :: [FilePath] -> E.UncheckedProg -> E.UncheckedProg
-prependRoots roots (E.Prog doc ds) =
-  E.Prog doc $ map mkImport roots ++ ds
+-- | Read (and parse) all source files (including the builtin prelude)
+-- corresponding to a set of root files.
+readUntypedLibrary ::
+  (MonadIO m, MonadError CompilerError m) =>
+  [FilePath] ->
+  m [(ImportName, E.UncheckedProg)]
+readUntypedLibrary = fmap (map f) . readUntypedLibraryExceptKnown []
   where
-    mkImport fp =
-      -- We do not use ImportDec here, because we do not want the
-      -- type checker to issue a warning about a redundant import.
-      E.LocalDec (E.OpenDec (E.ModImport fp E.NoInfo mempty) mempty) mempty
+    f lf = (lfImportName lf, lfMod lf)
diff --git a/src/Futhark/Construct.hs b/src/Futhark/Construct.hs
--- a/src/Futhark/Construct.hs
+++ b/src/Futhark/Construct.hs
@@ -445,7 +445,7 @@
         lambdaBody = body
       }
 
--- | Easily construct a 'Lambda' within a 'MonadBuilder'.
+-- | Easily construct a t'Lambda' within a 'MonadBuilder'.
 mkLambda ::
   MonadBuilder m =>
   [LParam (Rep m)] ->
@@ -547,7 +547,7 @@
 -- | Instantiate all existential parts dimensions of the given
 -- type, using a monadic action to create the necessary t'SubExp's.
 -- You should call this function within some monad that allows you to
--- collect the actions performed (say, 'Writer').
+-- collect the actions performed (say, 'State').
 instantiateShapes ::
   Monad m =>
   (Int -> m SubExp) ->
diff --git a/src/Futhark/IR/Aliases.hs b/src/Futhark/IR/Aliases.hs
--- a/src/Futhark/IR/Aliases.hs
+++ b/src/Futhark/IR/Aliases.hs
@@ -224,15 +224,15 @@
 removeLambdaAliases = runIdentity . rephraseLambda removeAliases
 
 removePatAliases ::
-  PatT (AliasDec, a) ->
-  PatT a
+  Pat (AliasDec, a) ->
+  Pat a
 removePatAliases = runIdentity . rephrasePat (return . snd)
 
 addAliasesToPat ::
   (ASTRep rep, CanBeAliased (Op rep), Typed dec) =>
-  PatT dec ->
+  Pat dec ->
   Exp (Aliases rep) ->
-  PatT (VarAliases, dec)
+  Pat (VarAliases, dec)
 addAliasesToPat pat e =
   Pat $ mkPatAliases pat e
 
@@ -247,9 +247,9 @@
 
 mkPatAliases ::
   (Aliased rep, Typed dec) =>
-  PatT dec ->
+  Pat dec ->
   Exp rep ->
-  [PatElemT (VarAliases, dec)]
+  [PatElem (VarAliases, dec)]
 mkPatAliases pat e =
   let als = expAliases e ++ repeat mempty
    in -- In case the pattern has
@@ -331,7 +331,7 @@
 
 mkAliasedLetStm ::
   (ASTRep rep, CanBeAliased (Op rep)) =>
-  Pat rep ->
+  Pat (LetDec rep) ->
   StmAux (ExpDec rep) ->
   Exp (Aliases rep) ->
   Stm (Aliases rep)
diff --git a/src/Futhark/IR/Mem.hs b/src/Futhark/IR/Mem.hs
--- a/src/Futhark/IR/Mem.hs
+++ b/src/Futhark/IR/Mem.hs
@@ -767,7 +767,7 @@
 
 matchPatToExp ::
   (Mem rep inner, LetDec rep ~ LetDecMem, TC.Checkable rep) =>
-  Pat (Aliases rep) ->
+  Pat (LetDec (Aliases rep)) ->
   Exp (Aliases rep) ->
   TC.TypeM rep ()
 matchPatToExp pat e = do
@@ -901,7 +901,7 @@
             ++ ")"
 
 bodyReturnsFromPat ::
-  PatT (MemBound NoUniqueness) -> [(VName, BodyReturns)]
+  Pat (MemBound NoUniqueness) -> [(VName, BodyReturns)]
 bodyReturnsFromPat pat =
   map asReturns $ patElems pat
   where
diff --git a/src/Futhark/IR/Mem/IxFun.hs b/src/Futhark/IR/Mem/IxFun.hs
--- a/src/Futhark/IR/Mem/IxFun.hs
+++ b/src/Futhark/IR/Mem/IxFun.hs
@@ -29,6 +29,7 @@
     existentialize,
     closeEnough,
     equivalent,
+    dynamicEqualsLMAD,
   )
 where
 
@@ -43,11 +44,6 @@
 import qualified Data.Map.Strict as M
 import Data.Maybe (isJust)
 import Futhark.Analysis.PrimExp
-  ( IntExp,
-    PrimExp (..),
-    TPrimExp (..),
-    primExpType,
-  )
 import Futhark.Analysis.PrimExp.Convert (substituteInPrimExp)
 import qualified Futhark.Analysis.PrimExp.Generalize as PEG
 import Futhark.IR.Prop
@@ -1067,3 +1063,25 @@
         == map ldStride (lmadDims lmad2)
         && map ldRotate (lmadDims lmad1)
         == map ldRotate (lmadDims lmad2)
+
+-- | Dynamically determine if two 'LMADDim' are equal.
+--
+-- True if the dynamic values of their constituents are equal.
+dynamicEqualsLMADDim :: Eq num => LMADDim (TPrimExp t num) -> LMADDim (TPrimExp t num) -> TPrimExp Bool num
+dynamicEqualsLMADDim dim1 dim2 =
+  ldStride dim1 .==. ldStride dim2
+    .&&. ldRotate dim1 .==. ldRotate dim2
+    .&&. ldShape dim1 .==. ldShape dim2
+    .&&. fromBool (ldPerm dim1 == ldPerm dim2)
+    .&&. fromBool (ldMon dim1 == ldMon dim2)
+
+-- | Dynamically determine if two 'LMAD' are equal.
+--
+-- True if offset and constituent 'LMADDim' are equal.
+dynamicEqualsLMAD :: Eq num => LMAD (TPrimExp t num) -> LMAD (TPrimExp t num) -> TPrimExp Bool num
+dynamicEqualsLMAD lmad1 lmad2 =
+  lmadOffset lmad1 .==. lmadOffset lmad2
+    .&&. foldr
+      ((.&&.) . uncurry dynamicEqualsLMADDim)
+      true
+      (zip (lmadDims lmad1) (lmadDims lmad2))
diff --git a/src/Futhark/IR/Parse.hs b/src/Futhark/IR/Parse.hs
--- a/src/Futhark/IR/Parse.hs
+++ b/src/Futhark/IR/Parse.hs
@@ -17,6 +17,7 @@
 import Data.Char (isAlpha)
 import Data.Functor
 import Data.List (zipWith5)
+import Data.List.NonEmpty (NonEmpty (..))
 import qualified Data.List.NonEmpty as NE
 import qualified Data.Set as S
 import qualified Data.Text as T
@@ -283,7 +284,9 @@
         d <- "@" *> L.decimal
         parens $ do
           w <- pSubExp <* pComma
-          Concat d <$> pVName <*> many (pComma *> pVName) <*> pure w,
+          x <- pVName
+          ys <- many (pComma *> pVName)
+          return $ Concat d (x :| ys) w,
       pIota,
       try $
         flip Update
@@ -312,6 +315,8 @@
       pConvOp "sitofp" SIToFP pIntType pFloatType,
       pConvOp "itob" (const . IToB) pIntType (keyword "bool"),
       pConvOp "btoi" (const BToI) (keyword "bool") pIntType,
+      pConvOp "ftob" (const . FToB) pFloatType (keyword "bool"),
+      pConvOp "btof" (const BToF) (keyword "bool") pFloatType,
       --
       pIndex,
       pFlatIndex,
@@ -381,11 +386,11 @@
 pLParams :: PR rep -> Parser [LParam rep]
 pLParams pr = braces $ pLParam pr `sepBy` pComma
 
-pPatElem :: PR rep -> Parser (PatElem rep)
+pPatElem :: PR rep -> Parser (PatElem (LetDec rep))
 pPatElem pr =
   (PatElem <$> pVName <*> (pColon *> pLetDec pr)) <?> "pattern element"
 
-pPat :: PR rep -> Parser (Pat rep)
+pPat :: PR rep -> Parser (Pat (LetDec rep))
 pPat pr = Pat <$> braces (pPatElem pr `sepBy` pComma)
 
 pResult :: Parser Result
diff --git a/src/Futhark/IR/Pretty.hs b/src/Futhark/IR/Pretty.hs
--- a/src/Futhark/IR/Pretty.hs
+++ b/src/Futhark/IR/Pretty.hs
@@ -18,6 +18,7 @@
 where
 
 import Data.Foldable (toList)
+import Data.List.NonEmpty (NonEmpty (..))
 import Data.Maybe
 import Futhark.IR.Syntax
 import Futhark.Util.Pretty
@@ -134,10 +135,10 @@
 instance Pretty Attrs where
   ppr = spread . attrAnnots
 
-instance Pretty (PatElemT dec) => Pretty (PatT dec) where
+instance Pretty t => Pretty (Pat t) where
   ppr (Pat xs) = braces $ commastack $ map ppr xs
 
-instance Pretty t => Pretty (PatElemT t) where
+instance Pretty t => Pretty (PatElem t) where
   ppr (PatElem name t) = ppr name <+> colon <+> align (ppr t)
 
 instance Pretty t => Pretty (Param t) where
@@ -217,8 +218,8 @@
     text "rearrange" <> apply [apply (map ppr perm), ppr e]
   ppr (Rotate es e) =
     text "rotate" <> apply [apply (map ppr es), ppr e]
-  ppr (Concat i x ys w) =
-    text "concat" <> text "@" <> ppr i <> apply (ppr w : ppr x : map ppr ys)
+  ppr (Concat i (x :| xs) w) =
+    text "concat" <> text "@" <> ppr i <> apply (ppr w : ppr x : map ppr xs)
   ppr (Copy e) = text "copy" <> parens (ppr e)
   ppr (Manifest perm e) = text "manifest" <> apply [apply (map ppr perm), ppr e]
   ppr (Assert e msg (loc, _)) =
diff --git a/src/Futhark/IR/Primitive.hs b/src/Futhark/IR/Primitive.hs
--- a/src/Futhark/IR/Primitive.hs
+++ b/src/Futhark/IR/Primitive.hs
@@ -102,6 +102,7 @@
     intByteSize,
     floatByteSize,
     commutativeBinOp,
+    associativeBinOp,
 
     -- * Prettyprinting
     convOpFun,
@@ -564,6 +565,12 @@
   | -- | Convert a boolean to an integer.  True is converted
     -- to 1 and False to 0.
     BToI IntType
+  | -- | Convert a float to a boolean value.  Zero becomes false;
+    -- | anything else is true.
+    FToB FloatType
+  | -- | Convert a boolean to a float.  True is converted
+    -- to 1 and False to 0.
+    BToF FloatType
   deriving (Eq, Ord, Show)
 
 -- | A list of all unary operators for all types.
@@ -640,7 +647,9 @@
       UIToFP <$> allIntTypes <*> allFloatTypes,
       SIToFP <$> allIntTypes <*> allFloatTypes,
       IToB <$> allIntTypes,
-      BToI <$> allIntTypes
+      BToI <$> allIntTypes,
+      FToB <$> allFloatTypes,
+      BToF <$> allFloatTypes
     ]
 
 -- | Apply an 'UnOp' to an operand.  Returns 'Nothing' if the
@@ -906,6 +915,8 @@
 doConvOp (SIToFP _ to) (IntValue v) = Just $ FloatValue $ doSIToFP v to
 doConvOp (IToB _) (IntValue v) = Just $ BoolValue $ intToInt64 v /= 0
 doConvOp (BToI to) (BoolValue v) = Just $ IntValue $ intValue to $ if v then 1 else 0 :: Int
+doConvOp (FToB _) (FloatValue v) = Just $ BoolValue $ floatToDouble v /= 0
+doConvOp (BToF to) (BoolValue v) = Just $ FloatValue $ floatValue to $ if v then 1 else 0 :: Double
 doConvOp _ _ = Nothing
 
 -- | Turn the conversion the other way around.  Note that most
@@ -921,6 +932,8 @@
 flipConvOp (SIToFP from to) = FPToSI to from
 flipConvOp (IToB from) = BToI from
 flipConvOp (BToI to) = IToB to
+flipConvOp (FToB from) = BToF from
+flipConvOp (BToF to) = FToB to
 
 -- | Zero-extend the given integer value to the size of the given
 -- type.  If the type is smaller than the given value, the result is a
@@ -1133,6 +1146,8 @@
 convOpType (SIToFP from to) = (IntType from, FloatType to)
 convOpType (IToB from) = (IntType from, Bool)
 convOpType (BToI to) = (Bool, IntType to)
+convOpType (FToB from) = (FloatType from, Bool)
+convOpType (BToF to) = (Bool, FloatType to)
 
 halfToWord :: Half -> Word16
 halfToWord (Half (CUShort x)) = x
@@ -1679,6 +1694,23 @@
 commutativeBinOp FMin {} = True
 commutativeBinOp _ = False
 
+-- | True if the given binary operator is associative.
+associativeBinOp :: BinOp -> Bool
+associativeBinOp Add {} = True
+associativeBinOp Mul {} = True
+associativeBinOp And {} = True
+associativeBinOp Or {} = True
+associativeBinOp Xor {} = True
+associativeBinOp LogOr {} = True
+associativeBinOp LogAnd {} = True
+associativeBinOp SMax {} = True
+associativeBinOp SMin {} = True
+associativeBinOp UMax {} = True
+associativeBinOp UMin {} = True
+associativeBinOp FMax {} = True
+associativeBinOp FMin {} = True
+associativeBinOp _ = False
+
 -- Prettyprinting instances
 
 instance Pretty BinOp where
@@ -1763,6 +1795,8 @@
 convOpFun SIToFP {} = "sitofp"
 convOpFun IToB {} = "itob"
 convOpFun BToI {} = "btoi"
+convOpFun FToB {} = "ftob"
+convOpFun BToF {} = "btof"
 
 taggedI :: String -> IntType -> Doc
 taggedI s Int8 = text $ s ++ "8"
diff --git a/src/Futhark/IR/Primitive/Parse.hs b/src/Futhark/IR/Primitive/Parse.hs
--- a/src/Futhark/IR/Primitive/Parse.hs
+++ b/src/Futhark/IR/Primitive/Parse.hs
@@ -1,5 +1,7 @@
 {-# LANGUAGE OverloadedStrings #-}
 
+-- | Parsers for primitive values and types.  Mostly useful for
+-- "Futhark.IR.Parse", but can perhaps come in handy elsewhere too.
 module Futhark.IR.Primitive.Parse
   ( pPrimValue,
     pPrimType,
@@ -24,27 +26,34 @@
 import Text.Megaparsec.Char
 import qualified Text.Megaparsec.Char.Lexer as L
 
-type Parser = Parsec Void T.Text
-
+-- | Is this character a valid member of an identifier?
 constituent :: Char -> Bool
 constituent c = isAlphaNum c || (c `elem` ("_/'+-=!&^.<>*|" :: String))
 
-whitespace :: Parser ()
+-- | Consume whitespace (including skipping line comments).
+whitespace :: Parsec Void T.Text ()
 whitespace = L.space space1 (L.skipLineComment "--") empty
 
-lexeme :: Parser a -> Parser a
+-- | Consume whitespace after the provided parser, if it succeeds.
+lexeme :: Parsec Void T.Text a -> Parsec Void T.Text a
 lexeme = try . L.lexeme whitespace
 
-keyword :: T.Text -> Parser ()
+-- | @keyword k@ parses @k@, which must not be immediately followed by
+-- a 'constituent' character.  This ensures that @iff@ is not seen as
+-- the @if@ keyword followed by @f@.  Sometimes called the "maximum
+-- munch" rule.
+keyword :: T.Text -> Parsec Void T.Text ()
 keyword s = lexeme $ chunk s *> notFollowedBy (satisfy constituent)
 
-pIntValue :: Parser IntValue
+-- | Parse an integer value.
+pIntValue :: Parsec Void T.Text IntValue
 pIntValue = try $ do
   x <- L.signed (pure ()) L.decimal
   t <- pIntType
   pure $ intValue t (x :: Integer)
 
-pFloatValue :: Parser FloatValue
+-- | Parse a floating-point value.
+pFloatValue :: Parsec Void T.Text FloatValue
 pFloatValue =
   choice
     [ pNum,
@@ -64,7 +73,8 @@
       t <- pFloatType
       pure $ floatValue t (x :: Double)
 
-pBoolValue :: Parser Bool
+-- | Parse a boolean value.
+pBoolValue :: Parsec Void T.Text Bool
 pBoolValue =
   choice
     [ keyword "true" $> True,
@@ -72,7 +82,7 @@
     ]
 
 -- | Defined in this module for convenience.
-pPrimValue :: Parser PrimValue
+pPrimValue :: Parsec Void T.Text PrimValue
 pPrimValue =
   choice
     [ FloatValue <$> pFloatValue,
@@ -82,17 +92,20 @@
     ]
     <?> "primitive value"
 
-pFloatType :: Parser FloatType
+-- | Parse a floating-point type.
+pFloatType :: Parsec Void T.Text FloatType
 pFloatType = choice $ map p allFloatTypes
   where
     p t = keyword (prettyText t) $> t
 
-pIntType :: Parser IntType
+-- | Parse an integer type.
+pIntType :: Parsec Void T.Text IntType
 pIntType = choice $ map p allIntTypes
   where
     p t = keyword (prettyText t) $> t
 
-pPrimType :: Parser PrimType
+-- | Parse a primitive type.
+pPrimType :: Parsec Void T.Text PrimType
 pPrimType =
   choice [p Bool, p Unit, FloatType <$> pFloatType, IntType <$> pIntType]
   where
diff --git a/src/Futhark/IR/Prop.hs b/src/Futhark/IR/Prop.hs
--- a/src/Futhark/IR/Prop.hs
+++ b/src/Futhark/IR/Prop.hs
@@ -233,11 +233,11 @@
   -- 'expExtTypesFromPat'.
   expTypesFromPat ::
     (HasScope rep m, Monad m) =>
-    Pat rep ->
+    Pat (LetDec rep) ->
     m [BranchType rep]
 
 -- | Construct the type of an expression that would match the pattern.
-expExtTypesFromPat :: Typed dec => PatT dec -> [ExtType]
+expExtTypesFromPat :: Typed dec => Pat dec -> [ExtType]
 expExtTypesFromPat pat =
   existentialiseExtTypes (patNames pat) $
     staticShapes $ map patElemType $ patElems pat
diff --git a/src/Futhark/IR/Prop/Aliases.hs b/src/Futhark/IR/Prop/Aliases.hs
--- a/src/Futhark/IR/Prop/Aliases.hs
+++ b/src/Futhark/IR/Prop/Aliases.hs
@@ -193,7 +193,7 @@
 consumedByLambda = consumedInBody . lambdaBody
 
 -- | The aliases of each pattern element (including the context).
-patAliases :: AliasesOf dec => PatT dec -> [Names]
+patAliases :: AliasesOf dec => Pat dec -> [Names]
 patAliases = map (aliasesOf . patElemDec) . patElems
 
 -- | Something that contains alias information.
@@ -204,7 +204,7 @@
 instance AliasesOf Names where
   aliasesOf = id
 
-instance AliasesOf dec => AliasesOf (PatElemT dec) where
+instance AliasesOf dec => AliasesOf (PatElem dec) where
   aliasesOf = aliasesOf . patElemDec
 
 -- | Also includes the name itself.
diff --git a/src/Futhark/IR/Prop/Names.hs b/src/Futhark/IR/Prop/Names.hs
--- a/src/Futhark/IR/Prop/Names.hs
+++ b/src/Futhark/IR/Prop/Names.hs
@@ -337,7 +337,7 @@
 instance FreeIn dec => FreeIn (Param dec) where
   freeIn' (Param attrs _ dec) = freeIn' attrs <> freeIn' dec
 
-instance FreeIn dec => FreeIn (PatElemT dec) where
+instance FreeIn dec => FreeIn (PatElem dec) where
   freeIn' (PatElem _ dec) = freeIn' dec
 
 instance FreeIn (LParamInfo rep) => FreeIn (LoopForm rep) where
@@ -362,7 +362,7 @@
 instance FreeIn SubExpRes where
   freeIn' (SubExpRes cs se) = freeIn' cs <> freeIn' se
 
-instance FreeIn dec => FreeIn (PatT dec) where
+instance FreeIn dec => FreeIn (Pat dec) where
   freeIn' (Pat xs) =
     fvBind bound_here $ freeIn' xs
     where
diff --git a/src/Futhark/IR/Prop/Patterns.hs b/src/Futhark/IR/Prop/Patterns.hs
--- a/src/Futhark/IR/Prop/Patterns.hs
+++ b/src/Futhark/IR/Prop/Patterns.hs
@@ -39,35 +39,35 @@
 paramIdent param = Ident (paramName param) (typeOf param)
 
 -- | An 'Ident' corresponding to a pattern element.
-patElemIdent :: Typed dec => PatElemT dec -> Ident
+patElemIdent :: Typed dec => PatElem dec -> Ident
 patElemIdent pelem = Ident (patElemName pelem) (typeOf pelem)
 
 -- | The type of a name bound by a t'PatElem'.
-patElemType :: Typed dec => PatElemT dec -> Type
+patElemType :: Typed dec => PatElem dec -> Type
 patElemType = typeOf
 
 -- | Set the rep of a t'PatElem'.
-setPatElemDec :: PatElemT oldattr -> newattr -> PatElemT newattr
+setPatElemDec :: PatElem oldattr -> newattr -> PatElem newattr
 setPatElemDec pe x = fmap (const x) pe
 
 -- | Return a list of the 'Ident's bound by the t'Pat'.
-patIdents :: Typed dec => PatT dec -> [Ident]
+patIdents :: Typed dec => Pat dec -> [Ident]
 patIdents = map patElemIdent . patElems
 
 -- | Return a list of the 'Name's bound by the t'Pat'.
-patNames :: PatT dec -> [VName]
+patNames :: Pat dec -> [VName]
 patNames = map patElemName . patElems
 
 -- | Return a list of the typess bound by the pattern.
-patTypes :: Typed dec => PatT dec -> [Type]
+patTypes :: Typed dec => Pat dec -> [Type]
 patTypes = map identType . patIdents
 
 -- | Return the number of names bound by the pattern.
-patSize :: PatT dec -> Int
+patSize :: Pat dec -> Int
 patSize (Pat xs) = length xs
 
 -- | Create a pattern using 'Type' as the attribute.
-basicPat :: [Ident] -> PatT Type
+basicPat :: [Ident] -> Pat Type
 basicPat values =
   Pat $ map patElem values
   where
diff --git a/src/Futhark/IR/Prop/Scope.hs b/src/Futhark/IR/Prop/Scope.hs
--- a/src/Futhark/IR/Prop/Scope.hs
+++ b/src/Futhark/IR/Prop/Scope.hs
@@ -16,6 +16,10 @@
 -- from variable names to 'NameInfo's.  Convenience facilities are
 -- also provided to communicate that some monad or applicative functor
 -- maintains type information.
+--
+-- A simple example of a monad that maintains such as environment is
+-- 'Reader'.  Indeed, 'HasScope' and 'LocalScope' instances for this
+-- monad are already defined.
 module Futhark.IR.Prop.Scope
   ( HasScope (..),
     NameInfo (..),
@@ -178,12 +182,12 @@
     M.insert i (IndexName it) $ scopeOfLParams (map fst xs)
 
 -- | The scope of a pattern.
-scopeOfPat :: LetDec rep ~ dec => PatT dec -> Scope rep
+scopeOfPat :: LetDec rep ~ dec => Pat dec -> Scope rep
 scopeOfPat =
   mconcat . map scopeOfPatElem . patElems
 
 -- | The scope of a pattern element.
-scopeOfPatElem :: LetDec rep ~ dec => PatElemT dec -> Scope rep
+scopeOfPatElem :: LetDec rep ~ dec => PatElem dec -> Scope rep
 scopeOfPatElem (PatElem name dec) = M.singleton name $ LetName dec
 
 -- | The scope of some lambda parameters.
diff --git a/src/Futhark/IR/Prop/TypeOf.hs b/src/Futhark/IR/Prop/TypeOf.hs
--- a/src/Futhark/IR/Prop/TypeOf.hs
+++ b/src/Futhark/IR/Prop/TypeOf.hs
@@ -35,6 +35,7 @@
   )
 where
 
+import Data.List.NonEmpty (NonEmpty (..))
 import Futhark.IR.Prop.Constants
 import Futhark.IR.Prop.Reshape
 import Futhark.IR.Prop.Scope
@@ -115,7 +116,7 @@
     result t = [rearrangeType perm t]
 basicOpType (Rotate _ e) =
   pure <$> lookupType e
-basicOpType (Concat i x _ ressize) =
+basicOpType (Concat i (x :| _) ressize) =
   result <$> lookupType x
   where
     result xt = [setDimSize i xt ressize]
diff --git a/src/Futhark/IR/Prop/Types.hs b/src/Futhark/IR/Prop/Types.hs
--- a/src/Futhark/IR/Prop/Types.hs
+++ b/src/Futhark/IR/Prop/Types.hs
@@ -554,7 +554,7 @@
 instance Typed dec => Typed (Param dec) where
   typeOf = typeOf . paramDec
 
-instance Typed dec => Typed (PatElemT dec) where
+instance Typed dec => Typed (PatElem dec) where
   typeOf = typeOf . patElemDec
 
 instance Typed b => Typed (a, b) where
@@ -594,7 +594,7 @@
 instance SetType b => SetType (a, b) where
   setType (a, b) t = (a, setType b t)
 
-instance SetType dec => SetType (PatElemT dec) where
+instance SetType dec => SetType (PatElem dec) where
   setType (PatElem name dec) t =
     PatElem name $ setType dec t
 
diff --git a/src/Futhark/IR/SOACS.hs b/src/Futhark/IR/SOACS.hs
--- a/src/Futhark/IR/SOACS.hs
+++ b/src/Futhark/IR/SOACS.hs
@@ -5,27 +5,12 @@
 module Futhark.IR.SOACS
   ( SOACS,
 
-    -- * Syntax types
-    Body,
-    Stm,
-    Pat,
-    Exp,
-    Lambda,
-    FParam,
-    LParam,
-    RetType,
-    PatElem,
-
     -- * Module re-exports
     module Futhark.IR.Prop,
     module Futhark.IR.Traversals,
     module Futhark.IR.Pretty,
     module Futhark.IR.Syntax,
     module Futhark.IR.SOACS.SOAC,
-    AST.LambdaT (Lambda),
-    AST.BodyT (Body),
-    AST.PatT (Pat),
-    AST.PatElemT (PatElem),
   )
 where
 
@@ -34,25 +19,10 @@
 import Futhark.IR.Pretty
 import Futhark.IR.Prop
 import Futhark.IR.SOACS.SOAC
-import Futhark.IR.Syntax hiding
-  ( Body,
-    Exp,
-    FParam,
-    LParam,
-    Lambda,
-    Pat,
-    PatElem,
-    RetType,
-    Stm,
-  )
-import qualified Futhark.IR.Syntax as AST
+import Futhark.IR.Syntax
 import Futhark.IR.Traversals
 import qualified Futhark.IR.TypeCheck as TC
 
--- This module could be written much nicer if Haskell had functors
--- like Standard ML.  Instead, we have to abuse the namespace/module
--- system.
-
 -- | The rep for the basic representation.
 data SOACS
 
@@ -62,31 +32,13 @@
 instance ASTRep SOACS where
   expTypesFromPat = return . expExtTypesFromPat
 
-type Exp = AST.Exp SOACS
-
-type Body = AST.Body SOACS
-
-type Stm = AST.Stm SOACS
-
-type Pat = AST.Pat SOACS
-
-type Lambda = AST.Lambda SOACS
-
-type FParam = AST.FParam SOACS
-
-type LParam = AST.LParam SOACS
-
-type RetType = AST.RetType SOACS
-
-type PatElem = AST.PatElem SOACS
-
 instance TC.CheckableOp SOACS where
   checkOp = typeCheckSOAC
 
 instance TC.Checkable SOACS
 
 instance Buildable SOACS where
-  mkBody = AST.Body ()
+  mkBody = Body ()
   mkExpPat merge _ = basicPat merge
   mkExpDec _ _ = ()
   mkLetNames = simpleMkLetNames
diff --git a/src/Futhark/IR/SOACS/Simplify.hs b/src/Futhark/IR/SOACS/Simplify.hs
--- a/src/Futhark/IR/SOACS/Simplify.hs
+++ b/src/Futhark/IR/SOACS/Simplify.hs
@@ -30,13 +30,13 @@
 import Data.Either
 import Data.Foldable
 import Data.List (partition, transpose, unzip6, zip6)
+import Data.List.NonEmpty (NonEmpty (..))
 import qualified Data.Map.Strict as M
 import Data.Maybe
 import qualified Data.Set as S
 import Futhark.Analysis.DataDependencies
 import qualified Futhark.Analysis.SymbolTable as ST
 import qualified Futhark.Analysis.UsageTable as UT
-import qualified Futhark.IR as AST
 import Futhark.IR.Prop.Aliases
 import Futhark.IR.SOACS
 import Futhark.MonadFreshNames
@@ -67,24 +67,18 @@
   Simplify.simplifyFun simpleSOACS soacRules Engine.noExtraHoistBlockers
 
 simplifyLambda ::
-  (HasScope SOACS m, MonadFreshNames m) =>
-  Lambda ->
-  m Lambda
+  (HasScope SOACS m, MonadFreshNames m) => Lambda SOACS -> m (Lambda SOACS)
 simplifyLambda =
   Simplify.simplifyLambda simpleSOACS soacRules Engine.noExtraHoistBlockers
 
 simplifyStms ::
-  (HasScope SOACS m, MonadFreshNames m) =>
-  Stms SOACS ->
-  m (Stms SOACS)
+  (HasScope SOACS m, MonadFreshNames m) => Stms SOACS -> m (Stms SOACS)
 simplifyStms stms = do
   scope <- askScope
   Simplify.simplifyStms simpleSOACS soacRules Engine.noExtraHoistBlockers scope stms
 
 simplifyConsts ::
-  MonadFreshNames m =>
-  Stms SOACS ->
-  m (Stms SOACS)
+  MonadFreshNames m => Stms SOACS -> m (Stms SOACS)
 simplifyConsts =
   Simplify.simplifyStms simpleSOACS soacRules Engine.noExtraHoistBlockers mempty
 
@@ -155,9 +149,9 @@
 
 fixLambdaParams ::
   (MonadBuilder m, Buildable (Rep m), BuilderOps (Rep m)) =>
-  AST.Lambda (Rep m) ->
+  Lambda (Rep m) ->
   [Maybe SubExp] ->
-  m (AST.Lambda (Rep m))
+  m (Lambda (Rep m))
 fixLambdaParams lam fixes = do
   body <- runBodyBuilder $
     localScope (scopeOfLParams $ lambdaParams lam) $ do
@@ -176,7 +170,7 @@
     maybeFix p (Just x) = letBindNames [paramName p] $ BasicOp $ SubExp x
     maybeFix _ Nothing = return ()
 
-removeLambdaResults :: [Bool] -> AST.Lambda rep -> AST.Lambda rep
+removeLambdaResults :: [Bool] -> Lambda rep -> Lambda rep
 removeLambdaResults keep lam =
   lam
     { lambdaBody = lam_body',
@@ -363,11 +357,11 @@
 removeReplicateInput ::
   Aliased rep =>
   ST.SymbolTable rep ->
-  AST.Lambda rep ->
+  Lambda rep ->
   [VName] ->
   Maybe
-    ( [([VName], Certs, AST.Exp rep)],
-      AST.Lambda rep,
+    ( [([VName], Certs, Exp rep)],
+      Lambda rep,
       [VName]
     )
 removeReplicateInput vtable fun arrs
@@ -472,11 +466,11 @@
           | otherwise = DimNew w
     certifying (stmAuxCerts aux1 <> cs) . letBind pat . BasicOp $
       Reshape (redim : newshape) arr
-  | Just (_, cs, _, BasicOp (Concat d arr arrs dw), ps, outer_arr : outer_arrs) <-
+  | Just (_, cs, _, BasicOp (Concat d (arr :| arrs) dw), ps, outer_arr : outer_arrs) <-
       isMapWithOp pat e,
     (arr : arrs) == map paramName ps =
     Simplify . certifying (stmAuxCerts aux1 <> cs) . letBind pat . BasicOp $
-      Concat (d + 1) outer_arr outer_arrs dw
+      Concat (d + 1) (outer_arr :| outer_arrs) dw
   | Just
       (map_pe, cs, _, BasicOp (Rearrange perm rearrange_arr), [p], [arr]) <-
       isMapWithOp pat e,
@@ -493,13 +487,13 @@
 mapOpToOp _ _ _ _ = Skip
 
 isMapWithOp ::
-  PatT dec ->
+  Pat dec ->
   SOAC (Wise SOACS) ->
   Maybe
-    ( PatElemT dec,
+    ( PatElem dec,
       Certs,
       SubExp,
-      AST.Exp (Wise SOACS),
+      Exp (Wise SOACS),
       [Param Type],
       [VName]
     )
@@ -597,7 +591,7 @@
     mix = concat . transpose
     incWrites r (w, n, a) = (w, n * r, a) -- ToDO: is it (n*r) or (n+r-1)??
     isConcat v = case ST.lookupExp v vtable of
-      Just (BasicOp (Concat 0 x ys _), cs) -> do
+      Just (BasicOp (Concat 0 (x :| ys) _), cs) -> do
         x_w <- sizeOf x
         y_ws <- mapM sizeOf ys
         guard $ all (x_w ==) y_ws
@@ -696,7 +690,7 @@
 arrayOpCerts (ArrayCopy cs _) = cs
 arrayOpCerts (ArrayVar cs _) = cs
 
-isArrayOp :: Certs -> AST.Exp rep -> Maybe ArrayOp
+isArrayOp :: Certs -> Exp rep -> Maybe ArrayOp
 isArrayOp cs (BasicOp (Index arr slice)) =
   Just $ ArrayIndexing cs arr slice
 isArrayOp cs (BasicOp (Rearrange perm arr)) =
@@ -708,7 +702,7 @@
 isArrayOp _ _ =
   Nothing
 
-fromArrayOp :: ArrayOp -> (Certs, AST.Exp rep)
+fromArrayOp :: ArrayOp -> (Certs, Exp rep)
 fromArrayOp (ArrayIndexing cs arr slice) = (cs, BasicOp $ Index arr slice)
 fromArrayOp (ArrayRearrange cs arr perm) = (cs, BasicOp $ Rearrange perm arr)
 fromArrayOp (ArrayRotate cs arr rots) = (cs, BasicOp $ Rotate rots arr)
@@ -718,8 +712,8 @@
 arrayOps ::
   forall rep.
   (Buildable rep, HasSOAC rep) =>
-  AST.Body rep ->
-  S.Set (AST.Pat rep, ArrayOp)
+  Body rep ->
+  S.Set (Pat (LetDec rep), ArrayOp)
 arrayOps = mconcat . map onStm . stmsToList . bodyStms
   where
     onStm (Let pat aux e) =
@@ -746,9 +740,9 @@
 replaceArrayOps ::
   forall rep.
   (Buildable rep, BuilderOps rep, HasSOAC rep) =>
-  M.Map (AST.Pat rep, ArrayOp) ArrayOp ->
-  AST.Body rep ->
-  AST.Body rep
+  M.Map (Pat (LetDec rep), ArrayOp) ArrayOp ->
+  Body rep ->
+  Body rep
 replaceArrayOps substs (Body _ stms res) =
   mkBody (fmap onStm stms) res
   where
diff --git a/src/Futhark/IR/SegOp.hs b/src/Futhark/IR/SegOp.hs
--- a/src/Futhark/IR/SegOp.hs
+++ b/src/Futhark/IR/SegOp.hs
@@ -151,7 +151,7 @@
     lambdaReturnType $ histOp op
 
 -- | Split reduction results returned by a 'KernelBody' into those
--- that correspond to indexes for the 'HistOps', and those that
+-- that correspond to indexes for the 'HistOp's, and those that
 -- correspond to value.
 splitHistResults :: [HistOp rep] -> [SubExp] -> [([SubExp], [SubExp])]
 splitHistResults ops res =
@@ -1327,7 +1327,7 @@
 topDownSegOp ::
   (HasSegOp rep, BuilderOps rep, Buildable rep) =>
   ST.SymbolTable rep ->
-  Pat rep ->
+  Pat (LetDec rep) ->
   StmAux (ExpDec rep) ->
   SegOp (SegOpLevel rep) rep ->
   Rule rep
@@ -1433,7 +1433,7 @@
 bottomUpSegOp ::
   (HasSegOp rep, BuilderOps rep) =>
   (ST.SymbolTable rep, UT.UsageTable) ->
-  Pat rep ->
+  Pat (LetDec rep) ->
   StmAux (ExpDec rep) ->
   SegOp (SegOpLevel rep) rep ->
   Rule rep
diff --git a/src/Futhark/IR/Syntax.hs b/src/Futhark/IR/Syntax.hs
--- a/src/Futhark/IR/Syntax.hs
+++ b/src/Futhark/IR/Syntax.hs
@@ -14,11 +14,11 @@
 -- The core language type system is much more restricted than the core
 -- language.  This is a theme that repeats often.  The only types that
 -- are supported in the core language are various primitive types
--- t'PrimType' which can be combined in arrays (ignore v'Mem' for
--- now).  Types are represented as t'TypeBase', which is parameterised
--- by the shape of the array and whether we keep uniqueness
--- information.  The t'Type' alias, which is the most commonly used,
--- uses t'Shape' and t'NoUniqueness'.
+-- t'PrimType' which can be combined in arrays (ignore v'Mem' and
+-- v'Acc' for now).  Types are represented as t'TypeBase', which is
+-- parameterised by the shape of the array and whether we keep
+-- uniqueness information.  The t'Type' alias, which is the most
+-- commonly used, uses t'Shape' and t'NoUniqueness'.
 --
 -- This means that the records, tuples, and sum types of the source
 -- language are represented merely as collections of primitives and
@@ -56,7 +56,7 @@
 -- the values of the variables indicated by the result.
 --
 -- A statement ('Stm') consists of a t'Pat' alongside an
--- expression 'ExpT'.  A pattern is a sequence of name/type pairs.
+-- expression 'Exp'.  A pattern is a sequence of name/type pairs.
 --
 -- For example, the source language expression @let z = x + y - 1 in
 -- z@ would in the core language be represented (in prettyprinted
@@ -70,9 +70,9 @@
 --
 -- == Representations
 --
--- Most AST types ('Stm', 'ExpT', t'Prog', etc) are parameterised by a
+-- Most AST types ('Stm', 'Exp', t'Prog', etc) are parameterised by a
 -- type parameter @rep@.  The representation specifies how to fill out
--- various polymorphic parts of the AST.  For example, 'ExpT' has a
+-- various polymorphic parts of the AST.  For example, 'Exp' has a
 -- constructor v'Op' whose payload depends on @rep@, via the use of a
 -- type family called t'Op' (a kind of type-level function) which is
 -- applied to the @rep@.  The SOACS representation
@@ -80,7 +80,7 @@
 -- that @Op SOACS@ is a SOAC, while the Kernels representation
 -- ("Futhark.IR.Kernels") defines @Op Kernels@ as some kind of kernel
 -- construct.  Similarly, various other decorations (e.g. what
--- information we store in a t'PatElemT') are also type families.
+-- information we store in a t'PatElem') are also type families.
 --
 -- The full list of possible decorations is defined as part of the
 -- type class 'RepTypes' (although other type families are also
@@ -91,6 +91,11 @@
 -- the different forms of decorations that we desire (it would easily
 -- become a type with a dozen type parameters).
 --
+-- Some AST elements (such as 'Pat') do not take a @rep@ type
+-- parameter, but instead immediately the single type of decoration
+-- that they contain.  We only use the more complicated machinery when
+-- needed.
+--
 -- Defining a new representation (or /rep/) thus requires you to
 -- define an empty datatype and implement a handful of type class
 -- instances for it.  See the source of "Futhark.IR.Seq"
@@ -113,17 +118,14 @@
     -- * Abstract syntax tree
     Ident (..),
     SubExp (..),
-    PatElem,
-    PatElemT (..),
-    PatT (..),
-    Pat,
+    PatElem (..),
+    Pat (..),
     StmAux (..),
     Stm (..),
     Stms,
     SubExpRes (..),
     Result,
-    BodyT (..),
-    Body,
+    Body (..),
     BasicOp (..),
     UnOp (..),
     BinOp (..),
@@ -133,14 +135,12 @@
     DimChange (..),
     ShapeChange,
     WithAccInput,
-    ExpT (..),
-    Exp,
+    Exp (..),
     LoopForm (..),
     IfDec (..),
     IfSort (..),
     Safety (..),
-    LambdaT (..),
-    Lambda,
+    Lambda (..),
 
     -- * Definitions
     Param (..),
@@ -166,6 +166,7 @@
 
 import Control.Category
 import Data.Foldable
+import Data.List.NonEmpty (NonEmpty (..))
 import qualified Data.Sequence as Seq
 import Data.String
 import Data.Traversable (fmapDefault, foldMapDefault)
@@ -175,32 +176,26 @@
 import Language.Futhark.Core
 import Prelude hiding (id, (.))
 
--- | A type alias for namespace control.
-type PatElem rep = PatElemT (LetDec rep)
-
 -- | A pattern is conceptually just a list of names and their types.
-newtype PatT dec = Pat {patElems :: [PatElemT dec]}
+newtype Pat dec = Pat {patElems :: [PatElem dec]}
   deriving (Ord, Show, Eq)
 
-instance Semigroup (PatT dec) where
+instance Semigroup (Pat dec) where
   Pat xs <> Pat ys = Pat (xs <> ys)
 
-instance Monoid (PatT dec) where
+instance Monoid (Pat dec) where
   mempty = Pat mempty
 
-instance Functor PatT where
+instance Functor Pat where
   fmap = fmapDefault
 
-instance Foldable PatT where
+instance Foldable Pat where
   foldMap = foldMapDefault
 
-instance Traversable PatT where
+instance Traversable Pat where
   traverse f (Pat xs) =
     Pat <$> traverse (traverse f) xs
 
--- | A type alias for namespace control.
-type Pat rep = PatT (LetDec rep)
-
 -- | Auxilliary Information associated with a statement.
 data StmAux dec = StmAux
   { stmAuxCerts :: !Certs,
@@ -216,7 +211,7 @@
 -- | A local variable binding.
 data Stm rep = Let
   { -- | Pat.
-    stmPat :: Pat rep,
+    stmPat :: Pat (LetDec rep),
     -- | Auxiliary information statement.
     stmAux :: StmAux (ExpDec rep),
     -- | Expression.
@@ -278,20 +273,17 @@
 
 -- | A body consists of a number of bindings, terminating in a result
 -- (essentially a tuple literal).
-data BodyT rep = Body
+data Body rep = Body
   { bodyDec :: BodyDec rep,
     bodyStms :: Stms rep,
     bodyResult :: Result
   }
 
-deriving instance RepTypes rep => Ord (BodyT rep)
-
-deriving instance RepTypes rep => Show (BodyT rep)
+deriving instance RepTypes rep => Ord (Body rep)
 
-deriving instance RepTypes rep => Eq (BodyT rep)
+deriving instance RepTypes rep => Show (Body rep)
 
--- | Type alias for namespace reasons.
-type Body = BodyT
+deriving instance RepTypes rep => Eq (Body rep)
 
 -- | The new dimension in a 'Reshape'-like operation.  This allows us to
 -- disambiguate "real" reshapes, that change the actual shape of the
@@ -371,8 +363,19 @@
     Update Safety VName (Slice SubExp) SubExp
   | FlatIndex VName (FlatSlice SubExp)
   | FlatUpdate VName (FlatSlice SubExp) VName
-  | -- | @concat@0([1],[2, 3, 4]) = [1, 2, 3, 4]@.
-    Concat Int VName [VName] SubExp
+  | -- | @
+    -- concat(0, [1] :| [[2, 3, 4], [5, 6]], 6) = [1, 2, 3, 4, 5, 6]@.
+    -- @
+    --
+    -- Concatenates the non-empty list of 'VName' resulting in an
+    -- array of length t'SubExp'. The 'Int' argument is used to
+    -- specify the dimension along which the arrays are
+    -- concatenated. For instance:
+    --
+    -- @
+    -- concat(1, [[1,2], [3, 4]] :| [[[5,6]], [[7, 8]]], 4) = [[1, 2, 5, 6], [3, 4, 7, 8]]
+    -- @
+    Concat Int (NonEmpty VName) SubExp
   | -- | Copy the given array.  The result will not alias anything.
     Copy VName
   | -- | Manifest an array with dimensions represented in the given
@@ -416,13 +419,13 @@
 -- | The root Futhark expression type.  The v'Op' constructor contains
 -- a rep-specific operation.  Do-loops, branches and function calls
 -- are special.  Everything else is a simple t'BasicOp'.
-data ExpT rep
+data Exp rep
   = -- | A simple (non-recursive) operation.
     BasicOp BasicOp
   | Apply Name [(SubExp, Diet)] [RetType rep] (Safety, SrcLoc, [SrcLoc])
-  | If SubExp (BodyT rep) (BodyT rep) (IfDec (BranchType rep))
+  | If SubExp (Body rep) (Body rep) (IfDec (BranchType rep))
   | -- | @loop {a} = {v} (for i < n|while b) do b@.
-    DoLoop [(FParam rep, SubExp)] (LoopForm rep) (BodyT rep)
+    DoLoop [(FParam rep, SubExp)] (LoopForm rep) (Body rep)
   | -- | Create accumulators backed by the given arrays (which are
     -- consumed) and pass them to the lambda, which must return the
     -- updated accumulators and possibly some extra values.  The
@@ -433,11 +436,11 @@
     WithAcc [WithAccInput rep] (Lambda rep)
   | Op (Op rep)
 
-deriving instance RepTypes rep => Eq (ExpT rep)
+deriving instance RepTypes rep => Eq (Exp rep)
 
-deriving instance RepTypes rep => Show (ExpT rep)
+deriving instance RepTypes rep => Show (Exp rep)
 
-deriving instance RepTypes rep => Ord (ExpT rep)
+deriving instance RepTypes rep => Ord (Exp rep)
 
 -- | For-loop or while-loop?
 data LoopForm rep
@@ -476,24 +479,18 @@
     IfEquiv
   deriving (Eq, Show, Ord)
 
--- | A type alias for namespace control.
-type Exp = ExpT
-
 -- | Anonymous function for use in a SOAC.
-data LambdaT rep = Lambda
+data Lambda rep = Lambda
   { lambdaParams :: [LParam rep],
-    lambdaBody :: BodyT rep,
+    lambdaBody :: Body rep,
     lambdaReturnType :: [Type]
   }
 
-deriving instance RepTypes rep => Eq (LambdaT rep)
-
-deriving instance RepTypes rep => Show (LambdaT rep)
+deriving instance RepTypes rep => Eq (Lambda rep)
 
-deriving instance RepTypes rep => Ord (LambdaT rep)
+deriving instance RepTypes rep => Show (Lambda rep)
 
--- | Type alias for namespacing reasons.
-type Lambda = LambdaT
+deriving instance RepTypes rep => Ord (Lambda rep)
 
 -- | A function and loop parameter.
 type FParam rep = Param (FParamInfo rep)
@@ -510,7 +507,7 @@
     funDefName :: Name,
     funDefRetType :: [RetType rep],
     funDefParams :: [FParam rep],
-    funDefBody :: BodyT rep
+    funDefBody :: Body rep
   }
 
 deriving instance RepTypes rep => Eq (FunDef rep)
diff --git a/src/Futhark/IR/Syntax/Core.hs b/src/Futhark/IR/Syntax/Core.hs
--- a/src/Futhark/IR/Syntax/Core.hs
+++ b/src/Futhark/IR/Syntax/Core.hs
@@ -60,7 +60,7 @@
     unitSlice,
     fixSlice,
     sliceSlice,
-    PatElemT (..),
+    PatElem (..),
 
     -- * Flat (LMAD) slices
     FlatSlice (..),
@@ -456,7 +456,7 @@
 -- | An element of a pattern - consisting of a name and an addditional
 -- parametric decoration.  This decoration is what is expected to
 -- contain the type of the resulting variable.
-data PatElemT dec = PatElem
+data PatElem dec = PatElem
   { -- | The name being bound.
     patElemName :: VName,
     -- | Pat element decoration.
@@ -464,13 +464,13 @@
   }
   deriving (Ord, Show, Eq)
 
-instance Functor PatElemT where
+instance Functor PatElem where
   fmap = fmapDefault
 
-instance Foldable PatElemT where
+instance Foldable PatElem where
   foldMap = foldMapDefault
 
-instance Traversable PatElemT where
+instance Traversable PatElem where
   traverse f (PatElem name dec) =
     PatElem name <$> f dec
 
diff --git a/src/Futhark/IR/Traversals.hs b/src/Futhark/IR/Traversals.hs
--- a/src/Futhark/IR/Traversals.hs
+++ b/src/Futhark/IR/Traversals.hs
@@ -44,6 +44,7 @@
 import Control.Monad.Identity
 import Data.Bitraversable
 import Data.Foldable (traverse_)
+import Data.List.NonEmpty (NonEmpty (..))
 import Futhark.IR.Prop.Scope
 import Futhark.IR.Prop.Types (mapOnType)
 import Futhark.IR.Syntax
@@ -146,13 +147,11 @@
   BasicOp <$> (Rearrange perm <$> mapOnVName tv e)
 mapExpM tv (BasicOp (Rotate es e)) =
   BasicOp <$> (Rotate <$> mapM (mapOnSubExp tv) es <*> mapOnVName tv e)
-mapExpM tv (BasicOp (Concat i x ys size)) =
-  BasicOp
-    <$> ( Concat i
-            <$> mapOnVName tv x
-            <*> mapM (mapOnVName tv) ys
-            <*> mapOnSubExp tv size
-        )
+mapExpM tv (BasicOp (Concat i (x :| ys) size)) = do
+  x' <- mapOnVName tv x
+  ys' <- mapM (mapOnVName tv) ys
+  size' <- mapOnSubExp tv size
+  return $ BasicOp $ Concat i (x' :| ys') size'
 mapExpM tv (BasicOp (Copy e)) =
   BasicOp <$> (Copy <$> mapOnVName tv e)
 mapExpM tv (BasicOp (Manifest perm e)) =
@@ -319,7 +318,7 @@
   walkOnVName tv e
 walkExpM tv (BasicOp (Rotate es e)) =
   mapM_ (walkOnSubExp tv) es >> walkOnVName tv e
-walkExpM tv (BasicOp (Concat _ x ys size)) =
+walkExpM tv (BasicOp (Concat _ (x :| ys) size)) =
   walkOnVName tv x >> mapM_ (walkOnVName tv) ys >> walkOnSubExp tv size
 walkExpM tv (BasicOp (Copy e)) =
   walkOnVName tv e
@@ -354,7 +353,7 @@
 -- given op for some representation.
 type OpStmsTraverser m op rep = (Scope rep -> Stms rep -> m (Stms rep)) -> op -> m op
 
--- | This representatin supports an 'OpStmsTraverser' for its 'Op'.
+-- | This representation supports an 'OpStmsTraverser' for its t'Op'.
 -- This is used for some simplification rules.
 class TraverseOpStms rep where
   -- | Transform every sub-'Stms' of this op.
diff --git a/src/Futhark/IR/TypeCheck.hs b/src/Futhark/IR/TypeCheck.hs
--- a/src/Futhark/IR/TypeCheck.hs
+++ b/src/Futhark/IR/TypeCheck.hs
@@ -21,7 +21,6 @@
     TypeM,
     bad,
     context,
-    message,
     Checkable (..),
     CheckableOp (..),
     lookupVar,
@@ -42,14 +41,12 @@
     matchExtPat,
     matchExtBranchType,
     argType,
-    argAliases,
     noArgAliases,
     checkArg,
     checkSOACArrayArgs,
     checkLambda,
     checkBody,
     consume,
-    consumeOnlyParams,
     binding,
     alternative,
   )
@@ -60,6 +57,7 @@
 import Control.Parallel.Strategies
 import Data.Bifunctor (second)
 import Data.List (find, intercalate, isPrefixOf, sort)
+import Data.List.NonEmpty (NonEmpty (..))
 import qualified Data.Map.Strict as M
 import Data.Maybe
 import qualified Data.Set as S
@@ -78,7 +76,7 @@
   | DupDefinitionError Name
   | DupParamError Name VName
   | DupPatError VName
-  | InvalidPatError (Pat (Aliases rep)) [ExtType] (Maybe String)
+  | InvalidPatError (Pat (LetDec (Aliases rep))) [ExtType] (Maybe String)
   | UnknownVariableError VName
   | UnknownFunctionError Name
   | ParameterMismatch (Maybe Name) [Type] [Type]
@@ -316,6 +314,7 @@
 runTypeM env (TypeM m) =
   second stateCons <$> runStateT (runReaderT m env) (TState mempty mempty)
 
+-- | Signal a type error.
 bad :: ErrorCase rep -> TypeM rep a
 bad e = do
   messages <- asks envContext
@@ -393,6 +392,9 @@
 checkConsumption (ConsumptionError e) = bad $ TypeError e
 checkConsumption (Consumption os) = return os
 
+-- | Type check two mutually control flow branches.  Think @if@.  This
+-- interacts with consumption checking, as it is OK for an array to be
+-- consumed in both branches.
 alternative :: TypeM rep a -> TypeM rep b -> TypeM rep (a, b)
 alternative m1 m2 = do
   (x, os1) <- collectOccurences m1
@@ -907,7 +909,7 @@
           ++ " dimensions of "
           ++ show rank
           ++ "-dimensional array."
-checkBasicOp (Concat i arr1exp arr2exps ressize) = do
+checkBasicOp (Concat i (arr1exp :| arr2exps) ressize) = do
   arr1t <- checkArrIdent arr1exp
   arr2ts <- mapM checkArrIdent arr2exps
   let success =
@@ -1230,7 +1232,7 @@
 
 checkPatElem ::
   Checkable rep =>
-  PatElemT (LetDec rep) ->
+  PatElem (LetDec rep) ->
   TypeM rep ()
 checkPatElem (PatElem name dec) =
   context ("When checking pattern element " ++ pretty name) $
@@ -1287,7 +1289,7 @@
 
 matchExtPat ::
   Checkable rep =>
-  Pat (Aliases rep) ->
+  Pat (LetDec (Aliases rep)) ->
   [ExtType] ->
   TypeM rep ()
 matchExtPat pat ts =
@@ -1490,7 +1492,7 @@
   checkLParamDec :: VName -> LParamInfo rep -> TypeM rep ()
   checkLetBoundDec :: VName -> LetDec rep -> TypeM rep ()
   checkRetType :: [RetType rep] -> TypeM rep ()
-  matchPat :: Pat (Aliases rep) -> Exp (Aliases rep) -> TypeM rep ()
+  matchPat :: Pat (LetDec (Aliases rep)) -> Exp (Aliases rep) -> TypeM rep ()
   primFParam :: VName -> PrimType -> TypeM rep (FParam (Aliases rep))
   matchReturnType :: [RetType rep] -> Result -> TypeM rep ()
   matchBranchType :: [BranchType rep] -> Body (Aliases rep) -> TypeM rep ()
@@ -1514,7 +1516,7 @@
   default checkRetType :: RetType rep ~ DeclExtType => [RetType rep] -> TypeM rep ()
   checkRetType = mapM_ $ checkExtType . declExtTypeOf
 
-  default matchPat :: Pat (Aliases rep) -> Exp (Aliases rep) -> TypeM rep ()
+  default matchPat :: Pat (LetDec (Aliases rep)) -> Exp (Aliases rep) -> TypeM rep ()
   matchPat pat = matchExtPat pat <=< expExtType
 
   default primFParam :: FParamInfo rep ~ DeclType => VName -> PrimType -> TypeM rep (FParam (Aliases rep))
diff --git a/src/Futhark/Internalise.hs b/src/Futhark/Internalise.hs
--- a/src/Futhark/Internalise.hs
+++ b/src/Futhark/Internalise.hs
@@ -5,6 +5,41 @@
 --
 -- This module implements a transformation from source to core
 -- Futhark.
+--
+-- The source and core language is similar in spirit, but the core
+-- language is much more regular (and mostly much simpler) in order to
+-- make it easier to write program transformations.
+--
+-- * "Language.Futhark.Syntax" contains the source language definition.
+--
+-- * "Futhark.IR.Syntax" contains the core IR definition.
+--
+-- Specifically, internalisation generates the SOACS dialect of the
+-- core IR ("Futhark.IR.SOACS").  This is then initially used by the
+-- compiler middle-end.  The main differences between the source and
+-- core IR are as follows:
+--
+-- * The core IR has no modules.  These are removed in
+--   "Futhark.Internalise.Defunctorise".
+--
+-- * The core IR is monomorphic.  Polymorphic functions are monomorphised in
+--   "Futhark.Internalise.Monomorphise"
+--
+-- * The core IR is first-order. "Futhark.Internalise.Defunctionalise"
+--   removes higher-order functions.
+--
+-- * The core IR is in [ANF](https://en.wikipedia.org/wiki/A-normal_form).
+--
+-- * The core IR does not have arrays of tuples (or tuples or records
+--   at all, really).  Arrays of tuples are turned into multiple
+--   arrays.  For example, a source language transposition of an array
+--   of pairs becomes a core IR that contains two transpositions of
+--   two distinct arrays.  The guts of this transformation is in
+--   "Futhark.Internalise.Exps".
+--
+-- * For the above reason, SOACs also accept multiple input arrays.
+--   The available primitive operations are also somewhat different
+--   than in the source language.  See 'Futhark.IR.SOACS.SOAC.SOAC'.
 module Futhark.Internalise (internaliseProg) where
 
 import qualified Data.Text as T
diff --git a/src/Futhark/Internalise/AccurateSizes.hs b/src/Futhark/Internalise/AccurateSizes.hs
--- a/src/Futhark/Internalise/AccurateSizes.hs
+++ b/src/Futhark/Internalise/AccurateSizes.hs
@@ -20,7 +20,7 @@
 
 shapeMapping ::
   (HasScope SOACS m, Monad m) =>
-  [FParam] ->
+  [FParam SOACS] ->
   [Type] ->
   m (M.Map VName SubExp)
 shapeMapping all_params value_arg_types =
@@ -42,7 +42,7 @@
     match (Var v, se) = Just (v, se)
     match _ = Nothing
 
-argShapes :: [VName] -> [FParam] -> [Type] -> InternaliseM [SubExp]
+argShapes :: [VName] -> [FParam SOACS] -> [Type] -> InternaliseM [SubExp]
 argShapes shapes all_params valargts = do
   mapping <- shapeMapping all_params valargts
   let addShape name =
diff --git a/src/Futhark/Internalise/Bindings.hs b/src/Futhark/Internalise/Bindings.hs
--- a/src/Futhark/Internalise/Bindings.hs
+++ b/src/Futhark/Internalise/Bindings.hs
@@ -36,7 +36,7 @@
 bindingFParams ::
   [E.TypeParam] ->
   [E.Pat] ->
-  ([I.FParam] -> [[I.FParam]] -> InternaliseM a) ->
+  ([I.FParam I.SOACS] -> [[I.FParam I.SOACS]] -> InternaliseM a) ->
   InternaliseM a
 bindingFParams tparams params m = do
   flattened_params <- mapM flattenPat params
@@ -65,7 +65,7 @@
   [E.TypeParam] ->
   E.Pat ->
   [I.Type] ->
-  ([I.FParam] -> [I.FParam] -> InternaliseM a) ->
+  ([I.FParam I.SOACS] -> [I.FParam I.SOACS] -> InternaliseM a) ->
   InternaliseM a
 bindingLoopParams tparams pat ts m = do
   pat_idents <- flattenPat pat
@@ -81,7 +81,7 @@
 bindingLambdaParams ::
   [E.Pat] ->
   [I.Type] ->
-  ([I.LParam] -> InternaliseM a) ->
+  ([I.LParam I.SOACS] -> InternaliseM a) ->
   InternaliseM a
 bindingLambdaParams params ts m = do
   params_idents <- concat <$> mapM flattenPat params
diff --git a/src/Futhark/Internalise/Exps.hs b/src/Futhark/Internalise/Exps.hs
--- a/src/Futhark/Internalise/Exps.hs
+++ b/src/Futhark/Internalise/Exps.hs
@@ -10,6 +10,7 @@
 
 import Control.Monad.Reader
 import Data.List (find, intercalate, intersperse, transpose)
+import Data.List.NonEmpty (NonEmpty (..))
 import qualified Data.List.NonEmpty as NE
 import qualified Data.Map.Strict as M
 import qualified Data.Set as S
@@ -128,7 +129,7 @@
 
 entryPoint ::
   Name ->
-  [(E.EntryParam, [I.FParam])] ->
+  [(E.EntryParam, [I.FParam SOACS])] ->
   ( E.EntryType,
     [[I.TypeBase ExtShape Uniqueness]]
   ) ->
@@ -187,20 +188,20 @@
        in (d + 1, te')
     withoutDims te = (0 :: Int, te)
 
-internaliseBody :: String -> E.Exp -> InternaliseM Body
+internaliseBody :: String -> E.Exp -> InternaliseM (Body SOACS)
 internaliseBody desc e =
   buildBody_ $ subExpsRes <$> internaliseExp (desc <> "_res") e
 
 bodyFromStms ::
   InternaliseM (Result, a) ->
-  InternaliseM (Body, a)
+  InternaliseM (Body SOACS, a)
 bodyFromStms m = do
   ((res, a), stms) <- collectStms m
   (,a) <$> mkBodyM stms res
 
 -- | Only returns those pattern names that are not used in the pattern
 -- itself (the "non-existential" part, you could say).
-letValExp :: String -> I.Exp -> InternaliseM [VName]
+letValExp :: String -> I.Exp SOACS -> InternaliseM [VName]
 letValExp name e = do
   e_t <- expExtType e
   names <- replicateM (length e_t) $ newVName name
@@ -208,11 +209,11 @@
   let ctx = shapeContext e_t
   pure $ map fst $ filter ((`S.notMember` ctx) . snd) $ zip names [0 ..]
 
-letValExp' :: String -> I.Exp -> InternaliseM [SubExp]
+letValExp' :: String -> I.Exp SOACS -> InternaliseM [SubExp]
 letValExp' _ (BasicOp (SubExp se)) = pure [se]
 letValExp' name ses = map I.Var <$> letValExp name ses
 
-eValBody :: [InternaliseM I.Exp] -> InternaliseM I.Body
+eValBody :: [InternaliseM (I.Exp SOACS)] -> InternaliseM (I.Body SOACS)
 eValBody es = buildBody_ $ do
   es' <- sequence es
   varsRes . concat <$> mapM (letValExp "x") es'
@@ -919,7 +920,7 @@
           ses''
         )
 
-generateCaseIf :: [I.SubExp] -> Case -> I.Body -> InternaliseM I.Exp
+generateCaseIf :: [I.SubExp] -> Case -> I.Body SOACS -> InternaliseM (I.Exp SOACS)
 generateCaseIf ses (CasePat p eCase _) bFail = do
   (cond, pertinent) <- generateCond p ses
   eCase' <- internalisePat' [] p pertinent eCase (internaliseBody "case")
@@ -1115,7 +1116,7 @@
 internaliseScanOrReduce ::
   String ->
   String ->
-  (SubExp -> I.Lambda -> [SubExp] -> [VName] -> InternaliseM (SOAC SOACS)) ->
+  (SubExp -> I.Lambda SOACS -> [SubExp] -> [VName] -> InternaliseM (SOAC SOACS)) ->
   (E.Exp, E.Exp, E.Exp, SrcLoc) ->
   InternaliseM [SubExp]
 internaliseScanOrReduce desc what f (lam, ne, arr, loc) = do
@@ -1526,7 +1527,7 @@
 
 -- The type of a body.  Watch out: this only works for the degenerate
 -- case where the body does not already return its context.
-bodyExtType :: Body -> InternaliseM [ExtType]
+bodyExtType :: Body SOACS -> InternaliseM [ExtType]
 bodyExtType (Body _ stms res) =
   existentialiseExtTypes (M.keys stmsscope) . staticShapes
     <$> extendedScope (traverse subExpResType res) stmsscope
@@ -1786,7 +1787,7 @@
           =<< mapM (fmap (arraysSize 0) . mapM lookupType) [ys]
 
       let conc xarr yarr =
-            I.BasicOp $ I.Concat 0 xarr [yarr] ressize
+            I.BasicOp $ I.Concat 0 (xarr :| [yarr]) ressize
       letSubExps desc $ zipWith conc xs ys
     handleRest [TupLit [offset, e] _] "rotate" = Just $ \desc -> do
       offset' <- internaliseExp1 "rotation_offset" offset
@@ -2084,7 +2085,7 @@
   return $ if check then I.Safe else I.Unsafe
 
 -- Implement partitioning using maps, scans and writes.
-partitionWithSOACS :: Int -> I.Lambda -> [I.VName] -> InternaliseM ([I.SubExp], [I.SubExp])
+partitionWithSOACS :: Int -> I.Lambda SOACS -> [I.VName] -> InternaliseM ([I.SubExp], [I.SubExp])
 partitionWithSOACS k lam arrs = do
   arr_ts <- mapM lookupType arrs
   let w = arraysSize 0 arr_ts
@@ -2178,7 +2179,7 @@
       [SubExp] ->
       SubExp ->
       Int ->
-      [I.LParam] ->
+      [I.LParam SOACS] ->
       InternaliseM SubExp
     mkOffsetLambdaBody _ _ _ [] =
       return $ constant (-1 :: Int64)
diff --git a/src/Futhark/Internalise/FreeVars.hs b/src/Futhark/Internalise/FreeVars.hs
--- a/src/Futhark/Internalise/FreeVars.hs
+++ b/src/Futhark/Internalise/FreeVars.hs
@@ -4,7 +4,6 @@
   ( freeVars,
     without,
     ident,
-    size,
     sizes,
     NameSet (..),
     patVars,
@@ -42,6 +41,7 @@
 size :: VName -> NameSet
 size v = NameSet $ M.singleton v $ Scalar $ Prim $ Signed Int64
 
+-- | A 'NameSet' with these names, considered to be sizes.
 sizes :: S.Set VName -> NameSet
 sizes = foldMap size
 
diff --git a/src/Futhark/Internalise/Lambdas.hs b/src/Futhark/Internalise/Lambdas.hs
--- a/src/Futhark/Internalise/Lambdas.hs
+++ b/src/Futhark/Internalise/Lambdas.hs
@@ -17,13 +17,13 @@
 
 -- | A function for internalising lambdas.
 type InternaliseLambda =
-  E.Exp -> [I.Type] -> InternaliseM ([I.LParam], I.Body, [I.Type])
+  E.Exp -> [I.Type] -> InternaliseM ([I.LParam SOACS], I.Body SOACS, [I.Type])
 
 internaliseMapLambda ::
   InternaliseLambda ->
   E.Exp ->
   [I.SubExp] ->
-  InternaliseM I.Lambda
+  InternaliseM (I.Lambda SOACS)
 internaliseMapLambda internaliseLambda lam args = do
   argtypes <- mapM I.subExpType args
   let rowtypes = map I.rowType argtypes
@@ -39,7 +39,7 @@
   InternaliseLambda ->
   E.Exp ->
   [I.SubExp] ->
-  InternaliseM I.Lambda
+  InternaliseM (I.Lambda SOACS)
 internaliseStreamMapLambda internaliseLambda lam args = do
   chunk_size <- newVName "chunk_size"
   let chunk_param = I.Param mempty chunk_size (I.Prim int64)
@@ -65,7 +65,7 @@
   E.Exp ->
   [I.Type] ->
   [I.Type] ->
-  InternaliseM I.Lambda
+  InternaliseM (I.Lambda SOACS)
 internaliseFoldLambda internaliseLambda lam acctypes arrtypes = do
   let rowtypes = map I.rowType arrtypes
   (params, body, rettype) <- internaliseLambda lam $ acctypes ++ rowtypes
@@ -86,7 +86,7 @@
   InternaliseLambda ->
   E.Exp ->
   [I.Type] ->
-  InternaliseM ([LParam], Body)
+  InternaliseM ([LParam SOACS], Body SOACS)
 internaliseStreamLambda internaliseLambda lam rowts = do
   chunk_size <- newVName "chunk_size"
   let chunk_param = I.Param mempty chunk_size $ I.Prim int64
@@ -109,7 +109,7 @@
   Int ->
   E.Exp ->
   [I.SubExp] ->
-  InternaliseM I.Lambda
+  InternaliseM (I.Lambda SOACS)
 internalisePartitionLambda internaliseLambda k lam args = do
   argtypes <- mapM I.subExpType args
   let rowtypes = map I.rowType argtypes
@@ -138,7 +138,7 @@
           (pure $ resultBody $ result i)
           (resultBody <$> mkResult eq_class (i + 1))
 
-    lambdaWithIncrement :: I.Body -> InternaliseM I.Body
+    lambdaWithIncrement :: I.Body SOACS -> InternaliseM (I.Body SOACS)
     lambdaWithIncrement lam_body = runBodyBuilder $ do
       eq_class <- resSubExp . head <$> bodyBind lam_body
       resultBody <$> mkResult eq_class 0
diff --git a/src/Futhark/Internalise/Monad.hs b/src/Futhark/Internalise/Monad.hs
--- a/src/Futhark/Internalise/Monad.hs
+++ b/src/Futhark/Internalise/Monad.hs
@@ -39,7 +39,7 @@
 type FunInfo =
   ( [VName],
     [DeclType],
-    [FParam],
+    [FParam SOACS],
     [(SubExp, Type)] -> Maybe [DeclExtType]
   )
 
diff --git a/src/Futhark/Optimise/CSE.hs b/src/Futhark/Optimise/CSE.hs
--- a/src/Futhark/Optimise/CSE.hs
+++ b/src/Futhark/Optimise/CSE.hs
@@ -248,7 +248,7 @@
 type ExpressionSubstitutions rep =
   M.Map
     (ExpDec rep, Exp rep)
-    (Pat rep)
+    (Pat (LetDec rep))
 
 type NameSubstitutions = M.Map VName VName
 
@@ -260,16 +260,16 @@
 newCSEState :: Bool -> CSEState rep
 newCSEState = CSEState (M.empty, M.empty)
 
-mkSubsts :: PatT dec -> PatT dec -> M.Map VName VName
+mkSubsts :: Pat dec -> Pat dec -> M.Map VName VName
 mkSubsts pat vs = M.fromList $ zip (patNames pat) (patNames vs)
 
-addNameSubst :: PatT dec -> PatT dec -> CSEState rep -> CSEState rep
+addNameSubst :: Pat dec -> Pat dec -> CSEState rep -> CSEState rep
 addNameSubst pat subpat (CSEState (esubsts, nsubsts) cse_arrays) =
   CSEState (esubsts, mkSubsts pat subpat `M.union` nsubsts) cse_arrays
 
 addExpSubst ::
   ASTRep rep =>
-  Pat rep ->
+  Pat (LetDec rep) ->
   ExpDec rep ->
   Exp rep ->
   CSEState rep ->
diff --git a/src/Futhark/Optimise/DoubleBuffer.hs b/src/Futhark/Optimise/DoubleBuffer.hs
--- a/src/Futhark/Optimise/DoubleBuffer.hs
+++ b/src/Futhark/Optimise/DoubleBuffer.hs
@@ -120,13 +120,13 @@
     doNotTouchLoop pat merge body = pure (mempty, pat, merge, body)
 
 type OptimiseLoop rep =
-  Pat rep ->
+  Pat (LetDec rep) ->
   [(FParam rep, SubExp)] ->
   Body rep ->
   DoubleBufferM
     rep
     ( Stms rep,
-      Pat rep,
+      Pat (LetDec rep),
       [(FParam rep, SubExp)],
       Body rep
     )
diff --git a/src/Futhark/Optimise/Fusion.hs b/src/Futhark/Optimise/Fusion.hs
--- a/src/Futhark/Optimise/Fusion.hs
+++ b/src/Futhark/Optimise/Fusion.hs
@@ -102,13 +102,13 @@
 binding :: [(Ident, Names)] -> FusionGM a -> FusionGM a
 binding vs = local (`bindVars` vs)
 
-gatherStmPat :: Pat -> Exp -> FusionGM FusedRes -> FusionGM FusedRes
+gatherStmPat :: Pat Type -> Exp SOACS -> FusionGM FusedRes -> FusionGM FusedRes
 gatherStmPat pat e = binding $ zip idents aliases
   where
     idents = patIdents pat
     aliases = expAliases (Alias.analyseExp mempty e)
 
-bindingPat :: Pat -> FusionGM a -> FusionGM a
+bindingPat :: Pat Type -> FusionGM a -> FusionGM a
 bindingPat = binding . (`zip` repeat mempty) . patIdents
 
 bindingParams :: Typed t => [Param t] -> FusionGM a -> FusionGM a
@@ -145,7 +145,7 @@
   let inspectKer k = k {inplace = aliases <> inplace k}
   return res' {kernels = M.map inspectKer $ kernels res'}
 
-checkForUpdates :: FusedRes -> Exp -> FusionGM FusedRes
+checkForUpdates :: FusedRes -> Exp SOACS -> FusionGM FusedRes
 checkForUpdates res (BasicOp (Update _ src slice _)) = do
   let ifvs = namesToList $ freeIn slice
   updateKerInPlaces res ([src], ifvs)
@@ -162,14 +162,14 @@
 --   variables in scope (map) by inserting each (pattern-array) name.
 --   Finally, if the binding is an in-place update, then the @inplace@ field
 --   of each (result) kernel is updated with the new in-place updates.
-bindingFamily :: Pat -> FusionGM FusedRes -> FusionGM FusedRes
+bindingFamily :: Pat Type -> FusionGM FusedRes -> FusionGM FusedRes
 bindingFamily pat = local bind
   where
     idents = patIdents pat
     family = patNames pat
     bind env = foldl (bindingFamilyVar family) env idents
 
-bindingTransform :: PatElem -> VName -> SOAC.ArrayTransform -> FusionGM a -> FusionGM a
+bindingTransform :: PatElem Type -> VName -> SOAC.ArrayTransform -> FusionGM a -> FusionGM a
 bindingTransform pe srcname trns = local $ \env ->
   case M.lookup srcname $ varsInScope env of
     Just (IsArray src' _ aliases input) ->
@@ -395,10 +395,10 @@
 --   @consumed@ is the set of names consumed by the SOAC.
 --   Output: a new Fusion Result (after processing the current SOAC binding)
 greedyFuse ::
-  [Stm] ->
+  [Stm SOACS] ->
   Names ->
   FusedRes ->
-  (Pat, StmAux (), SOAC, Names) ->
+  (Pat Type, StmAux (), SOAC, Names) ->
   FusionGM FusedRes
 greedyFuse rem_stms lam_used_nms res (out_idds, aux, orig_soac, consumed) = do
   soac <- inlineSOACInputs orig_soac
@@ -497,7 +497,7 @@
 
 prodconsGreedyFuse ::
   FusedRes ->
-  (Pat, StmAux (), SOAC, Names) ->
+  (Pat Type, StmAux (), SOAC, Names) ->
   FusionGM (Bool, [FusedKer], [KernName], [FusedKer], [KernName])
 prodconsGreedyFuse res (out_idds, aux, soac, consumed) = do
   let out_nms = patNames out_idds -- Extract VNames from output patterns
@@ -525,9 +525,9 @@
     certifyKer k = k {kerAux = kerAux k <> aux}
 
 horizontGreedyFuse ::
-  [Stm] ->
+  [Stm SOACS] ->
   FusedRes ->
-  (Pat, StmAux (), SOAC, Names) ->
+  (Pat Type, StmAux (), SOAC, Names) ->
   FusionGM (Bool, [FusedKer], [KernName], [FusedKer], [KernName])
 horizontGreedyFuse rem_stms res (out_idds, aux, soac, consumed) = do
   (inp_nms, _) <- soacInputs soac
@@ -671,11 +671,11 @@
 ------------------------------------------------------------------------
 ------------------------------------------------------------------------
 
-fusionGatherBody :: FusedRes -> Body -> FusionGM FusedRes
+fusionGatherBody :: FusedRes -> Body SOACS -> FusionGM FusedRes
 fusionGatherBody fres (Body _ stms res) =
   fusionGatherStms fres (stmsToList stms) res
 
-fusionGatherStms :: FusedRes -> [Stm] -> Result -> FusionGM FusedRes
+fusionGatherStms :: FusedRes -> [Stm SOACS] -> Result -> FusionGM FusedRes
 -- Some forms of do-loops can profitably be considered streamSeqs.  We
 -- are careful to ensure that the generated nested loop cannot itself
 -- be considered a stream, to avoid infinite recursion.
@@ -795,7 +795,7 @@
 fusionGatherStms fres [] res =
   foldM fusionGatherExp fres $ map (BasicOp . SubExp . resSubExp) res
 
-fusionGatherExp :: FusedRes -> Exp -> FusionGM FusedRes
+fusionGatherExp :: FusedRes -> Exp SOACS -> FusionGM FusedRes
 fusionGatherExp fres (DoLoop merge form loop_body) = do
   fres' <- addNamesToInfusible fres $ freeIn form <> freeIn merge
   let form_idents =
@@ -853,7 +853,7 @@
 
 -- Lambdas create a new scope.  Disallow fusing from outside lambda by
 -- adding inp_arrs to the infusible set.
-fusionGatherLam :: (Names, FusedRes) -> Lambda -> FusionGM (Names, FusedRes)
+fusionGatherLam :: (Names, FusedRes) -> Lambda SOACS -> FusionGM (Names, FusedRes)
 fusionGatherLam (u_set, fres) (Lambda idds body _) = do
   new_res <- bindingParams idds $ fusionGatherBody mempty body
   -- make the inpArr infusible, so that they
@@ -882,11 +882,11 @@
   | otherwise =
     pure mempty
 
-fuseInBody :: Body -> FusionGM Body
+fuseInBody :: Body SOACS -> FusionGM (Body SOACS)
 fuseInBody (Body _ stms res) =
   Body () <$> fuseInStms stms <*> pure res
 
-fuseInExp :: Exp -> FusionGM Exp
+fuseInExp :: Exp SOACS -> FusionGM (Exp SOACS)
 -- Handle loop specially because we need to bind the types of the
 -- merge variables.
 fuseInExp (DoLoop merge form loopbody) =
@@ -908,12 +908,12 @@
       mapOnOp = mapSOACM identitySOACMapper {mapOnSOACLambda = fuseInLambda}
     }
 
-fuseInLambda :: Lambda -> FusionGM Lambda
+fuseInLambda :: Lambda SOACS -> FusionGM (Lambda SOACS)
 fuseInLambda (Lambda params body rtp) = do
   body' <- bindingParams params $ fuseInBody body
   return $ Lambda params body' rtp
 
-replaceSOAC :: Pat -> StmAux () -> Exp -> FusionGM (Stms SOACS)
+replaceSOAC :: Pat Type -> StmAux () -> Exp SOACS -> FusionGM (Stms SOACS)
 replaceSOAC (Pat []) _ _ = return mempty
 replaceSOAC pat@(Pat (patElem : _)) aux e = do
   fres <- asks fusedRes
@@ -979,7 +979,7 @@
       lam' <- simplifyAndFuseInLambda lam
       return $ SOAC.Stream w form lam' nes inps
 
-simplifyAndFuseInLambda :: Lambda -> FusionGM Lambda
+simplifyAndFuseInLambda :: Lambda SOACS -> FusionGM (Lambda SOACS)
 simplifyAndFuseInLambda lam = do
   lam' <- simplifyLambda lam
   (_, nfres) <- fusionGatherLam (mempty, mkFreshFusionRes) lam'
diff --git a/src/Futhark/Optimise/Fusion/LoopKernel.hs b/src/Futhark/Optimise/Fusion/LoopKernel.hs
--- a/src/Futhark/Optimise/Fusion/LoopKernel.hs
+++ b/src/Futhark/Optimise/Fusion/LoopKernel.hs
@@ -263,7 +263,7 @@
       l'
         `SOAC.setLambda` (inps' `SOAC.setInputs` soac)
 
-removeUnusedParams :: Lambda -> [SOAC.Input] -> (Lambda, [SOAC.Input])
+removeUnusedParams :: Lambda SOACS -> [SOAC.Input] -> (Lambda SOACS, [SOAC.Input])
 removeUnusedParams l inps =
   (l {lambdaParams = ps'}, inps')
   where
@@ -700,17 +700,17 @@
 iswim _ _ _ =
   fail "ISWIM does not apply."
 
-removeParamOuterDim :: LParam -> LParam
+removeParamOuterDim :: LParam SOACS -> LParam SOACS
 removeParamOuterDim param =
   let t = rowType $ paramType param
    in param {paramDec = t}
 
-setParamOuterDimTo :: SubExp -> LParam -> LParam
+setParamOuterDimTo :: SubExp -> LParam SOACS -> LParam SOACS
 setParamOuterDimTo w param =
   let t = paramType param `setOuterSize` w
    in param {paramDec = t}
 
-setPatOuterDimTo :: SubExp -> Pat -> Pat
+setPatOuterDimTo :: SubExp -> Pat Type -> Pat Type
 setPatOuterDimTo w = fmap (`setOuterSize` w)
 
 -- Now for fiddling with transpositions...
diff --git a/src/Futhark/Optimise/InPlaceLowering/LowerIntoStm.hs b/src/Futhark/Optimise/InPlaceLowering/LowerIntoStm.hs
--- a/src/Futhark/Optimise/InPlaceLowering/LowerIntoStm.hs
+++ b/src/Futhark/Optimise/InPlaceLowering/LowerIntoStm.hs
@@ -103,13 +103,13 @@
 lowerUpdatesIntoSegMap ::
   MonadFreshNames m =>
   Scope (Aliases GPU) ->
-  Pat (Aliases GPU) ->
+  Pat (LetDec (Aliases GPU)) ->
   [DesiredUpdate (LetDec (Aliases GPU))] ->
   SegSpace ->
   KernelBody (Aliases GPU) ->
   Maybe
     ( m
-        ( Pat (Aliases GPU),
+        ( Pat (LetDec (Aliases GPU)),
           KernelBody (Aliases GPU),
           Stms (Aliases GPU)
         )
@@ -176,7 +176,7 @@
   ) =>
   Scope rep ->
   [DesiredUpdate (LetDec rep)] ->
-  Pat rep ->
+  Pat (LetDec rep) ->
   [(FParam rep, SubExp)] ->
   LoopForm rep ->
   Body rep ->
diff --git a/src/Futhark/Optimise/InliningDeadFun.hs b/src/Futhark/Optimise/InliningDeadFun.hs
--- a/src/Futhark/Optimise/InliningDeadFun.hs
+++ b/src/Futhark/Optimise/InliningDeadFun.hs
@@ -147,7 +147,7 @@
 
 inlineFunction ::
   MonadFreshNames m =>
-  Pat ->
+  Pat Type ->
   StmAux dec ->
   [(SubExp, Diet)] ->
   (Safety, SrcLoc, [SrcLoc]) ->
@@ -189,8 +189,8 @@
 inlineInBody ::
   MonadFreshNames m =>
   M.Map Name (FunDef SOACS) ->
-  Body ->
-  m Body
+  Body SOACS ->
+  m (Body SOACS)
 inlineInBody fdmap = onBody
   where
     inline (Let pat aux (Apply fname args _ what) : rest)
diff --git a/src/Futhark/Optimise/Simplify/Engine.hs b/src/Futhark/Optimise/Simplify/Engine.hs
--- a/src/Futhark/Optimise/Simplify/Engine.hs
+++ b/src/Futhark/Optimise/Simplify/Engine.hs
@@ -112,14 +112,18 @@
       envVtable = mempty
     }
 
-type Protect m = SubExp -> Pat (Rep m) -> Op (Rep m) -> Maybe (m ())
+-- | A function that protects a hoisted operation (if possible).  The
+-- first operand is the condition of the 'If' we have hoisted out of
+-- (or equivalently, a boolean indicating whether a loop has nonzero
+-- trip count).
+type Protect m = SubExp -> Pat (LetDec (Rep m)) -> Op (Rep m) -> Maybe (m ())
 
 type SimplifyOp rep op = op -> SimpleM rep (op, Stms (Wise rep))
 
 data SimpleOps rep = SimpleOps
   { mkExpDecS ::
       ST.SymbolTable (Wise rep) ->
-      Pat (Wise rep) ->
+      Pat (LetDec (Wise rep)) ->
       Exp (Wise rep) ->
       SimpleM rep (ExpDec (Wise rep)),
     mkBodyS ::
@@ -765,7 +769,7 @@
 simplifyExp ::
   SimplifiableRep rep =>
   UT.UsageTable ->
-  Pat (Wise rep) ->
+  Pat (LetDec (Wise rep)) ->
   Exp (Wise rep) ->
   SimpleM rep (Exp (Wise rep), Stms (Wise rep))
 simplifyExp usage (Pat pes) (If cond tbranch fbranch (IfDec ts ifsort)) = do
@@ -933,8 +937,8 @@
 
 simplifyPat ::
   (SimplifiableRep rep, Simplifiable dec) =>
-  PatT dec ->
-  SimpleM rep (PatT dec)
+  Pat dec ->
+  SimpleM rep (Pat dec)
 simplifyPat (Pat xs) =
   Pat <$> mapM inspect xs
   where
diff --git a/src/Futhark/Optimise/Simplify/Rep.hs b/src/Futhark/Optimise/Simplify/Rep.hs
--- a/src/Futhark/Optimise/Simplify/Rep.hs
+++ b/src/Futhark/Optimise/Simplify/Rep.hs
@@ -194,14 +194,14 @@
 removeExpWisdom :: CanBeWise (Op rep) => Exp (Wise rep) -> Exp rep
 removeExpWisdom = runIdentity . rephraseExp removeWisdom
 
-removePatWisdom :: PatT (VarWisdom, a) -> PatT a
+removePatWisdom :: Pat (VarWisdom, a) -> Pat a
 removePatWisdom = runIdentity . rephrasePat (return . snd)
 
 addWisdomToPat ::
   (ASTRep rep, CanBeWise (Op rep)) =>
-  Pat rep ->
+  Pat (LetDec rep) ->
   Exp (Wise rep) ->
-  Pat (Wise rep)
+  Pat (LetDec (Wise rep))
 addWisdomToPat pat e =
   Pat $ map f $ Aliases.mkPatAliases pat e
   where
@@ -227,7 +227,7 @@
 
 mkWiseLetStm ::
   (ASTRep rep, CanBeWise (Op rep)) =>
-  Pat rep ->
+  Pat (LetDec rep) ->
   StmAux (ExpDec rep) ->
   Exp (Wise rep) ->
   Stm (Wise rep)
@@ -237,7 +237,7 @@
 
 mkWiseExpDec ::
   (ASTRep rep, CanBeWise (Op rep)) =>
-  Pat (Wise rep) ->
+  Pat (LetDec (Wise rep)) ->
   ExpDec rep ->
   Exp (Wise rep) ->
   ExpDec (Wise rep)
diff --git a/src/Futhark/Optimise/Simplify/Rule.hs b/src/Futhark/Optimise/Simplify/Rule.hs
--- a/src/Futhark/Optimise/Simplify/Rule.hs
+++ b/src/Futhark/Optimise/Simplify/Rule.hs
@@ -110,7 +110,7 @@
 
 type RuleBasicOp rep a =
   ( a ->
-    Pat rep ->
+    Pat (LetDec rep) ->
     StmAux (ExpDec rep) ->
     BasicOp ->
     Rule rep
@@ -118,28 +118,28 @@
 
 type RuleIf rep a =
   a ->
-  Pat rep ->
+  Pat (LetDec rep) ->
   StmAux (ExpDec rep) ->
   ( SubExp,
-    BodyT rep,
-    BodyT rep,
+    Body rep,
+    Body rep,
     IfDec (BranchType rep)
   ) ->
   Rule rep
 
 type RuleDoLoop rep a =
   a ->
-  Pat rep ->
+  Pat (LetDec rep) ->
   StmAux (ExpDec rep) ->
   ( [(FParam rep, SubExp)],
     LoopForm rep,
-    BodyT rep
+    Body rep
   ) ->
   Rule rep
 
 type RuleOp rep a =
   a ->
-  Pat rep ->
+  Pat (LetDec rep) ->
   StmAux (ExpDec rep) ->
   Op rep ->
   Rule rep
diff --git a/src/Futhark/Optimise/Simplify/Rules/BasicOp.hs b/src/Futhark/Optimise/Simplify/Rules/BasicOp.hs
--- a/src/Futhark/Optimise/Simplify/Rules/BasicOp.hs
+++ b/src/Futhark/Optimise/Simplify/Rules/BasicOp.hs
@@ -12,6 +12,7 @@
 
 import Control.Monad
 import Data.List (find, foldl', isSuffixOf, sort)
+import Data.List.NonEmpty (NonEmpty (..))
 import Futhark.Analysis.PrimExp.Convert
 import qualified Futhark.Analysis.SymbolTable as ST
 import Futhark.Construct
@@ -81,14 +82,14 @@
 
 simplifyConcat :: BuilderOps rep => BottomUpRuleBasicOp rep
 -- concat@1(transpose(x),transpose(y)) == transpose(concat@0(x,y))
-simplifyConcat (vtable, _) pat _ (Concat i x xs new_d)
+simplifyConcat (vtable, _) pat _ (Concat i (x :| xs) new_d)
   | Just r <- arrayRank <$> ST.lookupType x vtable,
     let perm = [i] ++ [0 .. i -1] ++ [i + 1 .. r -1],
     Just (x', x_cs) <- transposedBy perm x,
     Just (xs', xs_cs) <- unzip <$> mapM (transposedBy perm) xs = Simplify $ do
     concat_rearrange <-
       certifying (x_cs <> mconcat xs_cs) $
-        letExp "concat_rearrange" $ BasicOp $ Concat 0 x' xs' new_d
+        letExp "concat_rearrange" $ BasicOp $ Concat 0 (x' :| xs') new_d
     letBind pat $ BasicOp $ Rearrange perm concat_rearrange
   where
     transposedBy perm1 v =
@@ -99,28 +100,28 @@
 
 -- Removing a concatenation that involves only a single array.  This
 -- may be produced as a result of other simplification rules.
-simplifyConcat _ pat aux (Concat _ x [] _) =
+simplifyConcat _ pat aux (Concat _ (x :| []) _) =
   Simplify $
     -- Still need a copy because Concat produces a fresh array.
     auxing aux $ letBind pat $ BasicOp $ Copy x
 -- concat xs (concat ys zs) == concat xs ys zs
-simplifyConcat (vtable, _) pat (StmAux cs attrs _) (Concat i x xs new_d)
+simplifyConcat (vtable, _) pat (StmAux cs attrs _) (Concat i (x :| xs) new_d)
   | x' /= x || concat xs' /= xs =
     Simplify $
       certifying (cs <> x_cs <> mconcat xs_cs) $
         attributing attrs $
           letBind pat $
-            BasicOp $ Concat i x' (zs ++ concat xs') new_d
+            BasicOp $ Concat i (x' :| zs ++ concat xs') new_d
   where
     (x' : zs, x_cs) = isConcat x
     (xs', xs_cs) = unzip $ map isConcat xs
     isConcat v = case ST.lookupBasicOp v vtable of
-      Just (Concat j y ys _, v_cs) | j == i -> (y : ys, v_cs)
+      Just (Concat j (y :| ys) _, v_cs) | j == i -> (y : ys, v_cs)
       _ -> ([v], mempty)
 
 -- Fusing arguments to the concat when possible.  Only done when
 -- concatenating along the outer dimension for now.
-simplifyConcat (vtable, _) pat aux (Concat 0 x xs outer_w)
+simplifyConcat (vtable, _) pat aux (Concat 0 (x :| xs) outer_w)
   | -- We produce the to-be-concatenated arrays in reverse order, so
     -- reverse them back.
     y : ys <-
@@ -133,7 +134,7 @@
       elem_type <- lookupType x
       y' <- fromConcatArg elem_type y
       ys' <- mapM (fromConcatArg elem_type) ys
-      auxing aux $ letBind pat $ BasicOp $ Concat 0 y' ys' outer_w
+      auxing aux $ letBind pat $ BasicOp $ Concat 0 (y' :| ys') outer_w
   where
     -- If we fuse so much that there is only a single input left, then
     -- it must have the right size.
diff --git a/src/Futhark/Optimise/Simplify/Rules/ClosedForm.hs b/src/Futhark/Optimise/Simplify/Rules/ClosedForm.hs
--- a/src/Futhark/Optimise/Simplify/Rules/ClosedForm.hs
+++ b/src/Futhark/Optimise/Simplify/Rules/ClosedForm.hs
@@ -41,7 +41,7 @@
 foldClosedForm ::
   (ASTRep rep, BuilderOps rep) =>
   VarLookup rep ->
-  Pat rep ->
+  Pat (LetDec rep) ->
   Lambda rep ->
   [SubExp] ->
   [VName] ->
@@ -79,7 +79,7 @@
 -- the do-loop can be expressed in a closed form.
 loopClosedForm ::
   (ASTRep rep, BuilderOps rep) =>
-  Pat rep ->
+  Pat (LetDec rep) ->
   [(FParam rep, SubExp)] ->
   Names ->
   IntType ->
diff --git a/src/Futhark/Optimise/Simplify/Rules/Index.hs b/src/Futhark/Optimise/Simplify/Rules/Index.hs
--- a/src/Futhark/Optimise/Simplify/Rules/Index.hs
+++ b/src/Futhark/Optimise/Simplify/Rules/Index.hs
@@ -7,6 +7,7 @@
   )
 where
 
+import Data.List.NonEmpty (NonEmpty (..))
 import Data.Maybe
 import Futhark.Analysis.PrimExp.Convert
 import qualified Futhark.Analysis.SymbolTable as ST
@@ -165,7 +166,7 @@
     Just (Reshape [_] v2, cs)
       | Just [_] <- arrayDims <$> seType (Var v2) ->
         Just $ pure $ IndexResult cs v2 $ Slice inds
-    Just (Concat d x xs _, cs)
+    Just (Concat d (x :| xs) _, cs)
       | -- HACK: simplifying the indexing of an N-array concatenation
         -- is going to produce an N-deep if expression, which is bad
         -- when N is large.  To try to avoid that, we use the
diff --git a/src/Futhark/Optimise/Simplify/Rules/Simple.hs b/src/Futhark/Optimise/Simplify/Rules/Simple.hs
--- a/src/Futhark/Optimise/Simplify/Rules/Simple.hs
+++ b/src/Futhark/Optimise/Simplify/Rules/Simple.hs
@@ -59,6 +59,15 @@
 simplifyBinOp _ _ (BinOp op (Constant v1) (Constant v2))
   | Just res <- doBinOp op v1 v2 =
     constRes res
+-- By normalisation, constants are always on the left.
+--
+-- x+(y+z) = (x+y)+z (where x and y are constants).
+simplifyBinOp look _ (BinOp op1 (Constant x1) (Var y1))
+  | associativeBinOp op1,
+    Just (BasicOp (BinOp op2 (Constant x2) y2), cs) <- look y1,
+    op1 == op2,
+    Just res <- doBinOp op1 x1 x2 =
+    Just (BinOp op1 (Constant res) y2, cs)
 simplifyBinOp look _ (BinOp Add {} e1 e2)
   | isCt0 e1 = resIsSubExp e2
   | isCt0 e2 = resIsSubExp e1
@@ -365,7 +374,7 @@
     improveReshape
   ]
 
--- | Try to simplify the given 'BasicOp', returning a new 'BasicOp'
+-- | Try to simplify the given t'BasicOp', returning a new t'BasicOp'
 -- and certificates that it must depend on.
 {-# NOINLINE applySimpleRules #-}
 applySimpleRules ::
diff --git a/src/Futhark/Optimise/TileLoops.hs b/src/Futhark/Optimise/TileLoops.hs
--- a/src/Futhark/Optimise/TileLoops.hs
+++ b/src/Futhark/Optimise/TileLoops.hs
@@ -344,7 +344,7 @@
   Names ->
   (Stms GPU, Tiling, TiledBody) ->
   [Type] ->
-  Pat GPU ->
+  Pat Type ->
   StmAux (ExpDec GPU) ->
   [(FParam GPU, SubExp)] ->
   VName ->
@@ -625,7 +625,7 @@
 postludeGeneric ::
   Tiling ->
   PrivStms ->
-  Pat GPU ->
+  Pat Type ->
   [VName] ->
   Stms GPU ->
   Result ->
@@ -655,7 +655,7 @@
   DoTiling gtids kdims ->
   SegLevel ->
   [Type] ->
-  Pat GPU ->
+  Pat Type ->
   gtids ->
   kdims ->
   SubExp ->
diff --git a/src/Futhark/Optimise/Unstream.hs b/src/Futhark/Optimise/Unstream.hs
--- a/src/Futhark/Optimise/Unstream.hs
+++ b/src/Futhark/Optimise/Unstream.hs
@@ -61,7 +61,7 @@
 type UnstreamM rep = ReaderT (Scope rep) (State VNameSource)
 
 type OnOp rep =
-  Pat rep -> StmAux (ExpDec rep) -> Op rep -> UnstreamM rep [Stm rep]
+  Pat (LetDec rep) -> StmAux (ExpDec rep) -> Op rep -> UnstreamM rep [Stm rep]
 
 optimiseStms ::
   ASTRep rep =>
diff --git a/src/Futhark/Pass/ExpandAllocations.hs b/src/Futhark/Pass/ExpandAllocations.hs
--- a/src/Futhark/Pass/ExpandAllocations.hs
+++ b/src/Futhark/Pass/ExpandAllocations.hs
@@ -601,9 +601,9 @@
   pure (scope', stm)
   where
     pick ::
-      PatElemT (MemInfo SubExp NoUniqueness MemBind) ->
+      PatElem (MemInfo SubExp NoUniqueness MemBind) ->
       ExpReturns ->
-      PatElemT (MemInfo SubExp NoUniqueness MemBind)
+      PatElem (MemInfo SubExp NoUniqueness MemBind)
     pick
       (PatElem name (MemArray pt s u _ret))
       (MemArray _ _ _ (Just (ReturnsInBlock m extixfun)))
@@ -617,7 +617,7 @@
         inst Ext {} = Nothing
         inst (Free x) = pure x
 
-offsetMemoryInPat :: Pat GPUMem -> [ExpReturns] -> OffsetM (Pat GPUMem)
+offsetMemoryInPat :: Pat LetDecMem -> [ExpReturns] -> OffsetM (Pat LetDecMem)
 offsetMemoryInPat (Pat pes) rets = do
   Pat <$> zipWithM onPE pes rets
   where
diff --git a/src/Futhark/Pass/ExplicitAllocations.hs b/src/Futhark/Pass/ExplicitAllocations.hs
--- a/src/Futhark/Pass/ExplicitAllocations.hs
+++ b/src/Futhark/Pass/ExplicitAllocations.hs
@@ -224,7 +224,7 @@
   [VName] ->
   Exp (Rep m) ->
   [ExpHint] ->
-  m (PatT LetDecMem)
+  m (Pat LetDecMem)
 patWithAllocations def_space chunkmap names e hints = do
   ts' <- instantiateShapes' names <$> expExtType e
   let idents = zipWith Ident names ts'
@@ -246,7 +246,7 @@
   [Ident] ->
   [ExpReturns] ->
   [ExpHint] ->
-  m [PatElemT LetDecMem]
+  m [PatElem LetDecMem]
 allocsForPat def_space chunkmap some_idents rts hints = do
   idents <- mkMissingIdents some_idents rts
 
@@ -1022,7 +1022,7 @@
           return (p {paramDec = MemAcc acc ispace ts u}, a)
 
 class SizeSubst op where
-  opSizeSubst :: PatT dec -> op -> ChunkMap
+  opSizeSubst :: Pat dec -> op -> ChunkMap
   opIsConst :: op -> Bool
   opIsConst = const False
 
diff --git a/src/Futhark/Pass/ExtractKernels.hs b/src/Futhark/Pass/ExtractKernels.hs
--- a/src/Futhark/Pass/ExtractKernels.hs
+++ b/src/Futhark/Pass/ExtractKernels.hs
@@ -165,8 +165,7 @@
 import Control.Monad.Reader
 import Data.Bifunctor (first)
 import Data.Maybe
-import qualified Futhark.IR.GPU as Out
-import Futhark.IR.GPU.Op
+import Futhark.IR.GPU
 import Futhark.IR.SOACS
 import Futhark.IR.SOACS.Simplify (simplifyStms)
 import Futhark.MonadFreshNames
@@ -186,7 +185,7 @@
 
 -- | Transform a program using SOACs to a program using explicit
 -- kernels, using the kernel extraction transformation.
-extractKernels :: Pass SOACS Out.GPU
+extractKernels :: Pass SOACS GPU
 extractKernels =
   Pass
     { passName = "extract kernels",
@@ -194,11 +193,11 @@
       passFunction = transformProg
     }
 
-transformProg :: Prog SOACS -> PassM (Prog Out.GPU)
+transformProg :: Prog SOACS -> PassM (Prog GPU)
 transformProg (Prog consts funs) = do
   consts' <- runDistribM $ transformStms mempty $ stmsToList consts
   funs' <- mapM (transformFunDef $ scopeOf consts') funs
-  return $ Prog consts' funs'
+  pure $ Prog consts' funs'
 
 -- In order to generate more stable threshold names, we keep track of
 -- the numbers used for thresholds separately from the ordinary name
@@ -208,13 +207,13 @@
     stateThresholdCounter :: Int
   }
 
-newtype DistribM a = DistribM (RWS (Scope Out.GPU) Log State a)
+newtype DistribM a = DistribM (RWS (Scope GPU) Log State a)
   deriving
     ( Functor,
       Applicative,
       Monad,
-      HasScope Out.GPU,
-      LocalScope Out.GPU,
+      HasScope GPU,
+      LocalScope GPU,
       MonadState State,
       MonadLogger
     )
@@ -232,29 +231,29 @@
     let (x, s, msgs) = runRWS m mempty (State src 0)
      in ((x, msgs), stateNameSource s)
   addLog msgs
-  return x
+  pure x
 
 transformFunDef ::
   (MonadFreshNames m, MonadLogger m) =>
-  Scope Out.GPU ->
+  Scope GPU ->
   FunDef SOACS ->
-  m (Out.FunDef Out.GPU)
+  m (FunDef GPU)
 transformFunDef scope (FunDef entry attrs name rettype params body) = runDistribM $ do
   body' <-
     localScope (scope <> scopeOfFParams params) $
       transformBody mempty body
-  return $ FunDef entry attrs name rettype params body'
+  pure $ FunDef entry attrs name rettype params body'
 
-type GPUStms = Stms Out.GPU
+type GPUStms = Stms GPU
 
-transformBody :: KernelPath -> Body -> DistribM (Out.Body Out.GPU)
+transformBody :: KernelPath -> Body SOACS -> DistribM (Body GPU)
 transformBody path body = do
   stms <- transformStms path $ stmsToList $ bodyStms body
-  return $ mkBody stms $ bodyResult body
+  pure $ mkBody stms $ bodyResult body
 
-transformStms :: KernelPath -> [Stm] -> DistribM GPUStms
+transformStms :: KernelPath -> [Stm SOACS] -> DistribM GPUStms
 transformStms _ [] =
-  return mempty
+  pure mempty
 transformStms path (stm : stms) =
   sequentialisedUnbalancedStm stm >>= \case
     Nothing -> do
@@ -264,7 +263,7 @@
     Just stms' ->
       transformStms path $ stmsToList stms' <> stms
 
-unbalancedLambda :: Lambda -> Bool
+unbalancedLambda :: Lambda SOACS -> Bool
 unbalancedLambda orig_lam =
   unbalancedBody (namesFromList $ map paramName $ lambdaParams orig_lam) $
     lambdaBody orig_lam
@@ -294,7 +293,7 @@
     unbalancedStm _ (Apply fname _ _ _) =
       not $ isBuiltInFunction fname
 
-sequentialisedUnbalancedStm :: Stm -> DistribM (Maybe (Stms SOACS))
+sequentialisedUnbalancedStm :: Stm SOACS -> DistribM (Maybe (Stms SOACS))
 sequentialisedUnbalancedStm (Let pat _ (Op soac@(Screma _ _ form)))
   | Just (_, lam2) <- isRedomapSOAC form,
     unbalancedLambda lam2,
@@ -302,13 +301,13 @@
     types <- asksScope scopeForSOACs
     Just . snd <$> runBuilderT (FOT.transformSOAC pat soac) types
 sequentialisedUnbalancedStm _ =
-  return Nothing
+  pure Nothing
 
 cmpSizeLe ::
   String ->
-  Out.SizeClass ->
+  SizeClass ->
   [SubExp] ->
-  DistribM ((SubExp, Name), Out.Stms Out.GPU)
+  DistribM ((SubExp, Name), Stms GPU)
 cmpSizeLe desc size_class to_what = do
   x <- gets stateThresholdCounter
   modify $ \s -> s {stateThresholdCounter = x + 1}
@@ -318,14 +317,14 @@
       letSubExp "comparatee"
         =<< foldBinOp (Mul Int64 OverflowUndef) (intConst Int64 1) to_what
     cmp_res <- letSubExp desc $ Op $ SizeOp $ CmpSizeLe size_key size_class to_what'
-    return (cmp_res, size_key)
+    pure (cmp_res, size_key)
 
 kernelAlternatives ::
-  (MonadFreshNames m, HasScope Out.GPU m) =>
-  Out.Pat Out.GPU ->
-  Out.Body Out.GPU ->
-  [(SubExp, Out.Body Out.GPU)] ->
-  m (Out.Stms Out.GPU)
+  (MonadFreshNames m, HasScope GPU m) =>
+  Pat Type ->
+  Body GPU ->
+  [(SubExp, Body GPU)] ->
+  m (Stms GPU)
 kernelAlternatives pat default_body [] = runBuilder_ $ do
   ses <- bodyBind default_body
   forM_ (zip (patNames pat) ses) $ \(name, SubExpRes cs se) ->
@@ -333,7 +332,7 @@
 kernelAlternatives pat default_body ((cond, alt) : alts) = runBuilder_ $ do
   alts_pat <- fmap Pat . forM (patElems pat) $ \pe -> do
     name <- newVName $ baseString $ patElemName pe
-    return pe {patElemName = name}
+    pure pe {patElemName = name}
 
   alt_stms <- kernelAlternatives alts_pat default_body alts
   let alt_body = mkBody alt_stms $ varsRes $ patNames alts_pat
@@ -341,13 +340,13 @@
   letBind pat $
     If cond alt alt_body $ IfDec (staticShapes (patTypes pat)) IfEquiv
 
-transformLambda :: KernelPath -> Lambda -> DistribM (Out.Lambda Out.GPU)
+transformLambda :: KernelPath -> Lambda SOACS -> DistribM (Lambda GPU)
 transformLambda path (Lambda params body ret) =
   Lambda params
     <$> localScope (scopeOfLParams params) (transformBody path body)
     <*> pure ret
 
-transformStm :: KernelPath -> Stm -> DistribM GPUStms
+transformStm :: KernelPath -> Stm SOACS -> DistribM GPUStms
 transformStm _ stm
   | "sequential" `inAttrs` stmAuxAttrs (stmAux stm) =
     runBuilder_ $ FOT.transformStmRecursively stm
@@ -358,7 +357,7 @@
 transformStm path (Let pat aux (If c tb fb rt)) = do
   tb' <- transformBody path tb
   fb' <- transformBody path fb
-  return $ oneStm $ Let pat aux $ If c tb' fb' rt
+  pure $ oneStm $ Let pat aux $ If c tb' fb' rt
 transformStm path (Let pat aux (WithAcc inputs lam)) =
   oneStm . Let pat aux
     <$> (WithAcc (map transformInput inputs) <$> transformLambda path lam)
@@ -388,7 +387,7 @@
     scan_ops <- forM scans $ \(Scan scan_lam nes) -> do
       (scan_lam', nes', shape) <- determineReduceOp scan_lam nes
       let scan_lam'' = soacsLambdaToGPU scan_lam'
-      return $ SegBinOp Noncommutative scan_lam'' nes' shape
+      pure $ SegBinOp Noncommutative scan_lam'' nes' shape
     let map_lam_sequential = soacsLambdaToGPU map_lam
     lvl <- segThreadCapped [w] "segscan" $ NoRecommendation SegNoVirt
     addStms . fmap (certify cs)
@@ -411,7 +410,7 @@
                   | commutativeLambda red_lam' = Commutative
                   | otherwise = comm
                 red_lam'' = soacsLambdaToGPU red_lam'
-            return $ SegBinOp comm' red_lam'' nes' shape
+            pure $ SegBinOp comm' red_lam'' nes' shape
           let map_lam_sequential = soacsLambdaToGPU map_lam
           lvl <- segThreadCapped [w] "segred" $ NoRecommendation SegNoVirt
           addStms . fmap (certify cs)
@@ -549,11 +548,11 @@
               foldMap (foldMap resCerts . fst) is_vs
                 <> foldMap (resCerts . snd) is_vs
             is_vs' = [(Slice $ map (DimFix . resSubExp) is, resSubExp v) | (is, v) <- is_vs]
-        return $ WriteReturns res_cs a_w a is_vs'
+        pure $ WriteReturns res_cs a_w a is_vs'
       body = KernelBody () kstms krets
       inputs = do
         (p, p_a) <- zip (lambdaParams lam') ivs
-        return $ KernelInput (paramName p) (paramType p) p_a [Var write_i]
+        pure $ KernelInput (paramName p) (paramType p) p_a [Var write_i]
   (kernel, stms) <-
     mapKernel
       segThreadCapped
@@ -583,14 +582,14 @@
   [SubExp] ->
   KernelPath ->
   Maybe Int64 ->
-  DistribM ((SubExp, Name), Out.Stms Out.GPU)
+  DistribM ((SubExp, Name), Stms GPU)
 sufficientParallelism desc ws path def =
-  cmpSizeLe desc (Out.SizeThreshold path def) ws
+  cmpSizeLe desc (SizeThreshold path def) ws
 
 -- | Intra-group parallelism is worthwhile if the lambda contains more
 -- than one instance of non-map nested parallelism, or any nested
 -- parallelism inside a loop.
-worthIntraGroup :: Lambda -> Bool
+worthIntraGroup :: Lambda SOACS -> Bool
 worthIntraGroup lam = bodyInterest (lambdaBody lam) > 1
   where
     bodyInterest body =
@@ -628,7 +627,7 @@
 
 -- | A lambda is worth sequentialising if it contains enough nested
 -- parallelism of an interesting kind.
-worthSequentialising :: Lambda -> Bool
+worthSequentialising :: Lambda SOACS -> Bool
 worthSequentialising lam = bodyInterest (lambdaBody lam) > 1
   where
     bodyInterest body =
@@ -665,7 +664,7 @@
 onTopLevelStms ::
   KernelPath ->
   Stms SOACS ->
-  DistNestT Out.GPU DistribM GPUStms
+  DistNestT GPU DistribM GPUStms
 onTopLevelStms path stms =
   liftInner $ transformStms path $ stmsToList stms
 
@@ -729,11 +728,11 @@
 onMap' ::
   KernelNest ->
   KernelPath ->
-  (KernelPath -> DistribM (Out.Stms Out.GPU)) ->
-  (KernelPath -> DistribM (Out.Stms Out.GPU)) ->
-  Pat ->
-  Lambda ->
-  DistribM (Out.Stms Out.GPU)
+  (KernelPath -> DistribM (Stms GPU)) ->
+  (KernelPath -> DistribM (Stms GPU)) ->
+  Pat Type ->
+  Lambda SOACS ->
+  DistribM (Stms GPU)
 onMap' loopnest path mk_seq_stms mk_par_stms pat lam = do
   -- Some of the control flow here looks a bit convoluted because we
   -- are trying to avoid generating unneeded threshold parameters,
@@ -745,7 +744,7 @@
     if onlyExploitIntra (stmAuxAttrs aux)
       || (worthIntraGroup lam && mayExploitIntra attrs)
       then flip runReaderT types $ intraGroupParallelise loopnest lam
-      else return Nothing
+      else pure Nothing
 
   case intra of
     _ | "sequential_inner" `inAttrs` attrs -> do
@@ -841,7 +840,7 @@
             addStms intra_prelude
 
             max_group_size <-
-              letSubExp "max_group_size" $ Op $ SizeOp $ Out.GetSizeMax Out.SizeGroup
+              letSubExp "max_group_size" $ Op $ SizeOp $ GetSizeMax SizeGroup
             fits <-
               letSubExp "fits" $
                 BasicOp $
@@ -850,16 +849,16 @@
             addStms check_suff_stms
 
             intra_ok <- letSubExp "intra_suff_and_fits" $ BasicOp $ BinOp LogAnd fits intra_suff
-            return (intra_ok, suff_key)
+            pure (intra_ok, suff_key)
 
         group_par_body <- renameBody $ mkBody intra_stms res
         pure (group_par_body, intra_ok, intra_suff_key, intra_suff_stms)
 
 removeUnusedMapResults ::
-  PatT Type ->
+  Pat Type ->
   [SubExpRes] ->
-  LambdaT rep ->
-  Maybe ([Int], PatT Type, LambdaT rep)
+  Lambda rep ->
+  Maybe ([Int], Pat Type, Lambda rep)
 removeUnusedMapResults (Pat pes) res lam = do
   let (pes', body_res) =
         unzip $ filter (used . fst) $ zip pes $ bodyResult (lambdaBody lam)
@@ -871,8 +870,8 @@
 onInnerMap ::
   KernelPath ->
   MapLoop ->
-  DistAcc Out.GPU ->
-  DistNestT Out.GPU DistribM (DistAcc Out.GPU)
+  DistAcc GPU ->
+  DistNestT GPU DistribM (DistAcc GPU)
 onInnerMap path maploop@(MapLoop pat aux w lam arrs) acc
   | unbalancedLambda lam,
     lambdaContainsParallelism lam =
@@ -933,10 +932,10 @@
             <$> onMap'
               nest'
               path
-              (const $ return $ oneStm sequentialised_kernel)
+              (const $ pure $ oneStm sequentialised_kernel)
               exploitInnerParallelism
               outer_pat
               lam''
 
       postStm stms
-      return acc'
+      pure acc'
diff --git a/src/Futhark/Pass/ExtractKernels/BlockedKernel.hs b/src/Futhark/Pass/ExtractKernels/BlockedKernel.hs
--- a/src/Futhark/Pass/ExtractKernels/BlockedKernel.hs
+++ b/src/Futhark/Pass/ExtractKernels/BlockedKernel.hs
@@ -76,7 +76,7 @@
 segRed ::
   (MonadFreshNames m, DistRep rep, HasScope rep m) =>
   SegOpLevel rep ->
-  Pat rep ->
+  Pat (LetDec rep) ->
   Certs ->
   SubExp -> -- segment size
   [SegBinOp rep] ->
@@ -95,7 +95,7 @@
 segScan ::
   (MonadFreshNames m, DistRep rep, HasScope rep m) =>
   SegOpLevel rep ->
-  Pat rep ->
+  Pat (LetDec rep) ->
   Certs ->
   SubExp -> -- segment size
   [SegBinOp rep] ->
@@ -114,7 +114,7 @@
 segMap ::
   (MonadFreshNames m, DistRep rep, HasScope rep m) =>
   SegOpLevel rep ->
-  Pat rep ->
+  Pat (LetDec rep) ->
   SubExp -> -- segment size
   Lambda rep ->
   [VName] ->
@@ -129,9 +129,9 @@
         SegMap lvl kspace (lambdaReturnType map_lam) kbody
 
 dummyDim ::
-  (MonadFreshNames m, MonadBuilder m, DistRep (Rep m)) =>
-  Pat (Rep m) ->
-  m (Pat (Rep m), [(VName, SubExp)], m ())
+  (MonadFreshNames m, MonadBuilder m) =>
+  Pat Type ->
+  m (Pat Type, [(VName, SubExp)], m ())
 dummyDim pat = do
   -- We add a unit-size segment on top to ensure that the result
   -- of the SegRed is an array, which we then immediately index.
@@ -158,7 +158,7 @@
 nonSegRed ::
   (MonadFreshNames m, DistRep rep, HasScope rep m) =>
   SegOpLevel rep ->
-  Pat rep ->
+  Pat Type ->
   SubExp ->
   [SegBinOp rep] ->
   Lambda rep ->
@@ -172,7 +172,7 @@
 segHist ::
   (DistRep rep, MonadFreshNames m, HasScope rep m) =>
   SegOpLevel rep ->
-  Pat rep ->
+  Pat Type ->
   SubExp ->
   -- | Segment indexes and sizes.
   [(VName, SubExp)] ->
diff --git a/src/Futhark/Pass/ExtractKernels/DistributeNests.hs b/src/Futhark/Pass/ExtractKernels/DistributeNests.hs
--- a/src/Futhark/Pass/ExtractKernels/DistributeNests.hs
+++ b/src/Futhark/Pass/ExtractKernels/DistributeNests.hs
@@ -42,6 +42,7 @@
 import Control.Monad.Writer.Strict
 import Data.Function ((&))
 import Data.List (find, partition, tails)
+import Data.List.NonEmpty (NonEmpty (..))
 import qualified Data.Map as M
 import Data.Maybe
 import Futhark.IR
@@ -65,7 +66,7 @@
 scopeForSOACs :: SameScope rep SOACS => Scope rep -> Scope SOACS
 scopeForSOACs = castScope
 
-data MapLoop = MapLoop SOACS.Pat (StmAux ()) SubExp SOACS.Lambda [VName]
+data MapLoop = MapLoop (Pat Type) (StmAux ()) SubExp (Lambda SOACS) [VName]
 
 mapLoopStm :: MapLoop -> Stm SOACS
 mapLoopStm (MapLoop pat aux w lam arrs) =
@@ -275,7 +276,7 @@
 
 mapNesting ::
   (MonadFreshNames m, DistRep rep) =>
-  PatT Type ->
+  Pat Type ->
   StmAux () ->
   SubExp ->
   Lambda SOACS ->
@@ -626,7 +627,7 @@
               =<< segmentedUpdateKernel nest' perm (stmAuxCerts aux) arr slice v
             return acc'
       _ -> addStmToAcc stm acc
-maybeDistributeStm stm@(Let _ aux (BasicOp (Concat d x xs w))) acc =
+maybeDistributeStm stm@(Let _ aux (BasicOp (Concat d (x :| xs) w))) acc =
   distributeSingleStm acc stm >>= \case
     Just (kernels, _, nest, acc') ->
       localScope (typeEnvFromDistAcc acc') $
@@ -639,7 +640,7 @@
       isSegmentedOp nest [0] mempty mempty [] (x : xs) $
         \pat _ _ _ (x' : xs') ->
           let d' = d + length (snd nest) + 1
-           in addStm $ Let pat aux $ BasicOp $ Concat d' x' xs' w
+           in addStm $ Let pat aux $ BasicOp $ Concat d' (x' :| xs') w
 maybeDistributeStm stm acc =
   addStmToAcc stm acc
 
@@ -648,7 +649,7 @@
   DistAcc rep ->
   Stm SOACS ->
   VName ->
-  (KernelNest -> PatT Type -> VName -> DistNestT rep m (Stms rep)) ->
+  (KernelNest -> Pat Type -> VName -> DistNestT rep m (Stms rep)) ->
   DistNestT rep m (DistAcc rep)
 distributeSingleUnaryStm acc stm stm_arr f =
   distributeSingleStm acc stm >>= \case
@@ -748,7 +749,7 @@
   (MonadFreshNames m, LocalScope rep m, DistRep rep) =>
   KernelNest ->
   [Int] ->
-  PatT Type ->
+  Pat Type ->
   Certs ->
   SubExp ->
   Lambda rep ->
@@ -964,7 +965,7 @@
   (MonadBuilder m, DistRep (Rep m)) =>
   (Lambda SOACS -> m (Lambda (Rep m))) ->
   SegOpLevel (Rep m) ->
-  PatT Type ->
+  Pat Type ->
   [(VName, SubExp)] ->
   [KernelInput] ->
   Certs ->
@@ -1072,7 +1073,7 @@
   Names ->
   [SubExp] ->
   [VName] ->
-  ( PatT Type ->
+  ( Pat Type ->
     [(VName, SubExp)] ->
     [KernelInput] ->
     [SubExp] ->
@@ -1135,7 +1136,7 @@
 
         m pat ispace kernel_inps nes' nested_arrs
 
-permutationAndMissing :: PatT Type -> Result -> Maybe ([Int], [PatElemT Type])
+permutationAndMissing :: Pat Type -> Result -> Maybe ([Int], [PatElem Type])
 permutationAndMissing (Pat pes) res = do
   let (_used, unused) =
         partition ((`nameIn` freeIn res) . patElemName) pes
@@ -1146,7 +1147,7 @@
 
 -- Add extra pattern elements to every kernel nesting level.
 expandKernelNest ::
-  MonadFreshNames m => [PatElemT Type] -> KernelNest -> m KernelNest
+  MonadFreshNames m => [PatElem Type] -> KernelNest -> m KernelNest
 expandKernelNest pes (outer_nest, inner_nests) = do
   let outer_size =
         loopNestingWidth outer_nest :
diff --git a/src/Futhark/Pass/ExtractKernels/Distribution.hs b/src/Futhark/Pass/ExtractKernels/Distribution.hs
--- a/src/Futhark/Pass/ExtractKernels/Distribution.hs
+++ b/src/Futhark/Pass/ExtractKernels/Distribution.hs
@@ -66,7 +66,7 @@
 import Futhark.Util
 import Futhark.Util.Log
 
-type Target = (PatT Type, Result)
+type Target = (Pat Type, Result)
 
 -- | First pair element is the very innermost ("current") target.  In
 -- the list, the outermost target comes first.  Invariant: Every
@@ -120,7 +120,7 @@
 targetsScope (Targets t ts) = mconcat $ map targetScope $ t : ts
 
 data LoopNesting = MapNesting
-  { loopNestingPat :: PatT Type,
+  { loopNestingPat :: Pat Type,
     loopNestingAux :: StmAux (),
     loopNestingWidth :: SubExp,
     loopNestingParamsAndArrs :: [(Param Type, VName)]
@@ -214,7 +214,7 @@
       [] -> nest
       n : _ -> n
 
-fixNestingPatOrder :: LoopNesting -> Target -> PatT Type -> LoopNesting
+fixNestingPatOrder :: LoopNesting -> Target -> Pat Type -> LoopNesting
 fixNestingPatOrder nest (_, res) inner_pat =
   nest {loopNestingPat = basicPat pat'}
   where
@@ -320,7 +320,7 @@
     distributionExpandTarget :: Target -> Target
   }
 
-distributionInnerPat :: DistributionBody -> PatT Type
+distributionInnerPat :: DistributionBody -> Pat Type
 distributionInnerPat = fst . innerTarget . distributionTarget
 
 distributionBodyFromStms ::
@@ -373,7 +373,7 @@
 
     distributeAtNesting ::
       Nesting ->
-      PatT Type ->
+      Pat Type ->
       (LoopNesting -> KernelNest, Names) ->
       M.Map VName Ident ->
       [Ident] ->
@@ -490,9 +490,9 @@
 
 removeIdentityMappingGeneral ::
   Names ->
-  PatT Type ->
+  Pat Type ->
   Result ->
-  ( PatT Type,
+  ( Pat Type,
     Result,
     M.Map VName Ident,
     Target -> Target
@@ -520,9 +520,9 @@
 
 removeIdentityMappingFromNesting ::
   Names ->
-  PatT Type ->
+  Pat Type ->
   Result ->
-  ( PatT Type,
+  ( Pat Type,
     Result,
     M.Map VName Ident,
     Target -> Target
diff --git a/src/Futhark/Pass/ExtractKernels/ISRWIM.hs b/src/Futhark/Pass/ExtractKernels/ISRWIM.hs
--- a/src/Futhark/Pass/ExtractKernels/ISRWIM.hs
+++ b/src/Futhark/Pass/ExtractKernels/ISRWIM.hs
@@ -19,9 +19,9 @@
 -- @map(scan)
 iswim ::
   (MonadBuilder m, Rep m ~ SOACS) =>
-  Pat ->
+  Pat Type ->
   SubExp ->
-  Lambda ->
+  Lambda SOACS ->
   [(SubExp, VName)] ->
   Maybe (m ())
 iswim res_pat w scan_fun scan_input
@@ -77,10 +77,10 @@
 -- @map(reduce)
 irwim ::
   (MonadBuilder m, Rep m ~ SOACS) =>
-  Pat ->
+  Pat Type ->
   SubExp ->
   Commutativity ->
-  Lambda ->
+  Lambda SOACS ->
   [(SubExp, VName)] ->
   Maybe (m ())
 irwim res_pat w comm red_fun red_input
@@ -136,8 +136,8 @@
 -- | Does this reduce operator contain an inner map, and if so, what
 -- does that map look like?
 rwimPossible ::
-  Lambda ->
-  Maybe (Pat, Certs, SubExp, Lambda)
+  Lambda SOACS ->
+  Maybe (Pat Type, Certs, SubExp, Lambda SOACS)
 rwimPossible fun
   | Body _ stms res <- lambdaBody fun,
     [stm] <- stmsToList stms, -- Body has a single binding
@@ -156,12 +156,12 @@
   let perm = [1, 0] ++ [2 .. arrayRank t -1]
   letExp (baseString arr) $ BasicOp $ Rearrange perm arr
 
-removeParamOuterDim :: LParam -> LParam
+removeParamOuterDim :: LParam SOACS -> LParam SOACS
 removeParamOuterDim param =
   let t = rowType $ paramType param
    in param {paramDec = t}
 
-setParamOuterDimTo :: SubExp -> LParam -> LParam
+setParamOuterDimTo :: SubExp -> LParam SOACS -> LParam SOACS
 setParamOuterDimTo w param =
   let t = setOuterDimTo w $ paramType param
    in param {paramDec = t}
@@ -175,7 +175,7 @@
 setOuterDimTo w t =
   arrayOfRow (rowType t) w
 
-setPatOuterDimTo :: SubExp -> Pat -> Pat
+setPatOuterDimTo :: SubExp -> Pat Type -> Pat Type
 setPatOuterDimTo w pat =
   basicPat $ map (setIdentOuterDimTo w) $ patIdents pat
 
@@ -187,6 +187,6 @@
 stripIdentOuterDim ident =
   ident {identType = rowType $ identType ident}
 
-stripPatOuterDim :: Pat -> Pat
+stripPatOuterDim :: Pat Type -> Pat Type
 stripPatOuterDim pat =
   basicPat $ map stripIdentOuterDim $ patIdents pat
diff --git a/src/Futhark/Pass/ExtractKernels/Interchange.hs b/src/Futhark/Pass/ExtractKernels/Interchange.hs
--- a/src/Futhark/Pass/ExtractKernels/Interchange.hs
+++ b/src/Futhark/Pass/ExtractKernels/Interchange.hs
@@ -35,12 +35,13 @@
 
 -- | An encoding of a sequential do-loop with no existential context,
 -- alongside its result pattern.
-data SeqLoop = SeqLoop [Int] Pat [(FParam, SubExp)] (LoopForm SOACS) Body
+data SeqLoop
+  = SeqLoop [Int] (Pat Type) [(FParam SOACS, SubExp)] (LoopForm SOACS) (Body SOACS)
 
 loopPerm :: SeqLoop -> [Int]
 loopPerm (SeqLoop perm _ _ _ _) = perm
 
-seqLoopStm :: SeqLoop -> Stm
+seqLoopStm :: SeqLoop -> Stm SOACS
 seqLoopStm (SeqLoop _ pat merge form body) =
   Let pat (defAux ()) $ DoLoop merge form body
 
@@ -182,9 +183,11 @@
              in pure $ snd $ manifestMaps ns names $ stms <> oneStm loop_stm
       | otherwise = pure $ oneStm $ seqLoopStm loop
 
-data Branch = Branch [Int] Pat SubExp Body Body (IfDec (BranchType SOACS))
+-- | An encoding of a branch with alongside its result pattern.
+data Branch
+  = Branch [Int] (Pat Type) SubExp (Body SOACS) (Body SOACS) (IfDec (BranchType SOACS))
 
-branchStm :: Branch -> Stm
+branchStm :: Branch -> Stm SOACS
 branchStm (Branch _ pat cond tbranch fbranch ret) =
   Let pat (defAux ()) $ If cond tbranch fbranch ret
 
@@ -227,10 +230,11 @@
     runBuilder $ foldM interchangeBranch1 loop $ reverse $ kernelNestLoops nest
   return $ stms <> oneStm (branchStm loop')
 
+-- | An encoding of a WithAcc with alongside its result pattern.
 data WithAccStm
-  = WithAccStm [Int] Pat [(Shape, [VName], Maybe (Lambda, [SubExp]))] Lambda
+  = WithAccStm [Int] (Pat Type) [(Shape, [VName], Maybe (Lambda SOACS, [SubExp]))] (Lambda SOACS)
 
-withAccStm :: WithAccStm -> Stm
+withAccStm :: WithAccStm -> Stm SOACS
 withAccStm (WithAccStm _ pat inputs lam) =
   Let pat (defAux ()) $ WithAcc inputs lam
 
diff --git a/src/Futhark/Pass/ExtractKernels/Intragroup.hs b/src/Futhark/Pass/ExtractKernels/Intragroup.hs
--- a/src/Futhark/Pass/ExtractKernels/Intragroup.hs
+++ b/src/Futhark/Pass/ExtractKernels/Intragroup.hs
@@ -13,8 +13,8 @@
 import qualified Data.Map.Strict as M
 import qualified Data.Set as S
 import Futhark.Analysis.PrimExp.Convert
-import qualified Futhark.IR.GPU as Out
-import Futhark.IR.GPU.Op hiding (HistOp)
+import Futhark.IR.GPU hiding (HistOp)
+import qualified Futhark.IR.GPU.Op as GPU
 import Futhark.IR.SOACS
 import Futhark.MonadFreshNames
 import Futhark.Pass.ExtractKernels.BlockedKernel
@@ -39,16 +39,16 @@
 -- We distinguish between "minimum group size" and "maximum
 -- exploitable parallelism".
 intraGroupParallelise ::
-  (MonadFreshNames m, LocalScope Out.GPU m) =>
+  (MonadFreshNames m, LocalScope GPU m) =>
   KernelNest ->
-  Lambda ->
+  Lambda SOACS ->
   m
     ( Maybe
         ( (SubExp, SubExp),
           SubExp,
           Log,
-          Out.Stms Out.GPU,
-          Out.Stms Out.GPU
+          Stms GPU,
+          Stms GPU
         )
     )
 intraGroupParallelise knest lam = runMaybeT $ do
@@ -103,7 +103,7 @@
           then
             eBinOp
               (SMin Int64)
-              (eSubExp =<< letSubExp "max_group_size" (Op $ SizeOp $ Out.GetSizeMax Out.SizeGroup))
+              (eSubExp =<< letSubExp "max_group_size" (Op $ SizeOp $ GetSizeMax SizeGroup))
               (eSubExp intra_avail_par)
           else foldBinOp' (SMax Int64) ws_min
 
@@ -113,7 +113,7 @@
       addStms w_stms
       read_input_stms <- runBuilder_ $ mapM readGroupKernelInput used_inps
       space <- mkSegSpace ispace
-      return (intra_avail_par, space, read_input_stms)
+      pure (intra_avail_par, space, read_input_stms)
 
   let kbody' = kbody {kernelBodyStms = read_input_stms <> kernelBodyStms kbody}
 
@@ -162,15 +162,15 @@
   mempty = IntraAcc mempty mempty mempty
 
 type IntraGroupM =
-  BuilderT Out.GPU (RWS () IntraAcc VNameSource)
+  BuilderT GPU (RWS () IntraAcc VNameSource)
 
 instance MonadLogger IntraGroupM where
   addLog log = tell mempty {accLog = log}
 
 runIntraGroupM ::
-  (MonadFreshNames m, HasScope Out.GPU m) =>
+  (MonadFreshNames m, HasScope GPU m) =>
   IntraGroupM () ->
-  m (IntraAcc, Out.Stms Out.GPU)
+  m (IntraAcc, Stms GPU)
 runIntraGroupM m = do
   scope <- castScope <$> askScope
   modifyNameSource $ \src ->
@@ -185,12 +185,12 @@
         accAvailPar = S.singleton ws
       }
 
-intraGroupBody :: SegLevel -> Body -> IntraGroupM (Out.Body Out.GPU)
+intraGroupBody :: SegLevel -> Body SOACS -> IntraGroupM (Body GPU)
 intraGroupBody lvl body = do
   stms <- collectStms_ $ intraGroupStms lvl $ bodyStms body
-  return $ mkBody stms $ bodyResult body
+  pure $ mkBody stms $ bodyResult body
 
-intraGroupStm :: SegLevel -> Stm -> IntraGroupM ()
+intraGroupStm :: SegLevel -> Stm SOACS -> IntraGroupM ()
 intraGroupStm lvl stm@(Let pat aux e) = do
   scope <- askScope
   let lvl' = SegThread (segNumGroups lvl) (segGroupSize lvl) SegNoVirt
@@ -232,7 +232,7 @@
                     liftInner . collectStms_ . intraGroupStms lvl,
                   distSegLevel = \minw _ _ -> do
                     lift $ parallelMin minw
-                    return lvl,
+                    pure lvl,
                   distOnSOACSStms =
                     pure . oneStm . soacsStmToGPU,
                   distOnSOACSLambda =
@@ -266,7 +266,7 @@
       ops' <- forM ops $ \(HistOp num_bins rf dests nes op) -> do
         (op', nes', shape) <- determineReduceOp op nes
         let op'' = soacsLambdaToGPU op'
-        return $ Out.HistOp num_bins rf dests nes' shape op''
+        pure $ GPU.HistOp num_bins rf dests nes' shape op''
 
       let bucket_fun' = soacsLambdaToGPU bucket_fun
       certifying (stmAuxCerts aux) $
@@ -295,10 +295,10 @@
                   foldMap (foldMap resCerts . fst) is_vs
                     <> foldMap (resCerts . snd) is_vs
                 is_vs' = [(Slice $ map (DimFix . resSubExp) is, resSubExp v) | (is, v) <- is_vs]
-            return $ WriteReturns cs a_w a is_vs'
+            pure $ WriteReturns cs a_w a is_vs'
           inputs = do
             (p, p_a) <- zip (lambdaParams lam') ivs
-            return $ KernelInput (paramName p) (paramType p) p_a [Var write_i]
+            pure $ KernelInput (paramName p) (paramType p) p_a [Var write_i]
 
       kstms <- runBuilder_ $
         localScope (scopeOfSegSpace space) $ do
@@ -318,10 +318,10 @@
 intraGroupStms lvl = mapM_ (intraGroupStm lvl)
 
 intraGroupParalleliseBody ::
-  (MonadFreshNames m, HasScope Out.GPU m) =>
+  (MonadFreshNames m, HasScope GPU m) =>
   SegLevel ->
-  Body ->
-  m ([[SubExp]], [[SubExp]], Log, Out.KernelBody Out.GPU)
+  Body SOACS ->
+  m ([[SubExp]], [[SubExp]], Log, KernelBody GPU)
 intraGroupParalleliseBody lvl body = do
   (IntraAcc min_ws avail_ws log, kstms) <-
     runIntraGroupM $ intraGroupStms lvl $ bodyStms body
diff --git a/src/Futhark/Pass/ExtractKernels/StreamKernel.hs b/src/Futhark/Pass/ExtractKernels/StreamKernel.hs
--- a/src/Futhark/Pass/ExtractKernels/StreamKernel.hs
+++ b/src/Futhark/Pass/ExtractKernels/StreamKernel.hs
@@ -121,7 +121,7 @@
   Lambda (Rep m) ->
   Int ->
   [VName] ->
-  m ([PatElemT Type], [PatElemT Type])
+  m ([PatElem Type], [PatElem Type])
 blockedPerThread thread_gtid w kernel_size ordering lam num_nonconcat arrs = do
   let (_, chunk_size, [], arr_params) =
         partitionChunkedKernelFoldParameters 0 $ lambdaParams lam
@@ -231,7 +231,7 @@
 streamRed ::
   (MonadFreshNames m, HasScope GPU m) =>
   MkSegLevel GPU m ->
-  Pat GPU ->
+  Pat Type ->
   SubExp ->
   Commutativity ->
   Lambda GPU ->
@@ -262,7 +262,7 @@
   (MonadFreshNames m, HasScope GPU m) =>
   MkSegLevel GPU m ->
   [String] ->
-  [PatElem GPU] ->
+  [PatElem Type] ->
   SubExp ->
   Commutativity ->
   Lambda GPU ->
diff --git a/src/Futhark/Pass/ExtractMulticore.hs b/src/Futhark/Pass/ExtractMulticore.hs
--- a/src/Futhark/Pass/ExtractMulticore.hs
+++ b/src/Futhark/Pass/ExtractMulticore.hs
@@ -280,7 +280,7 @@
       SegRed () space [red] (lambdaReturnType map_lam) kbody
   return (red_stms, op)
 
-transformSOAC :: Pat SOACS -> Attrs -> SOAC SOACS -> ExtractM (Stms MC)
+transformSOAC :: Pat Type -> Attrs -> SOAC SOACS -> ExtractM (Stms MC)
 transformSOAC pat _ (Screma w arrs form)
   | Just lam <- isMapSOAC form = do
     seq_op <- transformMap DoNotRename sequentialiseBody w lam arrs
diff --git a/src/Futhark/Pass/KernelBabysitting.hs b/src/Futhark/Pass/KernelBabysitting.hs
--- a/src/Futhark/Pass/KernelBabysitting.hs
+++ b/src/Futhark/Pass/KernelBabysitting.hs
@@ -8,6 +8,7 @@
 import Control.Monad.State.Strict
 import Data.Foldable
 import Data.List (elemIndex, isPrefixOf, sort)
+import Data.List.NonEmpty (NonEmpty (..))
 import qualified Data.Map.Strict as M
 import Data.Maybe
 import Futhark.IR
@@ -474,7 +475,7 @@
         letExp (baseString arr <> "_padding") $
           BasicOp $ Scratch (elemType arr_t) (shapeDims padding_shape)
       letExp (baseString arr <> "_padded") $
-        BasicOp $ Concat d arr [arr_padding] w_padded
+        BasicOp $ Concat d (arr :| [arr_padding]) w_padded
 
     rearrange num_chunks' w_padded per_chunk arr_name arr_padded arr_t = do
       let arr_dims = arrayDims arr_t
diff --git a/src/Futhark/Passes.hs b/src/Futhark/Passes.hs
--- a/src/Futhark/Passes.hs
+++ b/src/Futhark/Passes.hs
@@ -40,6 +40,9 @@
 import Futhark.Pass.Simplify
 import Futhark.Pipeline
 
+-- | A pipeline used by all current compilers.  Performs inlining,
+-- fusion, and various forms of cleanup.  This pipeline will be
+-- followed by another one that deals with parallelism and memory.
 standardPipeline :: Pipeline SOACS SOACS
 standardPipeline =
   passes
@@ -60,6 +63,8 @@
       removeDeadFunctions
     ]
 
+-- | The pipeline used by the CUDA and OpenCL backends, but before
+-- adding memory information.  Includes 'standardPipeline'.
 kernelsPipeline :: Pipeline SOACS GPU
 kernelsPipeline =
   standardPipeline
@@ -75,6 +80,8 @@
         inPlaceLoweringGPU
       ]
 
+-- | The pipeline used by the sequential backends.  Turns all
+-- parallelism into sequential loops.  Includes 'standardPipeline'.
 sequentialPipeline :: Pipeline SOACS Seq
 sequentialPipeline =
   standardPipeline
@@ -84,6 +91,8 @@
         inPlaceLoweringSeq
       ]
 
+-- | Run 'sequentialPipeline', then add memory information (and
+-- optimise it slightly).
 sequentialCpuPipeline :: Pipeline SOACS SeqMem
 sequentialCpuPipeline =
   sequentialPipeline
@@ -93,6 +102,8 @@
         simplifySeqMem
       ]
 
+-- | Run 'kernelsPipeline', then add memory information (and optimise
+-- it a lot).
 gpuPipeline :: Pipeline SOACS GPUMem
 gpuPipeline =
   kernelsPipeline
@@ -109,6 +120,8 @@
         simplifyGPUMem
       ]
 
+-- | Run 'standardPipeline' and then convert to multicore
+-- representation (and do a bunch of optimisation).
 mcPipeline :: Pipeline SOACS MC
 mcPipeline =
   standardPipeline
@@ -122,6 +135,7 @@
         inPlaceLoweringMC
       ]
 
+-- | Run 'mcPipeline' and then add memory information.
 multicorePipeline :: Pipeline SOACS MCMem
 multicorePipeline =
   mcPipeline
diff --git a/src/Futhark/Script.hs b/src/Futhark/Script.hs
--- a/src/Futhark/Script.hs
+++ b/src/Futhark/Script.hs
@@ -129,7 +129,7 @@
 inBraces sep = between (lexeme sep "{") (lexeme sep "}")
 
 -- | Parse a FutharkScript expression, given a whitespace parser.
-parseExp :: Parser () -> Parser Exp
+parseExp :: Parsec Void T.Text () -> Parsec Void T.Text Exp
 parseExp sep =
   choice
     [ lexeme sep "let" $> Let
diff --git a/src/Futhark/Test/Spec.hs b/src/Futhark/Test/Spec.hs
--- a/src/Futhark/Test/Spec.hs
+++ b/src/Futhark/Test/Spec.hs
@@ -126,6 +126,7 @@
   | ScriptFile FilePath
   deriving (Show)
 
+-- | How to generate a single random value.
 data GenValue
   = -- | Generate a value of the given rank and primitive
     -- type.  Scalars are considered 0-ary arrays.
diff --git a/src/Futhark/Tools.hs b/src/Futhark/Tools.hs
--- a/src/Futhark/Tools.hs
+++ b/src/Futhark/Tools.hs
@@ -33,10 +33,10 @@
     ExpDec rep ~ (),
     Op rep ~ SOAC rep
   ) =>
-  Pat rep ->
+  Pat (LetDec rep) ->
   ( SubExp,
     [Reduce rep],
-    LambdaT rep,
+    Lambda rep,
     [VName]
   ) ->
   m (Stm rep, Stm rep)
@@ -51,11 +51,11 @@
 
 splitScanOrRedomap ::
   (Typed dec, MonadFreshNames m) =>
-  [PatElemT dec] ->
+  [PatElem dec] ->
   SubExp ->
-  LambdaT rep ->
+  Lambda rep ->
   [[SubExp]] ->
-  m ([Ident], PatT dec, [VName])
+  m ([Ident], Pat dec, [VName])
 splitScanOrRedomap pes w map_lam nes = do
   let (acc_pes, arr_pes) =
         splitAt (length $ concat nes) pes
@@ -79,7 +79,7 @@
     Op (Rep m) ~ SOAC (Rep m),
     Buildable (Rep m)
   ) =>
-  Pat (Rep m) ->
+  Pat (LetDec (Rep m)) ->
   SubExp ->
   ScremaForm (Rep m) ->
   [VName] ->
@@ -103,10 +103,10 @@
 -- to the entire input.
 sequentialStreamWholeArray ::
   (MonadBuilder m, Buildable (Rep m)) =>
-  Pat (Rep m) ->
+  Pat (LetDec (Rep m)) ->
   SubExp ->
   [SubExp] ->
-  LambdaT (Rep m) ->
+  Lambda (Rep m) ->
   [VName] ->
   m ()
 sequentialStreamWholeArray pat w nes lam arrs = do
diff --git a/src/Futhark/Transform/FirstOrderTransform.hs b/src/Futhark/Transform/FirstOrderTransform.hs
--- a/src/Futhark/Transform/FirstOrderTransform.hs
+++ b/src/Futhark/Transform/FirstOrderTransform.hs
@@ -76,7 +76,7 @@
 
 transformBody ::
   (Transformer m, LetDec (Rep m) ~ LetDec SOACS) =>
-  Body ->
+  Body SOACS ->
   m (AST.Body (Rep m))
 transformBody (Body () stms res) = buildBody_ $ do
   mapM_ transformStmRecursively stms
@@ -85,9 +85,7 @@
 -- | First transform any nested t'Body' or t'Lambda' elements, then
 -- apply 'transformSOAC' if the expression is a SOAC.
 transformStmRecursively ::
-  (Transformer m, LetDec (Rep m) ~ LetDec SOACS) =>
-  Stm ->
-  m ()
+  (Transformer m, LetDec (Rep m) ~ LetDec SOACS) => Stm SOACS -> m ()
 transformStmRecursively (Let pat aux (Op soac)) =
   auxing aux $ transformSOAC pat =<< mapSOACM soacTransform soac
   where
@@ -122,7 +120,7 @@
 -- on the given rep.
 transformSOAC ::
   Transformer m =>
-  AST.Pat (Rep m) ->
+  Pat (LetDec (Rep m)) ->
   SOAC (Rep m) ->
   m ()
 transformSOAC pat (Screma w arrs form@(ScremaForm scans reds map_lam)) = do
@@ -365,7 +363,7 @@
     LetDec rep ~ LetDec SOACS,
     CanBeAliased (Op rep)
   ) =>
-  Lambda ->
+  Lambda SOACS ->
   m (AST.Lambda rep)
 transformLambda (Lambda params body rettype) = do
   body' <-
diff --git a/src/Futhark/Transform/Rename.hs b/src/Futhark/Transform/Rename.hs
--- a/src/Futhark/Transform/Rename.hs
+++ b/src/Futhark/Transform/Rename.hs
@@ -108,8 +108,8 @@
 -- a new name.
 renamePat ::
   (Rename dec, MonadFreshNames m) =>
-  PatT dec ->
-  m (PatT dec)
+  Pat dec ->
+  m (Pat dec)
 renamePat = modifyNameSource . runRenamer . rename'
   where
     rename' pat = renameBound (patNames pat) $ rename pat
@@ -224,10 +224,10 @@
   rename (Param attrs name dec) =
     Param <$> rename attrs <*> rename name <*> rename dec
 
-instance Rename dec => Rename (PatT dec) where
+instance Rename dec => Rename (Pat dec) where
   rename (Pat xs) = Pat <$> rename xs
 
-instance Rename dec => Rename (PatElemT dec) where
+instance Rename dec => Rename (PatElem dec) where
   rename (PatElem ident dec) = PatElem <$> rename ident <*> rename dec
 
 instance Rename Certs where
diff --git a/src/Futhark/Transform/Substitute.hs b/src/Futhark/Transform/Substitute.hs
--- a/src/Futhark/Transform/Substitute.hs
+++ b/src/Futhark/Transform/Substitute.hs
@@ -75,7 +75,7 @@
 instance Substitutable rep => Substitute (Exp rep) where
   substituteNames substs = mapExp $ replace substs
 
-instance Substitute dec => Substitute (PatElemT dec) where
+instance Substitute dec => Substitute (PatElem dec) where
   substituteNames substs (PatElem ident dec) =
     PatElem (substituteNames substs ident) (substituteNames substs dec)
 
@@ -100,7 +100,7 @@
   substituteNames substs (SubExpRes cs se) =
     SubExpRes (substituteNames substs cs) (substituteNames substs se)
 
-instance Substitute dec => Substitute (PatT dec) where
+instance Substitute dec => Substitute (Pat dec) where
   substituteNames substs (Pat xs) =
     Pat (substituteNames substs xs)
 
diff --git a/src/Futhark/Util.hs b/src/Futhark/Util.hs
--- a/src/Futhark/Util.hs
+++ b/src/Futhark/Util.hs
@@ -27,6 +27,7 @@
     hashText,
     unixEnvironment,
     isEnvVarAtLeast,
+    startupTime,
     fancyTerminal,
     runProgramWithExitCode,
     directoryContents,
@@ -46,6 +47,7 @@
     toPOSIX,
     trim,
     pmapIO,
+    interactWithFileSafely,
     readFileSafely,
     convFloat,
     UserString,
@@ -76,6 +78,7 @@
 import qualified Data.Text.Encoding as T
 import qualified Data.Text.Encoding.Error as T
 import qualified Data.Text.IO as T
+import Data.Time.Clock (UTCTime, getCurrentTime)
 import Data.Tuple (swap)
 import Numeric
 import qualified System.Directory.Tree as Dir
@@ -89,11 +92,11 @@
 import System.Process.ByteString
 import Text.Read (readMaybe)
 
--- | Like 'nub', but without the quadratic runtime.
+-- | Like @nub@, but without the quadratic runtime.
 nubOrd :: Ord a => [a] -> [a]
 nubOrd = nubByOrd compare
 
--- | Like 'nubBy', but without the quadratic runtime.
+-- | Like @nubBy@, but without the quadratic runtime.
 nubByOrd :: (a -> a -> Ordering) -> [a] -> [a]
 nubByOrd cmp = map NE.head . NE.groupBy eq . sortBy cmp
   where
@@ -200,6 +203,13 @@
     Just y -> y >= x
     _ -> False
 
+{-# NOINLINE startupTime #-}
+
+-- | The time at which the process started - or more accurately, the
+-- first time this binding was forced.
+startupTime :: UTCTime
+startupTime = unsafePerformIO getCurrentTime
+
 {-# NOINLINE fancyTerminal #-}
 
 -- | Are we running in a terminal capable of fancy commands and
@@ -355,11 +365,11 @@
         Left err -> throw (err :: SomeException)
         Right v -> pure v
 
--- | Read a file, returning 'Nothing' if the file does not exist, and
--- 'Left' if some other error occurs.
-readFileSafely :: FilePath -> IO (Maybe (Either String T.Text))
-readFileSafely filepath =
-  (Just . Right <$> T.readFile filepath) `catch` couldNotRead
+-- | Do some operation on a file, returning 'Nothing' if the file does
+-- not exist, and 'Left' if some other error occurs.
+interactWithFileSafely :: IO a -> IO (Maybe (Either String a))
+interactWithFileSafely m =
+  (Just . Right <$> m) `catch` couldNotRead
   where
     couldNotRead e
       | isDoesNotExistError e =
@@ -367,6 +377,12 @@
       | otherwise =
         return $ Just $ Left $ show e
 
+-- | Read a file, returning 'Nothing' if the file does not exist, and
+-- 'Left' if some other error occurs.
+readFileSafely :: FilePath -> IO (Maybe (Either String T.Text))
+readFileSafely filepath =
+  interactWithFileSafely $ T.readFile filepath
+
 -- | Convert between different floating-point types, preserving
 -- infinities and NaNs.
 convFloat :: (RealFloat from, RealFloat to) => from -> to
@@ -453,17 +469,22 @@
   where
     hex_str = showHex (ord c) "U"
 
+-- | Truncate to at most this many characters, making the last three
+-- characters "..." if truncation is necessary.
 atMostChars :: Int -> String -> String
 atMostChars n s
   | length s > n = take (n -3) s ++ "..."
   | otherwise = s
 
+-- | Invert a map, handling duplicate values (now keys) by
+-- constructing a set of corresponding values.
 invertMap :: (Ord v, Ord k) => M.Map k v -> M.Map v (S.Set k)
 invertMap m =
   M.toList m
     & fmap (swap . first S.singleton)
     & foldr (uncurry $ M.insertWith (<>)) mempty
 
+-- | Perform fixpoint iteration.
 fixPoint :: Eq a => (a -> a) -> a -> a
 fixPoint f x =
   let x' = f x
diff --git a/src/Language/Futhark/Interpreter.hs b/src/Language/Futhark/Interpreter.hs
--- a/src/Language/Futhark/Interpreter.hs
+++ b/src/Language/Futhark/Interpreter.hs
@@ -13,11 +13,12 @@
     interpretDec,
     interpretImport,
     interpretFunction,
+    ctxWithImports,
     ExtOp (..),
     BreakReason (..),
     StackFrame (..),
     typeCheckerEnv,
-    Value (ValuePrim, ValueArray, ValueRecord),
+    Value (ValuePrim, ValueRecord),
     fromTuple,
     isEmptyArray,
     prettyEmptyArray,
@@ -150,6 +151,7 @@
   | ShapeSum (M.Map Name [Shape d])
   deriving (Eq, Show, Functor, Foldable, Traversable)
 
+-- | The shape of an array.
 type ValueShape = Shape Int64
 
 instance Pretty d => Pretty (Shape d) where
@@ -721,7 +723,7 @@
       size_env <- extSizeEnv
       v $ evalType (size_env <> env) t
     Just (TermValue _ v) -> return v
-    _ -> error $ "`" <> pretty qv <> "` is not bound to a value."
+    _ -> error $ "\"" <> pretty qv <> "\" is not bound to a value."
 
 typeValueShape :: Env -> StructType -> EvalM ValueShape
 typeValueShape env t = do
@@ -1173,8 +1175,14 @@
 evalModExp :: Env -> ModExp -> EvalM Module
 evalModExp _ (ModImport _ (Info f) _) = do
   f' <- lookupImport f
+  known <- asks snd
   case f' of
-    Nothing -> error $ "Unknown import " ++ show f
+    Nothing ->
+      error $
+        unlines
+          [ "Unknown interpreter import: " ++ show f,
+            "Known: " ++ show (M.keys known)
+          ]
     Just m -> return $ Module m
 evalModExp env (ModDecs ds _) = do
   Env terms types _ <- foldM evalDec env ds
@@ -1955,6 +1963,11 @@
 interpretImport ctx (fp, prog) = do
   env <- runEvalM (ctxImports ctx) $ foldM evalDec (ctxEnv ctx) $ progDecs prog
   return ctx {ctxImports = M.insert fp env $ ctxImports ctx}
+
+-- | Produce a context, based on the one passed in, where all of
+-- the provided imports have been @open@ened in order.
+ctxWithImports :: [Env] -> Ctx -> Ctx
+ctxWithImports envs ctx = ctx {ctxEnv = mconcat (reverse envs) <> ctxEnv ctx}
 
 checkEntryArgs :: VName -> [F.Value] -> StructType -> Either String ()
 checkEntryArgs entry args entry_t
diff --git a/src/Language/Futhark/Parser.hs b/src/Language/Futhark/Parser.hs
--- a/src/Language/Futhark/Parser.hs
+++ b/src/Language/Futhark/Parser.hs
@@ -7,15 +7,11 @@
     parseValue,
     parseValues,
     parseDecOrExpIncrM,
-    ParseError (..),
-    scanTokensText,
-    L (..),
-    Token (..),
+    ParseError,
   )
 where
 
 import qualified Data.Text as T
-import Language.Futhark.Parser.Lexer
 import Language.Futhark.Parser.Parser
 import Language.Futhark.Prop
 import Language.Futhark.Syntax
diff --git a/src/Language/Futhark/Parser/Monad.hs b/src/Language/Futhark/Parser/Monad.hs
new file mode 100644
--- /dev/null
+++ b/src/Language/Futhark/Parser/Monad.hs
@@ -0,0 +1,292 @@
+{-# LANGUAGE FlexibleContexts #-}
+{-# LANGUAGE OverloadedStrings #-}
+
+-- | Utility functions and definitions used in the Happy-generated
+-- parser.  They are defined here because the @.y@ file is opaque to
+-- linters and other tools.  In particular, we cannot enable warnings
+-- for that file, because Happy-generated code is very dirty by GHC's
+-- standards.
+module Language.Futhark.Parser.Monad
+  ( ParserMonad,
+    ParserEnv,
+    ReadLineMonad (..),
+    parseInMonad,
+    parse,
+    getLinesFromM,
+    lexer,
+    mustBeEmpty,
+    arrayFromList,
+    combArrayElements,
+    binOp,
+    binOpName,
+    mustBe,
+    floatNegate,
+    intNegate,
+    primNegate,
+    primTypeFromName,
+    applyExp,
+    patternExp,
+    addDocSpec,
+    addAttrSpec,
+    addDoc,
+    addAttr,
+    twoDotsRange,
+    ParseError (..),
+    emptyArrayError,
+    parseError,
+    parseErrorAt,
+
+    -- * Reexports
+    L,
+    Token,
+  )
+where
+
+import Control.Applicative (liftA)
+import Control.Monad
+import Control.Monad.Except
+import Control.Monad.Trans.State
+import Data.Array hiding (index)
+import qualified Data.Map.Strict as M
+import Data.Monoid
+import qualified Data.Text as T
+import Futhark.Util.Loc hiding (L) -- Lexer has replacements.
+import Futhark.Util.Pretty hiding (line)
+import Language.Futhark.Parser.Lexer
+import Language.Futhark.Pretty ()
+import Language.Futhark.Prop
+import Language.Futhark.Syntax
+import Prelude hiding (mod)
+
+addDoc :: DocComment -> UncheckedDec -> UncheckedDec
+addDoc doc (ValDec val) = ValDec (val {valBindDoc = Just doc})
+addDoc doc (TypeDec tp) = TypeDec (tp {typeDoc = Just doc})
+addDoc doc (SigDec sig) = SigDec (sig {sigDoc = Just doc})
+addDoc doc (ModDec mod) = ModDec (mod {modDoc = Just doc})
+addDoc _ dec = dec
+
+addDocSpec :: DocComment -> SpecBase NoInfo Name -> SpecBase NoInfo Name
+addDocSpec doc (TypeAbbrSpec tpsig) = TypeAbbrSpec (tpsig {typeDoc = Just doc})
+addDocSpec doc (ValSpec name ps t _ loc) = ValSpec name ps t (Just doc) loc
+addDocSpec doc (TypeSpec l name ps _ loc) = TypeSpec l name ps (Just doc) loc
+addDocSpec doc (ModSpec name se _ loc) = ModSpec name se (Just doc) loc
+addDocSpec _ spec = spec
+
+addAttr :: AttrInfo Name -> UncheckedDec -> UncheckedDec
+addAttr attr (ValDec val) =
+  ValDec $ val {valBindAttrs = attr : valBindAttrs val}
+addAttr _ dec = dec
+
+-- We will extend this function once we actually start tracking these.
+addAttrSpec :: AttrInfo Name -> UncheckedSpec -> UncheckedSpec
+addAttrSpec _attr dec = dec
+
+mustBe :: L Token -> String -> ParserMonad ()
+mustBe (L _ (ID got)) expected
+  | nameToString got == expected = pure ()
+mustBe (L loc _) expected =
+  parseErrorAt loc . Just $
+    "Only the keyword '" <> expected <> "' may appear here."
+
+mustBeEmpty :: SrcLoc -> ValueType -> ParserMonad ()
+mustBeEmpty _ (Array _ _ _ (ShapeDecl dims))
+  | 0 `elem` dims = pure ()
+mustBeEmpty loc t =
+  parseErrorAt loc $ Just $ pretty t ++ " is not an empty array."
+
+newtype ParserEnv = ParserEnv
+  { parserFile :: FilePath
+  }
+
+type ParserMonad =
+  ExceptT String (StateT ParserEnv (StateT ([L Token], Pos) ReadLineMonad))
+
+data ReadLineMonad a
+  = Value a
+  | GetLine (Maybe T.Text -> ReadLineMonad a)
+
+readLineFromMonad :: ReadLineMonad (Maybe T.Text)
+readLineFromMonad = GetLine Value
+
+instance Monad ReadLineMonad where
+  return = pure
+  Value x >>= f = f x
+  GetLine g >>= f = GetLine $ g >=> f
+
+instance Functor ReadLineMonad where
+  fmap = liftA
+
+instance Applicative ReadLineMonad where
+  pure = Value
+  (<*>) = ap
+
+getLinesFromM :: Monad m => m T.Text -> ReadLineMonad a -> m a
+getLinesFromM _ (Value x) = pure x
+getLinesFromM fetch (GetLine f) = do
+  s <- fetch
+  getLinesFromM fetch $ f $ Just s
+
+getNoLines :: ReadLineMonad a -> Either String a
+getNoLines (Value x) = Right x
+getNoLines (GetLine f) = getNoLines $ f Nothing
+
+combArrayElements :: Value -> [Value] -> Either String Value
+combArrayElements = foldM comb
+  where
+    comb x y
+      | valueType x == valueType y = Right x
+      | otherwise =
+        Left $
+          "Elements " <> pretty x <> " and "
+            <> pretty y
+            <> " cannot exist in same array."
+
+arrayFromList :: [a] -> Array Int a
+arrayFromList l = listArray (0, length l -1) l
+
+applyExp :: [UncheckedExp] -> ParserMonad UncheckedExp
+applyExp all_es@((Constr n [] _ loc1) : es) =
+  pure $ Constr n es NoInfo (srcspan loc1 (last all_es))
+applyExp es =
+  foldM op (head es) (tail es)
+  where
+    op (AppExp (Index e is floc) _) (ArrayLit xs _ xloc) =
+      parseErrorAt (srcspan floc xloc) . Just . pretty $
+        "Incorrect syntax for multi-dimensional indexing."
+          </> "Use" <+> align (ppr index)
+      where
+        index = AppExp (Index e (is ++ map DimFix xs) xloc) NoInfo
+    op f x =
+      pure $ AppExp (Apply f x NoInfo (srcspan f x)) NoInfo
+
+patternExp :: UncheckedPat -> ParserMonad UncheckedExp
+patternExp (Id v _ loc) = pure $ Var (qualName v) NoInfo loc
+patternExp (TuplePat pats loc) = TupLit <$> mapM patternExp pats <*> pure loc
+patternExp (Wildcard _ loc) = parseErrorAt loc $ Just "cannot have wildcard here."
+patternExp (PatLit _ _ loc) = parseErrorAt loc $ Just "cannot have literal here."
+patternExp (PatConstr _ _ _ loc) = parseErrorAt loc $ Just "cannot have constructor here."
+patternExp (PatAttr _ p _) = patternExp p
+patternExp (PatAscription pat _ _) = patternExp pat
+patternExp (PatParens pat _) = patternExp pat
+patternExp (RecordPat fs loc) = RecordLit <$> mapM field fs <*> pure loc
+  where
+    field (name, pat) = RecordFieldExplicit name <$> patternExp pat <*> pure loc
+
+eof :: Pos -> L Token
+eof pos = L (SrcLoc $ Loc pos pos) EOF
+
+binOpName :: L Token -> (QualName Name, SrcLoc)
+binOpName (L loc (SYMBOL _ qs op)) = (QualName qs op, loc)
+binOpName t = error $ "binOpName: unexpected " ++ show t
+
+binOp :: UncheckedExp -> L Token -> UncheckedExp -> UncheckedExp
+binOp x (L loc (SYMBOL _ qs op)) y =
+  AppExp (BinOp (QualName qs op, loc) NoInfo (x, NoInfo) (y, NoInfo) (srcspan x y)) NoInfo
+binOp _ t _ = error $ "binOp: unexpected " ++ show t
+
+getTokens :: ParserMonad ([L Token], Pos)
+getTokens = lift $ lift get
+
+putTokens :: ([L Token], Pos) -> ParserMonad ()
+putTokens = lift . lift . put
+
+primTypeFromName :: SrcLoc -> Name -> ParserMonad PrimType
+primTypeFromName loc s = maybe boom pure $ M.lookup s namesToPrimTypes
+  where
+    boom = parseErrorAt loc $ Just $ "No type named " ++ nameToString s
+
+intNegate :: IntValue -> IntValue
+intNegate (Int8Value v) = Int8Value (- v)
+intNegate (Int16Value v) = Int16Value (- v)
+intNegate (Int32Value v) = Int32Value (- v)
+intNegate (Int64Value v) = Int64Value (- v)
+
+floatNegate :: FloatValue -> FloatValue
+floatNegate (Float16Value v) = Float16Value (- v)
+floatNegate (Float32Value v) = Float32Value (- v)
+floatNegate (Float64Value v) = Float64Value (- v)
+
+primNegate :: PrimValue -> PrimValue
+primNegate (FloatValue v) = FloatValue $ floatNegate v
+primNegate (SignedValue v) = SignedValue $ intNegate v
+primNegate (UnsignedValue v) = UnsignedValue $ intNegate v
+primNegate (BoolValue v) = BoolValue $ not v
+
+readLine :: ParserMonad (Maybe T.Text)
+readLine = lift $ lift $ lift readLineFromMonad
+
+lexer :: (L Token -> ParserMonad a) -> ParserMonad a
+lexer cont = do
+  (ts, pos) <- getTokens
+  case ts of
+    [] -> do
+      ended <- lift $ runExceptT $ cont $ eof pos
+      case ended of
+        Right x -> pure x
+        Left parse_e -> do
+          line <- readLine
+          ts' <-
+            case line of
+              Nothing -> throwError parse_e
+              Just line' -> pure $ scanTokensText (advancePos pos '\n') line'
+          (ts'', pos') <-
+            case ts' of
+              Right x -> pure x
+              Left lex_e -> throwError lex_e
+          case ts'' of
+            [] -> cont $ eof pos
+            xs -> do
+              putTokens (xs, pos')
+              lexer cont
+    (x : xs) -> do
+      putTokens (xs, pos)
+      cont x
+
+parseError :: (L Token, [String]) -> ParserMonad a
+parseError (L loc EOF, expected) =
+  parseErrorAt (srclocOf loc) . Just . unlines $
+    [ "unexpected end of file.",
+      "Expected one of the following: " ++ unwords expected
+    ]
+parseError (L loc DOC {}, _) =
+  parseErrorAt (srclocOf loc) $
+    Just "documentation comments ('-- |') are only permitted when preceding declarations."
+parseError (L loc tok, expected) =
+  parseErrorAt loc . Just . unlines $
+    [ "unexpected " ++ show tok,
+      "Expected one of the following: " ++ unwords expected
+    ]
+
+parseErrorAt :: SrcLoc -> Maybe String -> ParserMonad a
+parseErrorAt loc Nothing = throwError $ "Error at " ++ locStr loc ++ ": Parse error."
+parseErrorAt loc (Just s) = throwError $ "Error at " ++ locStr loc ++ ": " ++ s
+
+emptyArrayError :: SrcLoc -> ParserMonad a
+emptyArrayError loc =
+  parseErrorAt loc $
+    Just "write empty arrays as 'empty(t)', for element type 't'.\n"
+
+twoDotsRange :: SrcLoc -> ParserMonad a
+twoDotsRange loc = parseErrorAt loc $ Just "use '...' for ranges, not '..'.\n"
+
+--- Now for the parser interface.
+
+-- | A parse error.  Use 'show' to get a human-readable description.
+newtype ParseError = ParseError String
+
+instance Show ParseError where
+  show (ParseError s) = s
+
+parseInMonad :: ParserMonad a -> FilePath -> T.Text -> ReadLineMonad (Either ParseError a)
+parseInMonad p file program =
+  either (Left . ParseError) Right
+    <$> either
+      (pure . Left)
+      (evalStateT (evalStateT (runExceptT p) env))
+      (scanTokensText (Pos file 1 1 0) program)
+  where
+    env = ParserEnv {parserFile = file}
+
+parse :: ParserMonad a -> FilePath -> T.Text -> Either ParseError a
+parse p file program =
+  either (Left . ParseError) id $ getNoLines $ parseInMonad p file program
diff --git a/src/Language/Futhark/Parser/Parser.y b/src/Language/Futhark/Parser/Parser.y
--- a/src/Language/Futhark/Parser/Parser.y
+++ b/src/Language/Futhark/Parser/Parser.y
@@ -1,7 +1,7 @@
 {
-{-# LANGUAGE TupleSections #-}
 {-# LANGUAGE OverloadedStrings #-}
-{-# LANGUAGE Trustworthy #-}
+{-# LANGUAGE TupleSections #-}
+
 -- | Futhark parser written with Happy.
 module Language.Futhark.Parser.Parser
   ( prog
@@ -10,8 +10,6 @@
   , futharkType
   , anyValue
   , anyValues
-
-  , ParserMonad
   , parse
   , ParseError(..)
   , parseDecOrExpIncrM
@@ -40,6 +38,7 @@
 import Language.Futhark.Parser.Lexer
 import Futhark.Util.Pretty
 import Futhark.Util.Loc hiding (L) -- Lexer has replacements.
+import Language.Futhark.Parser.Monad
 
 }
 
@@ -382,7 +381,7 @@
       : BinOp {% let (QualName qs name, loc) = $1 in do
                    unless (null qs) $ parseErrorAt loc $
                      Just "Cannot use a qualified name in binding position."
-                   return name }
+                   pure name }
       | '-'   { nameFromString "-" }
 
 BindingId :: { (Name, SrcLoc) }
@@ -455,20 +454,20 @@
            }
 
 SumType :: { UncheckedTypeExp }
-SumType  : SumClauses %prec sumprec { let (cs, loc) = $1
-                        in TESum cs loc }
+SumType  : SumClauses %prec sumprec { let (cs, loc) = $1 in TESum cs loc }
 
 SumClauses :: { ([(Name, [UncheckedTypeExp])], SrcLoc) }
-            : SumClauses '|' SumClause %prec sumprec { let (cs, loc1) = $1;
-                                             (c, ts, loc2) = $3
-                                          in (cs++[(c, ts)], srcspan loc1 loc2) }
-            | SumClause  %prec sumprec { let (n, ts, loc) = $1
-                                        in ([(n, ts)], loc) }
+            : SumClauses '|' SumClause %prec sumprec
+              { let (cs, loc1) = $1; (c, ts, loc2) = $3
+                in (cs++[(c, ts)], srcspan loc1 loc2) }
+            | SumClause  %prec sumprec
+              { let (n, ts, loc) = $1 in ([(n, ts)], loc) }
 
 SumClause :: { (Name, [UncheckedTypeExp], SrcLoc) }
-           : SumClause TypeExpAtom { let (n, ts, loc) = $1
-                                     in (n, ts ++ [$2], srcspan loc $>)}
-           | Constr { (fst $1, [], snd $1) }
+           : SumClause TypeExpAtom
+             { let (n, ts, loc) = $1 in (n, ts ++ [$2], srcspan loc $>)}
+           | Constr
+            { (fst $1, [], snd $1) }
 
 TypeExpApply :: { UncheckedTypeExp }
               : TypeExpApply TypeArg
@@ -968,29 +967,29 @@
          | f64lit { let L loc (F64LIT num) = $1 in (Float64Value num, loc) }
          | QualName {% let (qn, loc) = $1 in
                        case qn of
-                         QualName ["f16"] "inf" -> return (Float16Value (1/0), loc)
-                         QualName ["f16"] "nan" -> return (Float16Value (0/0), loc)
-                         QualName ["f32"] "inf" -> return (Float32Value (1/0), loc)
-                         QualName ["f32"] "nan" -> return (Float32Value (0/0), loc)
-                         QualName ["f64"] "inf" -> return (Float64Value (1/0), loc)
-                         QualName ["f64"] "nan" -> return (Float64Value (0/0), loc)
+                         QualName ["f16"] "inf" -> pure (Float16Value (1/0), loc)
+                         QualName ["f16"] "nan" -> pure (Float16Value (0/0), loc)
+                         QualName ["f32"] "inf" -> pure (Float32Value (1/0), loc)
+                         QualName ["f32"] "nan" -> pure (Float32Value (0/0), loc)
+                         QualName ["f64"] "inf" -> pure (Float64Value (1/0), loc)
+                         QualName ["f64"] "nan" -> pure (Float64Value (0/0), loc)
                          _ -> parseErrorAt (snd $1) Nothing
                     }
          | floatlit { let L loc (FLOATLIT num) = $1 in (Float64Value num, loc) }
 
 ArrayValue :: { Value }
 ArrayValue :  '[' Value ']'
-             {% return $ ArrayValue (arrayFromList [$2]) $
+             {% pure $ ArrayValue (arrayFromList [$2]) $
                 arrayOf (valueType $2) (ShapeDecl [1]) Unique
              }
            |  '[' Value ',' Values ']'
              {% case combArrayElements $2 $4 of
                   Left e -> throwError e
-                  Right v -> return $ ArrayValue (arrayFromList $ $2:$4) $
+                  Right v -> pure $ ArrayValue (arrayFromList $ $2:$4) $
                              arrayOf (valueType v) (ShapeDecl [1+fromIntegral (length $4)]) Unique
              }
            | id '(' ValueType ')'
-             {% ($1 `mustBe` "empty") >> mustBeEmpty (srcspan $2 $4) $3 >> return (ArrayValue (listArray (0,-1) []) $3) }
+             {% ($1 `mustBe` "empty") >> mustBeEmpty (srcspan $2 $4) $3 >> pure (ArrayValue (listArray (0,-1) []) $3) }
 
            -- Errors
            | '[' ']'
@@ -1009,263 +1008,29 @@
        |                  { [] }
 
 {
-
-addDoc :: DocComment -> UncheckedDec -> UncheckedDec
-addDoc doc (ValDec val) = ValDec (val { valBindDoc = Just doc })
-addDoc doc (TypeDec tp) = TypeDec (tp { typeDoc = Just doc })
-addDoc doc (SigDec sig) = SigDec (sig { sigDoc = Just doc })
-addDoc doc (ModDec mod) = ModDec (mod { modDoc = Just doc })
-addDoc _ dec = dec
-
-addDocSpec :: DocComment -> SpecBase NoInfo Name -> SpecBase NoInfo Name
-addDocSpec doc (TypeAbbrSpec tpsig) = TypeAbbrSpec (tpsig { typeDoc = Just doc })
-addDocSpec doc val@(ValSpec {}) = val { specDoc = Just doc }
-addDocSpec doc (TypeSpec l name ps _ loc) = TypeSpec l name ps (Just doc) loc
-addDocSpec doc (ModSpec name se _ loc) = ModSpec name se (Just doc) loc
-addDocSpec _ spec = spec
-
-addAttr :: AttrInfo Name -> UncheckedDec -> UncheckedDec
-addAttr attr (ValDec val) =
-  ValDec $ val { valBindAttrs = attr : valBindAttrs val }
-addAttr attr dec =
-  dec
-
--- We will extend this function once we actually start tracking these.
-addAttrSpec :: AttrInfo Name -> UncheckedSpec -> UncheckedSpec
-addAttrSpec _attr dec = dec
-
-reverseNonempty :: (a, [a]) -> (a, [a])
-reverseNonempty (x, l) =
-  case reverse (x:l) of
-    x':rest -> (x', rest)
-    []      -> (x, [])
-
-mustBe (L loc (ID got)) expected
-  | nameToString got == expected = return ()
-mustBe (L loc _) expected =
-  parseErrorAt loc $ Just $
-  "Only the keyword '" ++ expected ++ "' may appear here."
-
-mustBeEmpty :: SrcLoc -> ValueType -> ParserMonad ()
-mustBeEmpty loc (Array _ _ _ (ShapeDecl dims))
-  | any (==0) dims = return ()
-mustBeEmpty loc t =
-  parseErrorAt loc $ Just $ pretty t ++ " is not an empty array."
-
-data ParserEnv = ParserEnv {
-                 parserFile :: FilePath
-               }
-
-type ParserMonad a =
-  ExceptT String (
-    StateT ParserEnv (
-       StateT ([L Token], Pos) ReadLineMonad)) a
-
-data ReadLineMonad a = Value a
-                     | GetLine (Maybe T.Text -> ReadLineMonad a)
-
-readLineFromMonad :: ReadLineMonad (Maybe T.Text)
-readLineFromMonad = GetLine Value
-
-instance Monad ReadLineMonad where
-  return = Value
-  Value x >>= f = f x
-  GetLine g >>= f = GetLine $ \s -> g s >>= f
-
-instance Functor ReadLineMonad where
-  f `fmap` m = do x <- m
-                  return $ f x
-
-instance Applicative ReadLineMonad where
-  (<*>) = ap
-
-getLinesFromM :: Monad m => m T.Text -> ReadLineMonad a -> m a
-getLinesFromM _ (Value x) = return x
-getLinesFromM fetch (GetLine f) = do
-  s <- fetch
-  getLinesFromM fetch $ f $ Just s
-
-getLinesFromTexts :: [T.Text] -> ReadLineMonad a -> Either String a
-getLinesFromTexts _ (Value x) = Right x
-getLinesFromTexts (x : xs) (GetLine f) = getLinesFromTexts xs $ f $ Just x
-getLinesFromTexts [] (GetLine f) = getLinesFromTexts [] $ f Nothing
-
-getNoLines :: ReadLineMonad a -> Either String a
-getNoLines (Value x) = Right x
-getNoLines (GetLine f) = getNoLines $ f Nothing
-
-combArrayElements :: Value
-                  -> [Value]
-                  -> Either String Value
-combArrayElements t ts = foldM comb t ts
-  where comb x y
-          | valueType x == valueType y = Right x
-          | otherwise                  = Left $ "Elements " ++ pretty x ++ " and " ++
-                                         pretty y ++ " cannot exist in same array."
-
-arrayFromList :: [a] -> Array Int a
-arrayFromList l = listArray (0, length l-1) l
-
-applyExp :: [UncheckedExp] -> ParserMonad UncheckedExp
-applyExp all@((Constr n [] _ loc1):es) =
-  return $ Constr n es NoInfo (srcspan loc1 (last all))
-applyExp es =
-  foldM ap (head es) (tail es)
-  where
-     ap (AppExp (Index e is floc) _) (ArrayLit xs _ xloc) =
-       parseErrorAt (srcspan floc xloc) $
-       Just $ pretty $ "Incorrect syntax for multi-dimensional indexing." </>
-       "Use" <+> align (ppr index)
-       where index = AppExp (Index e (is++map DimFix xs) xloc) NoInfo
-     ap f x =
-        return $ AppExp (Apply f x NoInfo (srcspan f x)) NoInfo
-
-patternExp :: UncheckedPat -> ParserMonad UncheckedExp
-patternExp (Id v _ loc) = return $ Var (qualName v) NoInfo loc
-patternExp (TuplePat pats loc) = TupLit <$> (mapM patternExp pats) <*> return loc
-patternExp (Wildcard _ loc) = parseErrorAt loc $ Just "cannot have wildcard here."
-patternExp (PatAscription pat _ _) = patternExp pat
-patternExp (PatParens pat _) = patternExp pat
-patternExp (RecordPat fs loc) = RecordLit <$> mapM field fs <*> pure loc
-  where field (name, pat) = RecordFieldExplicit name <$> patternExp pat <*> pure loc
-
-eof :: Pos -> L Token
-eof pos = L (SrcLoc $ Loc pos pos) EOF
-
-binOpName (L loc (SYMBOL _ qs op)) = (QualName qs op, loc)
-
-binOp x (L loc (SYMBOL _ qs op)) y =
-  AppExp (BinOp (QualName qs op, loc) NoInfo (x, NoInfo) (y, NoInfo) (srcspan x y)) NoInfo
-
-getTokens :: ParserMonad ([L Token], Pos)
-getTokens = lift $ lift get
-
-putTokens :: ([L Token], Pos) -> ParserMonad ()
-putTokens = lift . lift . put
-
-primTypeFromName :: SrcLoc -> Name -> ParserMonad PrimType
-primTypeFromName loc s = maybe boom return $ M.lookup s namesToPrimTypes
-  where boom = parseErrorAt loc $ Just $ "No type named " ++ nameToString s
-
-getFilename :: ParserMonad FilePath
-getFilename = lift $ gets parserFile
-
-intNegate :: IntValue -> IntValue
-intNegate (Int8Value v) = Int8Value (-v)
-intNegate (Int16Value v) = Int16Value (-v)
-intNegate (Int32Value v) = Int32Value (-v)
-intNegate (Int64Value v) = Int64Value (-v)
-
-floatNegate :: FloatValue -> FloatValue
-floatNegate (Float16Value v) = Float16Value (-v)
-floatNegate (Float32Value v) = Float32Value (-v)
-floatNegate (Float64Value v) = Float64Value (-v)
-
-primNegate :: PrimValue -> PrimValue
-primNegate (FloatValue v) = FloatValue $ floatNegate v
-primNegate (SignedValue v) = SignedValue $ intNegate v
-primNegate (UnsignedValue v) = UnsignedValue $ intNegate v
-primNegate (BoolValue v) = BoolValue $ not v
-
-readLine :: ParserMonad (Maybe T.Text)
-readLine = lift $ lift $ lift readLineFromMonad
-
-lexer :: (L Token -> ParserMonad a) -> ParserMonad a
-lexer cont = do
-  (ts, pos) <- getTokens
-  case ts of
-    [] -> do
-      ended <- lift $ runExceptT $ cont $ eof pos
-      case ended of
-        Right x -> return x
-        Left parse_e -> do
-          line <- readLine
-          ts' <-
-            case line of Nothing -> throwError parse_e
-                         Just line' -> return $ scanTokensText (advancePos pos '\n') line'
-          (ts'', pos') <-
-            case ts' of Right x -> return x
-                        Left lex_e  -> throwError lex_e
-          case ts'' of
-            [] -> cont $ eof pos
-            xs -> do
-              putTokens (xs, pos')
-              lexer cont
-    (x : xs) -> do
-      putTokens (xs, pos)
-      cont x
-
-parseError :: (L Token, [String]) -> ParserMonad a
-parseError (L loc EOF, expected) =
-  parseErrorAt (srclocOf loc) $ Just $
-  unlines ["unexpected end of file.",
-           "Expected one of the following: " ++ unwords expected]
-parseError (L loc DOC{}, _) =
-  parseErrorAt (srclocOf loc) $
-  Just "documentation comments ('-- |') are only permitted when preceding declarations."
-parseError (L loc tok, expected) =
-  parseErrorAt loc $ Just $
-  unlines ["unexpected " ++ show tok,
-          "Expected one of the following: " ++ unwords expected]
-
-parseErrorAt :: SrcLoc -> Maybe String -> ParserMonad a
-parseErrorAt loc Nothing = throwError $ "Error at " ++ locStr loc ++ ": Parse error."
-parseErrorAt loc (Just s) = throwError $ "Error at " ++ locStr loc ++ ": " ++ s
-
-emptyArrayError :: SrcLoc -> ParserMonad a
-emptyArrayError loc =
-  parseErrorAt loc $
-  Just "write empty arrays as 'empty(t)', for element type 't'.\n"
-
-twoDotsRange :: SrcLoc -> ParserMonad a
-twoDotsRange loc = parseErrorAt loc $ Just "use '...' for ranges, not '..'.\n"
-
---- Now for the parser interface.
-
--- | A parse error.  Use 'show' to get a human-readable description.
-data ParseError = ParseError String
-
-instance Show ParseError where
-  show (ParseError s) = s
-
-parseInMonad :: ParserMonad a -> FilePath -> T.Text
-             -> ReadLineMonad (Either ParseError a)
-parseInMonad p file program =
-  either (Left . ParseError) Right <$> either (return . Left)
-  (evalStateT (evalStateT (runExceptT p) env))
-  (scanTokensText (Pos file 1 1 0) program)
-  where env = ParserEnv file
-
-parseIncremental :: ParserMonad a -> FilePath -> T.Text
-                 -> Either ParseError a
-parseIncremental p file program =
-  either (Left . ParseError) id
-  $ getLinesFromTexts (T.lines program)
-  $ parseInMonad p file mempty
-
-parse :: ParserMonad a -> FilePath -> T.Text
-      -> Either ParseError a
-parse p file program =
-  either (Left . ParseError) id
-  $ getNoLines $ parseInMonad p file program
-
--- | Parse an Futhark expression incrementally from monadic actions, using the
+  -- | Parse an Futhark expression incrementally from monadic actions, using the
 -- 'FilePath' as the source name for error messages.
-parseExpIncrM :: Monad m =>
-                 m T.Text -> FilePath -> T.Text
-              -> m (Either ParseError UncheckedExp)
+parseExpIncrM ::
+  Monad m =>
+  m T.Text ->
+  FilePath ->
+  T.Text ->
+  m (Either ParseError UncheckedExp)
 parseExpIncrM fetch file program =
   getLinesFromM fetch $ parseInMonad expression file program
 
 -- | Parse either an expression or a declaration incrementally;
 -- favouring declarations in case of ambiguity.
-parseDecOrExpIncrM :: Monad m =>
-                      m T.Text -> FilePath -> T.Text
-                   -> m (Either ParseError (Either UncheckedDec UncheckedExp))
+parseDecOrExpIncrM ::
+  Monad m =>
+  m T.Text ->
+  FilePath ->
+  T.Text ->
+  m (Either ParseError (Either UncheckedDec UncheckedExp))
 parseDecOrExpIncrM fetch file input =
   case parseInMonad declaration file input of
-    Value Left{} -> fmap Right <$> parseExpIncrM fetch file input
-    Value (Right d) -> return $ Right $ Left d
+    Value Left {} -> fmap Right <$> parseExpIncrM fetch file input
+    Value (Right d) -> pure $ Right $ Left d
     GetLine c -> do
       l <- fetch
       parseDecOrExpIncrM fetch file $ input <> "\n" <> l
diff --git a/src/Language/Futhark/Semantic.hs b/src/Language/Futhark/Semantic.hs
--- a/src/Language/Futhark/Semantic.hs
+++ b/src/Language/Futhark/Semantic.hs
@@ -66,7 +66,7 @@
 -- | Produce a human-readable canonicalized string from an
 -- 'ImportName'.
 includeToString :: ImportName -> String
-includeToString (ImportName s _) = Posix.normalise $ Posix.makeRelative "/" s
+includeToString (ImportName s _) = Posix.normalise s
 
 -- | The result of type checking some file.  Can be passed to further
 -- invocations of the type checker.
diff --git a/src/Language/Futhark/Syntax.hs b/src/Language/Futhark/Syntax.hs
--- a/src/Language/Futhark/Syntax.hs
+++ b/src/Language/Futhark/Syntax.hs
@@ -5,10 +5,10 @@
 {-# LANGUAGE Strict #-}
 
 -- | The Futhark source language AST definition.  Many types, such as
--- 'ExpBase'@, are parametrised by type and name representation.
+-- 'ExpBase', are parametrised by type and name representation.
 -- E.g. in a value of type @ExpBase f vn@, annotations are wrapped in
--- the functor @f@, and all names are of type @vn@.  See the
--- @https://futhark.readthedocs.org@ for a language reference, or this
+-- the functor @f@, and all names are of type @vn@.  See
+-- https://futhark.readthedocs.org for a language reference, or this
 -- module may be a little hard to understand.
 module Language.Futhark.Syntax
   ( module Language.Futhark.Core,
@@ -237,7 +237,7 @@
 instance IsPrimValue Bool where
   primValue = BoolValue
 
--- | The value of an 'AttrAtom'.
+-- | The value of an v'AttrAtom'.
 data AttrAtom vn
   = AtomName Name
   | AtomInt Integer
diff --git a/src/Language/Futhark/TypeChecker.hs b/src/Language/Futhark/TypeChecker.hs
--- a/src/Language/Futhark/TypeChecker.hs
+++ b/src/Language/Futhark/TypeChecker.hs
@@ -14,6 +14,7 @@
     TypeError,
     Warnings,
     initialEnv,
+    envWithImports,
   )
 where
 
@@ -133,6 +134,13 @@
       Just (name, TypeAbbr l ps $ RetType [] t)
     addIntrinsicT _ =
       Nothing
+
+-- | Produce an environment, based on the one passed in, where all of
+-- the provided imports have been @open@ened in order.  This could in principle
+-- also be done with 'checkDec', but this is more precise.
+envWithImports :: Imports -> Env -> Env
+envWithImports imports env =
+  mconcat (map (fileEnv . snd) (reverse imports)) <> env
 
 checkProgM :: UncheckedProg -> TypeM FileModule
 checkProgM (Prog doc decs) = do
diff --git a/src/Language/Futhark/TypeChecker/Monad.hs b/src/Language/Futhark/TypeChecker/Monad.hs
--- a/src/Language/Futhark/TypeChecker/Monad.hs
+++ b/src/Language/Futhark/TypeChecker/Monad.hs
@@ -74,10 +74,9 @@
 import qualified Paths_futhark
 import Prelude hiding (mapM, mod)
 
--- | A note with extra information regarding a type error.
 newtype Note = Note Doc
 
--- | A collection of 'Note's.
+-- | A collection of extra information regarding a type error.
 newtype Notes = Notes [Note]
   deriving (Semigroup, Monoid)
 
diff --git a/src/Language/Futhark/TypeChecker/Terms/Monad.hs b/src/Language/Futhark/TypeChecker/Terms/Monad.hs
--- a/src/Language/Futhark/TypeChecker/Terms/Monad.hs
+++ b/src/Language/Futhark/TypeChecker/Terms/Monad.hs
@@ -17,7 +17,7 @@
     liftTypeM,
     ValBinding (..),
     Locality (..),
-    SizeSource (..),
+    SizeSource (SourceBound, SourceSlice),
     NameReason (..),
     InferredType (..),
     Checking (..),
@@ -399,7 +399,7 @@
 withEnv :: TermEnv -> Env -> TermEnv
 withEnv tenv env = tenv {termScope = termScope tenv <> envToTermScope env}
 
--- Wrap a function name to give it a vacuous Eq instance for SizeSource.
+-- | Wrap a function name to give it a vacuous Eq instance for SizeSource.
 newtype FName = FName (Maybe (QualName VName))
   deriving (Show)
 
diff --git a/src/Language/Futhark/TypeChecker/Types.hs b/src/Language/Futhark/TypeChecker/Types.hs
--- a/src/Language/Futhark/TypeChecker/Types.hs
+++ b/src/Language/Futhark/TypeChecker/Types.hs
@@ -327,6 +327,12 @@
           <+> "not valid for a type parameter"
           <+> ppr p <> "."
 
+-- | Check a type expression, producing:
+--
+-- * The checked expression.
+-- * Size variables for any anonymous sizes in the expression.
+-- * The elaborated type.
+-- * The liftedness of the type.
 checkTypeExp ::
   MonadTypeChecker m =>
   TypeExp Name ->
diff --git a/src/Language/Futhark/TypeChecker/Unify.hs b/src/Language/Futhark/TypeChecker/Unify.hs
--- a/src/Language/Futhark/TypeChecker/Unify.hs
+++ b/src/Language/Futhark/TypeChecker/Unify.hs
@@ -651,7 +651,10 @@
   Level ->
   StructType ->
   m ()
-linkVarToType onDims usage bound bcs vn lvl tp = do
+linkVarToType onDims usage bound bcs vn lvl tp_unnorm = do
+  -- We have to expand anyway for the occurs check, so we might as
+  -- well link the fully expanded type.
+  tp <- normTypeFully tp_unnorm
   occursCheck usage bcs vn tp
   scopeCheck usage bcs vn lvl tp
 
diff --git a/src/futhark.hs b/src/futhark.hs
deleted file mode 100644
--- a/src/futhark.hs
+++ /dev/null
@@ -1,141 +0,0 @@
-{-# LANGUAGE OverloadedStrings #-}
-
--- | The futhark command line tool.
-module Main (main) where
-
-import Control.Exception
-import Control.Monad
-import Data.List (sortOn)
-import Data.Maybe
-import qualified Data.Text as T
-import qualified Data.Text.IO as T
-import qualified Futhark.CLI.Autotune as Autotune
-import qualified Futhark.CLI.Bench as Bench
-import qualified Futhark.CLI.C as C
-import qualified Futhark.CLI.CUDA as CCUDA
-import qualified Futhark.CLI.Check as Check
-import qualified Futhark.CLI.Datacmp as Datacmp
-import qualified Futhark.CLI.Dataset as Dataset
-import qualified Futhark.CLI.Defs as Defs
-import qualified Futhark.CLI.Dev as Dev
-import qualified Futhark.CLI.Doc as Doc
-import qualified Futhark.CLI.Literate as Literate
-import qualified Futhark.CLI.Misc as Misc
-import qualified Futhark.CLI.Multicore as Multicore
-import qualified Futhark.CLI.MulticoreWASM as MulticoreWASM
-import qualified Futhark.CLI.OpenCL as OpenCL
-import qualified Futhark.CLI.Pkg as Pkg
-import qualified Futhark.CLI.PyOpenCL as PyOpenCL
-import qualified Futhark.CLI.Python as Python
-import qualified Futhark.CLI.Query as Query
-import qualified Futhark.CLI.REPL as REPL
-import qualified Futhark.CLI.Run as Run
-import qualified Futhark.CLI.Test as Test
-import qualified Futhark.CLI.WASM as WASM
-import Futhark.Error
-import Futhark.Util (maxinum)
-import Futhark.Util.Options
-import GHC.IO.Encoding (setLocaleEncoding)
-import GHC.IO.Exception (IOErrorType (..), IOException (..))
-import System.Environment
-import System.Exit
-import System.IO
-import Prelude
-
-type Command = String -> [String] -> IO ()
-
-commands :: [(String, (Command, String))]
-commands =
-  sortOn
-    fst
-    [ ("dev", (Dev.main, "Run compiler passes directly.")),
-      ("repl", (REPL.main, "Run interactive Read-Eval-Print-Loop.")),
-      ("run", (Run.main, "Run a program through the (slow!) interpreter.")),
-      ("c", (C.main, "Compile to sequential C.")),
-      ("opencl", (OpenCL.main, "Compile to C calling OpenCL.")),
-      ("cuda", (CCUDA.main, "Compile to C calling CUDA.")),
-      ("multicore", (Multicore.main, "Compile to multicore C.")),
-      ("python", (Python.main, "Compile to sequential Python.")),
-      ("pyopencl", (PyOpenCL.main, "Compile to Python calling PyOpenCL.")),
-      ("wasm", (WASM.main, "Compile to WASM with sequential C")),
-      ("wasm-multicore", (MulticoreWASM.main, "Compile to WASM with multicore C")),
-      ("test", (Test.main, "Test Futhark programs.")),
-      ("bench", (Bench.main, "Benchmark Futhark programs.")),
-      ("dataset", (Dataset.main, "Generate random test data.")),
-      ("datacmp", (Datacmp.main, "Compare Futhark data files for equality.")),
-      ("dataget", (Misc.mainDataget, "Extract test data.")),
-      ("doc", (Doc.main, "Generate documentation for Futhark code.")),
-      ("pkg", (Pkg.main, "Manage local packages.")),
-      ("check", (Check.main, "Type check a program.")),
-      ("imports", (Misc.mainImports, "Print all non-builtin imported Futhark files.")),
-      ("hash", (Misc.mainHash, "Print hash of program AST.")),
-      ("autotune", (Autotune.main, "Autotune threshold parameters.")),
-      ("defs", (Defs.main, "Show location and name of all definitions.")),
-      ("query", (Query.main, "Query semantic information about program.")),
-      ("literate", (Literate.main, "Process a literate Futhark program."))
-    ]
-
-msg :: String
-msg =
-  unlines $
-    ["<command> options...", "Commands:", ""]
-      ++ [ "   " <> cmd <> replicate (k - length cmd) ' ' <> desc
-           | (cmd, (_, desc)) <- commands
-         ]
-  where
-    k = maxinum (map (length . fst) commands) + 3
-
--- | Catch all IO exceptions and print a better error message if they
--- happen.
-reportingIOErrors :: IO () -> IO ()
-reportingIOErrors =
-  flip
-    catches
-    [ Handler onExit,
-      Handler onICE,
-      Handler onIOException,
-      Handler onError
-    ]
-  where
-    onExit :: ExitCode -> IO ()
-    onExit = throwIO
-
-    onICE :: InternalError -> IO ()
-    onICE (Error CompilerLimitation s) = do
-      T.hPutStrLn stderr "Known compiler limitation encountered.  Sorry."
-      T.hPutStrLn stderr "Revise your program or try a different Futhark compiler."
-      T.hPutStrLn stderr s
-      exitWith $ ExitFailure 1
-    onICE (Error CompilerBug s) = do
-      T.hPutStrLn stderr "Internal compiler error."
-      T.hPutStrLn stderr "Please report this at https://github.com/diku-dk/futhark/issues."
-      T.hPutStrLn stderr s
-      exitWith $ ExitFailure 1
-
-    onError :: SomeException -> IO ()
-    onError e
-      | Just UserInterrupt <- asyncExceptionFromException e =
-        return () -- This corresponds to CTRL-C, which is not an error.
-      | otherwise = do
-        T.hPutStrLn stderr "Internal compiler error (unhandled IO exception)."
-        T.hPutStrLn stderr "Please report this at https://github.com/diku-dk/futhark/issues"
-        T.hPutStrLn stderr $ T.pack $ show e
-        exitWith $ ExitFailure 1
-
-    onIOException :: IOException -> IO ()
-    onIOException e
-      | ioe_type e == ResourceVanished =
-        exitWith $ ExitFailure 1
-      | otherwise = throw e
-
-main :: IO ()
-main = reportingIOErrors $ do
-  hSetEncoding stdout utf8
-  hSetEncoding stderr utf8
-  setLocaleEncoding utf8
-  args <- getArgs
-  prog <- getProgName
-  case args of
-    cmd : args'
-      | Just (m, _) <- lookup cmd commands -> m (unwords [prog, cmd]) args'
-    _ -> mainWithOptions () [] msg (const . const Nothing) prog args
diff --git a/src/main.hs b/src/main.hs
new file mode 100644
--- /dev/null
+++ b/src/main.hs
@@ -0,0 +1,142 @@
+{-# LANGUAGE OverloadedStrings #-}
+
+-- | The @futhark@ command line program.
+module Main (main) where
+
+import Control.Exception
+import Control.Monad
+import Data.List (sortOn)
+import Data.Maybe
+import qualified Data.Text as T
+import qualified Data.Text.IO as T
+import qualified Futhark.CLI.Autotune as Autotune
+import qualified Futhark.CLI.Bench as Bench
+import qualified Futhark.CLI.C as C
+import qualified Futhark.CLI.CUDA as CCUDA
+import qualified Futhark.CLI.Check as Check
+import qualified Futhark.CLI.Datacmp as Datacmp
+import qualified Futhark.CLI.Dataset as Dataset
+import qualified Futhark.CLI.Defs as Defs
+import qualified Futhark.CLI.Dev as Dev
+import qualified Futhark.CLI.Doc as Doc
+import qualified Futhark.CLI.Literate as Literate
+import qualified Futhark.CLI.Misc as Misc
+import qualified Futhark.CLI.Multicore as Multicore
+import qualified Futhark.CLI.MulticoreWASM as MulticoreWASM
+import qualified Futhark.CLI.OpenCL as OpenCL
+import qualified Futhark.CLI.Pkg as Pkg
+import qualified Futhark.CLI.PyOpenCL as PyOpenCL
+import qualified Futhark.CLI.Python as Python
+import qualified Futhark.CLI.Query as Query
+import qualified Futhark.CLI.REPL as REPL
+import qualified Futhark.CLI.Run as Run
+import qualified Futhark.CLI.Test as Test
+import qualified Futhark.CLI.WASM as WASM
+import Futhark.Error
+import Futhark.Util (maxinum)
+import Futhark.Util.Options
+import GHC.IO.Encoding (setLocaleEncoding)
+import GHC.IO.Exception (IOErrorType (..), IOException (..))
+import System.Environment
+import System.Exit
+import System.IO
+import Prelude
+
+type Command = String -> [String] -> IO ()
+
+commands :: [(String, (Command, String))]
+commands =
+  sortOn
+    fst
+    [ ("dev", (Dev.main, "Run compiler passes directly.")),
+      ("repl", (REPL.main, "Run interactive Read-Eval-Print-Loop.")),
+      ("run", (Run.main, "Run a program through the (slow!) interpreter.")),
+      ("c", (C.main, "Compile to sequential C.")),
+      ("opencl", (OpenCL.main, "Compile to C calling OpenCL.")),
+      ("cuda", (CCUDA.main, "Compile to C calling CUDA.")),
+      ("multicore", (Multicore.main, "Compile to multicore C.")),
+      ("python", (Python.main, "Compile to sequential Python.")),
+      ("pyopencl", (PyOpenCL.main, "Compile to Python calling PyOpenCL.")),
+      ("wasm", (WASM.main, "Compile to WASM with sequential C")),
+      ("wasm-multicore", (MulticoreWASM.main, "Compile to WASM with multicore C")),
+      ("test", (Test.main, "Test Futhark programs.")),
+      ("bench", (Bench.main, "Benchmark Futhark programs.")),
+      ("dataset", (Dataset.main, "Generate random test data.")),
+      ("datacmp", (Datacmp.main, "Compare Futhark data files for equality.")),
+      ("dataget", (Misc.mainDataget, "Extract test data.")),
+      ("doc", (Doc.main, "Generate documentation for Futhark code.")),
+      ("pkg", (Pkg.main, "Manage local packages.")),
+      ("check", (Check.main, "Type-check a program.")),
+      ("check-syntax", (Misc.mainCheckSyntax, "Syntax-check a program.")),
+      ("imports", (Misc.mainImports, "Print all non-builtin imported Futhark files.")),
+      ("hash", (Misc.mainHash, "Print hash of program AST.")),
+      ("autotune", (Autotune.main, "Autotune threshold parameters.")),
+      ("defs", (Defs.main, "Show location and name of all definitions.")),
+      ("query", (Query.main, "Query semantic information about program.")),
+      ("literate", (Literate.main, "Process a literate Futhark program."))
+    ]
+
+msg :: String
+msg =
+  unlines $
+    ["<command> options...", "Commands:", ""]
+      ++ [ "   " <> cmd <> replicate (k - length cmd) ' ' <> desc
+           | (cmd, (_, desc)) <- commands
+         ]
+  where
+    k = maxinum (map (length . fst) commands) + 3
+
+-- | Catch all IO exceptions and print a better error message if they
+-- happen.
+reportingIOErrors :: IO () -> IO ()
+reportingIOErrors =
+  flip
+    catches
+    [ Handler onExit,
+      Handler onICE,
+      Handler onIOException,
+      Handler onError
+    ]
+  where
+    onExit :: ExitCode -> IO ()
+    onExit = throwIO
+
+    onICE :: InternalError -> IO ()
+    onICE (Error CompilerLimitation s) = do
+      T.hPutStrLn stderr "Known compiler limitation encountered.  Sorry."
+      T.hPutStrLn stderr "Revise your program or try a different Futhark compiler."
+      T.hPutStrLn stderr s
+      exitWith $ ExitFailure 1
+    onICE (Error CompilerBug s) = do
+      T.hPutStrLn stderr "Internal compiler error."
+      T.hPutStrLn stderr "Please report this at https://github.com/diku-dk/futhark/issues."
+      T.hPutStrLn stderr s
+      exitWith $ ExitFailure 1
+
+    onError :: SomeException -> IO ()
+    onError e
+      | Just UserInterrupt <- asyncExceptionFromException e =
+        return () -- This corresponds to CTRL-C, which is not an error.
+      | otherwise = do
+        T.hPutStrLn stderr "Internal compiler error (unhandled IO exception)."
+        T.hPutStrLn stderr "Please report this at https://github.com/diku-dk/futhark/issues"
+        T.hPutStrLn stderr $ T.pack $ show e
+        exitWith $ ExitFailure 1
+
+    onIOException :: IOException -> IO ()
+    onIOException e
+      | ioe_type e == ResourceVanished =
+        exitWith $ ExitFailure 1
+      | otherwise = throw e
+
+main :: IO ()
+main = reportingIOErrors $ do
+  hSetEncoding stdout utf8
+  hSetEncoding stderr utf8
+  setLocaleEncoding utf8
+  args <- getArgs
+  prog <- getProgName
+  case args of
+    cmd : args'
+      | Just (m, _) <- lookup cmd commands -> m (unwords [prog, cmd]) args'
+    _ -> mainWithOptions () [] msg (const . const Nothing) prog args