futhark 0.16.1 → 0.16.2

raw patch · 16 files changed

+138/−32 lines, 16 filesPVP: major bump suggested

API removals or changes: PVP suggests a major version bump

API changes (from Hackage documentation)

Files

• docs/c-api.rst +45/−0
• docs/language-reference.rst +8/−6
• docs/usage.rst +17/−0
• futhark.cabal +1/−1
• rts/c/util.h +1/−1
• rts/python/opencl.py +6/−1
• src/Futhark/CLI/Autotune.hs +6/−0
• src/Futhark/CLI/Bench.hs +8/−4
• src/Futhark/CodeGen/Backends/CCUDA/Boilerplate.hs +8/−0
• src/Futhark/CodeGen/Backends/COpenCL/Boilerplate.hs +8/−0
• src/Futhark/CodeGen/Backends/GenericC.hs +3/−1
• src/Futhark/CodeGen/Backends/GenericPython.hs +6/−6
• src/Futhark/CodeGen/ImpGen/Kernels/Base.hs +4/−2
• src/Futhark/IR/Mem/IxFun.hs +2/−2
• src/Futhark/IR/Prop/Types.hs +0/−1
• src/Language/Futhark/TypeChecker/Terms.hs +15/−7

@@ -81,6 +81,13 @@    you must not pass values between them.  They have the same C type,    so this is an easy mistake to make. +   After you have created a context object, you must immediately call+   :c:func:`futhark_context_get_error`, which will return non-``NULL``+   if initialisation failed.  If initialisation has failed, then you+   still need to call :c:func:`futhark_context_free` to release+   resources used for the context object, but you may not use the+   context object for anything else.+ .. c:function:: void futhark_context_free(struct futhark_context *ctx)     Free the context object.  It must not be used again.  The@@ -207,6 +214,17 @@    sure to call :c:func:`futhark_context_sync` before using the value    of ``out0``. +The exact behaviour of the exit code depends on the backend.  For the+sequential C backend, errors will always be available when the entry+point returns, and :c:func:`futhark_context_sync` will always return+success.  When using a GPU backend such as ``cuda`` or ``opencl``, the+entry point may still be running asynchronous operations when it+returns, in which case the entry point may return zero successfully,+even though execution has already (or will) fail.  These problems will+be reported when :c:func:`futhark_context_sync` is called.  When using+GPU backends, be careful to check the return code of *both* the entry+point itself, and :c:func:`futhark_context_sync`.+ GPU --- @@ -325,3 +343,30 @@     During :c:func:`futhark_context_new`, read PTX code from the given    file instead of using the embedded program.++General guarantees+------------------++Calling an entry point, or interacting with Futhark values through the+functions listed above, has no system-wide side effects, such as+writing to the file system, launching processes, or performing network+connections.  Defects in the program or Futhark compiler itself can+with high probability result only in the consumption of CPU or GPU+resources, or a process crash.++Using the ``#[unsafe]`` attribute with in-place updates can result in+writes to arbitrary memory locations.  A malicious program can likely+exploit this to obtain arbitrary code execution, just as with any+insecure C program.  If you must run untrusted code, consider using+the ``--safe`` command line option to instruct the compiler to disable+``#[unsafe]``.++Initialising a Futhark context likewise has no side effects, except if+explicitly configured differently, such as by using+:c:func:`futhark_context_config_dump_program_to`.  In its default+configuration, Futhark will not access the file system.++Note that for the GPU backends, the underlying API (such as CUDA or+OpenCL) may perform file system operations during startup, and perhaps+for caching GPU kernels in some cases.  This is beyond Futhark's+control.

@@ -728,12 +728,14 @@   ``^``, ``&``, ``|``, ``>>``, ``<<``      Bitwise operators, respectively bitwise xor, and, or, arithmetic-    shift right and left, and logical shift right.  Shift amounts-    must be non-negative and the operands must be integers.  Note-    that, unlike in C, bitwise operators have *higher* priority than-    arithmetic operators.  This means that ``x & y == z`` is-    understood as ``(x & y) == z``, rather than ``x & (y == z)`` as-    it would in C.  Note that the latter is a type error in Futhark+    shift right and left, and logical shift right.  **Shifting is+    undefined if the right operand is negative, or greater than or+    equal to the length in bits of the left operand.**++    Note that, unlike in C, bitwise operators have *higher* priority+    than arithmetic operators.  This means that ``x & y == z`` is+    understood as ``(x & y) == z``, rather than ``x & (y == z)`` as it+    would in C.  Note that the latter is a type error in Futhark     anyhow.    ``==``, ``!=``

@@ -19,6 +19,8 @@ useful mostly for testing and benchmarking.  Libraries can be called from non-Futhark code. +.. _executable:+ Compiling to Executable ----------------------- @@ -450,3 +452,18 @@ Futhark arrays are mapped to either the Numpy ``ndarray`` type or the `pyopencl.array <https://documen.tician.de/pyopencl/array.html>`_ type.  Scalars are mapped to Numpy scalar types.++Reproducibility+---------------++The Futhark compiler is deterministic by design, meaning that+repeatedly compiling the *same program* with the *same compilation+flags* and using the *same version* of the compiler will produce+identical output every time.++Note that this only applies to the code generated by the Futhark+compiler itself.  When compiling to an executable with one of the C+backends (see :ref:`executable`), Futhark will invoke a C compiler+that may not be perfectly reproducible.  In such cases the generated+``.c`` and ``.h`` files will be reproducible, but the final executable+may not.

@@ -1,7 +1,7 @@ cabal-version: 2.4  name:           futhark-version:        0.16.1+version:        0.16.2 synopsis:       An optimising compiler for a functional, array-oriented language.  description:    Futhark is a small programming language designed to be compiled to

@@ -2,7 +2,7 @@ // // Various helper functions that are useful in all generated C code. -static const char *fut_progname = "(some Futhark code)";+static const char *fut_progname = "(embedded Futhark)";  static void futhark_panic(int eval, const char *fmt, ...) {   va_list ap;

@@ -204,7 +204,7 @@     if (len(program_src) >= 0):         return cl.Program(self.ctx, program_src).build(             ["-DLOCKSTEP_WIDTH={}".format(lockstep_width)]-            + ["-D{}={}".format(s.replace('z', 'zz').replace('.', 'zi'),v) for (s,v) in self.sizes.items()])+            + ["-D{}={}".format(s.replace('z', 'zz').replace('.', 'zi').replace('#', 'zh'),v) for (s,v) in self.sizes.items()])  def opencl_alloc(self, min_size, tag):     min_size = 1 if min_size == 0 else min_size@@ -219,6 +219,11 @@     cl.enqueue_copy(self.queue, failure, self.global_failure, is_blocking=True)     self.failure_is_an_option = np.int32(0)     if failure[0] >= 0:+        # Reset failure information.+        cl.enqueue_fill_buffer(self.queue, self.global_failure, np.int32(-1), 0, np.int32().itemsize)++        # Read failure args.         failure_args = np.empty(self.global_failure_args_max+1, dtype=np.int32)         cl.enqueue_copy(self.queue, failure_args, self.global_failure_args, is_blocking=True)+         raise Exception(self.failure_msgs[failure[0]].format(*failure_args))

@@ -348,6 +348,12 @@     (ReqArg (\prog -> Right $ \config -> config { optFuthark = Just prog })      "PROGRAM")     "The binary used for operations (defaults to 'futhark')."+  , Option [] ["pass-option"]+    (ReqArg (\opt ->+               Right $ \config ->+               config { optExtraOptions = opt : optExtraOptions config })+     "OPT")+    "Pass this option to programs being run."   , Option [] ["tuning"]     (ReqArg (\s -> Right $ \config -> config { optTuning = Just s })     "EXTENSION")

@@ -165,15 +165,19 @@ progressBar :: Int -> Int -> Int -> String progressBar cur bound steps =   "[" ++ map cell [1..steps] ++ "] " ++ show cur ++ "/" ++ show bound-  where step_size = bound `div` steps+  where step_size :: Double+        step_size = fromIntegral bound / fromIntegral steps+        cur' = fromIntegral cur         chars = " ▏▎▍▍▌▋▊▉█"         char i = fromMaybe ' ' $ maybeNth (i::Int) chars+        num_chars = fromIntegral $ length chars          cell :: Int -> Char         cell i-          | i * step_size <= cur = char 9-          | otherwise = char (((cur - (i-1) * step_size) * length chars)-                              `div` step_size)+          | i' * step_size <= cur' = char 9+          | otherwise = char (floor (((cur' - (i'-1) * step_size) * num_chars)+                                     / step_size))+          where i' = fromIntegral i  descString :: String -> Int -> String descString desc pad_to = desc ++ ": " ++ replicate (pad_to - length desc) ' '

@@ -386,6 +386,14 @@                    ctx->failure_is_an_option = 0;                     if (failure_idx >= 0) {+                     // We have to clear global_failure so that the next entry point+                     // is not considered a failure from the start.+                     typename int32_t no_failure = -1;+                     CUDA_SUCCEED(+                       cuMemcpyHtoD(ctx->global_failure,+                                    &no_failure,+                                    sizeof(int32_t)));+                      typename int32_t args[$int:max_failure_args+1];                      CUDA_SUCCEED(                        cuMemcpyDtoH(&args,

@@ -416,6 +416,14 @@                  OPENCL_SUCCEED_OR_RETURN(clFinish(ctx->opencl.queue));                   if (failure_idx >= 0) {+                   // We have to clear global_failure so that the next entry point+                   // is not considered a failure from the start.+                   typename cl_int no_failure = -1;+                   OPENCL_SUCCEED_OR_RETURN(+                    clEnqueueWriteBuffer(ctx->opencl.queue, ctx->global_failure, CL_TRUE,+                                         0, sizeof(cl_int), &no_failure,+                                         0, NULL, NULL));+                    typename cl_int args[$int:max_failure_args+1];                    OPENCL_SUCCEED_OR_RETURN(                      clEnqueueReadBuffer(ctx->opencl.queue,

@@ -1656,6 +1656,7 @@                     mapM_ resetMemConst ps                     compileCode init_consts   libDecl [C.cedecl|int init_constants($ty:ctx_ty *ctx) {+      (void)ctx;       int err = 0;       $items:defs       $items:init_consts'@@ -1666,6 +1667,7 @@    free_consts <- collect $ mapM_ freeConst ps   libDecl [C.cedecl|int free_constants($ty:ctx_ty *ctx) {+      (void)ctx;       $items:free_consts       return 0;     }|]@@ -1988,7 +1990,7 @@   cached <- cacheMem name   case cached of     Just cur_size ->-      stm [C.cstm|if ($exp:cur_size < $exp:size) {+      stm [C.cstm|if ($exp:cur_size < (size_t)$exp:size) {                     $exp:name = realloc($exp:name, $exp:size);                     $exp:cur_size = $exp:size;                   }|]

@@ -294,7 +294,7 @@             -> [Option]             -> Imp.Definitions op             -> m String-compileProg module_name constructor imports defines ops userstate pre_timing options prog = do+compileProg module_name constructor imports defines ops userstate sync options prog = do   src <- getNameSource   let prog' = runCompilerM ops src userstate compileProg'       maybe_shebang =@@ -321,14 +321,14 @@           case module_name of             Just name -> do               (entry_points, entry_point_types) <--                unzip <$> mapM compileEntryFun (filter (Imp.functionEntry . snd) funs)+                unzip <$> mapM (compileEntryFun sync) (filter (Imp.functionEntry . snd) funs)               return [ClassDef $ Class name $                        Assign (Var "entry_points") (Dict entry_point_types) :                        map FunDef (constructor' : definitions ++ entry_points)]             Nothing -> do               let classinst = Assign (Var "self") $ simpleCall "internal" []               (entry_point_defs, entry_point_names, entry_points) <--                unzip3 <$> mapM (callEntryFun pre_timing)+                unzip3 <$> mapM (callEntryFun sync)                 (filter (Imp.functionEntry . snd) funs)               return (parse_options ++                       ClassDef (Class "internal" $ map FunDef $@@ -604,14 +604,14 @@                      [srcmem, srcidx, Var "ct.c_byte"]   stm $ Exp $ simpleCall "ct.memmove" [offset_call1, offset_call2, nbytes] -compileEntryFun :: (Name, Imp.Function op)+compileEntryFun :: [PyStmt] -> (Name, Imp.Function op)                 -> CompilerM op s (PyFunDef, (PyExp, PyExp))-compileEntryFun entry = do+compileEntryFun sync entry = do   (fname', params, prepareIn, body_lib, _, prepareOut, res, _) <- prepareEntry entry   let ret = Return $ tupleOrSingle $ map snd res       (pts, rts) = entryTypes $ snd entry   return (Def fname' ("self" : params) $-           prepareIn ++ body_lib ++ prepareOut ++ [ret],+           prepareIn ++ body_lib ++ prepareOut ++ sync ++ [ret],           (String fname', Tuple [List (map String pts), List (map String rts)]))  entryTypes :: Imp.Function op -> ([String], [String])

@@ -1425,7 +1425,8 @@    -- Avoid loop for the common case where each thread is statically   -- known to write at most one element.-  if toExp' int32 per_group_elems == kernelGroupSize constants+  localOps threadOperations $+    if toExp' int32 per_group_elems == kernelGroupSize constants     then sWhen (offset + ltid .<. toExp' int32 w) $          copyDWIMFix (patElemName pe) [ltid + offset] (Var what) [ltid]     else@@ -1441,7 +1442,8 @@   local_is <- localThreadIDs $ map snd dims   is_for_thread <- mapM (dPrimV "thread_out_index") $ zipWith (+) group_is local_is -  sWhen (isActive $ zip is_for_thread $ map fst dims) $+  localOps threadOperations $+    sWhen (isActive $ zip is_for_thread $ map fst dims) $     copyDWIMFix (patElemName pe) (map Imp.vi32 is_for_thread) (Var what) local_is  compileGroupResult space pe (Returns _ what) = do

@@ -421,8 +421,8 @@       mid_dims = take (length dims - num_coercions) $ drop hd_len dims'       num_rshps = length reshapes   guard (num_rshps == 0 || (num_rshps == 1 && length mid_dims == 1))-  let dims'' = map snd $ sortBy (compare `on` fst) $-               zipWith (\ld n -> (ldPerm ld, ld { ldShape = n }))+  let dims'' = permuteInv perm $+               zipWith (\ld n -> ld { ldShape = n })                dims' (newDims newshape)       lmad' = LMAD off dims''   return $ IxFun (lmad' :| lmads) (newDims newshape) cg

@@ -5,7 +5,6 @@          rankShaped        , arrayRank        , arrayShape-       , modifyArrayShape        , setArrayShape        , existential        , uniqueness

@@ -2193,7 +2193,6 @@   occur [observation as loc]    checkOccurences dflow-  occurs <- consumeArg argloc argtype' (diet tp1')    case anyConsumption dflow of     Just c ->@@ -2201,7 +2200,11 @@       in zeroOrderType (mkUsage argloc "potential consumption in expression") msg tp1     _ -> return () -  occur $ dflow `seqOccurences` occurs+  occurs <- (dflow `seqOccurences`) <$> consumeArg argloc argtype' (diet tp1')++  checkIfConsumable loc $ S.map AliasBound $ allConsumed occurs+  occur occurs+   (argext, parsubst) <-     case pname of       Named pname' -> do@@ -2928,9 +2931,8 @@   let als = AliasBound nm `S.insert` aliases t   in occur [observation als loc] --- | Proclaim that we have written to the given variable.-consume :: SrcLoc -> Aliasing -> TermTypeM ()-consume loc als = do+checkIfConsumable :: SrcLoc -> Aliasing -> TermTypeM ()+checkIfConsumable loc als = do   vtable <- asks $ scopeVtable . termScope   let consumable v = case M.lookup v vtable of                        Just (BoundV Local _ t)@@ -2940,9 +2942,15 @@                        _ -> False   case filter (not . consumable) $ map aliasVar $ S.toList als of     v:_ -> typeError loc mempty $-           "Attempt to consume variable" <+> pquote (pprName v)+           "Would consume variable" <+> pquote (pprName v)            <> ", which is not allowed."-    [] -> occur [consumption als loc]+    [] -> return ()++-- | Proclaim that we have written to the given variable.+consume :: SrcLoc -> Aliasing -> TermTypeM ()+consume loc als = do+  checkIfConsumable loc als+  occur [consumption als loc]  -- | Proclaim that we have written to the given variable, and mark -- accesses to it and all of its aliases as invalid inside the given