htvm 0.1.1 → 0.1.2

raw patch · 2 files changed

+220/−4 lines, 2 files

Files

• README.md +215/−0
• htvm.cabal +5/−4

@@ -0,0 +1,215 @@+HTVM+====++HTVM is a library which provides Haskell runtime and experimental frontend for+[TVM](https://tvm.ai/about) the Machine Learning framework.++**Both HTVM and TVM are under development. While TVM is somewhat stable, we+don't recommend to use HTVM in applications currently**++**[GitHub repository](https://github.com/grwlf/htvm) may contain newer version of HTVM**++TVM in a nutshell+-----------------++TVM framework extends [Halide](https://halide-lang.org) principles to Machine+Learning domain. It offers (a) EDSLs for defining and hand-optimizing ML models+(b) export/import facilities for translating models from other frameworks such+as TensorFlow and (c) compiler to binary code for a variety of supported+platforms, including LLVM (x86, arm), CUDA, OpenCL, Vulcan, ROCm, FPGAs and even+WebAssembly (note: level of support may vary). DSLs for C++ and Python are best+supported and also there are some support for Java, Go and Rust languages.++[Watch Halide introduction video](https://youtu.be/3uiEyEKji0M)++[Read more on TVM site](https://tvm.ai/about)++Originally, TVM aimed at increasing speed of model's inference by providing a+rich set of optimizing primitives called+['schedules'](https://docs.tvm.ai/tutorials/language/schedule_primitives.html#sphx-glr-tutorials-language-schedule-primitives-py)).+At the same time it had little support for training models. Recently,+training-related proposals were+[added](https://sea-region.github.com/dmlc/tvm/issues/1996).++TVM aims at compiling ML models in highly optimized binary code.++Important parts of TVM are:+  * `tvm` is a core library providing `compute` interface.+  * `topi` is a tensor operations collection. Most of the middle-layer+    primitives such as `matmul`, `conv2d` and `softmax` are defined there.+  * `relay` is a high-level library written in Python, providing+    functional-style interface and its own typechecker. Currently, relay is+    under active development and beyond the scope of HTVM.+  * `nnvm` is another high-level wrapper in Python, now deprecated in favor of+    `relay`.++Features and goals+------------------++In HTVM we are going to provide:++ 1. C Runtime, which makes it possible to run TVM models from Haskell.+ 2. Experimental EDSL for building TVM programs in Haskell.++Combined TVM/HTVM-stack features are:++### FFI++  * Not many dependencies: TVM is much easier to build than other frameworks (hi+    TensorFlow). Models are compiled to binary code, no interpreters required.+  * Performance: HTVM uses TVM, which is designed with performace in mind.+  * Simplicity of code.++### EDSL++  * Experimental status+  * Simplicity again. Pure ADT-based design.+  * Not much type-safety yet. Expect errors in runtime. Typechecker may be+    implemented in future.++Install+-------++### Installing dependencies++1. Make sure you have `g++` and `llvm` installed.++2. Build tvm from development repository located at+   https://github.com/grwlf/tvm, branch autodiff++   ```+   $ git clone https://github.com/grwlf/tvm+   $ cd tvm+   $ git checkout origin/autodiff+   .. follow up with the tvm build procedure+   ```++### Building HTVM++We use development environment specified in [Nix](https://nixos.org/nix)+language. In order to open it, please install the+[Nix package manager](https://nixos.org/nix/download.html).+Having Nix manager and `NIX_PATH` set, enter the environment, by running Nix+development shell from the project's root folder:++    $ nix-shell++It should get all the Haskell dependencies upon the first run.  Alternatively,+it should be possible to run Haskell distributions like [Haskell+Platform](https://www.haskell.org/platform/).++After nix-shell or Haskell distibution is ready, run `cabal` to build the+project.++    $ cabal configure --enable-tests+    $ cabal build++To run tests, execute the test suite. At this point you will need `g++`, `clang`+and `tvm` of the correct version (see above).++    $ cabal test++To enter the interactive shell, type++    $ cabal repl htvm+    *HTVM.EDSL.Types> :lo Demo++Usage examples may be found in [Tests](./test/Main.hs) and (possibly outdated)+[Demo](./src/Demo.hs).++TODO: Update demo, write more examples++Design notes+------------++### TVM C Runtime++FFI for TVM C Runtime library is a Haskell package, linked to+`libtvm_runtime.so`. This library contains functionality, required to load and+run ML code produced by TVM.++ 1. The module provide wrappers to `c_runtime_api.h` functions.+ 2. `TVMArray` is the main type describing Tensors in TVM. It is represented as+    ForeignPtr to internal representation and a set of accessor functions.+ 3. Currently, HTVM can marshal data from Haskell lists. Support for+    `Data.Array` is planned.+ 4. No backends besides LLVM are tested. Adding them should not be hard and is+    on the TODO list.++### TVM Haskell EDSL++EDSL has a proof-of-concept status. It may be used to declare ML models in+Haskell, convert them to TVM IR and finally compile.  Later, compiled model may be+loaded and run with Haskell FFI or with any other runtime supported by TVM.++Contrary to usual practices, we don't manipulate TVM IR by calling TVM functions+internally. Instead, we build AST in Haskell and print it to C++ program. After+that we compile the program with common instruments. This approach has its pros and+cons, which are described below.++ 1. `HTVM.EDSL.Types` module defines AST types which loosely corresponds to+    `Stmt` and `Expr` class hierarchies of TVM.+ 2. `HTVM.EDSL.Monad` provides monadic interface to AST builders. We favored+    simplicity over type-safety. We belive that overuse of Haskell type system+    ruined many good libraries. The interface relies on simple ADTs whenever+    possible.+ 3. `HTVM.EDSL.Print` contain functions which print AST to C++ program of Model+    Generator.+ 4. `HTVM.EDSL.Build` provides instruments to compile and run the model+    generator by executing `g++` and `clang` compilers:+    * The Model Generator program builds TVM IR and generates LLVM assembly.+      In HTVM, we support LLVM target, but more targets may be added later.+    * We execute `clang` to compile LLVM into x86 '.so' library. Resulting+      library may be loaded and executed by the Runtime code.++The whole data transformation pipeline goes as follows:++```++Monadic    --> AST --> C++ --> Model --> LLVM --> Model --> Runtime FFI+Interface   .       .       .  Gen    .  asm   .  Library+            .       .       .         .        .+            .     Print     .       Print      .+           Run    C++      g++               clang++```++Known disadvantages of C++ printing approach are:+- **Compilation speed is limited by the speed of `g++`, which is slow.** Gcc is+  used to compile C++ to binary which may take as long as 5 seconds. Little may+  be done about that without changing approaches. One possible way to overcome+  this limitation would be to provide direct FFI to TVM IR like+  [Halide-hs](https://github.com/cchalmers/halide-hs) does for Halide.+  Unfortunately, this approach has its own downsides:+  * Low-level IR API is not as stable as its high-level counterpart+  * TVM is in its early stages and sometimes crashes. FFI to IR would provide no+    isolation from this.+- **Calling construction-time procedures of TVM is non-trivial.** This is a+  consequence of previous limitation. For example, TVM may calculate Tensor+  shape in runtime and use it immediately to define new Tensors. In order to+  that in Haskell we would need to compile and run C++ program which is possible+  by slow. We try to avoid calling construction-time procedures.+- **User may face weird C++ errors**. TVM is quite a low-level library which+  offers little type-checking, so user may write bad programs easily. Other high+  level TVM wrappers like Relay in Python, does provide their own typecheckers+  to catch errors earlier. HTVM offers no typechecker currently but it is+  certainly possible to write one. Contributions are welcome!++The pros of this approach are:+- C++ printer is implemented in less than 300 lines of code. Easy to maintain.+- Easy to port to another TVM dialect such as Relay.+- Isolation from TVM crashes. Memory problems of TVM IR will be translated to error+  messages in Haskell.++Future plans+------------++ * We aim at supporting basic `import tvm` and `import topi` functionality.+ * Support for Scheduling is minimal, but should be enhanced in future.+ * Support for TOPI is minimal, but should be enhanced in future.+ * No targets besides LLVM are supported. Adding them should be as simple as+   adding them to C++ DSL.+ * We plan to support [Tensor-Level AD](https://sea-region.github.com/dmlc/tvm/issues/1996)+ * Adding support for [Relay](https://github.com/dmlc/tvm/issues/1673) is also+   possible but may require some efforts like writing Python printer.+

@@ -1,11 +1,11 @@ name:                htvm-synopsis:            TVM bindings.+synopsis:            Bindings for TVM machine learning framework description:-    This library provides interface to TVM Runtime, and experimental-    EDSL for building models with TVM.+    This library provides interface to TVM framework. Runtime FFI and+    experimental EDSL for defining models are included.  homepage:            https://github.com/grwlf/htvm-version:             0.1.1+version:             0.1.2 license:             GPL-3 license-file:        LICENSE category:            Machine Learning@@ -14,6 +14,7 @@ build-type:          Simple extra-source-files:  ChangeLog.md cabal-version:       >=1.10+extra-source-files:  README.md  library   build-depends: