diff --git a/LICENSE b/LICENSE new file mode 100644 --- /dev/null +++ b/LICENSE @@ -0,0 +1,203 @@ +Copyright 2016 The TensorFlow Authors. All rights reserved. + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright 2016, The TensorFlow Authors. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/Setup.hs b/Setup.hs new file mode 100644 --- /dev/null +++ b/Setup.hs @@ -0,0 +1,3 @@ +import Distribution.Simple + +main = defaultMain diff --git a/app/Main.hs b/app/Main.hs new file mode 100644 --- /dev/null +++ b/app/Main.hs @@ -0,0 +1,147 @@ +-- Copyright 2016 TensorFlow authors. +-- +-- Licensed under the Apache License, Version 2.0 (the "License"); +-- you may not use this file except in compliance with the License. +-- You may obtain a copy of the License at +-- +-- http://www.apache.org/licenses/LICENSE-2.0 +-- +-- Unless required by applicable law or agreed to in writing, software +-- distributed under the License is distributed on an "AS IS" BASIS, +-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +-- See the License for the specific language governing permissions and +-- limitations under the License. + +{-# LANGUAGE FlexibleContexts #-} +{-# LANGUAGE OverloadedLists #-} +{-# LANGUAGE TypeApplications #-} + +import Control.Monad (forM_, when) +import Control.Monad.IO.Class (liftIO) +import Data.Int (Int32, Int64) +import Data.List (genericLength) +import qualified Data.Text.IO as T +import qualified Data.Vector as V + +import qualified TensorFlow.Core as TF +import qualified TensorFlow.Ops as TF hiding (initializedVariable, zeroInitializedVariable) +import qualified TensorFlow.Variable as TF +import qualified TensorFlow.Minimize as TF + +import TensorFlow.Examples.MNIST.InputData +import TensorFlow.Examples.MNIST.Parse + +numPixels, numLabels :: Int64 +numPixels = 28*28 :: Int64 +numLabels = 10 :: Int64 + +-- | Create tensor with random values where the stddev depends on the width. +randomParam :: Int64 -> TF.Shape -> TF.Build (TF.Tensor TF.Build Float) +randomParam width (TF.Shape shape) = + (`TF.mul` stddev) <$> TF.truncatedNormal (TF.vector shape) + where + stddev = TF.scalar (1 / sqrt (fromIntegral width)) + +-- Types must match due to model structure. +type LabelType = Int32 + +data Model = Model { + train :: TF.TensorData Float -- ^ images + -> TF.TensorData LabelType + -> TF.Session () + , infer :: TF.TensorData Float -- ^ images + -> TF.Session (V.Vector LabelType) -- ^ predictions + , errorRate :: TF.TensorData Float -- ^ images + -> TF.TensorData LabelType + -> TF.Session Float + } + +createModel :: TF.Build Model +createModel = do + -- Use -1 batch size to support variable sized batches. + let batchSize = -1 + -- Inputs. + images <- TF.placeholder [batchSize, numPixels] + -- Hidden layer. + let numUnits = 500 + hiddenWeights <- + TF.initializedVariable =<< randomParam numPixels [numPixels, numUnits] + hiddenBiases <- TF.zeroInitializedVariable [numUnits] + let hiddenZ = (images `TF.matMul` TF.readValue hiddenWeights) + `TF.add` TF.readValue hiddenBiases + let hidden = TF.relu hiddenZ + -- Logits. + logitWeights <- + TF.initializedVariable =<< randomParam numUnits [numUnits, numLabels] + logitBiases <- TF.zeroInitializedVariable [numLabels] + let logits = (hidden `TF.matMul` TF.readValue logitWeights) + `TF.add` TF.readValue logitBiases + predict <- TF.render @TF.Build @LabelType $ + TF.argMax (TF.softmax logits) (TF.scalar (1 :: LabelType)) + + -- Create training action. + labels <- TF.placeholder [batchSize] + let labelVecs = TF.oneHot labels (fromIntegral numLabels) 1 0 + loss = + TF.reduceMean $ fst $ TF.softmaxCrossEntropyWithLogits logits labelVecs + params = [hiddenWeights, hiddenBiases, logitWeights, logitBiases] + trainStep <- TF.minimizeWith TF.adam loss params + + let correctPredictions = TF.equal predict labels + errorRateTensor <- TF.render $ 1 - TF.reduceMean (TF.cast correctPredictions) + + return Model { + train = \imFeed lFeed -> TF.runWithFeeds_ [ + TF.feed images imFeed + , TF.feed labels lFeed + ] trainStep + , infer = \imFeed -> TF.runWithFeeds [TF.feed images imFeed] predict + , errorRate = \imFeed lFeed -> TF.unScalar <$> TF.runWithFeeds [ + TF.feed images imFeed + , TF.feed labels lFeed + ] errorRateTensor + } + +main :: IO () +main = TF.runSession $ do + -- Read training and test data. + trainingImages <- liftIO (readMNISTSamples =<< trainingImageData) + trainingLabels <- liftIO (readMNISTLabels =<< trainingLabelData) + testImages <- liftIO (readMNISTSamples =<< testImageData) + testLabels <- liftIO (readMNISTLabels =<< testLabelData) + + -- Create the model. + model <- TF.build createModel + + -- Functions for generating batches. + let encodeImageBatch xs = + TF.encodeTensorData [genericLength xs, numPixels] + (fromIntegral <$> mconcat xs) + let encodeLabelBatch xs = + TF.encodeTensorData [genericLength xs] + (fromIntegral <$> V.fromList xs) + let batchSize = 100 + let selectBatch i xs = take batchSize $ drop (i * batchSize) (cycle xs) + + -- Train. + forM_ ([0..1000] :: [Int]) $ \i -> do + let images = encodeImageBatch (selectBatch i trainingImages) + labels = encodeLabelBatch (selectBatch i trainingLabels) + train model images labels + when (i `mod` 100 == 0) $ do + err <- errorRate model images labels + liftIO $ putStrLn $ "training error " ++ show (err * 100) + liftIO $ putStrLn "" + + -- Test. + testErr <- errorRate model (encodeImageBatch testImages) + (encodeLabelBatch testLabels) + liftIO $ putStrLn $ "test error " ++ show (testErr * 100) + + -- Show some predictions. + testPreds <- infer model (encodeImageBatch testImages) + liftIO $ forM_ ([0..3] :: [Int]) $ \i -> do + putStrLn "" + T.putStrLn $ drawMNIST $ testImages !! i + putStrLn $ "expected " ++ show (testLabels !! i) + putStrLn $ " got " ++ show (testPreds V.! i) diff --git a/data/MNIST.pb b/data/MNIST.pb new file mode 100644 Binary files /dev/null and b/data/MNIST.pb differ diff --git a/data/MNISTBias.ckpt b/data/MNISTBias.ckpt new file mode 100644 Binary files /dev/null and b/data/MNISTBias.ckpt differ diff --git a/data/MNISTWts.ckpt b/data/MNISTWts.ckpt new file mode 100644 Binary files /dev/null and b/data/MNISTWts.ckpt differ diff --git a/src-data/TensorFlow/Examples/MNIST/TrainedGraph.hs b/src-data/TensorFlow/Examples/MNIST/TrainedGraph.hs new file mode 100644 --- /dev/null +++ b/src-data/TensorFlow/Examples/MNIST/TrainedGraph.hs @@ -0,0 +1,30 @@ +-- Copyright 2016 TensorFlow authors. +-- +-- Licensed under the Apache License, Version 2.0 (the "License"); +-- you may not use this file except in compliance with the License. +-- You may obtain a copy of the License at +-- +-- http://www.apache.org/licenses/LICENSE-2.0 +-- +-- Unless required by applicable law or agreed to in writing, software +-- distributed under the License is distributed on an "AS IS" BASIS, +-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +-- See the License for the specific language governing permissions and +-- limitations under the License. + +{-# LANGUAGE OverloadedStrings #-} +-- | Paths to test helper files. +module TensorFlow.Examples.MNIST.TrainedGraph where + +import Paths_tensorflow_mnist (getDataFileName) +import Data.ByteString (ByteString) +import Data.ByteString.Char8 (pack) + +-- | File containing a Tensorflow serialized proto of MNIST. +mnistPb :: IO FilePath +mnistPb = getDataFileName "data/MNIST.pb" + +-- | Files containing pre-trained weights for MNIST. +wtsCkpt, biasCkpt :: IO ByteString +wtsCkpt = pack <$> getDataFileName "data/MNISTWts.ckpt" +biasCkpt = pack <$> getDataFileName "data/MNISTBias.ckpt" diff --git a/src/TensorFlow/Examples/MNIST/Parse.hs b/src/TensorFlow/Examples/MNIST/Parse.hs new file mode 100644 --- /dev/null +++ b/src/TensorFlow/Examples/MNIST/Parse.hs @@ -0,0 +1,96 @@ +-- Copyright 2016 TensorFlow authors. +-- +-- Licensed under the Apache License, Version 2.0 (the "License"); +-- you may not use this file except in compliance with the License. +-- You may obtain a copy of the License at +-- +-- http://www.apache.org/licenses/LICENSE-2.0 +-- +-- Unless required by applicable law or agreed to in writing, software +-- distributed under the License is distributed on an "AS IS" BASIS, +-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +-- See the License for the specific language governing permissions and +-- limitations under the License. + +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE OverloadedLists #-} +{-# LANGUAGE TypeSynonymInstances #-} +{-# LANGUAGE FlexibleInstances #-} +{-# LANGUAGE ViewPatterns #-} + +module TensorFlow.Examples.MNIST.Parse where + +import Control.Monad (when, liftM) +import Data.Binary.Get (Get, runGet, getWord32be, getLazyByteString) +import Data.ByteString.Lazy (toStrict, readFile) +import Data.List.Split (chunksOf) +import Data.Monoid ((<>)) +import Data.ProtoLens (Message, decodeMessageOrDie) +import Data.Text (Text) +import Data.Word (Word8, Word32) +import Prelude hiding (readFile) +import qualified Codec.Compression.GZip as GZip +import qualified Data.ByteString.Lazy as L +import qualified Data.Text as Text +import qualified Data.Vector as V + +-- | Utilities specific to MNIST. +type MNIST = V.Vector Word8 + +-- | Produces a unicode rendering of the MNIST digit sample. +drawMNIST :: MNIST -> Text +drawMNIST = chunk . block + where + block :: V.Vector Word8 -> Text + block (V.splitAt 1 -> ([0], xs)) = " " <> block xs + block (V.splitAt 1 -> ([n], xs)) = c `Text.cons` block xs + where c = "\9617\9618\9619\9608" !! fromIntegral (n `div` 64) + block (V.splitAt 1 -> _) = "" + chunk :: Text -> Text + chunk "" = "\n" + chunk xs = Text.take 28 xs <> "\n" <> chunk (Text.drop 28 xs) + +-- | Check's the file's endianess, throwing an error if it's not as expected. +checkEndian :: Get () +checkEndian = do + magic <- getWord32be + when (magic `notElem` ([2049, 2051] :: [Word32])) $ + fail "Expected big endian, but image file is little endian." + +-- | Reads an MNIST file and returns a list of samples. +readMNISTSamples :: FilePath -> IO [MNIST] +readMNISTSamples path = do + raw <- GZip.decompress <$> readFile path + return $ runGet getMNIST raw + where + getMNIST :: Get [MNIST] + getMNIST = do + checkEndian + -- Parse header data. + cnt <- liftM fromIntegral getWord32be + rows <- liftM fromIntegral getWord32be + cols <- liftM fromIntegral getWord32be + -- Read all of the data, then split into samples. + pixels <- getLazyByteString $ fromIntegral $ cnt * rows * cols + return $ V.fromList <$> chunksOf (rows * cols) (L.unpack pixels) + +-- | Reads a list of MNIST labels from a file and returns them. +readMNISTLabels :: FilePath -> IO [Word8] +readMNISTLabels path = do + raw <- GZip.decompress <$> readFile path + return $ runGet getLabels raw + where getLabels :: Get [Word8] + getLabels = do + checkEndian + -- Parse header data. + cnt <- liftM fromIntegral getWord32be + -- Read all of the labels. + L.unpack <$> getLazyByteString cnt + +readMessageFromFileOrDie :: Message m => FilePath -> IO m +readMessageFromFileOrDie path = do + pb <- readFile path + return $ decodeMessageOrDie $ toStrict pb + +-- TODO: Write a writeMessageFromFileOrDie and read/write non-lethal +-- versions. diff --git a/tensorflow-mnist.cabal b/tensorflow-mnist.cabal new file mode 100644 --- /dev/null +++ b/tensorflow-mnist.cabal @@ -0,0 +1,80 @@ +name: tensorflow-mnist +version: 0.1.0.0 +synopsis: TensorFlow demo application for learning MNIST model. +description: Please see README.md +homepage: https://github.com/tensorflow/haskell#readme +license: Apache +license-file: LICENSE +author: TensorFlow authors +maintainer: tensorflow-haskell@googlegroups.com +copyright: Google Inc. +category: Machine Learning +build-type: Simple +cabal-version: >=1.22 +data-files: data/*.ckpt + , data/*.pb + +library + hs-source-dirs: src + , src-data + exposed-modules: TensorFlow.Examples.MNIST.Parse + , TensorFlow.Examples.MNIST.TrainedGraph + other-modules: Paths_tensorflow_mnist + build-depends: proto-lens == 0.2.* + , base >= 4.7 && < 5 + , binary + , bytestring + , filepath + , lens-family + , containers + , split + , tensorflow-proto == 0.2.* + , tensorflow-core-ops == 0.2.* + , tensorflow == 0.2.* + , text + , vector + , zlib + default-language: Haskell2010 + +executable Main + default-language: Haskell2010 + main-is: Main.hs + hs-source-dirs: app + build-depends: base + , bytestring + , filepath + , lens-family + , proto-lens + , tensorflow + , tensorflow-mnist + , tensorflow-mnist-input-data + , tensorflow-ops + , tensorflow-proto + , text + , transformers + , vector + +Test-Suite ParseTest + default-language: Haskell2010 + type: exitcode-stdio-1.0 + main-is: ParseTest.hs + hs-source-dirs: tests + build-depends: HUnit + , base + , bytestring + , proto-lens + , lens-family + , tensorflow + , tensorflow-mnist + , tensorflow-mnist-input-data + , tensorflow-ops + , tensorflow-proto + , test-framework + , test-framework-hunit + , text + , transformers + , vector + +source-repository head + type: git + location: https://github.com/tensorflow/haskell diff --git a/tests/ParseTest.hs b/tests/ParseTest.hs new file mode 100644 --- /dev/null +++ b/tests/ParseTest.hs @@ -0,0 +1,175 @@ +-- Copyright 2016 TensorFlow authors. +-- +-- Licensed under the Apache License, Version 2.0 (the "License"); +-- you may not use this file except in compliance with the License. +-- You may obtain a copy of the License at +-- +-- http://www.apache.org/licenses/LICENSE-2.0 +-- +-- Unless required by applicable law or agreed to in writing, software +-- distributed under the License is distributed on an "AS IS" BASIS, +-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +-- See the License for the specific language governing permissions and +-- limitations under the License. + +{-# LANGUAGE OverloadedLists #-} +{-# LANGUAGE OverloadedStrings #-} + +module Main where + +import Control.Monad.IO.Class (liftIO) +import Data.Int (Int64) +import Data.Text (Text) +import qualified Data.Text.IO as Text +import Lens.Family2 ((&), (.~), (^.)) +import Prelude hiding (abs) +import Proto.Tensorflow.Core.Framework.Graph + ( GraphDef(..) + , version + , node ) +import Proto.Tensorflow.Core.Framework.NodeDef + ( NodeDef(..) + , op ) +import System.IO as IO +import TensorFlow.Examples.MNIST.InputData +import TensorFlow.Examples.MNIST.Parse +import TensorFlow.Examples.MNIST.TrainedGraph +import TensorFlow.Build + ( asGraphDef + , addGraphDef + , Build + ) +import TensorFlow.Tensor + ( Tensor(..) + , Ref + , feed + , render + , tensorFromName + , tensorValueFromName + ) +import TensorFlow.Ops +import TensorFlow.Session + (runSession, run, run_, runWithFeeds, build) +import TensorFlow.Types (TensorDataType(..), Shape(..), unScalar) +import Test.Framework (defaultMain, Test) +import Test.Framework.Providers.HUnit (testCase) +import Test.HUnit ((@=?), Assertion) +import qualified Data.Vector as V + +-- | Test that a file can be read and the GraphDef proto correctly parsed. +testReadMessageFromFileOrDie :: Test +testReadMessageFromFileOrDie = testCase "testReadMessageFromFileOrDie" $ do + -- Check the function on a known well-formatted file. + mnist <- readMessageFromFileOrDie =<< mnistPb :: IO GraphDef + -- Simple field read. + 1 @=? mnist^.version + -- Count the number of nodes. + let nodes :: [NodeDef] + nodes = mnist^.node + 100 @=? length nodes + -- Check that the expected op is found at an arbitrary index. + "Variable" @=? nodes!!6^.op + +-- | Parse the test set for label and image data. Will only fail if the file is +-- missing or incredibly corrupt. +testReadMNIST :: Test +testReadMNIST = testCase "testReadMNIST" $ do + imageData <- readMNISTSamples =<< testImageData + 10000 @=? length imageData + labelData <- readMNISTLabels =<< testLabelData + 10000 @=? length labelData + +testNodeName :: Text -> Tensor Build a -> Assertion +testNodeName n g = n @=? opName + where + opName = head (gDef^.node)^.op + gDef = asGraphDef $ render g + +testGraphDefGen :: Test +testGraphDefGen = testCase "testGraphDefGen" $ do + -- Test the inferred operation type. + let f0 :: Tensor Build Float + f0 = 0 + testNodeName "Const" f0 + testNodeName "Add" $ 1 + f0 + testNodeName "Mul" $ 1 * f0 + testNodeName "Sub" $ 1 - f0 + testNodeName "Abs" $ abs f0 + testNodeName "Sign" $ signum f0 + testNodeName "Neg" $ -f0 + -- Test the grouping. + testNodeName "Add" $ 1 + f0 * 2 + testNodeName "Add" $ 1 + (f0 * 2) + testNodeName "Mul" $ (1 + f0) * 2 + +-- | Convert a simple graph to GraphDef, load it, run it, and check the output. +testGraphDefExec :: Test +testGraphDefExec = testCase "testGraphDefExec" $ do + let graphDef = asGraphDef $ render $ scalar (5 :: Float) * 10 + runSession $ do + addGraphDef graphDef + x <- run $ tensorValueFromName "Mul_2" + liftIO $ (50 :: Float) @=? unScalar x + +-- | Load MNIST from a GraphDef and the weights from a checkpoint and run on +-- sample data. +testMNISTExec :: Test +testMNISTExec = testCase "testMNISTExec" $ do + -- Switch to unicode to enable pretty printing of MNIST digits. + IO.hSetEncoding IO.stdout IO.utf8 + + -- Parse the Graph definition, samples, & labels from files. + mnist <- readMessageFromFileOrDie =<< mnistPb :: IO GraphDef + mnistSamples <- readMNISTSamples =<< testImageData + mnistLabels <- readMNISTLabels =<< testLabelData + + -- Select a sample to run on and convert it into a TensorData of Floats. + let idx = 12 + sample :: MNIST + sample = mnistSamples !! idx + label = mnistLabels !! idx + tensorSample = encodeTensorData (Shape [1,784]) floatSample + where + floatSample :: V.Vector Float + floatSample = V.map fromIntegral sample + Text.putStrLn $ drawMNIST sample + + -- Execute the graph on the sample data. + runSession $ do + -- The version of this session is 0, but the version of the graph is 1. + -- Change the graph version to 0 so they're compatible. + build $ addGraphDef $ mnist & version .~ 0 + -- Define nodes that restore saved weights and biases. + let bias, wts :: Tensor Ref Float + bias = tensorFromName "Variable" + wts = tensorFromName "weights" + wtsCkptPath <- liftIO wtsCkpt + biasCkptPath <- liftIO biasCkpt + -- Run those restoring nodes on the graph in the current session. + run_ =<< (sequence :: Monad m => [m a] -> m [a]) + [ restore wtsCkptPath wts + , restoreFromName biasCkptPath "bias" bias + ] + -- Encode the expected sample data as one-hot data. + let ty = encodeTensorData [10] oneHotLabels + where oneHotLabels = V.replicate 10 (0 :: Float) V.// updates + updates = [(fromIntegral label, 1)] + let feeds = [ feed (tensorValueFromName "x-input") tensorSample + , feed (tensorValueFromName "y-input") ty + ] + -- Run the graph with the input feeds and read the ArgMax'd result from + -- the test (not training) side of the evaluation. + x <- runWithFeeds feeds $ tensorValueFromName "test/ArgMax" + -- Print the trained model's predicted outcome. + liftIO $ putStrLn $ "Expectation: " ++ show label ++ "\n" + ++ "Prediction: " ++ show (unScalar x :: Int64) + -- Check whether the prediction matches the expectation. + liftIO $ (fromInteger . toInteger $ label :: Int64) @=? unScalar x + +main :: IO () +main = defaultMain + [ testReadMessageFromFileOrDie + , testReadMNIST + , testGraphDefGen + , testGraphDefExec + , testMNISTExec ]