rc 0.1.0.0 → 0.1.0.1

raw patch · 6 files changed

+77/−18 lines, 6 filesPVP ok

version bump matches the API change (PVP)

API changes (from Hackage documentation)

Files

• ChangeLog.md +8/−2
• README.md +14/−3
• examples/NTC/Main.hs +7/−4
• rc.cabal +2/−2
• rc/RC/Helpers.hs +1/−0
• rc/RC/NTC.hs +45/−7

@@ -1,3 +1,9 @@-# Changelog for rc+# Changelog for rc (reservoir computing) -## Unreleased changes+## 0.1.0.1 *February 25th 2018*+  * Better documentation++## 0.1.0.0 *February 23th 2018*+  * Initial release featuring single-node approach to nonlinear+    transient computing (NTC)+  * Mackey-Glass time series forecasting demo

@@ -2,9 +2,11 @@  Facilitating RC research + ## Features  * [Nonlinear transient computing (NTC)](https://github.com/masterdezign/rc/tree/master/examples/NTC)+* Echo State Networks (ESNs) (upcoming)   ## Getting Started@@ -14,15 +16,24 @@      $ git clone https://github.com/masterdezign/rc.git && cd rc      $ stack build --install-ghc -### [Example 1](https://github.com/masterdezign/rc/tree/master/examples/NTC). NTC-+### [Example 1](https://github.com/masterdezign/rc/tree/master/examples/NTC). NTC, time series forecasting       $ stack exec ntc       Error: 3.1103181863915367e-3 -[Here](https://raw.githubusercontent.com/masterdezign/rc/master/examples/NTC/mg-prediction.png) is visualized prediction result.+Visualize the prediction results: +     $ python3 examples/NTC/plot.py +![Figure](https://raw.githubusercontent.com/masterdezign/rc/master/examples/NTC/mg-prediction.png)  Great!+++## Further reading++* Larger, L., et al. “Photonic Information Processing beyond Turing: an Optoelectronic Implementation of Reservoir Computing.” Optics Express, vol. 20, no. 3, 2012, p. 3241., doi:10.1364/oe.20.003241.+* Rabinovič, Mihail Izrailevič, et al. Principles of Brain Dynamics: Global State Interactions. The MIT Press, 2012.+* Jaeger, H. “Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication.” Science, vol. 304, no. 5667, Feb. 2004, pp. 78–80., doi:10.1126/science.1091277.+* Bogdan Penkovsky. Theory and Modeling of Complex Nonlinear Delay Dynamics Applied to Neuromorphic Computing. Artificial Intelligence [cs.AI]. Université Bourgogne Franche-Comté, 2017. English. 〈tel-01591441v2〉

@@ -14,6 +14,10 @@ takeFuture :: Int -> Matrix Double -> Matrix Double takeFuture horizon = (?? (All, Drop horizon)) +splitAt' :: Int -> Matrix Double -> (Matrix Double, Matrix Double)+splitAt' i m = (m ?? (All, Take i), m ?? (All, Drop i))++main :: IO () main = do   -- Load and transpose time series to predict   dta <- tr <$> loadMatrix "examples/data/mg.txt"@@ -21,11 +25,10 @@   let splitRatio = 0.50  -- Train on 50% of data       total = cols dta       spl = round $ splitRatio * fromIntegral total-      -- Split the data-      train = dta ?? (All, Take spl)-      validate = dta ?? (All, Drop spl)+      -- Split the data into training and validation data sets+      (train, validate) = splitAt' spl dta -  -- Configure new NTC network+  -- Configure a new NTC network   let p = RC.par0 { RC._inputWeightsRange = (0.1, 0.3) }       g = mkStdGen 1111       ntc = RC.new g p (1, 1000, 1)

@@ -2,10 +2,10 @@ -- -- see: https://github.com/sol/hpack ----- hash: 32b8af1a3f4db71e5262f476ae36f22cd703e7dfd8e047f0c0d7b61b9ee1f3e8+-- hash: 080028c4b027364671d6876826a008e2c27817e0f1b4394a51c0a890ac59f86e  name:           rc-version:        0.1.0.0+version:        0.1.0.1 synopsis:       Reservoir Computing, fast RNNs description:    Please see the README on Github at <https://github.com/masterdezign/rc#readme> category:       Machine Learning

@@ -23,6 +23,7 @@       | otherwise = β * (width - offset)  -- | Prepend a row of ones+-- -- >>> addBiases $ (2><3) [20..26] -- (3><3) --  [  1.0,  1.0,  1.0

@@ -1,3 +1,4 @@+-- | -- = Nonlinear transient computing -- -- This module was developed as a part of author's PhD project:@@ -27,9 +28,28 @@  import qualified RC.Helpers as H --- | DDE reservoir abstraction+-- | Reservoir abstraction.+--+-- Reservoir (a recurrent neural network) is an essential+-- component in the NTC framework.+--+-- In this project, we exploit an established analogy between+-- spatially extended and delay systems ^1-3. That makes possible+-- to employ DDEs as a reservoir substrate. The choice of substrate+-- does not affect the `Reservoir` definition below.+--+-- ^1 Arecchi, F. T., et al. “Two-Dimensional Representation of+--    a Delayed Dynamical System.” Physical Review A, vol. 45, no. 7,+--    Jan. 1992, doi:10.1103/physreva.45.r4225.+-- ^2 Appeltant, L., et al. “Information Processing Using a Single+--    Dynamical Node as Complex System.” Nature Communications, vol. 2,+--    2011, p. 468., doi:10.1038/ncomms1476.+-- ^3 Virtual Chimera States for Delayed-Feedback Systems - Laurent Larger,+--    Bogdan Penkovsky, Yuri Maistrenko. Physical Review Letters - 08 / 2013+-- newtype Reservoir = Reservoir { _transform :: Matrix Double -> Matrix Double } +-- | Customizable NTC parameters data NTCParameters = Par   { _preprocess :: Matrix Double -> Matrix Double     -- ^ Modify data before masking (e.g. compression)@@ -40,6 +60,7 @@   , _reservoirModel :: DDEModel.Par   } +-- | NTC network structure data NTC = NTC   { _inputWeights :: Matrix Double   , _reservoir :: Reservoir@@ -48,6 +69,7 @@   , _par :: NTCParameters   } +-- | Creates an untrained NTC network new   :: StdGen   -> NTCParameters@@ -82,8 +104,11 @@             , DDEModel._theta = recip 0.34375             } +-- | Substrate-specific low-level reservoir implementation genReservoir :: DDEModel.Par -> Reservoir-genReservoir par = Reservoir _r+genReservoir par@DDEModel.RC {+    DDEModel._filt = DDEModel.BandpassFiltering { DDEModel._tau = tau }+  } = Reservoir _r   where     _r sample = unflatten response       where@@ -101,12 +126,18 @@         -- Duplicate the last element (DDE.integHeun2_2D consumes one extra input)         trace = trace1 V.++ V.singleton (V.last trace1) -        tau = DDEModel._tau $ DDEModel._filt par+        -- Empirically chosen integration time step:+        -- twice faster than the system response time tau         hStep = tau / 2          !(_, !response) = DDE.integHeun2_2D delaySamples hStep (DDEModel.rhs par) (DDE.Input trace) -forwardPass :: NTC -> Matrix Double -> Matrix Double+genReservoir _ = error "Unsupported DDE model"++-- | Nonlinear transformation performed by an NTC network+forwardPass :: NTC  -- ^ NTC network+            -> Matrix Double  -- ^ Input information+            -> Matrix Double forwardPass NTC { _par = Par { _preprocess = prep, _postprocess = post }                 , _inputWeights = iw                 , _reservoir = Reservoir res@@ -114,7 +145,9 @@   let pipeline = post. res. (iw <>). prep   in pipeline sample --- | Offline NTC training+-- TODO: introduce an explicit `learnClassifier` function++-- | NTC training: learn the readout weights offline learn   :: NTC   -> Int@@ -132,10 +165,15 @@       Nothing -> Left "Cannot create a readout matrix"       w -> Right $ ntc { _outputWeights = w } -predict :: NTC-        -> Int+-- | Run prediction using a "clean" (uninitialized) reservoir and then+-- forget the reservoir's state.+-- This can be used for both forecasting and classification tasks.+predict :: NTC  -- ^ Trained network+        -> Int  -- ^ Washout (forget) points         -> Matrix Double+        -- ^ Input matrix where measurements are columns and features are rows         -> Either String (Matrix Double)+        -- ^ Either error string or predicted output predict ntc@NTC { _outputWeights = ow                 } forgetPts inp =   case ow of