Learning 0.0.2 → 0.0.3

raw patch · 7 files changed

+157/−153 lines, 7 files

Files

• ChangeLog.md +6/−0
• Learning.cabal +7/−9
• README.md +5/−5
• app/MainPCA.lhs +0/−62
• app/MainPCA2.lhs +0/−77
• examples/PCA/Main.lhs +62/−0
• examples/PCA2/Main.lhs +77/−0

@@ -1,5 +1,11 @@ # Changelog for Learning +## 0.0.2 *February 15th 2018*+  * Implement confusion matrices+  * Write PCA tutorials+  * Add `pcaVariance` function parametrizing variance to retain+  * Add `accuracy` function for classification evaluation+ ## 0.0.1 *February 9th 2018*   * Define core data structures   * Provide linear classifiers and regressors for the supervised learning

@@ -2,17 +2,17 @@ -- -- see: https://github.com/sol/hpack ----- hash: 6c4606a45d47a42344ba884c903f3f574904cf74f1bf8ba2b085dc33882900dd+-- hash: 17f24c6acefbeb01d1dbaf720f248d2ff7b3d82ff28f368f58a60e25cdc2a4d3  name:           Learning-version:        0.0.2+version:        0.0.3 synopsis:       The most frequently used machine learning tools description:    Please see the README on Github at <https://github.com/masterdezign/Learning#readme> category:       Machine Learning homepage:       https://github.com/masterdezign/Learning#readme bug-reports:    https://github.com/masterdezign/Learning/issues author:         Bogdan Penkovsky-maintainer:     dev at penkovsky [dot] com+maintainer:     dev () penkovsky dot com copyright:      Bogdan Penkovsky license:        BSD3 license-file:   LICENSE@@ -42,9 +42,9 @@   default-language: Haskell2010  executable learning-pca-  main-is: MainPCA.lhs+  main-is: Main.lhs   hs-source-dirs:-      app+      examples/PCA   ghc-options: -threaded -rtsopts -with-rtsopts=-N   build-depends:       Learning@@ -53,14 +53,13 @@     , hmatrix >=0.18.0.0     , vector   other-modules:-      MainPCA2       Paths_Learning   default-language: Haskell2010  executable learning-pca-advanced-  main-is: MainPCA2.lhs+  main-is: Main.lhs   hs-source-dirs:-      app+      examples/PCA2   ghc-options: -threaded -rtsopts -with-rtsopts=-N   build-depends:       Learning@@ -69,7 +68,6 @@     , hmatrix >=0.18.0.0     , vector   other-modules:-      MainPCA       Paths_Learning   default-language: Haskell2010 

@@ -5,7 +5,7 @@ The name of the package can be interpreted in two ways:  1. Either as "Learning" in "Machine Learning".-2. Or "Learning" meaning that [examples](https://github.com/masterdezign/Learning/tree/master/app)+2. Or "Learning" meaning that [examples](https://github.com/masterdezign/Learning/tree/master/examples) are written in [literate style](https://en.wikipedia.org/wiki/Literate_programming) and can be used to discover machine learning techniques. @@ -21,20 +21,20 @@  ## Getting Started -Use [Stack](http://haskellstack.org).+Use [Stack](http://haskellstack.org)       $ git clone https://github.com/masterdezign/Learning.git && cd Learning      $ stack build --install-ghc  ### Demo 1: principal components analysis (PCA) -Launch the [PCA demo](https://github.com/masterdezign/Learning/blob/master/app/MainPCA.lhs)+Launch the [PCA demo](https://github.com/masterdezign/Learning/blob/master/examples/PCA/Main.lhs)       $ stack exec learning-pca -### Demo 2: advanced principal components analysis (PCA)+### Demo 2: advanced PCA -Launch the advanced [PCA demo](https://github.com/masterdezign/Learning/blob/master/app/MainPCA2.lhs)+Launch the advanced [PCA demo](https://github.com/masterdezign/Learning/blob/master/examples/PCA2/Main.lhs)       $ stack exec learning-pca-advanced 

@@ -1,62 +0,0 @@-Principal Components Analysis (PCA) demo--------------------------------------------The tutorial is based on http://setosa.io/ev/principal-component-analysis/--Suppose, we study nutrition habits of the citizens of four countries.-Here, we provide the food consumption data among those countries.--> import           Learning-> import qualified Numeric.LinearAlgebra as LA--> england = [375, 57, 245, 1472, 105, 54, 193, 147, 1102,->            720, 253, 685, 488, 198, 360, 1374, 156]--> northernIreland = [135, 47, 267, 1494, 66, 41, 209, 93, 674,->                    1033, 143, 586, 355, 187, 334, 1506, 139]--> scotland = [458, 53, 242, 1462, 103, 62, 184, 122, 957,->             566, 171, 750, 418, 220, 337, 1572, 147]--> wales = [475, 73, 227, 1582, 103, 64, 235, 160, 1137,->          874, 265, 803, 570, 203, 365, 1256, 175]--We want to know how differ the countries based on those data.-For that purpose, we would like to reduce the redundant information-or, in other words, perform PCA.--We create a single list of feature vectors (each country) used later-for the analysis.--> countries = map LA.fromList [england, northernIreland, scotland, wales]--We perform PCA, i.e. calculate the compression (dimensionality reduction)-function `compress`. The `pca` function is given-`principalComponents` parameter. Here it's 1, that means that-`countries` vectors of 17 features will be reduced into scalars (1D vectors).--> compress = let principalComponents = 1->                pca1 = pca principalComponents countries->            in _compress pca1--Output the resulting scalar values for each country--> main = mapM_ (print. compress) countries--Here is a summary:--     England            -702.9850482521952-     Northern Ireland   -80.6002572540017-     Scotland           -649.8612350689822-     Wales              -798.521043705295--Wales- \/  England                       Northern Ireland-      \/                                   \/-..o....o..o.................................o-          /\-        Scotland--Now, we can clearly see that there exists a difference between-Northern Ireland and the rest of the countries.

@@ -1,77 +0,0 @@-Advanced Principal Components Analysis (PCA) demo-----------------------------------------------------The tutorial is a continuation of ./MainPCA.lhs--Previously, we were able to quickly determine that the food ration-in Northern Ireland is somewhat different comparing to the other three-countries. We have projected data from 17 dimensions into-one dimension. However, we could also make a projection into-two or more dimensions. So how do we determine which dimensionality-reduction does preserve the most of the information? How do we make-sure that only redundant information was removed?--For that purpose, we will calculate retained variance [1]-depending on the number of the principal components.--[1] http://www.dsc.ufcg.edu.br/~hmg/disciplinas/posgraduacao/rn-copin-2014.3/material/SignalProcPCA.pdf--Let's start with the imports and data definition.--> import           Learning ( pca' )-> import qualified Numeric.LinearAlgebra as LA-> import           Data.List ( scanl' )-> import           Text.Printf ( printf )--> england = [375, 57, 245, 1472, 105, 54, 193, 147, 1102,->            720, 253, 685, 488, 198, 360, 1374, 156]--> northernIreland = [135, 47, 267, 1494, 66, 41, 209, 93, 674,->                    1033, 143, 586, 355, 187, 334, 1506, 139]--> scotland = [458, 53, 242, 1462, 103, 62, 184, 122, 957,->             566, 171, 750, 418, 220, 337, 1572, 147]--> wales = [475, 73, 227, 1582, 103, 64, 235, 160, 1137,->          874, 265, 803, 570, 203, 365, 1256, 175]--> countries = map LA.fromList [england, northernIreland, scotland, wales]--In order to compute the eigenvectors u and eigenvalues eig of-a covariance matrix, we use function pca'. In fact, pca' was-already called under the hood in the previous tutorial.--> (u, eig) = pca' countries--Sums of the first N eigenvalues:--> cumul = drop 1 $ scanl' (+) 0 $ LA.toList eig--Sum of all eigenvalues:--> total = last cumul--> main = mapM_ (\(i, s) ->->                 let retained = s / total * 100 :: Double->                     msg = "%d principal component(s): Retained variance %.1f%%"->                 in putStrLn $ printf msg i retained)->              $ zip [1::Int ..] cumul--     1 principal component(s): Retained variance 67.4%-     2 principal component(s): Retained variance 96.5%-     3 principal component(s): Retained variance 100.0%-     4 principal component(s): Retained variance 100.0%--     ...--     17 principal component(s): Retained variance 100.0%--From this data we can conclude that the first two principal components-contain 96.5% of information. Therefore, we will loose 3.5% of information-after projecting into two orthogonal axes in the transformed coordinate system-obtained after PCA.--Hint: to compute compression (dimensionality reduction) and-decompression functions for specified variance to retain, use-(_compress. pcaVariance) and (_decompress. pcaVariance) functions.

@@ -0,0 +1,62 @@+Principal Components Analysis (PCA) demo+----------------------------------------+++The tutorial is based on http://setosa.io/ev/principal-component-analysis/++Suppose, we study nutrition habits of the citizens of four countries.+Here, we provide the food consumption data among those countries.++> import           Learning+> import qualified Numeric.LinearAlgebra as LA++> england = [375, 57, 245, 1472, 105, 54, 193, 147, 1102,+>            720, 253, 685, 488, 198, 360, 1374, 156]++> northernIreland = [135, 47, 267, 1494, 66, 41, 209, 93, 674,+>                    1033, 143, 586, 355, 187, 334, 1506, 139]++> scotland = [458, 53, 242, 1462, 103, 62, 184, 122, 957,+>             566, 171, 750, 418, 220, 337, 1572, 147]++> wales = [475, 73, 227, 1582, 103, 64, 235, 160, 1137,+>          874, 265, 803, 570, 203, 365, 1256, 175]++We want to know how differ the countries based on those data.+For that purpose, we would like to reduce the redundant information+or, in other words, perform PCA.++We create a single list of feature vectors (each country) used later+for the analysis.++> countries = map LA.fromList [england, northernIreland, scotland, wales]++We perform PCA, i.e. calculate the compression (dimensionality reduction)+function `compress`. The `pca` function is given+`principalComponents` parameter. Here it's 1, that means that+`countries` vectors of 17 features will be reduced into scalars (1D vectors).++> compress = let principalComponents = 1+>                pca1 = pca principalComponents countries+>            in _compress pca1++Output the resulting scalar values for each country++> main = mapM_ (print. compress) countries++Here is a summary:++     England            -702.9850482521952+     Northern Ireland   -80.6002572540017+     Scotland           -649.8612350689822+     Wales              -798.521043705295++Wales+ \/  England                       Northern Ireland+      \/                                   \/+..o....o..o.................................o+          /\+        Scotland++Now, we can clearly see that there exists a difference between+Northern Ireland and the rest of the countries.

@@ -0,0 +1,77 @@+Advanced Principal Components Analysis (PCA) demo+-------------------------------------------------+++The tutorial is a continuation of examples/PCA/Main.lhs++Previously, we were able to quickly determine that the food ration+in Northern Ireland is somewhat different comparing to the other three+countries. We have projected data from 17 dimensions into+one dimension. However, we could also make a projection into+two or more dimensions. So how do we determine which dimensionality+reduction does preserve the most of the information? How do we make+sure that only redundant information was removed?++For that purpose, we will calculate retained variance [1]+depending on the number of the principal components.++[1] http://www.dsc.ufcg.edu.br/~hmg/disciplinas/posgraduacao/rn-copin-2014.3/material/SignalProcPCA.pdf++Let's start with the imports and data definition.++> import           Learning ( pca' )+> import qualified Numeric.LinearAlgebra as LA+> import           Data.List ( scanl' )+> import           Text.Printf ( printf )++> england = [375, 57, 245, 1472, 105, 54, 193, 147, 1102,+>            720, 253, 685, 488, 198, 360, 1374, 156]++> northernIreland = [135, 47, 267, 1494, 66, 41, 209, 93, 674,+>                    1033, 143, 586, 355, 187, 334, 1506, 139]++> scotland = [458, 53, 242, 1462, 103, 62, 184, 122, 957,+>             566, 171, 750, 418, 220, 337, 1572, 147]++> wales = [475, 73, 227, 1582, 103, 64, 235, 160, 1137,+>          874, 265, 803, 570, 203, 365, 1256, 175]++> countries = map LA.fromList [england, northernIreland, scotland, wales]++In order to compute the eigenvectors u and eigenvalues eig of+a covariance matrix, we use function pca'. In fact, pca' was+already called under the hood in the previous tutorial.++> (u, eig) = pca' countries++Sums of the first N eigenvalues:++> cumul = drop 1 $ scanl' (+) 0 $ LA.toList eig++Sum of all eigenvalues:++> total = last cumul++> main = mapM_ (\(i, s) ->+>                 let retained = s / total * 100 :: Double+>                     msg = "%d principal component(s): Retained variance %.1f%%"+>                 in putStrLn $ printf msg i retained)+>              $ zip [1::Int ..] cumul++     1 principal component(s): Retained variance 67.4%+     2 principal component(s): Retained variance 96.5%+     3 principal component(s): Retained variance 100.0%+     4 principal component(s): Retained variance 100.0%++     ...++     17 principal component(s): Retained variance 100.0%++From this data we can conclude that the first two principal components+contain 96.5% of information. Therefore, we will loose 3.5% of information+after projecting into two orthogonal axes in the transformed coordinate system+obtained after PCA.++Hint: to compute compression (dimensionality reduction) and+decompression functions for specified variance to retain, use+(_compress. pcaVariance) and (_decompress. pcaVariance) functions.