lazyppl 1.0.1 → 1.1.0

raw patch · 34 files changed

+18955/−371 lines, 34 filesdep +QuickCheckdep +bytestringdep +deepseqdep −hmatrixdep ~basedep ~containersdep ~log-domainnew-uploaderPVP ok

version bump matches the API change (PVP)

Dependencies added: QuickCheck, bytestring, deepseq, lazyppl, typed-process, vector

Dependencies removed: hmatrix

Dependency ranges changed: base, containers, log-domain, math-functions, mtl, random, transformers

API changes (from Hackage documentation)

Files

• ChangeLog.md +54/−0
• README.md +74/−0
• lazyppl.cabal +192/−15
• src/LazyPPL.hs +124/−135
• src/LazyPPL/Distributions.hs +415/−78
• src/LazyPPL/Distributions/Counter.hs +9/−5
• src/LazyPPL/Distributions/DirichletP.hs +26/−20
• src/LazyPPL/Distributions/GP.hs +68/−63
• src/LazyPPL/Distributions/GP/Pure.hs +69/−0
• src/LazyPPL/Distributions/IBP.hs +44/−22
• src/LazyPPL/Distributions/Memoization.hs +37/−33
• src/LazyPPL/Distributions/Mondrian.hs +149/−0
• src/LazyPPL/Internal/History.hs +89/−0
• src/LazyPPL/Internal/Sampler.hs +32/−0
• test/Api.hs +91/−0
• test/Api/Distributions.hs +316/−0
• test/Api/Domains.hs +352/−0
• test/Api/Inference.hs +119/−0
• test/Api/Processes.hs +214/−0
• test/Api/Support.hs +105/−0
• test/Chain.hs +108/−0
• test/Concurrency.hs +143/−0
• test/GaussianCondition.hs +237/−0
• test/HiddenState.hs +101/−0
• test/KnownLimit.hs +76/−0
• test/Replay.hs +123/−0
• test/Stream.hs +50/−0
• test/StrictMemo.hs +24/−0
• test/TestLogPdf.hs +45/−0
• test/reference/beta-laws.tsv +4274/−0
• test/reference/beta-quantiles.tsv +7417/−0
• test/reference/dirichlet-sticks.tsv +57/−0
• test/reference/log-densities.tsv +3075/−0
• test/reference/reference.py +646/−0

@@ -1,5 +1,59 @@ # Changelog for lazyppl +## 1.1.0++### Added++- `lazyppl-examples`: separate examples and plotting package+- Playground: interactive browser models+- Example pages: locally served fonts and icons++### Changed++- Build: Cabal and GHC 9.6.7, 9.12.4, 9.14.1+- `gp`, `wiener`: pure Haskell without LAPACK+- `mh`, `mhirreducible`: lower stream memory retention+- `memrec`: `Ord` keys without `Show`+- Integer `memoize`: restricted to `Prob`+- Distributions: validate inputs and parameter domains+- `beta`, `dirichlet`: reject unsupported shape parameters+- Random labels and acceptance draws: open interval (0,1)+- `mh`, `mhirreducible`, `weightedsamples`, `wis`: separate generator per run+- `superuniformbounded`: bounded graph angle sampling++### Fixed++- `wis`: resample from all weighted samples+- Integer `memoize`: return values at their own indices+- `mh`, `mhirreducible`, `wis`: zero and invalid weights+- `mh1`: reject proposals without reverse sites+- `beta`, `dirichlet`: sampling at numerical boundaries+- `beta`, `dirichlet`: reject unsupported custom tree labels+- `categorical`: rounding with finite and infinite streams+- `categorical`: accept finite probabilities with a rounding-sized shortfall+- Dirichlet processes: small-stick accumulation+- `poisson`: sampling at extreme labels+- `ibp`: first customer's dish count+- `gp`: initial and conditional distributions+- `wiener`: variance at extreme times+- `normalLogPdf`, `betaLogPdf`, `dirichletLogPdf`: extreme-input accuracy+- `binomialLogPmf`: large-count accuracy+- `gammaLogPdf`, `poissonLogPmf`: extreme-input accuracy+- Memoization, Gaussian processes and counters: concurrent state+- `uniformbounded`, `uniformBoundedLogPdf`: overflowing bound differences+- Playground: retry failed WebAssembly downloads+- Playground: validate results before plotting+- Mondrian playground: reject overlaps and duplicate identifiers+- GraphDemo: graph probabilities and figure edges+- Example pages: correct maximum-likelihood sample labels+- Example pages: syntax colours in dark mode+- Website generation: preserve prior site on build failure++### Removed++- `LazyPPL.SingleSite`: moved to `lazyppl-singlesite`+- `lazyppl`: `hmatrix`, LAPACK and plotting dependencies+ ## 1.0.1  - Add support for GHC 9.8 and 9.10

@@ -0,0 +1,74 @@+# <img style="height:45px" src="https://user-images.githubusercontent.com/8027127/223598298-21dd4207-612d-4b4e-be9c-4daa2ae2de5b.png" /> LazyPPL++LazyPPL is a Haskell library for Bayesian probabilistic programming, drawing on the fact that lazy data structures are a natural idiom for programming with infinite-dimensional Bayesian methods such as Poisson/Gaussian/Dirichlet processes.++The crucial semantic idea, inspired by developments in synthetic probability theory, is to work with two separate monads: an affine monad of probability, which supports laziness, and a commutative, non-affine monad of measures, which does not. This gives a Metropolis-Hastings implementation that works with lazy programs.++The main LazyPPL modules are on [Hackage](https://hackage.haskell.org/package/lazyppl) and [Stackage](https://www.stackage.org/). Various worked examples in literate Haskell are at [https://lazyppl-team.github.io](https://lazyppl-team.github.io).++Laziness lets us specify non-parametric models cleanly: processes or infinite-dimensional functions only have their finite explored portion materialised. We have first-class++* stochastic memoization: `memoize :: (a -> Prob b) -> Prob (a -> b)`+* Gaussian processes: `wiener :: Prob (Double -> Double)`+* point processes: `poissonPP :: Double -> Double -> Prob [Double]` (in the example demos)+* Dirichlet processes: `dp :: Double -> Prob a -> Prob (Prob a)`+++## Examples++* [`WienerDemo`](https://lazyppl-team.github.io/WienerDemo.html) ([`lhs`](examples/models/WienerDemo.lhs)) is a simple implementation of regression using a Wiener process. Via maintaining a hidden table of previous calls, it appears to be a bona fide random function $R\to R$ constructed lazily. Some values are sampled during simulation, others during plotting.++![Wiener process regression](https://lazyppl-team.github.io/images/wiener-reg.svg)++* [`RegressionDemo`](https://lazyppl-team.github.io/RegressionDemo.html) ([`lhs`](examples/models/RegressionDemo.lhs)) demonstrates piecewise linear regression. The change points are drawn from a lazy Poisson process.++![Poisson-split piecewise linear regression](https://lazyppl-team.github.io/images/regression-piecewise-reg.svg)++* [`Clustering`](https://lazyppl-team.github.io/ClusteringDemo.html) ([`lhs`](examples/models/ClusteringDemo.lhs)) contains some simple clustering examples, where the number of clusters is unknown. Stick-breaking is lazy, and we use stochastic memoization.++![Dirichlet process clustering](https://lazyppl-team.github.io/images/clustering-map.svg)++* [`ProgramInduction`](https://lazyppl-team.github.io/ProgramInductionDemo.html) ([`lhs`](examples/models/ProgramInductionDemo.lhs)) is a simple example of program induction over an arithmetic language. Random expressions are represented as an infinite forest with a finite path through it.++![Program induction](https://lazyppl-team.github.io/images/programinduction-reg.svg)++* Feature extraction via the [Indian Buffet Process](examples/models/AdditiveClustering.hs), and relation inference via the [Mondrian Process](examples/models/MondrianExample.hs) and a Chinese-Restaurant-Process-based [Infinite Relational Model](examples/models/IrmTest.hs).+++## Library++* [`LazyPPL.hs`](src/LazyPPL.hs) provides the core monads and inference. `Prob` is an affine monad of probability measures. `Meas` is a non-affine monad of unnormalised measures with `sample` and `score`. The Metropolis-Hastings algorithm `mh` takes a probability `p` of changing any given site of the rose tree of random seeds. Setting `p=1/num_sites` approximates single-site MH and `p=1` recovers multi-site MH. We can't compute the number of active sites without forcing more of the lazy tree, so we use this `p`-parameterised form. A second algorithm, `mh1` in [`lazyppl-singlesite`](singlesite/), implements a single-site proposal kernel by inspecting the Haskell heap.+* [`LazyPPL/Distributions.hs`](src/LazyPPL/Distributions.hs) and `LazyPPL/Distributions/*` provide common distributions and abstract types capturing exchangeability. For example, `newCustomer :: Restaurant -> Prob Table` for the Chinese Restaurant Process ([`DirichletP.hs`](src/LazyPPL/Distributions/DirichletP.hs)) and `Restaurant -> Prob [Dish]` for the Indian Buffet Process ([`IBP.hs`](src/LazyPPL/Distributions/IBP.hs)).+++## Installation++Build with Cabal: [install GHCup](https://www.haskell.org/ghcup/install/) for GHC 9.14.1 and Cabal 3.18.2.0, then `cabal update` and `cabal build` in the repo root. First build takes a few minutes.++Run a demo: `cabal run -fdemos wiener-exe` (or `regression-exe`, `clustering-exe`, etc.), or explore in the REPL with `cabal repl lib:lazyppl`.++The library is pure Haskell and needs no LAPACK or BLAS installation. The plotting demos use the `matplotlib` Haskell wrapper, which calls into Python: `sudo apt-get install -y --no-install-recommends python3-matplotlib python3-numpy python3-scipy python-is-python3`.++To try lazyppl in your browser without installing anything, click [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/lazyppl-team/lazyppl). This opens a ready-to-use VS Code environment with GHC and Cabal already in place.+++## Writing your own probabilistic models++Under `executables:` in `examples/package.yaml`, add a stanza and run `hpack examples` (0.39.6) to regenerate `examples/lazyppl-examples.cabal`:++```yaml+executables:+  # ...+  mymodel-exe:+    main:                MyModel.hs+    source-dirs:         models+    ghc-options:+    - -threaded+    - -rtsopts+    - -with-rtsopts=-N+    - -main-is MyModel+    dependencies:+    - lazyppl+```++Run with `cabal run mymodel-exe`, or open in the REPL: `cabal repl exe:mymodel-exe`.

@@ -1,15 +1,15 @@ cabal-version: 1.12 --- This file has been generated from package.yaml by hpack version 0.38.1.+-- This file has been generated from package.yaml by hpack version 0.39.6. -- -- see: https://github.com/sol/hpack  name:           lazyppl-version:        1.0.1+version:        1.1.0 synopsis:       Lazy Probabilistic Programming Library description:    LazyPPL is a Haskell library for Bayesian probabilistic programming. It supports lazy use of probability, which is useful for specifying non-parametric models, and we provide new Metropolis-Hastings algorithms to allow this. For illustrations, see <https://lazyppl-team.github.io/>.                 .-                Note: the Gaussian process module (@LazyPPL.Distributions.GP@) uses @hmatrix@, which links against LAPACK. On Ubuntu install with @sudo apt-get install libgsl0-dev liblapack-dev@.+                LazyPPL is pure Haskell and needs no BLAS or LAPACK installation. category:       Statistics stability:      experimental homepage:       https://lazyppl-team.github.io/@@ -21,12 +21,17 @@ license-file:   LICENSE build-type:     Simple tested-with:-    GHC == 8.10.7-  , GHC == 9.6.7-  , GHC == 9.8.4-  , GHC == 9.10.1+    GHC == 9.6.7+  , GHC == 9.12.4+  , GHC == 9.14.1 extra-source-files:+    README.md     ChangeLog.md+    test/reference/beta-laws.tsv+    test/reference/beta-quantiles.tsv+    test/reference/dirichlet-sticks.tsv+    test/reference/log-densities.tsv+    test/reference/reference.py  source-repository head   type: git@@ -41,15 +46,187 @@       LazyPPL.Distributions.GP       LazyPPL.Distributions.IBP       LazyPPL.Distributions.Memoization+      LazyPPL.Distributions.Mondrian+      LazyPPL.Internal.History+      LazyPPL.Internal.Sampler+      LazyPPL.Distributions.GP.Pure   hs-source-dirs:       src   build-depends:-      base >=4.7 && <9-    , containers >=0.6.5.1-    , hmatrix >=0.20.2-    , log-domain >=0.13.2-    , math-functions >=0.3.4.2-    , mtl >=2.2.2-    , random >=1.2.0-    , transformers >=0.5.6.2+      base >=4.18.3.0 && <4.23+    , containers >=0.6.7 && <0.9+    , log-domain >=0.13.2 && <0.14+    , math-functions >=0.3.4.2 && <0.4+    , random >=1.3.1 && <1.4+    , transformers >=0.6.1.0 && <0.7+    , vector >=0.13.2.0 && <0.14+  default-language: Haskell2010++test-suite api-test+  type: exitcode-stdio-1.0+  main-is: Api.hs+  other-modules:+      StrictMemo+      Api.Support+      Api.Domains+      Api.Distributions+      Api.Processes+      Api.Inference+  hs-source-dirs:+      test+  build-depends:+      QuickCheck >=2.19.0.0 && <2.20+    , base >=4.18.3.0 && <4.23+    , bytestring >=0.11.5.4 && <0.13+    , containers >=0.6.7 && <0.9+    , lazyppl+    , log-domain >=0.13.2 && <0.14+    , math-functions >=0.3.4.2 && <0.4+    , random >=1.3.1 && <1.4+    , transformers >=0.6.1.0 && <0.7+    , typed-process >=0.2.13.0 && <0.3+    , vector >=0.13.2.0 && <0.14+  default-language: Haskell2010++test-suite chain-test+  type: exitcode-stdio-1.0+  main-is: Chain.hs+  hs-source-dirs:+      test+  ghc-options: -threaded -rtsopts+  build-depends:+      QuickCheck >=2.14.2 && <2.20+    , base >=4.18.3.0 && <4.23+    , containers >=0.6.7 && <0.9+    , deepseq >=1.4.8.1 && <1.6+    , lazyppl+    , log-domain >=0.13.2 && <0.14+    , math-functions >=0.3.4.2 && <0.4+    , mtl >=2.3.1 && <2.4+    , random >=1.3.1 && <1.4+    , transformers >=0.6.1.0 && <0.7+    , vector >=0.13.2.0 && <0.14+  default-language: Haskell2010++test-suite concurrency-test+  type: exitcode-stdio-1.0+  main-is: Concurrency.hs+  hs-source-dirs:+      test+  ghc-options: -threaded -rtsopts -with-rtsopts=-N2+  build-depends:+      base >=4.18.3.0 && <4.23+    , containers >=0.6.7 && <0.9+    , lazyppl+    , log-domain >=0.13.2 && <0.14+    , math-functions >=0.3.4.2 && <0.4+    , random >=1.3.1 && <1.4+    , transformers >=0.6.1.0 && <0.7+    , vector >=0.13.2.0 && <0.14+  default-language: Haskell2010++test-suite gp-test+  type: exitcode-stdio-1.0+  main-is: GaussianCondition.hs+  hs-source-dirs:+      test+  build-depends:+      QuickCheck >=2.14.2 && <2.20+    , base >=4.18.3.0 && <4.23+    , containers >=0.6.7 && <0.9+    , lazyppl+    , log-domain >=0.13.2 && <0.14+    , math-functions >=0.3.4.2 && <0.4+    , random >=1.3.1 && <1.4+    , transformers >=0.6.1.0 && <0.7+    , vector >=0.13.2.0 && <0.14+  default-language: Haskell2010++test-suite hidden-state-o2-test+  type: exitcode-stdio-1.0+  main-is: HiddenState.hs+  other-modules:+      KnownLimit+  hs-source-dirs:+      test+  ghc-options: -O2 -threaded -rtsopts -with-rtsopts=-N2+  build-depends:+      base >=4.18.3.0 && <4.23+    , containers >=0.6.7 && <0.9+    , lazyppl+    , log-domain >=0.13.2 && <0.14+    , math-functions >=0.3.4.2 && <0.4+    , random >=1.3.1 && <1.4+    , transformers >=0.6.1.0 && <0.7+    , vector >=0.13.2.0 && <0.14+  default-language: Haskell2010++test-suite hidden-state-test+  type: exitcode-stdio-1.0+  main-is: HiddenState.hs+  other-modules:+      KnownLimit+  hs-source-dirs:+      test+  ghc-options: -threaded -rtsopts -with-rtsopts=-N2+  build-depends:+      base >=4.18.3.0 && <4.23+    , containers >=0.6.7 && <0.9+    , lazyppl+    , log-domain >=0.13.2 && <0.14+    , math-functions >=0.3.4.2 && <0.4+    , random >=1.3.1 && <1.4+    , transformers >=0.6.1.0 && <0.7+    , vector >=0.13.2.0 && <0.14+  default-language: Haskell2010++test-suite logpdf-test+  type: exitcode-stdio-1.0+  main-is: TestLogPdf.hs+  hs-source-dirs:+      test+  build-depends:+      base >=4.18.3.0 && <4.23+    , containers >=0.6.7 && <0.9+    , lazyppl+    , log-domain >=0.13.2 && <0.14+    , math-functions >=0.3.4.2 && <0.4+    , random >=1.3.1 && <1.4+    , transformers >=0.6.1.0 && <0.7+    , vector >=0.13.2.0 && <0.14+  default-language: Haskell2010++test-suite replay-test+  type: exitcode-stdio-1.0+  main-is: Replay.hs+  hs-source-dirs:+      test+  ghc-options: -threaded -rtsopts -with-rtsopts=-N2+  build-depends:+      QuickCheck >=2.14.2 && <2.20+    , base >=4.18.3.0 && <4.23+    , containers >=0.6.7 && <0.9+    , lazyppl+    , log-domain >=0.13.2 && <0.14+    , math-functions >=0.3.4.2 && <0.4+    , random >=1.3.1 && <1.4+    , transformers >=0.6.1.0 && <0.7+    , vector >=0.13.2.0 && <0.14+  default-language: Haskell2010++test-suite stream-test+  type: exitcode-stdio-1.0+  main-is: Stream.hs+  hs-source-dirs:+      test+  ghc-options: -threaded -rtsopts -with-rtsopts=-M256M+  build-depends:+      base >=4.18.3.0 && <4.23+    , containers >=0.6.7 && <0.9+    , lazyppl+    , log-domain >=0.13.2 && <0.14+    , math-functions >=0.3.4.2 && <0.4+    , random >=1.3.1 && <1.4+    , transformers >=0.6.1.0 && <0.7+    , vector >=0.13.2.0 && <0.14   default-language: Haskell2010

@@ -1,4 +1,4 @@-{-# LANGUAGE GeneralizedNewtypeDeriving, ScopedTypeVariables #-}+{-# LANGUAGE GeneralizedNewtypeDeriving, ScopedTypeVariables, CPP #-} {-# OPTIONS_GHC -Wno-unrecognised-pragmas #-} {-# HLINT ignore "Use second" #-} @@ -23,7 +23,7 @@          c. 'wis' (simple reference weighted importance sampling) -        See also the SingleSite module for a separate single-site Metropolis-Hastings algorithm via GHC.Exts.Heap and System.IO.Unsafe.+        See also "LazyPPL.SingleSite" in @lazyppl-singlesite@ for a separate single-site Metropolis-Hastings algorithm via "GHC.Exts.Heap" and "System.IO.Unsafe".      3. Various useful helpful functions. @@ -35,7 +35,7 @@      Most of the structure here will not be needed in typical models. We expose more of the structure for more experimental uses.  -The `Distributions` module provides many useful distributions, and further non-parametric distributions are in `Distributions.DirichletP`, `Distributions.GP`, `Distr.IBP`, and `Distr.Memoization`. +The "LazyPPL.Distributions" module provides many useful distributions, and further non-parametric distributions are in "LazyPPL.Distributions.DirichletP", "LazyPPL.Distributions.GP", "LazyPPL.Distributions.IBP", and "LazyPPL.Distributions.Memoization".    -}@@ -43,7 +43,7 @@ module LazyPPL     ( -- * Rose tree type       ---      -- | Our source of randomness will be an infinitely wide and deep lazy [rose tree](https://en.wikipedia.org/wiki/Rose_tree), regarded as initialized with uniform [0,1] choices for each label.+      -- | Our source of randomness will be an infinitely wide and deep lazy [rose tree](https://en.wikipedia.org/wiki/Rose_tree), regarded as initialized with uniform choices from the open interval (0,1) for each label.       Tree(Tree),       -- * Monads       Prob(Prob), Meas(Meas),@@ -63,7 +63,9 @@ import Data.Monoid import System.Random hiding (uniform) import Control.Monad-import Control.Monad.State.Lazy (State, state , put, get, runState)+import System.IO.Unsafe (unsafeInterleaveIO)+import LazyPPL.Internal.History (History, newHistory, appendMutation, labelAt)+import LazyPPL.Internal.Sampler (randomLabel, checkedLogWeight) import Numeric.Log  @@ -103,7 +105,7 @@ newtype Meas a = Meas (WriterT (Product (Log Double)) Prob a)   deriving(Functor, Applicative, Monad) -{- | A uniform sample is a building block for probability distributions.+{- | A uniform sample from the open interval (0,1) is a building block for probability distributions. It is never 0 or 1, so an inverse transform such as @-log u@ stays finite.  This is implemented by getting the label at the head of the tree and discarding the rest.-} uniform :: Prob Double@@ -113,28 +115,48 @@ sample :: Prob a -> Meas a sample p = Meas $ lift p -{- | A one point measure with a given score (or weight, or mass, or likelihood), which should be a positive real number.+{- | A one point measure with a given score (or weight, or mass, or likelihood), which should be a non-negative real number. -A score of 0 describes impossibility. To avoid numeric issues, we encode it as @ exp(-300) @ instead.-}+A score of 0 describes impossibility. Negative and non-finite scores raise an error.-} score :: Double -> Meas ()-score r = Meas $ tell $ Product $ (Exp . log) (if r==0 then exp(-300) else r)+score r = scoreLog (Exp (log r)) +{- | 'score' with the weight given by its natural logarithm, so the weight may exceed the range of 'Double'. A logarithm of negative infinity is weight 0; a NaN or positive-infinite logarithm raises an error. -} scoreLog :: Log Double -> Meas ()-scoreLog r = Meas $ tell $ Product r+scoreLog r = scoreProductLog (Product r) +-- | 'scoreLog' with the weight wrapped in 'Product', the form in which 'Meas' accumulates weights. scoreProductLog :: Product (Log Double) -> Meas ()-scoreProductLog r = Meas $ tell r+scoreProductLog (Product r) = checkedLogWeight "score" r `seq` Meas (tell (Product r))  -{- | Generate a tree with uniform random labels.+{- | Generate a tree with labels drawn uniformly from the open interval (0,1).      This uses 'split' to split a random seed. -} randomTree :: RandomGen g => g -> Tree-randomTree g = let (a,g') = random g in Tree a (randomTrees g')+randomTree g = let (a,g') = randomLabel g in Tree a (randomTrees g') randomTrees :: RandomGen g => g -> [Tree] randomTrees g = let (g1,g2) = split g in randomTree g1 : randomTrees g2 -{- | 'runProb' runs a probability deterministically, given a source of randomness. -}+{- | 'runProb' runs a probability deterministically, given a source of randomness.++Some results of 'runProb' hold hidden state: a counter of "LazyPPL.Distributions.Counter", a+restaurant of 'LazyPPL.Distributions.IBP.newRestaurant', a function of+'LazyPPL.Distributions.Memoization.generalmemoize' or 'LazyPPL.Distributions.Memoization.memrec',+and the random functions of 'LazyPPL.Distributions.GP.wiener' and 'LazyPPL.Distributions.GP.gp'.+Inside a model, bind such a value with @>>=@ and use it there. Outside a model, evaluate the value+itself once, with 'Control.Exception.evaluate', before an IO action that runs more than once (the+body of a loop or of a thread) uses it:++> f <- evaluate (runProb wiener tree)+> ys <- mapM (\x -> evaluate (f x)) xs++Bound with @let@ instead, or returned inside a function from a 'Prob' computation, such a value can+lose its state when an IO action that runs more than once uses it: GHC may copy the binding into+the action, so that every run of the action starts from new state, and a counter then returns the+same number to every read. Evaluating a pair or a list does not evaluate the values inside it, so+evaluate each value with hidden state. @cabal run lazyppl:test:hidden-state-test -- known-limit@+runs these unsupported uses. -} runProb :: Prob a -> Tree -> a runProb (Prob a) = a @@ -149,29 +171,28 @@           (x, w) <- runWriterT m           rest <- helper           return $ (x, w) : rest-    newStdGen-    g <- getStdGen+    g <- newStdGen     let rs = randomTree g     let xws = runProb helper rs-    return $ map (\(x,w) -> (x, getProduct w)) xws+    return $ map (\(x,w) -> (x, checkedLogWeight "weightedsamples" (getProduct w))) xws  {- | Weighted importance sampling first draws n weighted samples,     and then samples a stream of results from that, regarded as an empirical distribution. Sometimes called "likelihood weighted importance sampling".  -This is a reference implementation. It will not usually be very efficient at all, but may be useful for debugging. +This is a reference implementation. It will not usually be very efficient at all, but may be useful for debugging. Raises an error if all @n@ samples have weight 0.  -}-wis :: Int -- ^ @n@, the number of samples to base on+wis :: Int -- ^ @n@, the number of samples to base on, at least 1     -> Meas a -- ^ @m@, the measure to normalize     -> IO [a] -- ^ Returns a stream of samples wis n m = do   xws <- weightedsamples m   let xws' = take n $ accumulate xws 0   let max = snd $ last xws'-  newStdGen-  g <- getStdGen+  g <- newStdGen   let rs = (randoms g :: [Double])-  return $ map (\r -> fst $ head $ filter (\(x, w) -> w >= Exp (log r) * max) xws') rs-  where accumulate ((x, w) : xws) a = (x, w + a) : (x, w + a) : accumulate xws (w + a)+  if max == 0 then error "wis: every sample has weight 0"+    else return $ map (\r -> fst $ head $ filter (\(x, w) -> w > 0 && w >= Exp (log r) * max) xws') rs+  where accumulate ((x, w) : xws) a = (x, w + a) : accumulate xws (w + a)         accumulate [] a = []  {- | Produce a stream of samples, using Metropolis Hastings simulation.@@ -191,86 +212,98 @@      __Tip:__ if @m :: Prob a@ then use @fmap (map fst) (mh 1 $ sample m)@ to get a stream of samples from a probability distribution without conditioning.  --}-{-- The algorithm is as follows: -    Top level: produces a stream of samples.-    * Split the random number generator in two-    * One part is used as the first seed for the simulation,-    * and one part is used for the randomness in the MH algorithm.--    Then, run 'step' over and over to get a stream of '(tree, result, weight)'s.-    The stream of seeds is used to produce a stream of result/weight pairs.--    NB There are three kinds of randomness in the step function.-    1. The start tree 't', which is the source of randomness for simulating the-    program m to start with. This is sort-of the point in the "state space".-    2. The randomness needed to propose a new tree ('g1')-    3. The randomness needed to decide whether to accept or reject that ('g2')-    The tree t is an argument and result,-    but we use a state monad ('get'/'put') to deal with the other randomness '(g, g1, g2)'--    Steps of the 'step' function:-    1. Randomly change some sites, -    2. Rerun the model with the new tree, to get a new weight 'w''.-    3. Compute the MH acceptance ratio. This is the probability of either returning the new seed or the old one.-    4. Accept or reject the new sample based on the MH ratio.  ---} mh :: forall a.-   Double -- ^ The chance @p@ of changing any site+   Double -- ^ The chance @p@ of changing any site, in [0,1]    -> Meas a -- ^ The unnormalized measure to sample from    -> IO [(a,Product (Log Double))] -- ^ Returns a stream of (result,weight) pairs-mh p (Meas m) = do-    -- Top level: produce a stream of samples.-    -- Split the random number generator in two-    -- One part is used as the first seed for the simulation,-    -- and one part is used for the randomness in the MH algorithm.-    newStdGen-    g <- getStdGen-    let (g1,g2) = split g-    let t = randomTree g1-    let (x, w) = runProb (runWriterT m) t-    -- Now run step over and over to get a stream of (tree,result,weight)s.-    let (samples,_) = runState (iterateM step (t,x,w)) g2-    -- The stream of seeds is used to produce a stream of result/weight pairs.-    return $ map (\(_,x,w) -> (x,w)) samples-    {- NB There are three kinds of randomness in the step function.-    1. The start tree 't', which is the source of randomness for simulating the-    program m to start with. This is sort-of the point in the "state space".-    2. The randomness needed to propose a new tree ('g1')-    3. The randomness needed to decide whether to accept or reject that ('g2')-    The tree t is an argument and result,-    but we use a state monad ('get'/'put') to deal with the other randomness '(g,g1,g2)' -}-    where step :: RandomGen g => (Tree,a,Product (Log Double)) -> State g (Tree,a,Product (Log Double))-          step (t, x, w) = do-            -- Randomly change some sites-            g <- get-            let (g1, g2) = split g-            let t' = mutateTree p g1 t-            -- Rerun the model with the new tree, to get a new-            -- weight w'.-            let (x', w') = runProb (runWriterT m) t'-            -- MH acceptance ratio. This is the probability of either-            -- returning the new seed or the old one.-            let ratio = getProduct w' / getProduct w-            let (r, g2') = random g2-            put g2'-            if r < min 1 (exp $ ln ratio) -- (trace ("-- Ratio: " ++ show ratio) ratio))-              then return (t', x', w') -- trace ("---- Weight: " ++ show w') w')-              else return (t, x, w) -- trace ("---- Weight: " ++ show w) w)+mh p = metropolis p Nothing +-- A label that the model never reads stays unevaluated, and its delayed+-- mutations keep the earlier trees alive. So every 32 accepted mutateTree+-- proposals we swap the tree for a checkpoint with the same labels, which+-- labelAt computes from the stored seeds. The seed store in+-- LazyPPL.Internal.History appends one 16-byte record per accepted mutateTree+-- proposal, and its vector doubles when full. An accepted all-sites change+-- starts a new store.+metropolis :: Double -> Maybe Double -> Meas a -> IO [(a, Product (Log Double))]+metropolis p refresh (Meas m)+  -- The comparisons are false for NaN, so NaN is rejected too.+  | not (p >= 0 && p <= 1) = error ("mh: the chance p of changing a site must be in [0,1], got " ++ show p)+  | Just q <- refresh, not (q > 0 && q <= 1) =+      error ("mhirreducible: the chance q of an all-sites change must be in (0,1], got " ++ show q)+  | otherwise = do+    g <- newStdGen+    let (g1, g2) = split g+        t = randomTree g1+        (x, w) = checkedSample (runProb (runWriterT m) t)+    history <- newHistory g1+    let go generator tree value weight history count = unsafeInterleaveIO $ do+          let (proposalSeed, acceptanceSeed) = split generator+              (fresh, seed) = case refresh of+                Nothing -> (False, proposalSeed)+                Just q -> let (r, next) = randomLabel proposalSeed+                          in (r < q, next)+              proposed = if fresh then randomTree seed else mutateTree p seed tree+              (value', weight') = checkedSample (runProb (runWriterT m) proposed)+              ratio = getProduct weight' / getProduct weight+              (r, nextGenerator) = randomLabel acceptanceSeed+          -- At weight 0 the ratio can be 0/0 = NaN and the test then fails,+          -- so we accept every move until the weight is positive. r is a label, in+          -- (0,1), as for the tree: random's 1 would reject a move of ratio 1 and+          -- an all-sites change at q = 1.+          if getProduct weight == 0 || r < min 1 (exp (ln ratio))+            then do+              (history', count') <- if fresh+                then do h <- newHistory seed; pure (h, 0)+                else do appendMutation history count seed; pure (history, count+1)+              let tree' = if count' > 0 && count' `rem` 32 == 0+                            then checkpoint p history' count'+                            else proposed+              tree' `seq` pure ()+              rest <- go nextGenerator tree' value' weight' history' count'+              pure ((value', weight') : rest)+            else do+              rest <- go nextGenerator tree value weight history count+              pure ((value, weight) : rest)+    go g2 t x w history 0+  where+    -- A product of valid weights can still overflow to a logarithm of +Infinity.+    checkedSample (v, Product l) = (v, Product (checkedLogWeight "mh" l)) +checkpoint :: Double -> History -> Int -> Tree+checkpoint p history count = node []+  where+    node path = Tree (labelAt p history count path) (children path 0)+    children path index = node (index:path) : children path (index+1)++ -- | Replace the labels of a tree randomly, with probability p mutateTree :: forall g. RandomGen g => Double -> g -> Tree -> Tree mutateTree p g (Tree a ts) =-  let (a',g') = (random g :: (Double,g)) in-  let (a'',g'') = random g' in+  let (a',g') = randomLabel g in+  let (a'',g'') = randomLabel g' in+#ifdef LAZYPPL_BROWSER+  -- Wasm has a 32-bit address space, so we evaluate each visited label to stop+  -- its thunk from keeping the old trees alive.+  let label = if a'<p then a'' else a in+  label `seq` Tree label (mutateTrees p g'' ts)+#else   Tree (if a'<p then a'' else a) (mutateTrees p g'' ts)+#endif mutateTrees :: RandomGen g => Double -> g -> [Tree] -> [Tree]+#ifdef LAZYPPL_BROWSER+mutateTrees p g (t:ts) =+  let (g1,g2) = split g+      t' = mutateTree p g1 t+  in t' `seq` (t' : mutateTrees p g2 ts)+#else mutateTrees p g (t:ts) = let (g1,g2) = split g in mutateTree p g1 t : mutateTrees p g2 ts+#endif   -{- | Irreducible form of 'mh'. Takes @p@ like 'mh', but also @q@, which is the chance of proposing an all-sites change. Irreducibility means that, asymptotically, the sequence of samples will converge in distribution to the renormalized version of @m@. +{- | Irreducible form of 'mh'. Takes @p@ like 'mh', but also @q@ in (0,1], which is the chance of proposing an all-sites change. Irreducibility means that, asymptotically, the sequence of samples will converge in distribution to the renormalized version of @m@, provided that the total weight of @m@ is positive and finite and that computing each weight terminates. The parts of a sample that are never demanded may be infinite and stay unevaluated. With @q = 0@ there would be no all-sites change, and so no such guarantee; @q = 0@ raises an error.   The kernel in `mh` is not formally irreducible in the usual sense, although it is an open question whether this is a problem for asymptotic convergence in any definable model. In any case, convergence is only asymptotic, and so it can be helpful to use `mhirreducible` is that in some situations. @@ -279,49 +312,11 @@  -}  mhirreducible :: forall a.-              Double -- ^ The chance @p@ of changing any given site-              -> Double -- ^ The chance @q@ of doing an all-sites change+              Double -- ^ The chance @p@ of changing any given site, in [0,1]+              -> Double -- ^ The chance @q@ of doing an all-sites change, in (0,1]               -> Meas a -- ^ The unnormalized measure @m@ to sample from               -> IO [(a,Product (Log Double))] -- ^ Returns a stream of (result,weight) pairs-mhirreducible p q (Meas m) = do-    -- Top level: produce a stream of samples.-    -- Split the random number generator in two-    -- One part is used as the first seed for the simulation,-    -- and one part is used for the randomness in the MH algorithm.-    newStdGen-    g <- getStdGen-    let (g1,g2) = split g-    let t = randomTree g1-    let (x, w) = runProb (runWriterT m) t-    -- Now run step over and over to get a stream of (tree,result,weight)s.-    let (samples,_) = runState (iterateM step (t,x,w)) g2-    -- The stream of seeds is used to produce a stream of result/weight pairs.-    return $ map (\(t,x,w) -> (x,w)) samples-    {- NB There are three kinds of randomness in the step function.-    1. The start tree 't', which is the source of randomness for simulating the-    program m to start with. This is sort-of the point in the "state space".-    2. The randomness needed to propose a new tree ('g1')-    3. The randomness needed to decide whether to accept or reject that ('g2')-    The tree t is an argument and result,-    but we use a state monad ('get'/'put') to deal with the other randomness '(g,g1,g2)' -}-    where step :: RandomGen g => (Tree,a,Product (Log Double)) -> State g (Tree,a,Product (Log Double))-          step (t,x,w) = do-            -- Randomly change some sites-            g <- get-            let (g1,g2) = split g-            -- Decide whether to resample all sites (r<q) or just some of them-            let (r,g1') = random g1-            let t' = if r<q then randomTree g1' else mutateTree p g1' t-            -- Rerun the model with the new tree, to get a new-            -- weight w'.-            let (x', w') = runProb (runWriterT m) t'-            -- MH acceptance ratio. This is the probability of either-            -- returning the new seed or the old one.-            let ratio = getProduct w' / getProduct w-            let (r,g2') = random g2-            put g2'-            if r < min 1 (exp $ ln ratio) then return (t',x',w') else return (t,x,w)-+mhirreducible p q = metropolis p (Just q)   {- | Useful function which thins out a stream of results, as is common in Markov Chain Monte Carlo simulation.@@ -331,10 +326,4 @@ every n xs = case drop (n -1) xs of   (y : ys) -> y : every n ys   [] -> []---- | Monadic iterate (inlined from monad-extras).-iterateM :: Monad m => (a -> m a) -> a -> m [a]-iterateM f x = do-    x' <- f x-    (x':) `liftM` iterateM f x' 

@@ -1,16 +1,17 @@-{- | Handy distributions for the `LazyPPL` library, based on the `uniform` distribution. Mostly defined using the `Statistics.Distribution` module and family. +{- | Handy distributions for the `LazyPPL` library, based on the `uniform` distribution.  Sometimes both a distribution (type @Prob a@) and pdf (type @a -> Double@) are given. Distributions are useful for sampling, densities are used for scoring.   For more distributions, see the Statistics.Distribution in the statistics package. -} {-# OPTIONS_GHC -Wno-unrecognised-pragmas #-} {-# HLINT ignore "Avoid lambda using `infix`" #-}+{-# LANGUAGE BangPatterns #-}   module LazyPPL.Distributions (         -- * Continuous distributions         normal,normalPdf,normalLogPdf,exponential,expPdf,exponentialLogPdf,-        gamma, gammaLogPdf, beta, betaLogPdf, dirichlet, dirichletLogPdf,+        gamma, gammaLogPdf, beta, BetaLabelError(..), betaLogPdf, dirichlet, dirichletLogPdf,         uniformbounded, uniformBoundedLogPdf,         logNormalLogPdf,         -- * Discrete distributions@@ -22,9 +23,13 @@       where  import LazyPPL (Prob,uniform)-import Data.List (findIndex)+import Control.Exception (Exception, throw)+import Data.Bits (bit, shift)+import Data.List (findIndex, zipWith5)+import qualified Numeric.Sum as Sum import Numeric.SpecFunctions import Numeric.MathFunctions.Constants+import Numeric.RootFinding (Root(..), RiddersParam(..), Tolerance(..), ridders)  {-|   [Normal distribution](https://en.wikipedia.org/wiki/Normal_distribution)@@ -32,11 +37,15 @@ normal :: Double -- ^ mu, mean         -> Double -- ^ sigma, standard deviation         -> Prob Double-normal m s = do-  x <- uniform-  return $ (- invErfc (2 * x)) * (m_sqrt_2 * s) + m+normal m s+  | not (finite m && positiveFinite s) = error "normal: the mean must be finite and the standard deviation finite and positive"+  | otherwise = do+    x <- uniform+    -- The standard normal quantile is finite on the open interval; scaling it+    -- last avoids overflowing m_sqrt_2 * s.+    return $ m + s * (- invErfc (2 * x) * m_sqrt_2) normalPdf :: Double -> Double -> Double -> Double-normalPdf m s x = exp (- ((x - m) * (x - m) / (2 * s * s)) - log (m_sqrt_2_pi * s))+normalPdf m s x = exp (normalLogPdf m s x)   {-|@@ -44,11 +53,13 @@ -} exponential :: Double -- ^ lambda, rate             -> Prob Double-exponential rate = do-  x <- uniform-  return $ - (log x / rate)+exponential rate+  | not (positiveFinite rate) = error "exponential: the rate must be finite and positive"+  | otherwise = do+    x <- uniform+    return $ - (log x / rate) expPdf :: Double -> Double -> Double-expPdf rate x = exp (- (rate * x)) * rate+expPdf rate x = exp (exponentialLogPdf rate x)  {-|   [Gamma distribution](https://en.wikipedia.org/wiki/Gamma_distribution)@@ -56,149 +67,475 @@ gamma :: Double -- ^ k, shape       -> Double -- ^ theta, scale       -> Prob Double-gamma a b = do-  x <- uniform-  return $ b * invIncompleteGamma a x+gamma a b+  | not (positiveFinite a && positiveFinite b) = error "gamma: the shape and scale must be finite and positive"+  | otherwise = do+    x <- uniform+    return $ b * invIncompleteGamma a x  {-|   [Beta distribution](https://en.wikipedia.org/wiki/Beta_distribution)++  Each parameter must be at least 0.001, and their sum at most 2000, unless one+  of them is 1: Beta(a,1) and Beta(1,b) take every finite positive a and b.+  Other parameters raise an error. For other shapes, custom tree labels below+  @2^-53@ raise 'BetaLabelBelowGeneratorRange'; the default generator's+  smallest label is @2^-53@. The analytic one-shape cases accept all labels in+  @(0,1)@. On the supported rows of the reference grid, the drawn u-quantile has a distribution+  function within @1e-9 * min u (1 - u)@ of u, or is one of the two doubles+  next to the exact quantile. -} beta :: Double -- ^ alpha       -> Double -- ^ beta       -> Prob Double-beta a b = do-  x <- uniform-  return $ invIncompleteBeta a b x+beta a b+  | not (positiveFinite a && positiveFinite b && (betaSupported [a, b] || a == 1 || b == 1)) =+      error ("beta: the parameters must each be at least " ++ show betaLeast ++ " and sum to at most " ++ show betaSumMost+             ++ ", or be 1 and a finite positive number, got " ++ show (a, b))+  | otherwise = do+    x <- uniform+    return $ fst (betaQuantile a b x) +-- | A custom tree label lies below the supported range of the general Beta+-- sampler. The default random-tree generator never produces such a label.+data BetaLabelError = BetaLabelBelowGeneratorRange Double deriving (Eq, Show)+instance Exception BetaLabelError+ {-|   [Poisson distribution](https://en.wikipedia.org/wiki/Poisson_distribution) -} poisson :: Double -- ^ lambda, rate         -> Prob Integer-poisson lambda = do-  x <- uniform-  let cmf = map (\x -> 1 - incompleteGamma (fromIntegral (x + 1)) lambda) [0,1..]-  let (Just n) = findIndex (> x) cmf-  return $ fromIntegral n+poisson lambda+  | not (lambda >= 0 && finite lambda) = error "poisson: the rate must be finite and non-negative"+  | otherwise = do+    x <- uniform+    -- Compare the smaller tail directly, so subtracting it from 1 cannot erase it.+    let tails = map (\n -> incompleteGamma (fromIntegral (n + 1)) lambda) [0,1..]+        cumulative = map Sum.kbn $ drop 1 $ scanl Sum.add Sum.zero (map (exp . poissonLogPmf lambda) [0..])+        (Just n) = if x >= 0.5 then findIndex (< 1 - x) tails else findIndex (> x) cumulative+    return $ fromIntegral n  poissonPdf :: Double -> Integer -> Double-poissonPdf rate n = let result = exp(-rate) * rate ^^ fromIntegral n / factorial (fromIntegral n) in-  if isInfinite result || isNaN result then exp (-rate + fromIntegral n * log rate - logGamma (fromIntegral (n+1))) else result+poissonPdf rate n = exp (poissonLogPmf rate n)   {-|   [Dirichlet distribution](https://en.wikipedia.org/wiki/Dirichlet_distribution)++  Drawn by stick breaking: the first coordinate is Beta(a1, a2 + ... + ak), each+  later coordinate takes a Beta(ai, a(i+1) + ... + ak) share of the stick that+  remains, and the last coordinate is the rest. This is the Dirichlet law, and no+  coordinate is NaN even when the concentrations are so small that Gamma draws+  underflow to 0.++  Each concentration must be at least 0.001, and their sum at most 2000; other+  concentrations raise an error. Each share is a quantile of its Beta law, with+  the accuracy of 'beta'. -} dirichlet :: [Double] -- ^ vector of alphas; length is dimension           -> Prob[Double]-dirichlet as = do-  xs <- mapM (\a -> gamma a 1) as-  let s = Prelude.sum xs-  let ys = map (/ s) xs-  return ys+dirichlet as+  | not (betaSupported as) = error ("dirichlet: the concentrations must each be at least " ++ show betaLeast+                                    ++ " and sum to at most " ++ show betaSumMost ++ ", got " ++ show as)+  | otherwise = breakStick 1 as (drop 1 (scanr (+) 0 as))+  where+    breakStick rest [_] _ = return [rest]+    breakStick rest (a : rest') (later : laters) = do+      u <- uniform+      let (v, w) = betaQuantile a later u+      xs <- breakStick (rest * w) rest' laters+      return (rest * v : xs)+    breakStick _ _ _ = return [] +-- | The supported parameters of dirichlet, and of beta apart from Beta(a,1) and+-- Beta(1,b) (see betaQuantile): each at least betaLeast, with a sum of at most+-- betaSumMost. Every stick of dirichlet as is then a Beta law whose two+-- parameters lie in the domain, as their sum is a suffix sum of as.+betaSupported :: [Double] -> Bool+betaSupported as = all (>= betaLeast) as && sum as <= betaSumMost++-- The bounds are measured. test/reference/reference.py writes, with mpmath, the+-- rows of test/reference/beta-quantiles.tsv: Beta(a,b) at a label u, with the+-- interval of doubles that a supported draw may take (see betaQuantile), or a+-- rejection marker when both samplers reject the input. It covers 23+-- parameters from 1e-10 to 1e10 at 13 labels from 1e-300 to 1 - 2^-53, and for+-- the corners of the bounds, the doubles just outside them and random pairs+-- inside. On 1 Nov 2026 (math-functions 0.3.4.4, GHC 9.14.1), before the label+-- restriction and with the former first-order intervals, the bounds removed,+-- and each row run in a process killed after 2 s, beta a b and dirichlet [a,b]+-- landed in the intervals at all 2057 rows inside the shape bounds,+-- the slowest in 0.35 s (Beta(1999.999,0.001) at 1e-300). Outside, rows landed+-- outside the intervals where both parameters exceed 3000, where math-functions+-- computes I by quadrature (Beta(1e4,1e4) at 1e-300), and ran past 2 s with a+-- parameter of 1e-5 or below (Beta(1000,1e-5) at 2^-53). Draws with a parameter+-- near 0.001 slow down as the sum grows: Beta(1e4,0.001) took 1.6 s at 1e-20.+betaLeast, betaSumMost :: Double+betaLeast = 1e-3+betaSumMost = 2000++-- | The u-quantile x of Beta(a,b), with 1 - x, for parameters in the supported+-- domain. The quantile is a root of the regularized incomplete Beta function I+-- of math-functions: its inverse, invIncompleteBeta (0.3.4.4), starts from+-- guesses that fail outside a few regimes (its issues #29, #44 and #47), and+-- returns 0 for Beta(1, 0.001) at 1/4, whose quantile is 1 - 0.75^1000. Of x+-- and 1 - x, the one at most 1/2 is solved for, through+-- I_x(a,b) = 1 - I_(1-x)(b,a), so that each keeps its relative precision even+-- when the other rounds to 1. Both probabilities u and 1 - u are passed on:+-- 1 - u is exact when u >= 1/2, and u when it is the smaller.+betaQuantile :: Double -> Double -> Double -> (Double, Double)+betaQuantile a b u+  -- Beta(a,1) has the distribution function x^a and Beta(1,b) has 1 - (1-x)^b,+  -- so their u-quantiles are u^(1/a) and 1 - (1-u)^(1/b). With t = log u / a or+  -- t = log1p (-u) / b, exp t and -expm1 t retain the smaller share without+  -- subtracting it from 1. If division overflows to negative infinity, that+  -- smaller share rounds to 0.+  | b == 1 = let t = log u / a in (exp t, - expm1 t)+  | a == 1 = let t = log1p (- u) / b in (- expm1 t, exp t)+  | u < encodeFloat 1 (-53) = throw (BetaLabelBelowGeneratorRange u)+  | u <= incompleteBeta a b 0.5 = let x = smallQuantile a b u (1 - u) in (x, 1 - x)+  | otherwise = let y = smallQuantile b a (1 - u) u in (1 - y, y)++-- | The v-quantile z <= 1/2 of Beta(p,q), given I_(1/2)(p,q) >= v and w = 1 - v:+-- the root of a residual in t = log z, between the least positive double and+-- 1/2, by math-functions' Ridders method, which keeps the bracket. Solving for+-- log z keeps full relative precision for quantiles far below 1, such as+-- 0.75^1000. The residual compares the logarithms of the smaller of the two+-- probabilities: for w at least 2^-19, log I_z(p,q) with log v; below, the+-- upper tail 1 - I_z(p,q) = I_(1-z)(q,p) with log w. Near 1, I_z(p,q) is a+-- double within 2^-53 of 1 - w, so comparing it with v would leave the root+-- anywhere on a plateau of relative width 2^-53 / w in w, at most 2^-34 for w+-- at least 2^-19. Where a probability underflows, its logarithm is held at that+-- of the least positive double, so the residual stays finite and keeps its+-- sign; a probability that is not a number in [0,1] raises an error. A quantile+-- below the least positive double is 0, the nearer double or the one below it.+-- The search stops within one ulp of t (AbsTol 0) and may take 200 steps,+-- twice math-functions' default, because the bracket spans the whole double+-- range on the log scale; a search that does not converge raises an error+-- rather than returning a guess.+smallQuantile :: Double -> Double -> Double -> Double -> Double+smallQuantile p q v w+  | f hi <= 0 = 0.5+  | f lo > 0 = 0+  | otherwise = case ridders RiddersParam { riddersMaxIter = 200, riddersTol = AbsTol 0 } (lo, hi) f of+      Root t -> exp t+      _ -> error ("beta: no quantile found for parameters " ++ show (p, q) ++ " and probability " ++ show v)+  where+    f t | w < encodeFloat 1 (-19) = logProbability w - logProbability (incompleteBeta q p (1 - exp t))+        | otherwise = logProbability (incompleteBeta p q (exp t)) - logProbability v+    logProbability c+      | c >= 0 && c <= 1 = log (max leastPositive c)+      | otherwise = error ("beta: the incomplete Beta function gave " ++ show c ++ " for parameters " ++ show (p, q))+    leastPositive = encodeFloat 1 (-1074)+    lo = log leastPositive+    hi = log 0.5+ -- | [Continuous uniform distribution on a bounded interval](https://en.wikipedia.org/wiki/Continuous_uniform_distribution) uniformbounded :: Double -- ^ lower                 -> Double -- ^ upper                 -> Prob Double-uniformbounded lower upper = do-  x <- uniform-  return $ (upper - lower) * x + lower+uniformbounded lower upper+  | not (finite lower && finite upper && lower < upper) = error "uniformbounded: the bounds must be finite, with lower < upper"+  | otherwise = do+    x <- uniform+    -- The width overflows for finite bounds of opposite signs near the largest+    -- double; the weighted sum of the bounds does not (random 1.3.1's+    -- scaleFloating handles an infinite width the same way).+    return $ if isInfinite (upper - lower) then x * upper + (1 - x) * lower else (upper - lower) * x + lower  -- | [Bernoulli distribution](https://en.wikipedia.org/wiki/Bernoulli_distribution) bernoulli :: Double -- ^ bias           -> Prob Bool-bernoulli r = do-  x <- uniform-  return $ x < r+bernoulli r+  | not (probability r) = error "bernoulli: the probability must be in [0,1]"+  | otherwise = do+    x <- uniform+    return $ x < r  {-|   [Discrete uniform distribution](https://en.wikipedia.org/wiki/Discrete_uniform_distribution) on [0, ..., n-1] -} uniformdiscrete :: Int -- ^ n                 -> Prob Int-uniformdiscrete n =-  do-    let upper = fromIntegral n-    r <- uniformbounded 0 upper-    return $ floor r+uniformdiscrete n+  | n < 1 = error "uniformdiscrete: n must be at least 1"+  | otherwise = do+      let upper = fromIntegral n+      r <- uniformbounded 0 upper+      return $ floor r  {-| [Categorical distribution](https://www.google.com/search?client=safari&rls=en&q=categorical+distribution&ie=UTF-8&oe=UTF-8): Takes a list of k numbers that sum to 1, -    and returns a random number between 0 and (k-1), weighted accordingly -}+    and returns a random number between 0 and (k-1), weighted accordingly.+    Only the probabilities up to the chosen index are read, so the list may be infinite, and each one read must be in [0,1]. -} categorical :: [Double] -> Prob Int categorical xs = do   r <- uniform-  case findIndex (>r) $ tail $ scanl (+) 0 xs of-    Just i -> return i-    Nothing -> error "categorical: probabilities do not sum to 1"+  return $! pickCategorical r xs +pickCategorical :: Double -> [Double] -> Int+pickCategorical r = pick 0 Sum.zero 0 Nothing+  where+      -- The upper bounds below are added as exact integers, in units of+      -- 2^-1075 (half the smallest positive Double). For example, 0.5 is+      -- 2^1074 units and the smallest positive Double is 2 units. Every+      -- bound from upperWeight is a whole number of these units, so their+      -- sum never rounds. Probability 1 is 2^1075 units. The probabilities+      -- themselves are added as Doubles (a compensated sum), which can+      -- round, as in the example further down.+      limit = bit 1075 :: Integer+      -- The largest exact probability that rounds to the Double q:+      -- q plus half the gap to the next Double, in the same units.+      upperWeight q+        | q == 0 = 1+        | otherwise = let (mantissa, exponent) = decodeFloat q+                      in shift mantissa (exponent + 1075) + bit (max 0 (exponent + 1074))+      pick !i !total !upper !previous (q:qs)+        | not (probability q) = error "categorical: probabilities must be in [0,1]"+        | q > 0 && Sum.kbn next > r = i+        | otherwise = pick (i + 1) next (min limit (upper + upperWeight q))+          -- Capped at 1, so an infinite list cannot grow it.+                           (if q > 0 then Just i else previous) qs+        where next = Sum.add total q+      -- The list ended before the running sum passed r. Valid input can do+      -- this. For example, with 49 weights of 1/49:+      --   stored Double:  0.02040816326530612  (just below 1/49)+      --   sum of all 49:  0.9999999999999999   (not 1)+      --   draw r:         0.9999999999999999   (the largest draw below 1)+      -- No running sum passes r. Each true weight is at most upperWeight of+      -- its Double, so if the true weights sum to 1, the upper bounds sum to+      -- at least 1. If they do, the gap between the sum and 1 (1.1e-16 in+      -- the example) comes from rounding: return the last index with+      -- positive weight (index 48 in the example). If not, the weights sum+      -- to less than 1 by more than rounding can explain.+      pick _ _ !upper (Just i) []+        | upper >= limit = i+        | otherwise = error "categorical: probabilities do not sum to 1"+      pick _ _ _ Nothing [] = error "categorical: probabilities do not sum to 1"+ {-| Returns an infinite stream of samples from the given distribution. --} iid :: Prob a -> Prob [a] iid p = do r <- p; rs <- iid p; return $ r : rs  normalLogPdf :: Double -> Double -> Double -> Double-normalLogPdf mu sigma x =-  let d = x - mu-  in -0.5 * log (2 * pi) - log sigma - (d * d) / (2 * sigma * sigma)+normalLogPdf mu sigma x+  | not (finite mu && positiveFinite sigma) = error "normalLogPdf: the mean must be finite and the standard deviation finite and positive"+  | otherwise = -0.5 * log (2 * pi) - log sigma - (0.5 * z) * z+  where+    d = x - mu+    -- Standardise before squaring, so (x - mu)^2 and sigma^2 cannot overflow or+    -- underflow on their own. x - mu overflows only for huge x and mu of opposite+    -- signs, and then x/sigma - mu/sigma loses nothing.+    z | isInfinite d && not (isInfinite x) = x / sigma - mu / sigma+      | otherwise = d / sigma  exponentialLogPdf :: Double -> Double -> Double exponentialLogPdf rate x-  | x < 0    = -1e300+  | rate <= 0 || isNaN rate || isInfinite rate = error "exponentialLogPdf: rate must be finite and positive"+  | x < 0    = m_neg_inf   | otherwise = log rate - rate * x  gammaLogPdf :: Double -> Double -> Double -> Double gammaLogPdf k theta x-  | x <= 0   = -1e300-  | otherwise = (k - 1) * log x - x / theta - k * log theta - logGamma k+  | not (positiveFinite k && positiveFinite theta) = error "gammaLogPdf: the shape and scale must be finite and positive"+  | x <= 0 || isInfinite x = m_neg_inf+  | isInfinite lambda && isInfinite relativeRate = m_neg_inf+  | isInfinite lambda = k * log1pmx relativeRate - stirlingError k+      - m_ln_sqrt_2_pi - 0.5 * log k + log k - log x+  -- The density is the Poisson mass of lambda = x/theta at k - 1, divided by+  -- theta (at k when k < 1, times k/x), so the Poisson kernel applies.+  | lambda == 0 = (k - 1) * log x - k * log theta - logGamma k+  | k == 1 = - lambda - log theta+  | k > 1 = poissonKernel (k - 1) lambda (nearResidual k (lambda - (k - 1)) (rate - (toRational k - 1))) - log theta+  | otherwise = poissonKernel k lambda (nearResidual k (lambda - k) (rate - toRational k)) + log k - log x+  where+    lambda = x / theta+    rate = toRational x / toRational theta+    relativeRate = fromRational ((rate - toRational k) / toRational k) +-- | Log density of the Beta distribution, or negative infinity outside (0,1).+-- Parameters that are not positive, or whose sum is not finite, raise an error. betaLogPdf :: Double -> Double -> Double -> Double betaLogPdf a b x-  | x <= 0 || x >= 1 = -1e300-  | otherwise = (a - 1) * log x + (b - 1) * log (1 - x) - logBeta a b+  | not (positiveFinite a && positiveFinite b && positiveFinite (a + b)) = error "betaLogPdf: both parameters must be positive, with a finite sum"+  | x <= 0 || x >= 1 = m_neg_inf+  | otherwise = betaKernel a b x (1 - x) (log x) (log1p (-x)) (toRational x)  bernoulliLogPmf :: Double -> Bool -> Double-bernoulliLogPmf p True  = log (max 1e-300 p)-bernoulliLogPmf p False = log (max 1e-300 (1 - p))+bernoulliLogPmf p outcome+  | isNaN p || p < 0 || p > 1 = error "bernoulliLogPmf: probability must be in [0,1]"+  | outcome = log p+  | otherwise = log1p (-p) +-- | Log-probability of the k-th category. Only the probabilities at indices+-- 0..k are inspected, so an infinite probability stream, as the Dirichlet+-- process supplies, is answered after reading its needed prefix. categoricalLogPmf :: [Double] -> Int -> Double categoricalLogPmf probs k-  | k < 0 || k >= length probs = -1e300-  | otherwise = log (max 1e-300 (probs !! k))+  | k < 0 = m_neg_inf+  | otherwise = case drop k probs of+      [] -> m_neg_inf+      (p:_)+        | any (\q -> isNaN q || q < 0 || q > 1) (take (k + 1) probs) ->+            error "categoricalLogPmf: probabilities must be in [0,1]"+        | otherwise -> log p  uniformBoundedLogPdf :: Double -> Double -> Double -> Double uniformBoundedLogPdf lower upper x-  | x < lower || x > upper = -1e300+  | not (finite lower && finite upper && lower < upper) = error "uniformBoundedLogPdf: the bounds must be finite, with lower < upper"+  | x < lower || x > upper = m_neg_inf+  | isInfinite (upper - lower) = -(log (upper / 2 - lower / 2) + log 2)   | otherwise = -log (upper - lower) +-- | Log density of the Dirichlet distribution, or negative infinity outside+-- the open simplex. The coordinates must sum to 1 within+-- gamma_n = n*u/(1-n*u), with n the dimension and u = 2^-53, which bounds the+-- rounding error of the coordinates and their sum. Concentrations that are not+-- positive, or whose sum is not finite, raise an error. dirichletLogPdf :: [Double] -> [Double] -> Double dirichletLogPdf alphas xs-  | length alphas /= length xs = -1e300-  | any (<= 0) xs              = -1e300-  | otherwise =-      let lnB = sum (map logGamma alphas) - logGamma (sum alphas)-      in sum (zipWith (\a x -> (a - 1) * log x) alphas xs) - lnB+  | null xs || length alphas /= length xs = m_neg_inf+  | not (all positiveFinite alphas && positiveFinite (sum alphas)) = error "dirichletLogPdf: concentrations must be positive, with a finite sum"+  | any (\x -> x <= 0 || isNaN x || isInfinite x) xs = m_neg_inf+  | abs (sum xs - 1) > roundingBound = m_neg_inf+  | otherwise = sum (zipWith5 factor alphas laters xs rests (drop 1 rests))+    where+      nu = fromIntegral (length xs) * encodeFloat 1 (negate (floatDigits (0 :: Double)))+      roundingBound = nu / (1 - nu)+      -- With r_i = x_i + ... + x_k, the share x_i/r_i of the remaining stick is+      -- Beta(a_i, a_(i+1) + ... + a_k), independently (the aggregation property),+      -- so the density is the product of those Beta densities over the Jacobian+      -- r_i of x_i = r_i * (x_i/r_i). Suffix sums form 1 - x_i/r_i without cancelling.+      rests = scanr1 (+) xs+      laters = drop 1 (scanr (+) 0 alphas)+      factor a later x r next =+        betaKernel a later (x / r) (next / r) (log x - log r) (log next - log r) (toRational x / toRational r) - log r  logNormalLogPdf :: Double -> Double -> Double -> Double logNormalLogPdf mu sigma x-  | x <= 0   = -1e300-  | otherwise =-      let lx = log x-          d  = lx - mu-      in -0.5 * log (2 * pi) - log sigma - lx - (d * d) / (2 * sigma * sigma)+  | not (finite mu && positiveFinite sigma) = error "logNormalLogPdf: the mean must be finite and the standard deviation finite and positive"+  | x <= 0   = m_neg_inf+  | otherwise = normalLogPdf mu sigma (log x) - log x  binomialLogPmf :: Int -> Double -> Int -> Double binomialLogPmf n p k-  | k < 0 || k > n  = -1e300-  | p <= 0 && k > 0 = -1e300-  | p >= 1 && k < n = -1e300-  | otherwise =-      logGamma (fromIntegral n + 1)-      - logGamma (fromIntegral k + 1)-      - logGamma (fromIntegral (n - k) + 1)-      + fromIntegral k * log (max 1e-300 p)-      + fromIntegral (n - k) * log (max 1e-300 (1 - p))+  | n < 0 || isNaN p || p < 0 || p > 1 = error "binomialLogPmf: n must be non-negative and probability in [0,1]"+  | k < 0 || k > n  = m_neg_inf+  | p == 0 = if k == 0 then 0 else m_neg_inf+  | p == 1 = if k == n then 0 else m_neg_inf+  | k == 0 = fromIntegral n * log1p (-p)+  | k == n = fromIntegral n * log p+  | otherwise = binomialKernel (fromIntegral k) (fromIntegral (n - k)) p (1 - p)+      (nearResidual (fromIntegral n) (fromIntegral n * p - fromIntegral k) (fromIntegral n * toRational p - fromIntegral k))  poissonLogPmf :: Double -> Integer -> Double poissonLogPmf lambda k-  | k < 0    = -1e300-  | otherwise = fromIntegral k * log lambda - lambda - logGamma (fromIntegral k + 1)+  | lambda < 0 || isNaN lambda || isInfinite lambda = error "poissonLogPmf: rate must be finite and non-negative"+  | k < 0    = m_neg_inf+  -- At rate 0 the formula below gives NaN for k = 0 (0 * log 0), as does+  -- statistics-0.16.5.0.+  | lambda == 0 = if k == 0 then 0 else m_neg_inf+  | k == 0 = - lambda+  | isInfinite count = largeCountPoisson lambda k+  | otherwise = poissonKernel (fromIntegral k) lambda (countResidual lambda k)+  where count = fromInteger k :: Double++-- For a count above Double's finite range, keep one quarter of it in range.+-- The exact Integer residual retains central counts that round to the rate.+largeCountPoisson :: Double -> Integer -> Double+largeCountPoisson lambda k+  | isInfinite quarter = m_neg_inf+  | relative > -0.5 = quarter * (4 * log1pmx relative) - correction+  | otherwise = - quarter * (4 * (logRatio - 1) + lambda / quarter) - correction+  where+    quarter = fromInteger (k `div` 4)+    relative = fromRational ((toRational lambda - fromInteger k) / fromInteger k)+    logCount = log quarter + log 4+    ratio = (lambda / quarter) / 4+    logRatio | ratio >= m_tiny = - log ratio+             | otherwise = logCount - log lambda+    correction = stirlingError quarter / 4 + m_ln_sqrt_2_pi + 0.5 * logCount++-- | lambda - k, rounded once. A count above 2^53 can differ from the nearest+-- double by more than the spread of the distribution (10^32 and the double 1e32+-- differ by 5.4e15, half a standard deviation), so its difference from lambda is+-- formed exactly.+countResidual :: Double -> Integer -> Double+countResidual lambda k+  | k < 2 ^ (53 :: Int) = lambda - fromInteger k+  | otherwise = fromRational (toRational lambda - fromInteger k)++-- Loader's saddle-point forms (C. Loader, "Fast and accurate computation of+-- binomial probabilities", 2000). log(n!) is the Stirling formula plus+-- stirlingError n (from math-functions), and each power is a deviance bd0, so+-- no two large terms cancel, as they did in sums of logGamma values. Logarithms+-- of products are taken as sums of logarithms, so no product leaves the range.++-- | The logarithm of lambda^m e^(-lambda) / Gamma(m+1), for real m > 0 and+-- lambda > 0, given lambda - m as d.+poissonKernel :: Double -> Double -> Double -> Double+poissonKernel m lambda d = - stirlingError m - bd0 m lambda d - m_ln_sqrt_2_pi - 0.5 * log m++-- | The logarithm of Gamma(m+l+1) / (Gamma(m+1) Gamma(l+1)) p^m q^l, for real+-- m, l > 0 and q = 1 - p given separately, given np - m as d; nq - l is -d.+binomialKernel :: Double -> Double -> Double -> Double -> Double -> Double+binomialKernel m l p q d =+  stirlingError n - stirlingError m - stirlingError l+    - bd0 m (n * p) d - bd0 l (n * q) (- d)+    + 0.5 * (log n - log m - log l) - m_ln_sqrt_2_pi+  where n = m + l++-- | The residual np - x of bd0 near the centre, rounded once, for a count or+-- shape `size`. Rounding np, a product or quotient of the inputs, by a+-- relative 2^-53 moves the deviance by at most |np - x| 2^-53, which stays+-- below 1e-12 of the log density while size is below 2^25: so below, the+-- residual comes from Double arithmetic (`fast`), and above, from the exact+-- rational value of the same expression. At gammaLogPdf 1e300 1e-300 1, the+-- rounded x / theta is one ulp, 1.5e284, from 1e300, and the deviance 1.1e268+-- that it gives is 3.7 times the true 3.0e267.+nearResidual :: Double -> Double -> Rational -> Double+nearResidual size fast exact+  | size < 33554432 = fast+  | otherwise = fromRational exact++-- | The deviance x log(x/np) + np - x, for positive finite x and np, given+-- d = np - x. With r = np/x it is -x (log r - (r - 1)) = -x log1pmx (r - 1).+-- For r in [1/2, 2], r - 1 = d/x, and math-functions' log1pmx is accurate near+-- 0, where the deviance is small; between doubles, np - x is exact there. Elsewhere+-- |log(x/np)| >= log 2 and the deviance is at least 0.15 max(x, np): it is+-- x (log(x/np) - 1) + np, whose two terms sum in magnitude to at most 13.4+-- times its value, and neither exceeds it in magnitude by more than np, so+-- neither overflows where the deviance does not (x log(x/np) itself is 2.1e308+-- at x = 1e308, np = 1.25e307, where the deviance is 1.2e308). It takes+-- log x - log np when x/np is not a normal double. math-functions' own bd0+-- (0.3.4.4) forms x/np and x + np: it gives Infinity at x = 1, np = 1e-320, and+-- at x = np = 1e308 its series starts from Infinity * 0 and never ends.+bd0 :: Double -> Double -> Double -> Double+bd0 x np d+  | x / 2 <= np && np / 2 <= x = - x * log1pmx (d / x)+  | otherwise = x * (logRatio - 1) + np+  where+    ratio = x / np+    logRatio | ratio >= m_tiny && ratio <= m_huge = log ratio+             | otherwise = log x - log np++-- | The log density of Beta(a,b) at p, with q = 1 - p and both logarithms given,+-- so that none of the four is formed by cancellation, and p also exactly.+betaKernel :: Double -> Double -> Double -> Double -> Double -> Double -> Rational -> Double+betaKernel a b p q logP logQ exactP+  -- (a + b - 1) times the binomial mass of a - 1 successes in a + b - 2 trials.+  -- With both counts at least 1 the kernel's own Stirling terms do not cancel.+  | a >= 2 && b >= 2 = log (a + b - 1) + binomialKernel (a - 1) (b - 1) p q+      (nearResidual (a + b) ((a + b - 2) * p - (a - 1)) ((toRational a + toRational b - 2) * exactP - (toRational a - 1)))+  -- Below 2 the direct form has no large terms to cancel; logBeta corrects a+  -- large argument with Stirling terms.+  | otherwise = (a - 1) * logP + (b - 1) * logQ - logBeta a b++-- Parameter domains. A comparison with NaN is false, so each test rejects NaN.+positiveFinite :: Double -> Bool+positiveFinite x = x > 0 && x < m_pos_inf++finite :: Double -> Bool+finite x = x > m_neg_inf && x < m_pos_inf++probability :: Double -> Bool+probability p = p >= 0 && p <= 1

@@ -1,4 +1,3 @@-{-# LANGUAGE BangPatterns #-} module LazyPPL.Distributions.Counter (Counter,newCounter,readAndIncrement) where  import LazyPPL@@ -18,16 +17,21 @@  data Counter = C (IORef Int) +-- | A counter that starts at 0. A counter holds hidden state: see 'LazyPPL.runProb' for its use outside a model. newCounter :: Prob Counter+-- Both operations stay out of the caller's code. Inlined there, full laziness+-- (on at GHC's default -O) gave a counter created and read inside one Prob+-- computation the number 0 at every read (test/HiddenState.hs, -O1 and -O2).+{-# NOINLINE newCounter #-} newCounter = do r <- uniform                 return $ C $ unsafePerformIO $ newIORef (round (r-r)) +-- | The counter's next number: 0, then 1, and so on. readAndIncrement :: Counter -> Prob Int +{-# NOINLINE readAndIncrement #-} readAndIncrement (C ref) = do     r <- uniform-    return $ unsafePerformIO $ do -        !i <- readIORef ref -        () <- writeIORef ref (i + 1 + round (r - r))-        return i +    -- One atomic step, so two threads never receive the same number.+    return $ unsafePerformIO $ atomicModifyIORef' ref (\i -> (i + 1 + round (r - r), i))  

@@ -18,8 +18,8 @@ -- * Random distribution interface dp) where -import Data.List-import Data.Maybe+import Data.Coerce (coerce)+import qualified Numeric.Sum as Sum import LazyPPL import LazyPPL.Distributions import LazyPPL.Distributions.Memoization (MonadMemo)@@ -33,35 +33,41 @@  {-| A customer enters the restaurant and is assigned a table. -} newCustomer :: Restaurant -> Prob Table-newCustomer (R restaurant) =-  do-    r <- uniform-    return $ T $ fromJust $ findIndex (> r) (scanl1 (+) restaurant)+-- Coercing the newtype preserves categorical's random-tree allocation.+newCustomer (R restaurant) = coerce (categorical restaurant) -{-| Create a new restaurant with concentration parameter alpha. -}+{-| Create a new restaurant with concentration parameter alpha, which may be any finite positive number; other values raise an error. -} newRestaurant :: Double -- ^ Concentration parameter, alpha               -> Prob Restaurant-newRestaurant alpha = do-  sticks <- stickBreaking alpha 0-  return $ R sticks+newRestaurant alpha+  | not (concentration alpha) = error ("newRestaurant: the concentration alpha must be finite and positive, got " ++ show alpha)+  | otherwise = do+      sticks <- stickBreaking alpha Sum.zero+      return $ R sticks +-- | Whether alpha is a concentration that beta 1 alpha takes: finite and positive.+concentration :: Double -> Bool+concentration alpha = alpha > 0 && not (isInfinite alpha)+ {- | Stick breaking breaks the unit interval into an     infinite number of parts (lazily) --}-stickBreaking :: Double -> Double -> Prob [Double]+stickBreaking :: Double -> Sum.KBNSum -> Prob [Double] stickBreaking alpha lower =   do     r <- beta 1 alpha-    let v = r * (1 - lower)-    vs <- stickBreaking alpha (lower + v)+    let v = r * (1 - Sum.kbn lower)+    vs <- stickBreaking alpha (Sum.add lower v)     return (v : vs) -{-| [Dirichlet Process](https://en.wikipedia.org/wiki/Dirichlet_process) as a random distribution. -}+{-| [Dirichlet Process](https://en.wikipedia.org/wiki/Dirichlet_process) as a random distribution. The concentration alpha may be any finite positive number; other values raise an error. -} dp :: Double -- ^ Concentration parameter, alpha    -> Prob a -- ^ Base distribution    -> Prob (Prob a)-dp alpha p = do-  xs <- iid p-  vs <- stickBreaking alpha 0-  return $ do-    r <- uniform-    return $ xs !! fromJust (findIndex (> r) (scanl1 (+) vs))+dp alpha p+  | not (concentration alpha) = error ("dp: the concentration alpha must be finite and positive, got " ++ show alpha)+  | otherwise = do+      xs <- iid p+      vs <- stickBreaking alpha Sum.zero+      -- Prob's fmap adds a tree split. Map this result on the same tree so the+      -- shared selector also preserves the existing fixed-seed distribution.+      return $ Prob $ \tree -> xs !! runProb (categorical vs) tree

@@ -9,15 +9,13 @@ -}  -module LazyPPL.Distributions.GP (wiener, gp) where+module LazyPPL.Distributions.GP (wiener, gp, GPError(..)) where  import LazyPPL (Prob,Tree(Tree),Prob(Prob),runProb) import LazyPPL.Distributions (normal,iid)-import Numeric.LinearAlgebra hiding (step,size,find)-import Numeric.LinearAlgebra.Data hiding (step,size,find)-import Data.List (find)-import Data.Map (empty,lookup,insert,size,keys,elems,Map)-import Data.IORef+import LazyPPL.Distributions.GP.Pure (GPError(..), emptyGPState, sampleGP)+import qualified Data.Map+import Control.Concurrent.MVar (modifyMVar, newMVar) import System.IO.Unsafe import Prelude hiding ((<>)) @@ -25,71 +23,78 @@  This is a random function.  -The Wiener process is implemented by using a Brownian bridge and a hidden memo table. -}+The Wiener process is implemented by using a Brownian bridge and a hidden memo table.+A query reads the table, draws and stores under one lock, so concurrent queries+condition on each other; if a query raises an error, the table is unchanged.+The returned function holds hidden state: see 'LazyPPL.runProb' for its use outside a model. -} wiener :: Prob (Double -> Double)-wiener = Prob $ \(Tree r gs) ->-                   unsafePerformIO $ do-                                 ref <- newIORef Data.Map.empty-                                 modifyIORef' ref (Data.Map.insert 0 0)-                                 return $ \x -> unsafePerformIO $ do-                                        table <- readIORef ref-                                        case Data.Map.lookup x table of-                                             Just y -> do {return y}-                                             Nothing -> do let lower = do {l <- findMaxLower x (keys table) ;-                                                                           v <- Data.Map.lookup l table ; return (l,v) }-                                                           let upper = do {u <- find (> x) (keys table) ;-                                                                           v <- Data.Map.lookup u table ; return (u,v) }-                                                           let m = bridge lower x upper-                                                           let y = runProb m (gs !! (1 + size table))-                                                           modifyIORef' ref (Data.Map.insert x y)-                                                           return y+wiener = Prob $ \(Tree _ gs) ->+  unsafePerformIO $ do+    ref <- newMVar (Data.Map.singleton 0 0)+    return $ \x -> unsafePerformIO $ modifyMVar ref $ \table ->+      if isNaN x || isInfinite x then error ("wiener: the time must be finite, got " ++ show x) else+      case Data.Map.lookup x table of+        Just y -> return (table, y)+        Nothing -> do+          let m = bridge (Data.Map.lookupLT x table) x (Data.Map.lookupGT x table)+              y = runProb m (gs !! (1 + Data.Map.size table))+          y `seq` return (Data.Map.insert x y table, y) +-- | The value at time y given its nearest neighbours in the table: the+-- conditional mean plus the conditional standard deviation times a standard+-- normal draw. It reads the same label of the tree as normal mean sd would, so+-- each point keeps its subtree. bridge :: Maybe (Double,Double) -> Double -> Maybe (Double,Double) -> Prob Double--- not needed since the table is always initialized with (0, 0)--- bridge Nothing y Nothing = if y==0 then return 0 else normal 0 (sqrt y) -bridge (Just (x,x')) y Nothing = normal x' (sqrt (y-x))-bridge Nothing y (Just (z,z')) = normal z' (sqrt (z-y))-bridge (Just (x,x')) y (Just (z,z')) = normal (x' + ((y-x)*(z'-x')/(z-x))) (sqrt ((z-y)*(y-x)/(z-x)))--findMaxLower :: Double -> [Double] -> Maybe Double -findMaxLower d [] = Nothing-findMaxLower d (x:xs) = let y = findMaxLower d xs in-                       case y of -                           Nothing -> if x < d then Just x else Nothing -                           Just m -> do -                                          if x > m && x < d then Just x else Just m ---+bridge lower y upper = Prob $ \t -> mean + sd * runProb (normal 0 1) t+  where+    (mean, sd) = case (lower, upper) of+      (Just (x,x'), Nothing) -> (x', sqrt (y - x))+      (Nothing, Just (z,z')) -> (z', sqrt (z - y))+      -- Between x < y < z the variance is left * right / width, with left = y - x,+      -- right = z - y and width = z - x. As sqrt (min left right) times+      -- sqrt (max left right / width), whose second factor lies between sqrt (1/2)+      -- and 1, the standard deviation is representable whenever its true value is, even+      -- where the variance or left / width is not. The mean moves from the nearer+      -- neighbour by that neighbour's share of the width, at most 1/2.+      (Just (x,x'), Just (z,z')) ->+        let left = y - x+            right = z - y+            width = z - x+        in ( if left <= right then x' + (left / width) * (z' - x') else z' - (right / width) * (z' - x')+           , sqrt (min left right) * sqrt (max left right / width) )+      -- The table always holds the point 0.+      (Nothing, Nothing) -> error "wiener: empty table"  {-| [Gaussian process](https://en.wikipedia.org/wiki/Gaussian_process) with given covariance function.  This is a random function.  The function is defined by using linear algebra and a hidden memo table.-Although it uses hidden state, it is still safe, i.e. statistically commutative and discardable. --}-gp :: (Double -> Double -> Double) -- ^ Covariance function-      -> Prob (Double -> Double) -- ^ Returns a random function-gp cov = do ns <- iid $ normal 0 1-            return $ unsafePerformIO $ do-                                 ref <- newIORef Data.Map.empty-                                 modifyIORef' ref (Data.Map.insert 0 0)-                                 return $ \x -> unsafePerformIO $ do-                                        table <- readIORef ref-                                        case Data.Map.lookup x table of-                                             Just y -> do {return y}-                                             Nothing -> do let y = step cov table x $ ns !! (1 + size table)-                                                           modifyIORef' ref (Data.Map.insert x y)-                                                           return y -step :: (Double -> Double -> Double) -> Data.Map.Map Double Double -> Double -> Double -> Double-step cov table x seed =-       let sig11 = cov x x -           sig12 = matrix (size table) [cov x b | b <-keys table] -           sig21 = matrix 1 [cov a x | a <-keys table] -           sig22 = matrix (size table) [cov a b | a <- keys table , b <-keys table] -           regCoeff = sig12 <> (pinvTol 1E8 sig22)-           mu = (regCoeff <> (matrix 1 $ elems table)) `atIndex` (0,0)-           var = sig11 - ((regCoeff <> sig21) `atIndex` (0,0)) in-       mu + seed * (sqrt $ if var > -0.01 then (abs var) else error (show var))+At distinct points the values have covariance+@C(x,y) = cov(x,y) + 2^-33 * cov(x,x) * [x == y]@, and a repeated query returns+the same value. The covariance function must be symmetric, which is not+checked. A point with zero variance returns 0, and its covariance with every+other queried point must be 0. +A covariance value that is NaN or infinite raises 'GPCovarianceNotFinite'. Any+other failure, including an overflow or a NaN in the+[Cholesky factorization](https://en.wikipedia.org/wiki/Cholesky_decomposition)+of @C@, raises 'GPCovarianceNotPositiveDefinite'. A query reads the table,+draws and stores under one lock, so concurrent queries condition on each other;+a failed query leaves the table unchanged. The returned function holds hidden+state: see 'LazyPPL.runProb' for its use outside a model. -}+gp :: (Double -> Double -> Double) -- ^ Covariance function+   -> Prob (Double -> Double) -- ^ Returns a random function+gp cov = do+  ns <- iid (normal 0 1)+  return $ unsafePerformIO $ do+    ref <- newMVar (Data.Map.empty, emptyGPState)+    return $ \x -> unsafePerformIO $ modifyMVar ref $ \(table, state) ->+      case Data.Map.lookup x table of+        Just value -> return ((table, state), value)+        Nothing -> do+          let seed = ns !! (1 + Data.Map.size table)+              (value, next) = sampleGP cov state x seed+          -- Force sampleGP's checks inside modifyMVar, so a failed query changes nothing.+          value `seq` return ((Data.Map.insert x value table, next), value)

@@ -0,0 +1,69 @@+{-# LANGUAGE BangPatterns #-}+{-# OPTIONS_GHC -O2 #-}+module LazyPPL.Distributions.GP.Pure (GPError(..), GPState, emptyGPState, sampleGP) where++import Control.Exception (Exception, throw)+import Data.List (foldl')++-- | A 'GPError' holds the point that could not be sampled, and its constructor says why.+data GPError+  = GPCovarianceNotFinite Double+  | GPCovarianceNotPositiveDefinite Double+  deriving (Eq, Show)++instance Exception GPError++-- Rows, and each row's entries for earlier points, are newest first.+data Row = Row !Double !Double !Double [Double]++-- | The checked rows of the [Cholesky factor](https://en.wikipedia.org/wiki/Cholesky_decomposition),+-- and the zero-variance points, which have no row.+data GPState = GPState [Row] [Double]++-- | Nothing sampled yet.+emptyGPState :: GPState+emptyGPState = GPState [] []++-- | @sampleGP cov state x seed@ samples at a new point @x@ from an independent+-- standard normal draw @seed@. The values have covariance+-- @C(x,y) = cov(x,y) + 2^-33 * cov(x,x) * [x == y]@, and zero-variance points+-- get 0.+sampleGP :: (Double -> Double -> Double) -> GPState -> Double -> Double -> (Double, GPState)+sampleGP cov (GPState rows zeros) x seed+  | not (finite prior && finite diagonal) || any nonfiniteCross cross =+      throw (GPCovarianceNotFinite x)+  | prior < 0 || any invalidZero cross =+      throw (GPCovarianceNotPositiveDefinite x)+  | prior == 0 = (0, GPState rows (x:zeros))+  | pivot <= 0 || isNaN pivot =+      throw (GPCovarianceNotPositiveDefinite x)+  | otherwise = (sum (zipWith (*) lower seeds) + pivot*seed,+                 GPState (Row x seed pivot lower:rows) zeros)+  where+    prior = cov x x+    diagonal = prior + nugget * prior+    cross = [(False,cov x a,cov a x) | Row a _ _ _ <- rows] +++            [(True,cov x a,cov a x) | a <- zeros]+    nonfiniteCross (_,a,b) = not (finite a && finite b)+    invalidZero (zero,a,b) = (prior == 0 || zero) && (a /= 0 || b /= 0)+    seeds = [z | Row _ z _ _ <- rows]+    lower = foldl' append [] (reverse rows)+    append previous (Row a _ d coefficients) =+      let !value = (cov x a - sum (zipWith (*) coefficients previous)) / d+      in value:previous+    pivot = sqrt (diagonal - sum (map (\a -> a*a) lower))++-- The nugget is a fixed fraction of each point's variance, so C does not+-- depend on the query order. It stops the factorization failing on nearly+-- dependent rows, as for an RBF kernel on a fine grid.+nugget :: Double+nugget = 2 ** (-33)++-- `lower` and `pivot` in sampleGP are one Cholesky-Banachiewicz step, forward+-- substitution against the earlier rows followed by a positive pivot. `matrix`+-- 0.3.6.4 has no incremental Cholesky API, so its `cholDecomp` would factor the+-- whole matrix again at each new point, O(m^3) work per query against O(m^2)+-- here, with m points already queried.++finite :: Double -> Bool+finite value = not (isNaN value || isInfinite value)

@@ -5,12 +5,15 @@ {-| An implementation of the Indian buffet process by [Griffiths and Ghahramani](https://papers.nips.cc/paper_files/paper/2005/file/2ef35a8b78b572a47f56846acbeef5d3-Paper.pdf). -We are using abstract types to hide the implementation details, inspired by [Exchangeable Random Processes and Data Abstraction](https://www.cs.ox.ac.uk/people/hongseok.yang/paper/pps17a.pdf). +The restaurant types hide their implementation state, inspired by [Exchangeable Random Processes and Data Abstraction](https://www.cs.ox.ac.uk/people/hongseok.yang/paper/pps17a.pdf). The `Dish` constructor exposes the column index used by feature-matrix examples.  Illustration: [Feature extraction example](https://lazyppl-team.github.io/AdditiveClusteringDemo.html).  -} -module LazyPPL.Distributions.IBP where+module LazyPPL.Distributions.IBP+  ( Restaurant, Dish(..), newCustomer, newRestaurant, ibp+  , RestaurantS, DishS, newCustomerS, newRestaurantS+  ) where  import LazyPPL import LazyPPL.Distributions@@ -31,6 +34,8 @@     return [ D k | k <- [0..(length (matrix!!i) - 1)], matrix!!i!!k ]  +-- | A restaurant of the Indian buffet process with concentration alpha. Each 'newCustomer' takes the next row+-- of its matrix, so a restaurant holds hidden state: see 'LazyPPL.runProb' for its use outside a model. newRestaurant :: Double -> Prob Restaurant newRestaurant alpha = do         r <- uniform@@ -46,9 +51,8 @@         existingDishes <- mapM (\m -> bernoulli (fromIntegral m / i)) features         let newFeatures = zipWith (\ a b -> if b then a + 1 else a) features existingDishes         nNewDishes     <- fromIntegral <$> poisson (alpha / i)-        let fixZero = if features == [] && nNewDishes == 0 then 1 else nNewDishes-        let newRow = existingDishes ++ take fixZero (repeat True)-        rest           <- matrix alpha (index + 1) (newFeatures ++ take fixZero (repeat 1))+        let newRow = existingDishes ++ take nNewDishes (repeat True)+        rest           <- matrix alpha (index + 1) (newFeatures ++ take nNewDishes (repeat 1))         return $ newRow : rest  -- the distribution on matrices @@ -57,35 +61,53 @@   -{---Another possible implementation of the indian buffet process -which uses a truncated stickbreaking construction. -It is only an approximation to the true IBP, but doesn't need IO.   --See also -Stick-breaking Construction for the Indian Buffet Process-Teh, Gorur, Ghahramani. AISTATS 2007.--A stochastic programming perspective on nonparametric Bayes-Daniel M. Roy, Vikash Mansinghka, Noah Goodman, and Joshua Tenenbaum-ICML Workshop on Nonparametric Bayesian, 2008. ---}+-- | A restaurant for the approximation 'newRestaurantS' to the Indian buffet process. It holds the+-- finite list of sticks, the probabilities with which a customer takes its dishes. newtype RestaurantS = RS [Double] +-- | A dish of a 'RestaurantS' restaurant: one dish for each stick, in the order of the sticks. newtype DishS = DS Int deriving (Eq,Ord,Show) +-- | A new customer of a 'RestaurantS' restaurant. It takes each dish independently, with the+-- probability given by the dish's stick. newCustomerS :: RestaurantS -> Prob [DishS] newCustomerS (RS rs) = do    fs <- mapM bernoulli rs   return $ map DS $ findIndices id fs +{-| An /approximation/ to 'newRestaurant': the Indian buffet process built by stick breaking and cut+short. The concentration alpha may be any finite positive number; other values raise an error. Its+law is not the law of 'newRestaurant'; the difference is bounded as follows.++The full construction draws @nu_1, nu_2, ...@ independently from @Beta(alpha, 1)@ and forms the sticks+@r_1 = nu_1@, @r_2 = nu_1 * nu_2@, and so on. A customer takes the @k@-th dish with probability @r_k@,+independently for each @k@.++This function generates the sticks in order and stops at the first one below 0.01. That stick is kept+as the last one: every stick before it is at least 0.01, and it is below 0.01. The later sticks are+not generated.++The later sticks are the error. Given the last kept stick @r@, they are @r * nu'_1@,+@r * nu'_1 * nu'_2@, and so on, where the @nu'_j@ are further independent draws from @Beta(alpha, 1)@,+whose mean is @alpha / (alpha + 1)@. Each later stick is the probability of one dish, so the expected+number of dishes that this function leaves out of one customer's row, compared with the exact process,+is the expected sum of the later sticks:++> r * (alpha / (alpha + 1)) / (1 - alpha / (alpha + 1)) = alpha * r++As @r < 0.01@, that is fewer than @0.01 * alpha@ dishes. The bound grows with @alpha@: it is below 0.1+when @alpha = 10@ and below 10 when @alpha = 1000@. Use 'newRestaurant' when that error matters.++See also /Stick-breaking Construction for the Indian Buffet Process/, Teh, Gorur and Ghahramani,+AISTATS 2007, and /A stochastic programming perspective on nonparametric Bayes/, Roy, Mansinghka,+Goodman and Tenenbaum, ICML Workshop on Nonparametric Bayesian, 2008.+-} newRestaurantS :: Double -> Prob RestaurantS-newRestaurantS a = RS <$> stickScale 1+newRestaurantS a+  | not (a > 0 && not (isInfinite a)) = error ("newRestaurantS: the concentration alpha must be finite and positive, got " ++ show a)+  | otherwise = RS <$> stickScale 1   where stickScale p = do r' <- beta a 1                           let r = p * r'                           -- Truncate when the probabilities are getting small                           rs <- if r < 0.01 then return [] else stickScale r                           return $ r : rs---

@@ -24,13 +24,15 @@  module LazyPPL.Distributions.Memoization (MonadMemo, memoize, generalmemoize, memrec) where -import Control.Monad-import Data.IORef-import Data.Map (empty, insert, keys, lookup, size)+import Data.IORef (atomicModifyIORef', newIORef)+import Data.Map (empty, insert, lookup, size)+import Data.Tuple (Solo(MkSolo)) import LazyPPL import System.IO.Unsafe -{-| Type class for memoizable argument types @a@ under a monad @m@ -}+{-| Type class for memoizable argument types @a@ under a monad @m@.++The default 'memoize', for an enumerable @a@, samples an infinite trie of results inside @m@, so it returns only in a lazy monad such as 'Prob'. A strict monad such as 'Maybe' or 'IO' would never return the memo function, so give no instance for one. -} class (Monad m) => MonadMemo m a where   memoize :: (a -> m b) -> m (a -> b)   default memoize :: (Enum a) => (a -> m b) -> m (a -> b)@@ -40,18 +42,25 @@       return $ \x -> look t (fromEnum x)  {-- Basic trie based integer-indexed memo table.-    NB Currently ignores negative integers --}+    Negative indices are rejected. --} data BinTree a = Branch a (BinTree a) (BinTree a)  ini :: (Monad m) => Int -> (Int -> m a) -> m (BinTree a) ini n f = do x <- f n; l <- ini (2 * n + 1) f; r <- ini (2 * n + 2) f; return $ Branch x l r  look :: BinTree a -> Int -> a-look (Branch x l r) 0 = x-look (Branch _ l r) n = if even n then look r (n `div` 2) else look l (n `div` 2)+look tree n+  | n < 0 = error "memoize: negative indices are not supported"+  | otherwise = follow tree (reverse (path (toInteger n + 1)))+  where+    path :: Integer -> [Bool]+    path 1 = []+    path k = odd k : path (k `div` 2)+    follow (Branch x _ _) [] = x+    follow (Branch _ l r) (goRight : rest) = follow (if goRight then r else l) rest -{-| Implementation for enumerable types using tries -}-instance (Monad m) => MonadMemo m Int+{-| Implementation for enumerable types using tries, in the lazy 'Prob' monad. Indices must be non-negative. -}+instance MonadMemo Prob Int  {-| Implementation for pair types using currying -} instance (Monad m, MonadMemo m a, MonadMemo m b) => MonadMemo m (a, b) where@@ -63,40 +72,35 @@     a table of calls that have already been made.     If @a@ is finite, we could just sample all values of @a@ in advance     and avoid unsafePerformIO. If @a@ is enumerable, we can use the trie method. ++    The returned function holds hidden state: see 'LazyPPL.runProb' for its use outside a model. -} generalmemoize :: Ord a => (a -> Prob b) -> Prob (a -> b)-generalmemoize f = Prob $ \(Tree _ gs) ->-  unsafePerformIO $ do-    ref <- newIORef Data.Map.empty-    return $ \x -> unsafePerformIO $ do-      m <- liftM (Data.Map.lookup x) (readIORef ref)-      case m of-        Just y -> return y-        Nothing -> do-          n <- readIORef ref-          let y = runProb (f x) (gs !! (1 + size n))-          modifyIORef' ref (Data.Map.insert x y)-          return y+generalmemoize f = memrec (const f) -{-- Stochastic memoization for recursive functions.+{-| Stochastic memoization for recursive functions.     Applying 'memoize' to a recursively defined function only memoizes at the     top-level: recursive calls are calls to the non-memoized function.     'memrec' is an alternative implementation which resolves recursion and     memoization at the same time, so that recursive calls are also memoized.---}-memrec :: Ord a => Show a => ((a -> b) -> (a -> Prob b)) -> Prob (a -> b)++    The returned function holds hidden state: see 'LazyPPL.runProb' for its use outside a model.+-}+memrec :: Ord a => ((a -> b) -> (a -> Prob b)) -> Prob (a -> b) memrec f =   Prob $ \(Tree _ gs) ->     unsafePerformIO $ do       ref <- newIORef Data.Map.empty+      -- One atomic step looks up the key and, for a new key, stores its result+      -- with the next unused subtree, so two threads never share a subtree or+      -- lose an entry. The result stays lazy (MkSolo keeps atomicModifyIORef'+      -- from forcing it), so a recursive call evaluates after the step.       let memoized_fixpoint = \x -> unsafePerformIO $ do-            m <- liftM (Data.Map.lookup x) (readIORef ref)-            case m of-              Just y -> return y-              Nothing -> do-                n <- readIORef ref-                let fix = f memoized_fixpoint-                let y = runProb (fix x) (gs !! (1 + size n))-                modifyIORef' ref (Data.Map.insert x y)-                return y+            MkSolo y <- atomicModifyIORef' ref $ \table ->+              case Data.Map.lookup x table of+                Just y -> (table, MkSolo y)+                Nothing ->+                  let y = runProb (f memoized_fixpoint x) (gs !! (1 + size table))+                  in (Data.Map.insert x y table, MkSolo y)+            return y       return memoized_fixpoint

@@ -0,0 +1,149 @@+{-# LANGUAGE ExtendedDefaultRules, ScopedTypeVariables, BangPatterns #-}+{-# OPTIONS_GHC -Wno-unrecognised-pragmas #-}+{-# HLINT ignore "Use newtype instead of data" #-}++module LazyPPL.Distributions.Mondrian where++import LazyPPL+import Data.IORef+-- import Debug.Trace+import LazyPPL.Distributions+import LazyPPL.Distributions.Counter+import System.IO.Unsafe+import Data.Map (Map,empty,lookup,insert,size,keys,findWithDefault,fromList,(!))+-- import Text.RawString.QQ+-- import qualified Data.List+-- import Control.DeepSeq+++-- | From Roy-Teh "The Mondrian Process", NIPS 2009++-- | 'oneDimMondrian' corresponds to a Poisson point process on [low, high] with uniform intensity+oneDimMondrian :: Double -> (Double, Double) -> Prob [Double]+oneDimMondrian budget (low, high) = do+  cutCost <- exponential (high - low)+  if budget < cutCost+    then return []+    else do+      let remaining = budget - cutCost+      cut <- uniformbounded low high+      leftcuts <- oneDimMondrian remaining (low, cut)+      rightcuts <- oneDimMondrian remaining (cut, high)+      return $ leftcuts ++ [cut] ++ rightcuts++-- | Some data types. Intuitively, Matrix is a type of 2D exchangeable arrays, and 'Mondrian Double' would be the type of block-structured graphons.++data Mondrian a+  = Block a [(Double, Double)] -- ^ args:  and intervals making up the block+  | Partition Int Double [(Double, Double)] (Mondrian a) (Mondrian a) -- ^ args: dimension cutPosition intervals (hypercube over which the Mondrian is defined) subtree1 +  deriving (Eq, Show)++newtype Row = Row Int -- ^ in [0, 1]++newtype Col = Col Int++data MatrixIO = MatrixIO Counter Counter [[Bool]]++lookup :: MatrixIO -> Row -> Col -> Bool+lookup (MatrixIO _ _ matrix) (Row r) (Col c) = matrix !! r !! c++newRow :: MatrixIO -> Prob Row+newRow (MatrixIO c _ _) = readAndIncrement c >>= (return . Row)++newCol :: MatrixIO -> Prob Col+newCol (MatrixIO _ c _) = readAndIncrement c >>= (return . Col)++data Matrix = Matrix [[Bool]]++{- | Generates a random Mondrian tree: a random block partition of the rectangle [a1, b1] × ⋯ × [an, bn]+where each block has an associated random draw from the base distribution.+-}+randomMondrian :: Prob a -> Double -> [(Double, Double)] -> Prob (Mondrian a)+randomMondrian base budget abs = do+  let lengths = map (\(a, b) -> b - a) abs+  let sumLengths = sum lengths+  cutCost <- exponential sumLengths+  if budget < cutCost+    then do p <- base; return $ Block p abs+    else do+      let remaining = budget - cutCost+      dim <- categorical $ map (/sumLengths) lengths -- if dim is true then cut is perpendicular to (a_d, b_d)+      let (a_d, b_d) = abs !! dim+      cut <- uniformbounded a_d b_d+      leftMondrian <- randomMondrian base remaining+        $ zipWith (\ab i -> if i == dim then (a_d, cut) else ab) abs [0 ..]+      rightMondrian <- randomMondrian base remaining+        $ zipWith (\ab i -> if i == dim then (cut, b_d) else ab) abs [0 ..]+      return $ Partition dim cut abs leftMondrian rightMondrian++-- | Given a Mondrian tree and two data points, apply a function on the biais of the corresponding block. +applyOnBiaisFromMondrian2D :: Mondrian Double -> Double -> Double -> (Double -> a) -> a+applyOnBiaisFromMondrian2D mondrian x y f = do+  case mondrian of+    Block p _ -> f p+    Partition !dim !cut _ left right ->+      let !xy = case dim of {0 -> x; _ -> y}+          !cond = xy < cut+          !lr = if cond then left else right in+      applyOnBiaisFromMondrian2D lr x y f++-- | Given a Mondrian tree and two data points, sample an 'edge'.+sampleFromMondrian2D :: Mondrian Double -> Double -> Double -> Prob Bool+sampleFromMondrian2D mondrian x y = applyOnBiaisFromMondrian2D mondrian x y bernoulli++-- | Probability of getting a given truth value 'val' at a (r, c) pair from a Mondrian. +likelihoodFromMondrian2D :: Mondrian Double -> Double -> Double -> Bool -> Double+likelihoodFromMondrian2D mondrian x y val =+  let !f = if val then id else (1-) in+    applyOnBiaisFromMondrian2D mondrian x y f++{- | Sample a block structure over [0, 1]^2 from the Mondrian process,+and use it to generate an infinite exchangeable array.+-}+sampleRelationFromRandomMondrian2D :: Prob Double -> Double -> Prob Matrix+sampleRelationFromRandomMondrian2D base budget = do+  mondrian <- randomMondrian base budget [(0, 1), (0, 1)]+  sampleRelationFromMondrian2D mondrian++sampleRelationFromMondrian2D :: Mondrian Double -> Prob Matrix+sampleRelationFromMondrian2D mondrian = do+  rs <- iid uniform+  cs <- iid uniform+  matrix <- mapM (\r -> mapM (sampleFromMondrian2D mondrian r) cs) rs+  return $ Matrix matrix++sampleMapRelationFromMondrian2D :: Mondrian Double -> Int -> Prob (Map (Double, Double) Bool)+sampleMapRelationFromMondrian2D mondrian size = do+  xs <- iid uniform+  ys <- iid uniform+  matrix <- mapM (\x ->+    mapM (\y -> (do b <- sampleFromMondrian2D mondrian x y; return ((x, y), b))) (take size xs))+    (take size ys)+  return $ fromList $ concat matrix++sampleMatrixRelationFromMondrian2D :: Mondrian Double -> Prob Matrix+sampleMatrixRelationFromMondrian2D mondrian = do+  rs <- iid uniform+  cs <- iid uniform+  matrix <- mapM (\r ->+    mapM (sampleFromMondrian2D mondrian r) cs) rs+  return $ Matrix matrix+++data BinTree a = Leaf | Node a (BinTree a) (BinTree a)+  deriving (Eq, Show)++{- | Create a Mondrian tree over a hypercube `abs` from a tree of (dimension, cut) given in the chronological order, and a map associating to each block its bernoulli biais.+-}+mondrianFromCuts :: [(Double, Double)] -> BinTree (Int, Double) -> Map [(Double, Double)] Double -> Double -> Mondrian Double+mondrianFromCuts abs dcs biais defaultBiais =+  case dcs of+    Leaf -> Block (findWithDefault defaultBiais abs biais) abs+    Node (dim, cut) left right ->+      Partition dim cut abs+      (mondrianFromCuts leftAbs left biais defaultBiais)+      (mondrianFromCuts rightAbs right biais defaultBiais)+      where+        leftAbs = zipWith (\ab i -> if i == dim then (a0, cut) else ab) abs [0 ..]+        rightAbs = zipWith (\ab i -> if i == dim then (cut, b0) else ab) abs [0 ..]+        (a0, b0) = abs !! dim

@@ -0,0 +1,89 @@+{-# LANGUAGE BangPatterns #-}+{-# OPTIONS_GHC -fno-cse -fno-full-laziness #-}++module LazyPPL.Internal.History+  (History, newHistory, appendMutation, labelAt) where++import Control.Concurrent.MVar+import Control.Monad (unless)+import Data.List.NonEmpty (NonEmpty(..))+import qualified Data.Vector.Unboxed.Mutable as Vector+import Data.Word (Word64)+import System.IO.Unsafe (unsafePerformIO)+import LazyPPL.Internal.Sampler (randomLabel)+import System.Random (SeedGen(fromSeed64, toSeed64), StdGen, split)++-- A record holds a StdGen as the two Word64 values (16 bytes) of its public+-- SeedGen instance in `random` 1.3. Stored unboxed, the records contain no+-- pointers, so the garbage collector skips them, and the `vector` package does+-- the allocation and copying.+data Cursor = Cursor !(Vector.IOVector (Word64,Word64)) !Int+data History = History !StdGen !(MVar (Maybe Cursor))++newHistory :: StdGen -> IO History+newHistory seed = History seed <$> newMVar Nothing++appendMutation :: History -> Int -> StdGen -> IO ()+appendMutation (History _ lock) index generator =+  modifyMVar_ lock $ \old -> do+    Cursor records count <- case old of+      Nothing -> do+        unless (index == 0) (ioError (userError "missing MH history"))+        records <- Vector.new 1+        pure (Cursor records 0)+      Just cursor -> pure cursor+    unless (count == index) (ioError (userError "out-of-order MH history"))+    unless (index < (maxBound :: Int))+      (ioError (userError "MH history exceeds addressable record count"))+    let capacity = Vector.length records+    grown <- if index < capacity+      then pure records+      -- Grow by doubling, without overflowing Int.+      else Vector.grow records (min capacity (maxBound-capacity))+    case toSeed64 generator of+      seed :| [gamma] -> Vector.write grown index (seed,gamma)+      _ -> ioError (userError "StdGen seed is not 128 bits")+    pure (Just (Cursor grown (index+1)))++-- `labelAt p history count path` is the label at `path` (deepest index first)+-- after the first `count` mutations, where a mutation replaces each label+-- independently with probability `p`.+--+-- readMVar waits for an append in progress, so those `count` records are+-- initialized. Later appends write only after them, and growth copies them to+-- a new vector. A reader's vector and count therefore stay valid.+{-# NOINLINE labelAt #-}+labelAt :: Double -> History -> Int -> [Int] -> Double+labelAt p (History initial lock) count reversedPath = unsafePerformIO $+  readMVar lock >>= \cursor -> do+    let path = reverse reversedPath+        original = fst (randomLabel (atPath 1 path initial))+    if count == 0 || not (p > 0)+      then pure original+      else case cursor of+        Nothing -> ioError (userError "missing MH checkpoint records")+        Just (Cursor records written) -> do+          unless (count <= written) (ioError (userError "incomplete MH checkpoint"))+          let search :: Int -> IO Double+              search !index+                | index < 0 = pure original+                | otherwise = do+                    (seed,gamma) <- Vector.read records index+                    let generator = atPath 2 path (fromSeed64 (seed :| [gamma]))+                        (decision, next) = randomLabel generator+                    if decision < p+                      then pure (fst (randomLabel next))+                      else search (index-1)+          value <- search (count-1)+          value `seq` pure value++-- At each node, randomTree draws once before splitting its seed and mutateTree+-- draws twice.+atPath :: Int -> [Int] -> StdGen -> StdGen+atPath _ [] generator = generator+atPath draws (index:path) generator = atPath draws path (child index (advance draws generator))+  where+    advance 0 g = g+    advance n g = advance (n-1) (snd (randomLabel g))+    child 0 g = fst (split g)+    child n g = child (n-1) (snd (split g))

@@ -0,0 +1,32 @@+{- | Rules that every sampler of the random tree shares: how a label is drawn,+and which weights inference accepts. This module is internal. -}+module LazyPPL.Internal.Sampler (randomLabel, checkedLogWeight) where++import Data.Bits (complement, shiftR)+import Numeric.Log (Log, ln)+import System.Random (RandomGen, genWord64)++{- | A label of the random tree, uniform on the open interval (0,1).++@random@ from random 1.3.1 gives a 'Double' in the closed interval [0,1], and+an inverse transform fails at an end point: @-log 0@ is infinite, and a+Bernoulli(1) coin compares a label of 1 with 1. With k the 52 high bits of+the complement of one 'Data.Word.Word64', this gives @(k + 1/2) / 2^52@, in+@[2^-53, 1 - 2^-53]@. It reads one word, as @random@ does, so every node of a+tree gets the same generator as before; both labels decrease as the word+grows, and they differ by at most @2^-52@. -}+randomLabel :: RandomGen g => g -> (Double, g)+randomLabel g =+  let (w, g') = genWord64 g+  in (encodeFloat (2 * toInteger (complement w `shiftR` 12) + 1) (-53), g')++{- | A weight is a non-negative real number, kept as its logarithm so that it+may exceed the range of 'Double': the logarithm is finite, or negative+infinity for weight 0. A NaN or positive-infinite logarithm raises an error+that names the caller. -}+checkedLogWeight :: String -> Log Double -> Log Double+checkedLogWeight caller w+  | isNaN l || (l > 0 && isInfinite l) =+      error (caller ++ ": the weight must be non-negative and finite, but its logarithm is " ++ show l)+  | otherwise = w+  where l = ln w

@@ -0,0 +1,91 @@+{-# OPTIONS_GHC -fno-cse -fno-full-laziness #-}+module Main where++import LazyPPL (Meas(..), Prob, Tree(..), mh, mhirreducible, randomTree, runProb, sample, score, scoreLog, scoreProductLog, uniform, weightedsamples, wis)+import StrictMemo (strictMemoChecks)+import LazyPPL.Distributions+import LazyPPL.Distributions.GP (gp, wiener)+import LazyPPL.Distributions.IBP (ibp)+import LazyPPL.Distributions.Memoization (memoize, memrec)+import System.Environment (getArgs, getExecutablePath)+import System.Exit (exitFailure, exitSuccess)++import Api.Support+import Api.Domains+import Api.Distributions+import Api.Processes+import Api.Inference++-- Accuracy of samplers, densities and processes at extreme parameters.+accuracyChecks :: IO [Bool]+accuracyChecks = concat <$> sequence [dirichletSamplingChecks, lawChecks, gridLawChecks, betaQuantileChecks False, dirichletStickChecks, densityChecks, wienerBridgeChecks, numericChecks, processChecks, strictMemoChecks]++-- Sampler and density domains: tree labels, log weights, chances, distribution+-- parameters and the concentrations of the nonparametric priors.+domainChecks :: IO [Bool]+domainChecks = concat <$> sequence [endpointChecks, logWeightChecks, chanceChecks, parameterChecks, betaLabelChecks, concentrationChecks]++main :: IO ()+main = do+  args <- getArgs+  case args of+    ["categorical-boundary"] -> categoricalBoundaryChecks >>= finish+    ["categorical-properties"] -> categoricalPropertyChecks >>= finish+    ["dirichlet-stick-rounding"] -> dirichletStickRoundingChecks >>= finish+    ["poisson-boundary"] -> poissonBoundaryChecks >>= finish+    ["memo"] -> memoStochasticChecks >>= finish+    ["mondrian"] -> mondrianChecks >>= finish+    ["zero-weight"] -> zeroAcceptanceChecks >>= finish+    ["domains"] -> domainChecks >>= finish+    ["accuracy"] -> accuracyChecks >>= finish+    ["laws"] -> concat <$> sequence [dirichletSamplingChecks, lawChecks, gridLawChecks, betaQuantileChecks False, dirichletStickChecks] >>= finish+    ["quantiles"] -> betaQuantileChecks False >>= finish+    ["quantiles-inside"] -> betaQuantileChecks True >>= finish+    ["sticks"] -> dirichletStickChecks >>= finish+    ["grid-laws"] -> gridLawChecks >>= finish+    ["densities"] -> densityChecks >>= finish+    ["density-rows"] -> densityRowsChild+    ["wiener"] -> concat <$> sequence [wienerBridgeChecks, processChecks] >>= finish+    ["posterior"] -> posteriorChecks >>= finish+    _ -> do+      additional <- concat <$> sequence [categoricalBoundaryChecks, categoricalPropertyChecks, dirichletStickRoundingChecks, poissonBoundaryChecks, densityBoundaryChecks, lazyCategoricalChecks, memoStochasticChecks, mondrianChecks, recursiveMemoCheck, dirichletProcessChecks, zeroAcceptanceChecks, domainChecks, accuracyChecks, posteriorChecks]+      if args == ["additional"] then finish additional else existingChecks additional++finish :: [Bool] -> IO ()+finish results = do+  let passed = length (filter id results)+      total = length results+  putStrLn $ show passed ++ "/" ++ show total ++ " checks passed."+  if passed == total then exitSuccess else exitFailure++existingChecks :: [Bool] -> IO ()+existingChecks additional = do+  results <-+    sequence+      [ approxEq "normalPdf 0 1 0" 0.3989422804 (normalPdf 0 1 0) 1e-9+      , approxEq "normalPdf 0 1 1" 0.2419707245 (normalPdf 0 1 1) 1e-9+      , approxEq "expPdf 2 0.5" (2 * exp (-1)) (expPdf 2 0.5) 1e-9+      , approxEq "poissonPdf 3 2" (4.5 * exp (-3)) (poissonPdf 3 2) 1e-9+      , approxEq "poissonPdf 1 0" (exp (-1)) (poissonPdf 1 0) 1e-9+      , -- Six standard errors of the sampling distribution of the mean of 5000 draws.+        approxEq "normal 3 2 sample mean" 3 (fst normalSampleMoments) (6 * 2 / sqrt 5000)+      , -- Six standard deviations of the sampling distribution of the variance.+        approxEq "normal 3 2 sample variance" 4 (snd normalSampleMoments) (6 * sqrt (2 * 4 * 4 / 4999))+      , assertBool "mh samples stay in [0,1]" =<< smokeMhWithinUnitInterval+      , assertBool "mhirreducible samples stay in [0,1]" =<< smokeMhIrreducibleWithinUnitInterval+      , assertBool "wis samples stay in [0,1]" =<< smokeWisWithinUnitInterval+      , (\n -> approxEq "wis 4 resamples from all four samples" 4 n 0.5) =<< wisDistinctResamples+      , approxEq "memoize returns f n at n for n in 0..40 (mismatches)" 0 memoizeMismatches 0.5+      , approxEq "gp with cov 0 0 = 1 has sd 1 at 0" 1 gpStandardDeviationAtZero 0.15+      , (\d -> approxEq "gp constant covariance 1e8: gap within six nugget standard deviations" 0 d (6*sqrt (2*2**(-33)))) =<< gpConstantKernelGap 1e8+      , (\d -> approxEq "gp constant covariance 1e20: gap within six nugget standard deviations" 0 d (6*sqrt (2*2**(-33)))) =<< gpConstantKernelGap 1e20+      , (\d -> approxEq "gp with an RBF kernel scaled by 1e20 stays finite at 31 points" 0 d 0.5) =<< gpScaledKernelFinite 1e20+      , (\d -> approxEq "gp rank-two covariance 1e16: line residual within six nugget standard deviations" 0 d (6*sqrt (92*2**(-33)))) =<< gpRankTwoLineResidual 1e16+      , (\d -> approxEq "gp rank-two covariance 1e100: line residual within six nugget standard deviations" 0 d (6*sqrt (92*2**(-33)))) =<< gpRankTwoLineResidual 1e100+      , (\d -> approxEq "gp with covariance 1e100 cos (x - y) stays finite at 0..5" 0 d 0.5) =<< gpCosineKernelFinite 1e100+      , (\d -> approxEq "gp scale equivariance for four kernels at 1e-300 to 1e300" 0 d 1e-6) =<< gpScaleEquivarianceError+      , (\d -> assertBool "gp admits the smallest positive first-point variance" (d > 0 && not (isNaN d || isInfinite d))) =<< gpSmallestVarianceSample+      , approxEq "ibp 1 first row empty with probability exp (-1)" (exp (-1)) ibpFirstRowEmptyFraction 0.05+      , assertBool "gp samples stay finite" =<< smokeGpFinite+      ]+  finish (additional ++ results)

@@ -0,0 +1,316 @@+{-# OPTIONS_GHC -fno-cse -fno-full-laziness #-}+module Api.Distributions where++import Control.Exception (ErrorCall, SomeException, evaluate, try)+import Data.List (isInfixOf, isPrefixOf, nub, sort, tails)+import LazyPPL (Meas(..), Prob, Tree(..), mh, mhirreducible, randomTree, runProb, sample, score, scoreLog, scoreProductLog, uniform, weightedsamples, wis)+import Numeric.Log (Log(..), ln)+import Numeric (expm1, log1p)+import System.Timeout (timeout)+import LazyPPL.Distributions+import qualified LazyPPL.Distributions.IBP as IBP+import System.Environment (getArgs, getExecutablePath)+import System.Process.Typed (proc, readProcessStdout)+import qualified Data.ByteString.Lazy.Char8 as BL+import System.Random (RandomGen(..), SplitGen(..), mkStdGen, setStdGen)+import Control.Monad (replicateM)++import Api.Support++-- 5000 draws of normal 3 2 on a fixed seed, so the sampler is checked, not just+-- the density.+normalSampleMoments :: (Double, Double)+normalSampleMoments =+  let draws = runProb (replicateM 5000 (normal 3 2)) (randomTree (mkStdGen 7))+      n = fromIntegral (length draws) :: Double+      mean = sum draws / n+      variance = sum [(x - mean) ^ (2 :: Int) | x <- draws] / (n - 1)+  in (mean, variance)++-- Draws on the simplex. With every label 1/4, normalised Gamma draws underflow+-- to 0 for small concentrations; a draw must still be a point of the simplex.+dirichletSamplingChecks :: IO [Bool]+dirichletSamplingChecks = do+  let quarter = let t = Tree 0.25 (repeat t) in t+      sparse = runProb (dirichlet [0.001, 0.001]) quarter+      single = runProb (dirichlet [0.001]) quarter+      onSimplex xs = all (\x -> x >= 0 && not (isNaN x || isInfinite x)) xs+        && abs (sum xs - 1) <= fromIntegral (length xs) * 2.220446049250313e-16+  points <- sequence+    [ assertBool ("dirichlet [0.001,0.001] at labels 1/4 is on the simplex, got " ++ show sparse) (onSimplex sparse)+    , assertBool ("dirichlet [0.001] is exactly [1], got " ++ show single) (single == [1])+    ]+  moments <- concat <$> mapM ensemble [[0.5, 1.5, 2], [2, 1.5, 0.5], [0.05, 0.05, 0.1], [0.001, 0.001, 0.002]]+  pure (points ++ moments)+  where+    -- 20000 seeded draws. The means, the variances and the variance of x0 + x1,+    -- whose law is Beta(a0 + a1, a2), lie within six standard errors of their+    -- exact values. For a variable in [0,1], E[X^2] bounds n times the variance+    -- of its sample variance.+    ensemble alphas = do+      let n = 20000 :: Int+          draws = runProb (replicateM n (dirichlet alphas)) (randomTree (mkStdGen 11))+          total = sum alphas+          mean xs = sum xs / fromIntegral n+          variance xs = let m = mean xs in sum [(x - m) ^ (2 :: Int) | x <- xs] / fromIntegral (n - 1)+          betaVariance a b = a * b / ((a + b) ^ (2 :: Int) * (a + b + 1))+          coordinate i = map (!! i) draws+          pairSum = map (\d -> d !! 0 + d !! 1) draws+          varianceError xs = 6 * sqrt (mean (map (^ (2 :: Int)) xs) / fromIntegral n)+          name = "dirichlet " ++ show alphas+      sequence $+        [ approxEq (name ++ " mean of x" ++ show i) (a / total) (mean (coordinate i))+            (6 * sqrt (betaVariance a (total - a) / fromIntegral n))+        | (i, a) <- zip [0 :: Int ..] alphas ] +++        [ approxEq (name ++ " variance of x" ++ show i) (betaVariance a (total - a)) (variance (coordinate i))+            (varianceError (coordinate i))+        | (i, a) <- zip [0 :: Int ..] alphas ] +++        [ approxEq (name ++ " variance of x0 + x1") (betaVariance (alphas !! 0 + alphas !! 1) (alphas !! 2))+            (variance pairSum) (varianceError pairSum) ]++-- A distribution function on [0,1], extended by 0 below and 1 above.+onUnit :: (Double -> Double) -> Double -> Double+onUnit f x = f (max 0 (min 1 x))++-- Beta(1,b) has the distribution function 1 - (1-x)^b and the quantile+-- 1 - (1-u)^(1/b); Beta(a,1) has x^a and u^(1/a).+beta1Cdf, betaA1Cdf :: Double -> Double -> Double+beta1Cdf b = onUnit (\x -> - expm1 (b * log1p (- x)))+betaA1Cdf a = onUnit (\x -> exp (a * log x))++-- Laws of the Beta and Dirichlet samplers. Each check draws n = 10000 values on+-- one seeded tree and computes ksDistance from an exact distribution function F.+-- For n independent draws from F, the Dvoretzky-Kiefer-Wolfowitz inequality with+-- Massart's constant gives P(sup |F_n - F| > e) <= 2 exp(-2 n e^2): at+-- e = 0.0269 below 1e-6, and for draws from a law whose distribution function+-- differs from F by at least 0.0269 + 0.0195 = 0.0464 at a point, a distance+-- above e with probability at least 0.999. These figures hold for ideal+-- independent draws; the draws here come from one fixed seed of a pseudorandom+-- generator. ksDistance compares each draw with F at the draw and at the double+-- before it, so a difference between two laws inside that interval is not+-- measured.+lawChecks :: IO [Bool]+lawChecks = do+  let n = 10000 :: Int+      threshold = sqrt (log (2 / 1e-6) / (2 * fromIntegral n))+      law name seed cdf p = do+        let d = ksDistance cdf (runProb (replicateM n p) (randomTree (mkStdGen seed)))+        assertBool (name ++ ": distance " ++ show d ++ " from the exact distribution function (threshold " ++ show threshold ++ ")")+          (d <= threshold)+      coordinate i alphas = (!! i) <$> dirichlet alphas+  sequence+    [ law "beta 1 0.001" 21 (beta1Cdf 0.001) (beta 1 0.001)+    , law "beta 1 0.1" 22 (beta1Cdf 0.1) (beta 1 0.1)+    , law "beta 0.001 1" 23 (betaA1Cdf 0.001) (beta 0.001 1)+    , law "beta 0.5 0.5" 24 (onUnit (\x -> 2 / pi * asin (sqrt x))) (beta 0.5 0.5)+    , law "beta 2 3" 25 (onUnit (\x -> x * x * (6 - 8 * x + 3 * x * x))) (beta 2 3)+      -- The coordinate of concentration 1 next to 0.001 is Beta(1, 0.001), in either order.+    , law "dirichlet [1,0.001] x0" 26 (beta1Cdf 0.001) (coordinate 0 [1, 0.001])+    , law "dirichlet [0.001,1] x1" 27 (beta1Cdf 0.001) (coordinate 1 [0.001, 1])+      -- x_i is Beta(a_i, a - a_i) with a the total; Beta(2,2) has 3x^2 - 2x^3.+    , law "dirichlet [0.5,1.5,2] x2" 28 (onUnit (\x -> x * x * (3 - 2 * x))) (coordinate 2 [0.5, 1.5, 2])+    , law "dirichlet [1,0.01,0.01] x0" 29 (beta1Cdf 0.02) (coordinate 0 [1, 0.01, 0.01])+    ]++-- The documented domains (LazyPPL.Distributions): dirichlet takes concentrations+-- of at least 0.001 each with a sum of at most 2000; beta takes the same+-- parameters, and Beta(a,1) and Beta(1,b) for every finite positive a and b.+dirichletInDomain, betaInDomain :: Double -> Double -> Bool+dirichletInDomain a b = a >= 0.001 && b >= 0.001 && a + b <= 2000+betaInDomain a b = dirichletInDomain a b || (a == 1 && positive b) || (b == 1 && positive a)+  where positive c = c > 0 && not (isInfinite c)++-- The Beta sampler on constant trees against test/reference/beta-quantiles.tsv.+-- Inside the documented domain of each, beta a b and the two coordinates of+-- dirichlet [a,b] at a supported label u lie in the row's intervals: the doubles+-- whose distribution function differs from u by at most 1e-9 min(u, 1 - u),+-- joined with the two doubles next to the exact quantile. A row marked reject+-- has no intervals and checks both samplers' errors instead. Outside the shape+-- domain or below 2^-53 for general shapes, each raises an error. With+-- insideOnly, only the rows inside beta's shape domain are read; the+-- samplers of an earlier version, which have no domain, can take minutes+-- outside it.+betaQuantileChecks :: Bool -> IO [Bool]+betaQuantileChecks insideOnly = do+  rows <- referenceTable "beta-quantiles.tsv"+  results <- mapM check [r | r@(a : b : _) <- rows, not insideOnly || betaInDomain (read a) (read b)]+  tableReport "beta and dirichlet [a,b] in the quantile intervals of beta-quantiles.tsv" results+  where+    check [a', b', u', "reject"] = do+      let (a, b, u) = (read a', read b', read u') :: (Double, Double, Double)+          tree = constantLabel u+          unsupportedLabel = u < encodeFloat 1 (-53) && a /= 1 && b /= 1+          betaSupportedHere = betaInDomain a b && not unsupportedLabel+          dirichletSupportedHere = dirichletInDomain a b && not unsupportedLabel+      single <- if betaInDomain a b+        then raisesBetaLabel (runProb (beta a b) tree)+        else raisesError (runProb (beta a b) tree)+      pair <- if dirichletInDomain a b+        then raisesBetaLabel (sum (runProb (dirichlet [a,b]) tree))+        else raisesError (sum (runProb (dirichlet [a,b]) tree))+      pure (not betaSupportedHere && not dirichletSupportedHere && single && pair,+            "beta and dirichlet must reject " ++ show (a,b,u))+    check (a' : b' : u' : xlo' : xhi' : ylo' : yhi' : _) = do+      let (a, b, u, xlo, xhi, ylo, yhi) = (read a', read b', read u', read xlo', read xhi', read ylo', read yhi') :: (Double, Double, Double, Double, Double, Double, Double)+          tree = constantLabel u+          name = "beta " ++ show a ++ " " ++ show b ++ " at label " ++ show u+          inside lo hi v = lo <= v && v <= hi+          answer v = maybe "an error or no answer in 2 s" show v+          intervals = "x in [" ++ show xlo ++ "," ++ show xhi ++ "] and 1-x in [" ++ show ylo ++ "," ++ show yhi ++ "]"+      let unsupportedLabel = u < encodeFloat 1 (-53) && a /= 1 && b /= 1+      single <- if betaInDomain a b && unsupportedLabel+        then (\e -> (e, "general beta rejects the custom label below 2^-53")) <$> raisesBetaLabel (runProb (beta a b) tree)+        else if betaInDomain a b+        then do+          x <- within 2000000 (runProb (beta a b) tree)+          pure (maybe False (inside xlo xhi) x, "beta: " ++ intervals ++ ", got " ++ answer x)+        else (\e -> (e, "beta is outside its domain and must raise an error")) <$> raisesError (runProb (beta a b) tree)+      pair <- if dirichletInDomain a b && unsupportedLabel+        then (\e -> (e, "dirichlet rejects the custom label below 2^-53")) <$> raisesBetaLabel (sum (runProb (dirichlet [a, b]) tree))+        else if dirichletInDomain a b+        then do+          xs <- within 2000000 (let xs = runProb (dirichlet [a, b]) tree in sum xs `seq` xs)+          pure (case xs of+                  Just [v, w] -> inside xlo xhi v && inside ylo yhi w+                  _ -> False, "dirichlet: " ++ intervals ++ ", got " ++ answer xs)+        else (\e -> (e, "dirichlet is outside its domain and must raise an error")) <$> raisesError (sum (runProb (dirichlet [a, b]) tree))+      pure (fst single && fst pair, name ++ ": " ++ snd single ++ "; " ++ snd pair)+    check row = pure (False, "malformed row " ++ show row)++-- dirichlet as at the constant label u against the exact stick-breaking+-- coordinates of test/reference/dirichlet-sticks.tsv. Coordinate i is the+-- remaining stick times the u-quantile of Beta(a_i, a_(i+1) + ... + a_k), and+-- its interval adds the relative tolerances of the shares that form it.+dirichletStickChecks :: IO [Bool]+dirichletStickChecks = do+  rows <- referenceTable "dirichlet-sticks.tsv"+  results <- mapM check rows+  tableReport "dirichlet at the exact stick-breaking coordinates of dirichlet-sticks.tsv" results+  where+    check (alphas' : u' : bounds') = do+      let alphas = read ("[" ++ alphas' ++ "]") :: [Double]+          u = read u' :: Double+          bounds = intervals (map read bounds')+          unsupportedLabel = u < encodeFloat 1 (-53) && any (\(a, rest) -> a /= 1 && foldr (+) 0 rest /= 1)+            [(a, rest) | (a : rest) <- tails alphas, not (null rest)]+      if unsupportedLabel+        then do+          rejected <- raisesBetaLabel (sum (runProb (dirichlet alphas) (constantLabel u)))+          pure (rejected, "dirichlet " ++ show alphas ++ " must reject label " ++ show u)+        else do+          xs <- within 2000000 (let v = runProb (dirichlet alphas) (constantLabel u) in sum v `seq` v)+          let ok = maybe False (\v -> length v == length bounds && and (zipWith (\c (lo, hi) -> lo <= c && c <= hi) v bounds)) xs+          pure (ok, "dirichlet " ++ show alphas ++ " at label " ++ show u ++ ": coordinates in " ++ show bounds+                      ++ ", got " ++ maybe "an error or no answer in 2 s" show xs)+    check row = pure (False, "malformed row " ++ show row)+    intervals (lo : hi : rest) = (lo, hi) : intervals rest+    intervals _ = [] :: [(Double, Double)]++-- For sorted xs and sorted points, how many xs are at most each point.+atOrBelow :: [Double] -> [Double] -> [Int]+atOrBelow = go 0+  where+    go _ _ [] = []+    go k xs (p : ps) = let (le, rest) = span (<= p) xs; k' = k + length le in k' : go k' rest ps++-- Laws of the Beta sampler: 10000 seeded draws against the exact distribution+-- function of test/reference/beta-laws.tsv at its grid of points, for two shapes+-- without a closed form and for Beta(1,b) and Beta(a,1) at 1e-4 and 1e4, the+-- sticks of the Dirichlet process and of IBP's newRestaurantS. The largest distance over the grid is at most the Kolmogorov-Smirnov+-- statistic, so the threshold of lawChecks keeps its false-alarm bound; the+-- grid's points are close enough that the distribution function rises by less+-- than 0.005 between neighbours.+gridLawChecks :: IO [Bool]+gridLawChecks = do+  rows <- referenceTable "beta-laws.tsv"+  let grid = [((read a, read b), (read x, read f)) | [a, b, x, f] <- rows] :: [((Double, Double), (Double, Double))]+      n = 10000 :: Int+      threshold = sqrt (log (2 / 1e-6) / (2 * fromIntegral n))+      shapes = [(2.5, 3.5), (1000, 1000), (1, 1e-4), (1, 1e4), (1e-4, 1), (1e4, 1)]+  coverage <- assertBool "beta-laws.tsv covers both general shapes and all four nonparametric stick laws"+                (sort (nub (map fst grid)) == sort shapes)+  results <- sequence+    [ do let points = [p | (s, p) <- grid, s == (a, b)]+             draws = sort (runProb (replicateM n (beta a b)) (randomTree (mkStdGen seed)))+             d = maximum (zipWith (\k (_, f) -> abs (fromIntegral k / fromIntegral n - f)) (atOrBelow draws (map fst points)) points)+         result <- try (evaluate d)+         case result of+           Right distance -> assertBool ("beta " ++ show a ++ " " ++ show b ++ ": distance " ++ show distance ++ " from the exact distribution function at "+                                            ++ show (length points) ++ " points (threshold " ++ show threshold ++ ")") (distance <= threshold)+           Left err -> assertBool ("beta " ++ show a ++ " " ++ show b ++ ": " ++ takeWhile (/= '\n') (show (err :: ErrorCall))) False+    | (seed, (a, b)) <- zip [41 ..] (nub (map fst grid)) ]+  pure (coverage : results)++-- One row of test/reference/log-densities.tsv: the function at the arguments is+-- within 1e-12 max(1, |exact|) of the exact value, or is the same infinity where+-- the exact value leaves the double range.+densityRow :: [String] -> (Bool, String)+densityRow [function, arguments, exact'] =+  let expected = read exact' :: Double+      actual = case (function, words arguments) of+        ("poissonLogPmf", [l, k]) -> poissonLogPmf (read l) (read k)+        ("gammaLogPdf", [k, t, x]) -> gammaLogPdf (read k) (read t) (read x)+        ("betaLogPdf", [a, b, x]) -> betaLogPdf (read a) (read b) (read x)+        ("binomialLogPmf", [n, p, k]) -> binomialLogPmf (read n) (read p) (read k)+        ("dirichletLogPdf", [as, xs]) -> dirichletLogPdf (read as) (read xs)+        _ -> error ("log-densities.tsv: unknown row " ++ function ++ " " ++ arguments)+      ok | isInfinite expected = actual == expected+         | otherwise = abs (actual - expected) <= 1e-12 * max 1 (abs expected)+  in (ok, function ++ " " ++ arguments ++ " is " ++ show expected ++ ", got " ++ show actual)+densityRow row = (False, "malformed row " ++ show row)++-- The rows of test/reference/log-densities.tsv run in a child process, this+-- program with the argument "density-rows", which prints one verdict per row.+-- A computation that loops without allocating cannot be interrupted inside its+-- own process (poissonLogPmf 1e308 (10^308) once did), so the deadline is kept+-- from outside: a child that has not exited within 120 s is terminated, and+-- every row without a verdict fails.+densityChecks :: IO [Bool]+densityChecks = do+  rows <- referenceTable "log-densities.tsv"+  out <- externally 120000000 ["density-rows"]+  let reported = [l | l <- maybe [] lines out, take 4 l `elem` ["PASS", "FAIL"]]+      missing = length rows - length reported+  mapM_ putStrLn [l | l <- reported, take 4 l == "FAIL"]+  putStrLn ((if missing == 0 && all ((== "PASS") . take 4) reported then "PASS " else "FAIL ")+            ++ "log densities of log-densities.tsv: " ++ show (length (filter ((== "PASS") . take 4) reported))+            ++ " of " ++ show (length rows) ++ " rows"+            ++ (if missing > 0 then ", " ++ show missing ++ " without a verdict" ++ maybe " (no exit within 120 s)" (const "") out else ""))+  pure (map ((== "PASS") . take 4) reported ++ replicate missing False)++densityRowsChild :: IO ()+densityRowsChild = do+  rows <- referenceTable "log-densities.tsv"+  mapM_ (\row -> let (ok, detail) = densityRow row in putStrLn ((if ok then "PASS " else "FAIL ") ++ detail)) rows++-- Runs this test program with the given arguments and returns its standard+-- output, or Nothing when it has not exited within the given microseconds, in+-- which case it is terminated. typed-process reads the output while the child+-- runs, so a full pipe cannot stop it.+externally :: Int -> [String] -> IO (Maybe String)+externally micros args = do+  self <- getExecutablePath+  fmap (BL.unpack . snd) <$> timeout micros (readProcessStdout (proc self args))++-- Log densities and draws at extreme but valid parameters. Each expected value+-- comes from the closed form or asymptotic expansion named beside it.+numericChecks :: IO [Bool]+numericChecks = do+  let half = let t = Tree 0.5 (repeat t) in t+      relative name expected actual = approxEq name expected actual (1e-12 * max 1 (abs expected))+      exactly name expected actual = assertBool (name ++ " is " ++ show expected ++ ", got " ++ show actual) (actual == expected)+  sequence+    [ -- z = 1: -log(2 pi)/2 - log 1e200 - 1/2+      relative "normalLogPdf 0 1e200 1e200" (-461.9359571320138) (normalLogPdf 0 1e200 1e200)+      -- z = 0: -log(2 pi)/2 + log 1e200+    , relative "normalLogPdf 0 1e-200 0" 459.5980800656045 (normalLogPdf 0 1e-200 0)+    , relative "logNormalLogPdf 0 1e-200 1" 459.5980800656045 (logNormalLogPdf 0 1e-200 1)+      -- z = 2 while x - mu exceeds the largest double: -log(2 pi)/2 - log 1e308 - 2+    , relative "normalLogPdf -1e308 1e308 1e308" (-712.1151471753708) (normalLogPdf (-1e308) 1e308 1e308)+      -- the median of a normal is its mean, up to the quantile's absolute accuracy near 0+    , let v = runProb (normal 0 1.5e308) half+      in assertBool ("normal 0 1.5e308 at label 1/2 is within 1e-15 sigma of 0, got " ++ show v) (abs v <= 1e-15 * 1.5e308)+      -- width 2e308: -log 2 - log 1e308+    , approxEq "uniformBoundedLogPdf -1e308 1e308 0" (-709.8893558227261) (uniformBoundedLogPdf (-1e308) 1e308 0) 1e-9+    , exactly "uniformbounded -1e308 1e308 at label 1/2" 0 (runProb (uniformbounded (-1e308) 1e308) half)+    ]

@@ -0,0 +1,352 @@+{-# OPTIONS_GHC -fno-cse -fno-full-laziness #-}+module Api.Domains where++import Control.Exception (ErrorCall, SomeException, evaluate, try)+import Data.List (foldl', isInfixOf, isPrefixOf, nub, sort, tails)+import Data.Word (Word64)+import LazyPPL (Meas(..), Prob, Tree(..), mh, mhirreducible, randomTree, runProb, sample, score, scoreLog, scoreProductLog, uniform, weightedsamples, wis)+import Control.Monad.Trans.Writer (runWriterT)+import Data.Monoid (Product(..), getProduct)+import Numeric.Log (Log(..), ln)+import qualified Numeric.Sum as Sum+import System.Timeout (timeout)+import LazyPPL.Distributions+import qualified LazyPPL.Distributions.IBP as IBP+import qualified LazyPPL.Distributions.DirichletP as DP+import System.Random (RandomGen(..), SplitGen(..), mkStdGen, setStdGen)+import Control.Monad (replicateM)+import Test.QuickCheck (NonEmptyList(..), NonNegative(..), Positive(..), Property, ioProperty, isSuccess, quickCheckResult)++import Api.Support++-- An impossible event has mass 0, so its log is negative infinity.+densityBoundaryChecks :: IO [Bool]+densityBoundaryChecks = sequence+  [ assertBool name (isInfinite actual && actual < 0)+  | (name, actual) <-+      [ ("Bernoulli impossible success", bernoulliLogPmf 0 True)+      , ("Bernoulli impossible failure", bernoulliLogPmf 1 False)+      , ("categorical zero mass", categoricalLogPmf [1,0] 1)+      , ("categorical index outside support", categoricalLogPmf [1] 1)+      , ("Poisson positive count at rate zero", poissonLogPmf 0 1)+      , ("Dirichlet outside simplex", dirichletLogPdf [1,1,1] [0.2,0.2,0.2])+      , ("Dirichlet empty simplex", dirichletLogPdf [] [])+      , ("exponential log density outside support", exponentialLogPdf 1 (-1))+      ]+  ] >>= \results -> (results ++) <$> sequence+  [ approxEq "Poisson zero rate and zero count" 0 (poissonLogPmf 0 0) 1e-15+  , approxEq "Poisson density zero rate and zero count" 1 (poissonPdf 0 0) 1e-15+  , do+      result <- try (evaluate (poissonPdf 1 (-1))) :: IO (Either ErrorCall Double)+      case result of+        Left err -> putStrLn ("FAIL Poisson negative count: " ++ show err) >> pure False+        Right value -> approxEq "Poisson negative count" 0 value 1e-15+  , approxEq "exponential density outside support" 0 (expPdf 1 (-1)) 1e-15+  , approxEq "Dirichlet(1,1,1) simplex density" (log 2) (dirichletLogPdf [1,1,1] [0.25,0.25,0.5]) 1e-12+  , approxEq "binomial certain failure" 0 (binomialLogPmf 5 0 0) 1e-15+  , approxEq "binomial certain success" 0 (binomialLogPmf 5 1 5) 1e-15+  , let Meas action = score 0+        (_, weight) = runProb (runWriterT action) (randomTree (mkStdGen 1))+    in assertBool "score zero stays impossible" (ln (getProduct weight) == -1/0)+  , and <$> mapM (\x -> do+      let Meas action = score x+          (_, weight) = runProb (runWriterT action) (randomTree (mkStdGen 1))+      result <- try (evaluate (ln (getProduct weight))) :: IO (Either ErrorCall Double)+      assertBool "invalid score is rejected" (either (const True) (const False) result)) [-1, 0/0, 1/0]+  , isSuccess <$> quickCheckResult dirichletNormalizedInput+  ]++dirichletNormalizedInput :: NonEmptyList (Positive Int) -> Bool+dirichletNormalizedInput (NonEmpty positive) =+  let values = map (fromIntegral . getPositive) positive :: [Double]+      xs = map (/ sum values) values+      density = dirichletLogPdf (replicate (length xs) 1) xs+  in not (isNaN density || isInfinite density)++-- The Dirichlet process scores atoms on an infinite stick-breaking probability+-- stream, so `categoricalLogPmf` must inspect only the prefix up to the selected+-- element and return on an infinite stream.+lazyCategoricalChecks :: IO [Bool]+lazyCategoricalChecks = do+  let stream = 0 : 1 : repeat 0+  selected <- evaluate (categoricalLogPmf stream 1)+  zeroMass <- evaluate (categoricalLogPmf stream 0)+  invalid <- try (evaluate (categoricalLogPmf (1.5 : repeat 0) 0)) :: IO (Either ErrorCall Double)+  sequence+    [ assertBool "categoricalLogPmf scores an infinite probability stream" (selected == log 1)+    , assertBool "categoricalLogPmf gives negative infinity for zero mass on an infinite stream"+        (isInfinite zeroMass && zeroMass < 0)+    , assertBool "categoricalLogPmf rejects an invalid inspected probability"+        (either (const True) (const False) invalid)+    ]++-- A generator that returns one word forever, so every label of the tree is the+-- label of that word.+newtype ConstWord = ConstWord Word64++instance RandomGen ConstWord where+  genWord64 g@(ConstWord w) = (w, g)++instance SplitGen ConstWord where+  splitGen g = (g, g)++categoricalBoundaryChecks :: IO [Bool]+categoricalBoundaryChecks = do+  let e = encodeFloat 1 (-54)+      weights = [0.5, e, e, e, e, 0.5 - 4 * e]+      tree = randomTree (ConstWord 0)+  result <- try (timeout 500000 (evaluate (runProb (categorical weights) tree))) :: IO (Either SomeException (Maybe Int))+  sequence+    [ assertBool ("categorical selects the final positive weight at the maximum label, got " ++ show result)+        (case result of Right (Just 5) -> True; _ -> False)+    ]++categoricalFiniteProperty :: NonEmptyList Word64 -> Word64 -> Property+categoricalFiniteProperty (NonEmpty values) word = ioProperty $ do+  let masses = 1 : map (fromIntegral . (`mod` 1024)) values+      weights = map (/ sum masses) masses+      labels = [0, 1 - encodeFloat 1 (-53), runProb uniform (randomTree (ConstWord word))]+      valid (Just i) = i >= 0 && i < length weights && weights !! i > 0+      valid Nothing = False+  draws <- mapM (\label -> within 500000 (runProb (categorical weights) (constantLabel label))) labels+  pure (all valid draws)++categoricalInfiniteProperty :: NonEmptyList Word64 -> NonNegative Int -> Word64 -> Property+categoricalInfiniteProperty (NonEmpty values) (NonNegative prefix) word = ioProperty $ do+  let splitMass n value =+        let mass = encodeFloat 1 (-n)+            fraction = fromIntegral (value `mod` 17) / 16+        in [mass * fraction, mass * (1 - fraction)]+      weights = replicate (prefix `mod` 100) 0 ++ concat (zipWith splitMass [1..] (cycle values))+      labels = [0, 1 - encodeFloat 1 (-53), runProb uniform (randomTree (ConstWord word))]+      valid (Just i) = i >= 0 && weights !! i > 0+      valid Nothing = False+  draws <- mapM (\label -> within 500000 (runProb (categorical weights) (constantLabel label))) labels+  pure (all valid draws)++categoricalPropertyChecks :: IO [Bool]+categoricalPropertyChecks = do+  finite <- quickCheckResult categoricalFiniteProperty+  infinite <- quickCheckResult categoricalInfiniteProperty+  shortDraw <- try (evaluate (runProb (categorical [0.5, 0.3]) (constantLabel 0.9))) :: IO (Either ErrorCall Int)+  quarterShort <- try (evaluate (runProb (categorical [0.5, 0.25]) (constantLabel 0.9999999999999999))) :: IO (Either ErrorCall Int)+  tailDraw <- try (evaluate (runProb (categorical [0.2, 0, 0]) (constantLabel 0.9))) :: IO (Either ErrorCall Int)+  let nearOne = [0.5, 0.5 - encodeFloat 1 (-40)]+      maxLabel = constantLabel (1 - encodeFloat 1 (-53))+  oneUlpDraw <- try (evaluate (runProb (categorical [1 - encodeFloat 1 (-53)]) maxLabel)) :: IO (Either ErrorCall Int)+  paddedDraw <- try (evaluate (runProb (categorical (nearOne ++ replicate 8190 0)) maxLabel)) :: IO (Either ErrorCall Int)+  tinyDraw <- try (evaluate (runProb (categorical (nearOne ++ replicate 8190 (encodeFloat 1 (-1074)))) maxLabel)) :: IO (Either ErrorCall Int)+  earlyDraw <- within 500000 (runProb (categorical [0.5, 0.3]) (constantLabel 0.4))+  lazyDraw <- within 500000 (runProb (categorical (0.8 : error "categorical forced the unchosen tail")) (constantLabel 0.4))+  roundedDraw <- within 500000 (runProb (categorical [1/590, 459/590, 130/590]) (constantLabel (1 - encodeFloat 1 (-53))))+  decimalDraw <- within 500000 (runProb (categorical [0.1, 0.2, 0.7]) (constantLabel (1 - encodeFloat 1 (-53))))+  decimalFallback <- within 500000 (runProb (categorical [0.50000000000000005, 0.250000000000000025, 0.249999999999999925]) maxLabel)+  let equalWeights = replicate 49 (1/49) :: [Double]+      equalTotal = Sum.kbn (foldl' Sum.add Sum.zero equalWeights)+  equalDraw <- within 500000 (runProb (categorical equalWeights) maxLabel)+  negativeZeroDraw <- within 500000 (runProb (categorical ((-0.0) : replicate 49 (1/49))) maxLabel)+  allZero <- rejects "categorical [0,0]" (runProb (categorical [0, 0]) (constantLabel 0.9))+  sequence+    [ assertBool "categorical chooses positive finite weights" (isSuccess finite)+    , assertBool "categorical chooses positive weights from an infinite stream" (isSuccess infinite)+    , assertBool ("categorical rejects missing mass in [0.5,0.3], got " ++ show shortDraw)+        (case shortDraw of Left err -> "probabilities do not sum to 1" `isInfixOf` show err; _ -> False)+    , assertBool "categorical rejects [0.5,0.25] at the largest label"+        (case quarterShort of Left err -> "probabilities do not sum to 1" `isInfixOf` show err; _ -> False)+    , assertBool ("categorical rejects missing mass before trailing zeros, got " ++ show tailDraw)+        (case tailDraw of Left err -> "probabilities do not sum to 1" `isInfixOf` show err; _ -> False)+    , assertBool "categorical rejects a singleton one ulp below one"+        (case oneUlpDraw of Left err -> "probabilities do not sum to 1" `isInfixOf` show err; _ -> False)+    , assertBool "categorical rejects a clear deficit despite many trailing zeros"+        (case paddedDraw of Left err -> "probabilities do not sum to 1" `isInfixOf` show err; _ -> False)+    , assertBool "categorical rejects a clear deficit despite many tiny weights"+        (case tinyDraw of Left err -> "probabilities do not sum to 1" `isInfixOf` show err; _ -> False)+    , assertBool "categorical stops before a later deficit" (earlyDraw == Just 0)+    , assertBool "categorical does not force an unchosen tail" (lazyDraw == Just 0)+    , assertBool "categorical draws from a rounded normalized vector" (roundedDraw == Just 2)+    , assertBool "categorical accepts rounded decimal probabilities" (decimalDraw == Just 2)+    , assertBool "categorical accepts rounded decimal probabilities at exhaustion" (decimalFallback == Just 2)+    , assertBool ("categorical 49-weight compensated sum is 0.9999999999999999, got " ++ show equalTotal)+        (equalTotal == 0.9999999999999999)+    , assertBool "categorical selects index 48 for 49 rounded equal probabilities at 0.9999999999999999" (equalDraw == Just 48)+    , assertBool "categorical accepts negative zero before rounded probabilities" (negativeZeroDraw == Just 49)+    , pure allZero+    ]++dirichletStickRoundingChecks :: IO [Bool]+dirichletStickRoundingChecks = do+  let e = encodeFloat 1 (-54)+      tree = Tree 0.5 (branch (0.5 : replicate 8 e ++ repeat 0.5) : repeat (constantLabel 0.5))+      branch (u:us) = Tree 0.5 (stickTree u : branch us : repeat (constantLabel 0.5))+      branch [] = constantLabel 0.5+      stickTree u = Tree 0.5 (constantLabel u : repeat (constantLabel 0.5))+      restaurant = runProb (DP.newRestaurant 1) tree+      draw u = runProb (DP.newCustomer restaurant) (constantLabel u)+  atBoundary <- within 500000 (draw (0.75 + 2 * e))+  insideNext <- within 500000 (draw 0.8)+  beforeBoundary <- within 500000 (draw 0.75)+  sequence [assertBool ("Dirichlet process retains the small sticks before the boundary: " ++ show beforeBoundary ++ ", " ++ show atBoundary ++ ", " ++ show insideNext) (atBoundary == insideNext && atBoundary /= Nothing && atBoundary /= beforeBoundary && beforeBoundary /= Nothing)]++poissonBoundaryChecks :: IO [Bool]+poissonBoundaryChecks = do+  result <- within 500000 (runProb (poisson 1) (constantLabel (1 - encodeFloat 1 (-53))))+  low <- within 500000 (runProb (poisson 36.5) (constantLabel (encodeFloat 1 (-53))))+  sequence+    [ assertBool ("Poisson(1) at the largest label gives 17, got " ++ show result) (result == Just 17)+    , assertBool ("Poisson(36.5) at the smallest label gives 0, got " ++ show low) (low == Just 0)+    ]++-- Words 0 and 2^64-1 are the two ends of the generator's range. Every label of+-- the tree lies strictly between 0 and 1, so each inverse transform returns a+-- value in its support.+endpointChecks :: IO [Bool]+endpointChecks = concat <$> mapM check [("word 0", 0), ("word 2^64-1", maxBound)]+  where+    check (name, word) = do+      let tree = randomTree (ConstWord word)+          draw p = within 500000 (runProb p tree)+          label = runProb uniform tree+          finite x = not (isNaN x || isInfinite x)+      discrete <- draw (uniformdiscrete 1)+      sure <- draw (bernoulli 1)+      never <- draw (bernoulli 0)+      single <- draw (categorical [1])+      count <- draw (poisson 0)+      waiting <- draw (exponential 1)+      gaussian <- draw (normal 0 1)+      sequence+        [ assertBool (name ++ ": label " ++ show label ++ " lies in (0,1)") (label > 0 && label < 1)+        , assertBool (name ++ ": uniformdiscrete 1 gives 0, got " ++ show discrete) (discrete == Just 0)+        , assertBool (name ++ ": bernoulli 1 gives True, got " ++ show sure) (sure == Just True)+        , assertBool (name ++ ": bernoulli 0 gives False, got " ++ show never) (never == Just False)+        , assertBool (name ++ ": categorical [1] gives 0, got " ++ show single) (single == Just 0)+        , assertBool (name ++ ": poisson 0 gives 0, got " ++ show count) (count == Just 0)+        , assertBool (name ++ ": exponential 1 is finite, got " ++ show waiting) (maybe False finite waiting)+        , assertBool (name ++ ": normal 0 1 is finite, got " ++ show gaussian) (maybe False finite gaussian)+        ]++-- A log weight is finite, or negative infinity for weight 0. NaN and positive+-- infinity are not weights, and a sampler must not run on them.+logWeightChecks :: IO [Bool]+logWeightChecks = do+  let firstLogWeight model = do+        setStdGen (mkStdGen 1)+        draws <- mh 1 model+        evaluate (ln (getProduct (snd (head draws))))+      scoredBy scorer = do x <- sample uniform; scorer; pure x+      invalid = [("NaN", Exp (0/0)), ("+Infinity", Exp (1/0))]+  single <- sequence+    [ rejectsIO ("mh rejects scoreLog " ++ name) (firstLogWeight (scoredBy (scoreLog w))) | (name, w) <- invalid ]+  products <- sequence+    [ rejectsIO ("mh rejects scoreProductLog " ++ name) (firstLogWeight (scoredBy (scoreProductLog (Product w))))+    | (name, w) <- invalid ]+  overflow <- rejectsIO "mh rejects a total log weight that overflows to +Infinity"+    (firstLogWeight (scoredBy (scoreLog (Exp 1e308) >> scoreLog (Exp 1e308))))+  large <- firstLogWeight (scoredBy (scoreLog (Exp 1000)))+  zero <- firstLogWeight (scoredBy (scoreLog (Exp (-1/0))))+  valid <- sequence+    [ assertBool ("mh keeps a log weight of 1000, got " ++ show large) (large == 1000)+    , assertBool ("mh keeps weight 0 as log weight -Infinity, got " ++ show zero) (isInfinite zero && zero < 0)+    ]+  pure (single ++ products ++ [overflow] ++ valid)++-- The chances given to mh and mhirreducible are probabilities, and the+-- irreducible kernel needs a positive chance q of an all-sites change.+chanceChecks :: IO [Bool]+chanceChecks = do+  let firstValue sampler = do+        setStdGen (mkStdGen 1)+        draws <- sampler (sample uniform)+        evaluate (fst (head draws))+  invalid <- sequence+    [ rejectsIO (name ++ " is rejected") (firstValue sampler)+    | (name, sampler) <-+        [ ("mhirreducible 0 0", mhirreducible 0 0)+        , ("mhirreducible 0.5 NaN", mhirreducible 0.5 (0/0))+        , ("mhirreducible 0.5 1.5", mhirreducible 0.5 1.5)+        , ("mh NaN", mh (0/0))+        , ("mh -0.5", mh (-0.5))+        , ("mh 1.5", mh 1.5) ] ]+  setStdGen (mkStdGen 1)+  moving <- take 16 <$> mhirreducible 0 0.5 (sample uniform)+  valid <- assertBool "mhirreducible 0 0.5 moves through its all-sites changes"+    (length (nub (map fst moving)) > 1)+  pure (invalid ++ [valid])++betaLabelChecks :: IO [Bool]+betaLabelChecks = do+  let least = encodeFloat 1 (-1074)+      at u p = runProb p (constantLabel u)+      rejected name value = raisesBetaLabel value >>= assertBool (name ++ " raises BetaLabelError")+  sequence+    [ rejected "beta 2 2 at least subnormal" (at least (beta 2 2))+    , rejected "beta 1000 1000 at least subnormal" (at least (beta 1000 1000))+    , rejected "beta 2 2 at 1e-320" (at 1e-320 (beta 2 2))+    , rejected "dirichlet [2,2] at 1e-320" (sum (at 1e-320 (dirichlet [2,2])))+    , assertBool "beta 1 2 rounds the least-label quantile to zero" (at least (beta 1 2) == 0)+    , approxEq "beta 2 1 at the least label" (encodeFloat 1 (-537)) (at least (beta 2 1)) (encodeFloat 1 (-537) * 1e-12)+    ]++-- The Dirichlet process and IBP's newRestaurantS draw their sticks from+-- beta 1 alpha and beta alpha 1. They sample at every finite positive+-- concentration, 1e-4 and 1e4 among them, and raise an error that names the+-- function and alpha for any other.+concentrationChecks :: IO [Bool]+concentrationChecks = do+  let tree = randomTree (mkStdGen 7)+      samples name p = do+        r <- within 2000000 (let xs = runProb p tree in length (show xs) `seq` xs)+        assertBool (name ++ " samples") (maybe False (const True) r)+      namesItself function name x = do+        result <- try (timeout 2000000 (evaluate x))+        let message = either (\err -> takeWhile (/= '\n') (show (err :: ErrorCall))) (const "") result+        assertBool (name ++ " raises an error naming " ++ function ++ " and alpha"+                      ++ either (const (": " ++ message)) (const " (it returned a value or no answer)") result)+          (function `isPrefixOf` message && "alpha" `isInfixOf` message)+  sequence $+    concat [ [ samples ("DirichletP.newRestaurant " ++ show alpha ++ " with 20 customers")+                 (DP.newRestaurant alpha >>= \r -> replicateM 20 (DP.newCustomer r))+             , samples ("dp " ++ show alpha ++ " (normal 0 1), 20 draws")+                 (DP.dp alpha (normal 0 1) >>= \d -> replicateM 20 d)+             , samples ("IBP.newRestaurantS " ++ show alpha ++ ", the first 10 dishes of 3 customers")+                 (IBP.newRestaurantS alpha >>= \r -> replicateM 3 (take 10 <$> IBP.newCustomerS r)) ]+           | alpha <- [1e-4, 1e4] ] +++    [ namesItself "newRestaurant" "DirichletP.newRestaurant 0" (runProb (DP.newRestaurant 0 >>= DP.newCustomer) tree)+    , namesItself "dp" "dp (-1) (normal 0 1)" (runProb (DP.dp (-1) (normal 0 1) >>= id) tree)+    , namesItself "newRestaurantS" "IBP.newRestaurantS NaN" (runProb (IBP.newRestaurantS (0 / 0) >>= IBP.newCustomerS) tree)+    , namesItself "dp" "dp Infinity (normal 0 1)" (runProb (DP.dp (1 / 0) (normal 0 1) >>= id) tree) ]++-- Each distribution checks its parameters, so an invalid parameter raises an+-- error instead of silently changing the law; samplers and densities agree.+parameterChecks :: IO [Bool]+parameterChecks = do+  let tree = randomTree (mkStdGen 1)+      sampled :: Prob a -> a+      sampled p = runProb p tree+  sequence+    [ rejects "bernoulli 2" (sampled (bernoulli 2))+    , rejects "bernoulli -0.5" (sampled (bernoulli (-0.5)))+    , rejects "exponential -1" (sampled (exponential (-1)))+    , rejects "exponential 0" (sampled (exponential 0))+    , rejects "gamma 1 -1" (sampled (gamma 1 (-1)))+    , rejects "gamma -1 1" (sampled (gamma (-1) 1))+    , rejects "beta 0 1" (sampled (beta 0 1))+      -- Outside the supported domain of beta and dirichlet: large shapes, and+      -- concentrations whose sum overflows.+    , rejects "beta 1e16 1e16" (sampled (beta 1e16 1e16))+    , rejects "dirichlet [1e308,1e308,1e308]" (sampled (dirichlet [1e308, 1e308, 1e308]))+    , rejects "normal 0 -1" (sampled (normal 0 (-1)))+    , rejects "normal 0 0" (sampled (normal 0 0))+    , rejects "poisson -1" (sampled (poisson (-1)))+    , rejects "uniformdiscrete 0" (sampled (uniformdiscrete 0))+    , rejects "uniformbounded 1 0" (sampled (uniformbounded 1 0))+    , rejects "categorical [-1,2]" (sampled (categorical [-1, 2]))+    , rejects "categorical [2,-1]" (sampled (categorical [2, -1]))+    , rejects "normalLogPdf 0 -1 0" (normalLogPdf 0 (-1) 0)+    , rejects "logNormalLogPdf 0 0 1" (logNormalLogPdf 0 0 1)+    , rejects "gammaLogPdf -1 1 1" (gammaLogPdf (-1) 1 1)+    , rejects "betaLogPdf 0 1 0.5" (betaLogPdf 0 1 0.5)+    , rejects "betaLogPdf 1e308 1e308 0.5" (betaLogPdf 1e308 1e308 0.5)+    , rejects "dirichletLogPdf [1e308,1e308] [0.5,0.5]" (dirichletLogPdf [1e308, 1e308] [0.5, 0.5])+    , rejects "uniformBoundedLogPdf 1 0 0.5" (uniformBoundedLogPdf 1 0 0.5)+    ]

@@ -0,0 +1,119 @@+{-# OPTIONS_GHC -fno-cse -fno-full-laziness #-}+module Api.Inference where++import Control.Exception (ErrorCall, SomeException, evaluate, try)+import Data.List (isInfixOf, isPrefixOf, nub, sort, tails)+import LazyPPL (Meas(..), Prob, Tree(..), mh, mhirreducible, randomTree, runProb, sample, score, scoreLog, scoreProductLog, uniform, weightedsamples, wis)+import Data.Monoid (Product(..), getProduct)+import Numeric.Log (Log(..), ln)+import LazyPPL.Distributions+import System.Random (RandomGen(..), SplitGen(..), mkStdGen, setStdGen)+import Control.Monad (replicateM)++import Api.Support++smokeMhWithinUnitInterval :: IO Bool+smokeMhWithinUnitInterval = do+  draws <- take 25 <$> mh 0.5 (sample uniform)+  pure $ all (\(x, _) -> x >= 0 && x <= 1) draws++smokeMhIrreducibleWithinUnitInterval :: IO Bool+smokeMhIrreducibleWithinUnitInterval = do+  draws <- take 25 <$> mhirreducible 0.5 0.1 (sample uniform)+  pure $ all (\(x, _) -> x >= 0 && x <= 1) draws++wisDistinctResamples :: IO Double+wisDistinctResamples = do+  draws <- take 2000 <$> wis 4 (sample uniform)+  pure $ fromIntegral (length (nub draws))++smokeWisWithinUnitInterval :: IO Bool+smokeWisWithinUnitInterval = do+  draws <- take 25 <$> wis 50 (sample uniform)+  pure $ all (\x -> x >= 0 && x <= 1) draws++-- Only the all-true sequence has weight (prior mass 1/1024). From weight 0 the chain accepts+-- every move until it lands there, at sample index 408 with seed 1.+zeroAcceptanceChecks :: IO [Bool]+zeroAcceptanceChecks = do+  let constrainedCoins = do+        coins <- sample (replicateM 10 (bernoulli 0.5))+        score (if and coins then 1 else 0)+        pure coins+      impossible = do x <- sample uniform; score 0; pure x+  setStdGen (mkStdGen 1)+  draws <- mh 0.5 constrainedCoins+  escaped <- assertBool "mh reaches the support from a zero-weight initialization within 1024 transitions"+    (any (and . fst) (take 1024 draws))+  retained <- assertBool "mh rejects impossible proposals after entering the support"+    (all (and . fst) (dropWhile (not . and . fst) (take 1024 draws)))+  moving <- mapM (\(name, sampler) -> do+      setStdGen (mkStdGen 1)+      states <- take 16 <$> sampler impossible+      assertBool (name ++ " accepts zero-to-zero initialization moves")+        (length (nub (map fst states)) > 1 && all ((== 0) . getProduct . snd) states))+    [("mh", mh 0.5), ("mhirreducible", mhirreducible 0.5 0.1)]+  setStdGen (mkStdGen 1)+  zeroBatch <- try (wis 16 impossible >>= evaluate . head) :: IO (Either ErrorCall Double)+  noZeroNormalization <- assertBool "wis rejects an all-zero empirical measure"+    (either (const True) (const False) zeroBatch)+  setStdGen (mkStdGen 1)+  positive <- take 32 <$> wis 256 (do+    x <- sample uniform+    score (if x > 0.5 then 1 else 0)+    pure x)+  positiveSupport <- assertBool "wis returns supported samples from a batch containing zero weights"+    (all (>0.5) positive)+  pure (escaped : retained : moving ++ [noZeroNormalization, positiveSupport])++-- A fair coin b. On true, a uniform u is read and scored 6u, which integrates+-- to 3; on false, the score is 1. So P(b) = 3/4 exactly, and the site u exists+-- only on the true branch.+coinModel :: Meas Bool+coinModel = do+  b <- sample (bernoulli 0.5)+  if b then sample uniform >>= \u -> score (6 * u) else score 1+  pure b++-- The fraction of true draws is within five standard errors of 3/4. The+-- standard error comes from the means of 40 consecutive batches, and it must be+-- below 0.01 for the check to say anything.+posteriorFraction :: String -> [Bool] -> IO Bool+posteriorFraction name draws = do+  let n = length draws+      batches = 40+      size = n `div` batches+      batchMean k = fromIntegral (length (filter id (take size (drop (k * size) draws)))) / fromIntegral size :: Double+      means = map batchMean [0 .. batches - 1]+      p = sum means / fromIntegral batches+      se = sqrt (sum [(m - p) ^ (2 :: Int) | m <- means] / fromIntegral (batches - 1) / fromIntegral batches)+      ok = se < 0.01 && abs (p - 0.75) < 5 * se+  putStrLn $ (if ok then "PASS " else "FAIL ") ++ name ++ ": P(true) = " ++ show p ++ ", standard error " ++ show se ++ ", exact 3/4"+  pure ok++-- wis n resamples from its n weighted particles, so each particle's share of+-- the resamples is its weight over the total. With the same global seed,+-- weightedsamples gives wis's particles, and the weights here are 1 and 3.+wisResamplingCheck :: IO Bool+wisResamplingCheck = do+  let model = do x <- sample uniform; score (if x < 0.5 then 1 else 3); pure x+      n = 5+      draws = 20000 :: Int+  setStdGen (mkStdGen 24)+  particles <- take n <$> weightedsamples model+  setStdGen (mkStdGen 24)+  resamples <- take draws <$> wis n model+  let total = sum [exp (ln w) | (_, w) <- particles]+      shares = [ (fromIntegral (length (filter (== x) resamples)) / fromIntegral draws, exp (ln w) / total)+               | (x, w) <- particles ]+      within5 (observed, expected) = abs (observed - expected) <= 5 * sqrt (expected * (1 - expected) / fromIntegral draws)+  assertBool ("wis 5 resamples each particle in proportion to its weight, (observed, expected) = " ++ show shares)+    (all within5 shares && length (filter (`elem` map fst particles) resamples) == draws)++posteriorChecks :: IO [Bool]+posteriorChecks = do+  setStdGen (mkStdGen 21)+  chain <- map fst . take 40000 . drop 1000 <$> mh 0.5 coinModel+  setStdGen (mkStdGen 22)+  irreducible <- map fst . take 40000 . drop 1000 <$> mhirreducible 0.5 0.1 coinModel+  sequence [posteriorFraction "mh 0.5 coin posterior" chain, posteriorFraction "mhirreducible 0.5 0.1 coin posterior" irreducible, wisResamplingCheck]

@@ -0,0 +1,214 @@+{-# OPTIONS_GHC -fno-cse -fno-full-laziness #-}+-- The Wiener and memo checks need a fresh hidden table per call, which floating+-- a constant random function out of its lambda would share.+module Api.Processes where++import Control.Exception (ErrorCall, SomeException, evaluate, try)+import LazyPPL (Meas(..), Prob, Tree(..), mh, mhirreducible, randomTree, runProb, sample, score, scoreLog, scoreProductLog, uniform, weightedsamples, wis)+import Numeric.SpecFunctions (erfc)+import LazyPPL.Distributions+import LazyPPL.Distributions.GP (gp, wiener)+import LazyPPL.Distributions.IBP (ibp)+import LazyPPL.Distributions.Memoization (memoize, memrec)+import qualified LazyPPL.Distributions.DirichletP as DP+import qualified LazyPPL.Distributions.Mondrian as Mondrian+import System.Random (RandomGen(..), SplitGen(..), mkStdGen, setStdGen)+import Test.QuickCheck (NonEmptyList(..), NonNegative(..), Positive(..), Property, ioProperty, isSuccess, quickCheckResult)++import Api.Support++memoizeMismatches :: Double+memoizeMismatches =+  let g = runProb (memoize (\n -> pure (n :: Int))) (randomTree (mkStdGen 1))+  in fromIntegral (length (filter (\n -> g n /= n) [0 .. 40]))++standardDeviation :: [Double] -> Double+standardDeviation xs =+  let n = fromIntegral (length xs)+      m = sum xs / n+  in sqrt (sum [(x - m) ^ (2 :: Int) | x <- xs] / (n - 1))++-- Since cov 0 0 = 1, f 0 has standard deviation 1 up to the nugget.+gpStandardDeviationAtZero :: Double+gpStandardDeviationAtZero =+  standardDeviation [runProb (gp (\x y -> exp (negate ((x - y) ^ (2 :: Int))))) (randomTree (mkStdGen s)) 0 | s <- [1 .. 400]]++-- With a constant kernel c, f 1 - f 0 is only nugget noise, of variance 2*c*2^-33.+gpConstantKernelGap :: Double -> IO Double+gpConstantKernelGap c = do+  let f = runProb (gp (\_ _ -> c)) (randomTree (mkStdGen 3))+  r <- try (evaluate (f 0 `seq` abs (f 1 - f 0) / sqrt c)) :: IO (Either SomeException Double)+  pure (either (const (1 / 0)) id r)++-- Scaling the kernel by s scales the draw by sqrt s, so even a huge s gives a finite draw.+gpScaledKernelFinite :: Double -> IO Double+gpScaledKernelFinite s = do+  let f = runProb (gp (\x y -> s * exp (negate ((x - y) ^ (2 :: Int))))) (randomTree (mkStdGen 4))+  r <- try (evaluate (sum [abs (f x) | x <- [0, 0.1 .. 3]] / sqrt s)) :: IO (Either SomeException Double)+  pure (either (const (1 / 0)) (\v -> if isNaN v || isInfinite v then 1 / 0 else 0) r)++-- The rank-two kernel gives a line plus nugget noise. At x = 5 the residual has nugget+-- variance 92*c*2^-33, the 92 in the bound.+gpRankTwoLineResidual :: Double -> IO Double+gpRankTwoLineResidual c = do+  let f = runProb (gp (\x y -> c * (1 + x * y))) (randomTree (mkStdGen 4))+      line x = f 0 + x * (f 1 - f 0)+  r <- try (evaluate (maximum [abs (f x - line x) | x <- [0 .. 5]] / sqrt c)) :: IO (Either SomeException Double)+  pure (either (const (1 / 0)) id r)++-- c cos (x - y) has rank two as well, and its draw must still be finite.+gpCosineKernelFinite :: Double -> IO Double+gpCosineKernelFinite c = do+  let f = runProb (gp (\x y -> c * cos (x - y))) (randomTree (mkStdGen 5))+  r <- try (evaluate (sum [abs (f x) | x <- [0 .. 5]] / sqrt c)) :: IO (Either SomeException Double)+  pure (either (const (1 / 0)) (\v -> if isNaN v || isInfinite v then 1 / 0 else 0) r)++-- On the same random tree, gp (s * k) is sqrt s times gp k, nugget included.+gpScaleEquivarianceError :: IO Double+gpScaleEquivarianceError = do+  let kernels = [\_ _ -> 1, \x y -> 1 + x * y, \x y -> cos (x - y), \x y -> exp (negate ((x - y) ^ (2 :: Int)))]+      draw s k = runProb (gp (\x y -> s * k x y)) (randomTree (mkStdGen 6))+      gap s k = let f1 = draw 1 k; fs = draw s k+                in maximum [abs (fs x / sqrt s - f1 x) / (1 + abs (f1 x)) | x <- [0 .. 5]]+  r <- try (evaluate (maximum [gap s k | k <- kernels,+                               s <- [1e-300,1e-100,1e8,1e16,1e100,1e300]])) :: IO (Either SomeException Double)+  pure (either (const (1 / 0)) id r)++-- Even a variance of 2^-1074, the smallest positive double, gives the first point a draw, as+-- there is no absolute cutoff. Later pivots may underflow.+gpSmallestVarianceSample :: IO Double+gpSmallestVarianceSample = do+  let s = encodeFloat 1 (-1074)+      f = runProb (gp (\_ _ -> s)) (randomTree (mkStdGen 4))+  r <- try (evaluate (abs (f 0) / sqrt s)) :: IO (Either SomeException Double)+  pure (either (const (1 / 0)) id r)++-- The first customer takes Poisson(1) dishes, so the first row is empty with probability exp(-1).+ibpFirstRowEmptyFraction :: Double+ibpFirstRowEmptyFraction =+  let firstRowEmpty s = case runProb (ibp 1) (randomTree (mkStdGen s)) of+        row : _ -> null row+        [] -> error "ibp returned no rows"+  in fromIntegral (length (filter firstRowEmpty [1 .. 2000 :: Int])) / 2000++smokeGpFinite :: IO Bool+smokeGpFinite = do+  let cov x y = exp (negate ((x - y) * (x - y)))+      f = runProb (gp cov) (randomTree (mkStdGen 12345))+      ys = [f 0, f 0.25, f 1.0]+  pure $ all (\y -> not (isNaN y) && not (isInfinite y)) ys++memoStochasticChecks :: IO [Bool]+memoStochasticChecks = do+  result <- quickCheckResult $ \seed (NonNegative key) -> ioProperty $ do+    let f = runProb (memoize (const uniform :: Int -> Prob Double)) (randomTree (mkStdGen seed))+        otherKey = if key == maxBound then 0 else key + 1+    values <- mapM (evaluate . f) [key, otherKey, key]+    pure $ case values of+      [first, other, repeated] -> first == repeated && first /= other+      _ -> False+  sequence [assertBool "memoize reuses sampled values and distinguishes keys" (isSuccess result)]++newtype OrdOnlyKey = OrdOnlyKey Int deriving (Eq, Ord)++recursiveMemoCheck :: IO [Bool]+recursiveMemoCheck = do+  let f = runProb (memrec (\self (OrdOnlyKey n) ->+        pure (if n == 0 then 1 else self (OrdOnlyKey (n-1)) + 1))) (randomTree (mkStdGen 1))+  sequence [assertBool "memrec accepts keys with Ord and no Show" (f (OrdOnlyKey 3) == (4 :: Int))]++dirichletProcessChecks :: IO [Bool]+dirichletProcessChecks = do+  result <- quickCheckResult $ \seed ->+    let tree = randomTree (mkStdGen seed)+        constantDraw = do distribution <- DP.dp 1 (pure (7 :: Int)); distribution+    in runProb constantDraw tree == 7 && runProb (categorical (0:1:undefined)) tree == 1+  sequence [assertBool "Dirichlet process preserves constant atoms and lazy categorical selection" (isSuccess result)]++-- x = 1/2 itself goes to the right child. The blocks have bias 0 and 1, so a wrong traversal+-- or likelihood shows.+mondrianChecks :: IO [Bool]+mondrianChecks = do+  let box = [(0,1),(0,1)]+      tree = Mondrian.Partition 0 0.5 box+        (Mondrian.Block 0 [(0,0.5),(0,1)]) (Mondrian.Block 1 [(0.5,1),(0,1)])+      draw seed p = runProb p (randomTree (mkStdGen seed))+      expected = [(0.25,False),(0.5,True),(0.75,True)]+      halfTree = let t = Tree 0.5 (repeat t) in t+      -- With every label 1/2, the root (cost log(2)/2 < 0.4) is cut along dimension 1 at 1/2,+      -- and each child (cost log(2)/1.5) exceeds the budget left.+      expectedPrior = Mondrian.Partition 1 0.5 box+        (Mondrian.Block 0.25 [(0,1),(0,0.5)]) (Mondrian.Block 0.25 [(0,1),(0.5,1)])+  result <- quickCheckResult $ \seed -> and+        [ Mondrian.likelihoodFromMondrian2D tree x y value == 1 &&+          Mondrian.likelihoodFromMondrian2D tree x y (not value) == 0 &&+          draw seed (Mondrian.sampleFromMondrian2D tree x y) == value+        | (x,value) <- expected, y <- [0,1] ]+  prior <- quickCheckResult $ \seed ->+    draw seed (Mondrian.randomMondrian (pure (7 :: Int)) 0 box) == Mondrian.Block 7 box+  sequence+    [ assertBool "Mondrian likelihood and samples follow the cut" (isSuccess result)+    , assertBool "zero-budget Mondrian draws its base without cuts" (isSuccess prior)+    , assertBool "Mondrian samples the independently specified first cut and child boxes"+        (runProb (Mondrian.randomMondrian (pure 0.25) 0.4 box) halfTree == expectedPrior)+    ]++-- The Wiener bridge at unequal gaps. Given W(0) = 0 and W(T) = w, W(s) for+-- 0 < s < T is normal with mean (s/T) w and standard deviation+-- sqrt(s (T - s) / T), representable here even where s/T or s (T - s) / T is+-- not. On the tree whose every label is 1/4, every standard normal draw is the+-- lower quartile z = -0.6744897501960817, so w = sqrt T z and W(s) has an exact+-- value. Over seeds, the standardized W(s) is standard normal; that law is+-- checked with the Kolmogorov-Smirnov threshold of lawChecks.+wienerBridgeChecks :: IO [Bool]+wienerBridgeChecks = do+  let z = -0.6744897501960817+      leastPositive = encodeFloat 1 (-1074) :: Double+      afterT s t tree = let f = runProb wiener tree in f t `seq` (f t, f s)+      exactAt s t = do+        let expected = z * sqrt s * sqrt ((t - s) / t) + s * (z / sqrt t)+        r <- within 500000 (snd (afterT s t (constantLabel 0.25)))+        assertBool ("wiener at " ++ show s ++ " after " ++ show t ++ " on labels 1/4 is " ++ show expected+                      ++ ", got " ++ maybe "an error or no answer" show r)+          (maybe False (\v -> abs (v - expected) <= 1e-12 * abs expected) r)+      n = 10000 :: Int+      threshold = sqrt (log (2 / 1e-6) / (2 * fromIntegral n))+      standardNormalCdf x = erfc (- x / sqrt 2) / 2+      lawAt s t = do+        let residual seed = let (w, v) = afterT s t (randomTree (mkStdGen seed))+                            in (v - s * (w / t)) / (sqrt s * sqrt ((t - s) / t))+            d = ksDistance standardNormalCdf (map residual [1 .. n])+        assertBool ("wiener at " ++ show s ++ " after " ++ show t ++ ": standardized draws at distance " ++ show d+                      ++ " from the standard normal (threshold " ++ show threshold ++ ")")+          (d <= threshold)+  sequence+    [ exactAt 1e-200 1e200+    , exactAt leastPositive (2 * leastPositive)+    , exactAt 1 4+    , lawAt 1e-200 1e200+    , lawAt leastPositive (2 * leastPositive)+    , lawAt 1 4+    ]++-- Given f(2s), the Wiener bridge at s has mean f(2s)/2 and variance s/2, so on+-- one tree f(s)/sqrt(s) does not depend on s. A query at an invalid time is an+-- error and leaves the process unchanged.+processChecks :: IO [Bool]+processChecks = do+  let scaled s = let f = runProb wiener (randomTree (mkStdGen 13)) in f (2 * s) `seq` f s / sqrt s+      reference = scaled 1+      f = runProb wiener (randomTree (mkStdGen 14))+  scales <- sequence+    [ do value <- within 500000 (scaled s)+         assertBool ("wiener f(s)/sqrt(s) at s = " ++ show s ++ " is " ++ show reference ++ ", got " ++ show value)+           (maybe False (\v -> abs (v - reference) <= 1e-12 * abs reference) value)+    | s <- [1e-200, 1e-100, 1e100, 1e200] ]+  invalid <- try (evaluate (f (0/0)))+  after <- within 500000 (f 1)+  atomic <- sequence+    [ assertBool ("wiener rejects time NaN" ++ either (const "") (\v -> ", got " ++ show (v :: Double)) invalid)+        (either (\e -> const True (e :: ErrorCall)) (const False) invalid)+    , assertBool ("wiener after a rejected query gives a finite f 1, got " ++ show after)+        (maybe False (\v -> not (isNaN v || isInfinite v)) after)+    ]+  pure (scales ++ atomic)

@@ -0,0 +1,105 @@+{-# OPTIONS_GHC -fno-cse -fno-full-laziness #-}+module Api.Support where++import Control.Exception (ErrorCall, SomeException, evaluate, try)+import Data.List (isInfixOf, isPrefixOf, nub, sort, tails)+import LazyPPL (Meas(..), Prob, Tree(..), mh, mhirreducible, randomTree, runProb, sample, score, scoreLog, scoreProductLog, uniform, weightedsamples, wis)+import Numeric.MathFunctions.Comparison (addUlps)+import System.Timeout (timeout)+import LazyPPL.Distributions++approxEq :: String -> Double -> Double -> Double -> IO Bool+approxEq name expected actual tol = do+  let ok = abs (expected - actual) < tol+  putStrLn $+    (if ok then "PASS" else "FAIL")+      ++ " "+      ++ name+      ++ " expected="+      ++ show expected+      ++ " actual="+      ++ show actual+  pure ok++assertBool :: String -> Bool -> IO Bool+assertBool name ok = do+  putStrLn $ (if ok then "PASS" else "FAIL") ++ " " ++ name+  pure ok++-- Evaluates a value with a time limit. An exception or a timeout gives Nothing.+within :: Int -> a -> IO (Maybe a)+within micros x = do+  result <- try (timeout micros (evaluate x))+  pure $ case result of+    Left err -> const Nothing (err :: SomeException)+    Right value -> value++-- Passes when evaluating the value raises an error call.+rejects :: String -> a -> IO Bool+rejects name x = do+  result <- try (timeout 500000 (evaluate x))+  assertBool (name ++ " is rejected" ++ case result of+      Left err -> const "" (err :: ErrorCall)+      Right Nothing -> " (no answer within 0.5 s)"+      Right (Just _) -> " (it returned a value)")+    (either (const True) (const False) result)++-- Passes when running the action raises an error call.+rejectsIO :: String -> IO Double -> IO Bool+rejectsIO name action = do+  result <- try action+  assertBool (name ++ case result of+      Left err -> const "" (err :: ErrorCall)+      Right value -> " (it returned " ++ show value ++ ")")+    (either (const True) (const False) result)++-- The tree whose every label is u.+constantLabel :: Double -> Tree+constantLabel u = let t = Tree u (repeat t) in t++-- The largest distance between the empirical distribution function of the+-- sample and cdf, with an allowance for rounding: at a draw x the exact function+-- is taken at x (at the least positive double for a draw that underflowed to+-- 0), and just below x at the double before x. A draw is a rounded double, so+-- equal draws can be an atom: Beta(0.001,1) puts 0.475 below the least positive+-- double, which all round to 0. statistics-0.16.5.0's kolmogorovSmirnovCdfD+-- computes the statistic for a continuous law, without such atoms.+ksDistance :: (Double -> Double) -> [Double] -> Double+ksDistance cdf xs = maximum+  [ max (i / n - cdf (if x == 0 then encodeFloat 1 (-1074) else x)) (cdf (addUlps (-1) x) - (i - 1) / n)+  | (i, x) <- zip [1 ..] (sort xs) ]+  where n = fromIntegral (length xs)++-- The rows of a table that test/reference/reference.py wrote with mpmath, each+-- split at its tabs; lines that start with # are comments. Cabal runs the test+-- in the package root, where the path below starts.+referenceTable :: FilePath -> IO [[String]]+referenceTable name = do+  text <- readFile ("test/reference/" ++ name)+  pure [cells line | line <- lines text, not (null line), take 1 line /= "#"]+  where+    cells s = case break (== '\t') s of+      (cell, []) -> [cell]+      (cell, _ : rest) -> cell : cells rest++-- Prints the failures of a table check and one summary line.+tableReport :: String -> [(Bool, String)] -> IO [Bool]+tableReport name results = do+  mapM_ (\(_, detail) -> putStrLn ("FAIL " ++ detail)) (filter (not . fst) results)+  let passed = length (filter fst results)+  putStrLn ((if passed == length results then "PASS " else "FAIL ") ++ name ++ ": "+            ++ show passed ++ " of " ++ show (length results) ++ " rows")+  pure (map fst results)++-- Whether evaluating the value raises an error call within 2 s.+raisesError :: a -> IO Bool+raisesError x = do+  result <- try (timeout 2000000 (evaluate x))+  pure (either (\err -> const True (err :: ErrorCall)) (const False) result)++raisesBetaLabel :: a -> IO Bool+raisesBetaLabel x = do+  result <- try (timeout 2000000 (evaluate x))+  pure (check result)+  where check :: Either BetaLabelError b -> Bool+        check = either (const True) (const False)

@@ -0,0 +1,108 @@+{-# LANGUAGE ScopedTypeVariables #-}+module Main where++import Control.DeepSeq (force)+import Control.Exception (SomeException, evaluate, try)+import Control.Monad (liftM, unless, forM, forM_)+import Control.Monad.State.Lazy (State, get, put, runState)+import Control.Monad.Trans.Writer (runWriterT)+import Data.Monoid (Product(..))+import GHC.Float (castDoubleToWord64)+import LazyPPL+import LazyPPL.Internal.Sampler (randomLabel)+import Numeric.Log (Log, ln)+import System.Random hiding (uniform)+import Test.QuickCheck hiding (sample)+import Test.QuickCheck.Random (mkQCGen)++-- The tree-keeping transition and lazy State traversal are from+-- 8168aff:src/LazyPPL.hs. They do not use the replay history or checkpoints.+reference :: forall a. Double -> Maybe Double -> Meas a -> IO [(a, Product (Log Double))]+reference p refresh (Meas m) = do+  g <- newStdGen+  let (g1,g2) = split g+      t = randomTree g1+      (x,w) = runProb (runWriterT m) t+      (samples,_) = runState (iterateM step (t,x,w)) g2+  pure (map (\(_,v,weight) -> (v,weight)) samples)+  where+    step :: (Tree,a,Product (Log Double)) -> State StdGen (Tree,a,Product (Log Double))+    step (t,x,w) = do+      g <- get+      let (g1,g2) = split g+          t' = case refresh of+            Nothing -> mutate p g1 t+            Just q -> let (r,g1') = random g1+                      in if r < q then randomTree g1' else mutate p g1' t+          (x',w') = runProb (runWriterT m) t'+          ratio = getProduct w' / getProduct w+          (r,g2') = random g2+      put g2'+      if r < min 1 (exp (ln ratio)) then pure (t',x',w') else pure (t,x,w)+    iterateM f x = do+      x' <- f x+      (x':) `liftM` iterateM f x'++mutate :: Double -> StdGen -> Tree -> Tree+mutate p g (Tree a ts) =+  let (a',g') = randomLabel g+      (a'',g'') = randomLabel g'+      children seed (t:rest) = let (g1,g2) = split seed+                              in mutate p g1 t : children g2 rest+      children _ [] = []+  in Tree (if a' < p then a'' else a) (children g'' ts)++at :: [Int] -> Tree -> Double+at [] (Tree x _) = x+at (i:is) (Tree _ ts) = at is (ts !! i)++model :: Int -> Meas ([Int] -> Double)+model kind = do+  tree <- sample (Prob id)+  let root = at [] tree+      branch = if root < 0.5 then [0,2] else [2,0]+  case kind of+    0 -> pure ()+    1 -> score (if root < 0.3 then 1 else 1e-12)+    _ -> score (0.01 + at branch tree)+  pure (\path -> if path == [99] then error "delayed payload" else at path tree)++chainProperty :: Property+chainProperty = forAll arbitrary $ \seed ->+  forAll (elements [0,0.001,0.2,1]) $ \rate ->+  forAll (elements [Nothing,Just 0.05,Just 1]) $ \refresh ->+  forAll (chooseInt (0,2)) $ \kind ->+  forAll (elements [33,97,257,4101]) $ \count -> ioProperty $ do+    let measure = model kind+        sampler = maybe (mh rate) (mhirreducible rate) refresh+        paths = [[],[0],[1,0],[0,2],[2,0],[5,3,1]]+        observe (f,w) = map (castDoubleToWord64 . f) paths ++ [castDoubleToWord64 (ln (getProduct w))]+    setStdGen (mkStdGen seed)+    actual <- sampler measure+    setStdGen (mkStdGen seed)+    expected <- reference rate refresh measure+    -- Advance before demanding the old payloads, including across seed-store growth.+    _ <- evaluate (snd (actual !! (count-1)))+    _ <- evaluate (snd (expected !! (count-1)))+    let indices = [0 .. min 256 (count-1)] ++ [count-1]+    observed <- evaluate (force (map (observe . (actual !!)) indices))+    oracle <- evaluate (force (map (observe . (expected !!)) indices))+    failures <- forM [actual,expected] $ \xs -> do+      outcome <- try (evaluate (fst (head xs) [99])) :: IO (Either SomeException Double)+      pure (either (const True) (const False) outcome)+    pure $ counterexample (show (seed,rate,refresh,kind,count)) $+      observed === oracle .&&. failures === [True,True]++main :: IO ()+main = do+  forM_ [Nothing, Just 0.05, Just 1] $ \refresh -> do+    let sampler = maybe (mh 0.2) (mhirreducible 0.2) refresh+    forM_ [sampler, reference 0.2 refresh] $ \run -> do+      xs <- run (pure (error "undemanded payload" :: Bool))+      count <- evaluate (length (take 128 xs))+      unless (count == 128) (fail "Sampler forced an undemanded payload")+      bad <- run (score (error "demanded score") >> pure True)+      outcome <- try (evaluate (head bad)) :: IO (Either SomeException (Bool,Product (Log Double)))+      unless (either (const True) (const False) outcome) (fail "Sampler hid a score exception")+  result <- quickCheckWithResult stdArgs {maxSuccess=200, replay=Just (mkQCGen 20261026,0)} chainProperty+  unless (isSuccess result) (fail "Seeded chains differ from the tree-keeping sampler")

@@ -0,0 +1,143 @@+{-# OPTIONS_GHC -fno-cse -fno-full-laziness #-}+module Main where++import Control.Concurrent (forkFinally, newEmptyMVar, putMVar, takeMVar)+import Control.Concurrent.MVar (MVar)+import Control.Exception (evaluate, throwIO)+import Control.Monad (forM, replicateM, unless, when)+import Data.IORef (IORef, atomicModifyIORef', newIORef, writeIORef)+import qualified Data.Set as Set+import LazyPPL (Prob, mh, randomTree, runProb, sample, uniform, weightedsamples)+import LazyPPL.Distributions.Counter (newCounter, readAndIncrement)+import LazyPPL.Distributions.GP (gp)+import LazyPPL.Distributions.Memoization (generalmemoize)+import System.Exit (exitFailure)+import System.IO.Unsafe (unsafePerformIO)+import System.Random (mkStdGen)+import System.Timeout (timeout)++-- Runs `draw` `runs` times in each of two threads at once.+concurrentDraws :: Int -> IO a -> IO [a]+concurrentDraws runs draw = do+  slots <- forM [1, 2 :: Int] $ \_ -> do+    slot <- newEmptyMVar+    _ <- forkFinally (replicateM runs (draw >>= evaluate)) (putMVar slot)+    pure slot+  concat <$> mapM (\slot -> takeMVar slot >>= either throwIO pure) slots++report :: String -> Bool -> String -> IO Bool+report name ok detail = do+  putStrLn $ (if ok then "PASS " else "FAIL ") ++ name ++ ": " ++ detail+  pure ok++-- Each inference run takes its own generator from the global one. Two runs+-- that start at the same moment must still get different generators, so no+-- two runs share their first draw.+distinctFirstDraws :: String -> IO Double -> IO Bool+distinctFirstDraws name draw = do+  draws <- concurrentDraws 20000 draw+  let distinct = Set.size (Set.fromList draws)+  report name (distinct == length draws)+    (show distinct ++ " distinct first draws in " ++ show (length draws) ++ " concurrent runs")++-- The pause point of the interleaving tests below: the thread that makes the+-- `pauseAt`-th marked call signals `inside`, then waits for `release`. The+-- wait has a time limit, because a correct implementation can make the other+-- thread wait for this one.+{-# NOINLINE pauseHook #-}+pauseHook :: IORef (IO ())+pauseHook = unsafePerformIO (newIORef (pure ()))++markedCall :: IO ()+markedCall = do+  action <- atomicModifyIORef' pauseHook (\a -> (a, a))+  action++pauseAt :: Int -> IO (MVar (), MVar ())+pauseAt target = do+  inside <- newEmptyMVar+  release <- newEmptyMVar+  calls <- newIORef (0 :: Int)+  writeIORef pauseHook $ do+    n <- atomicModifyIORef' calls (\c -> (c + 1, c + 1))+    when (n == target) $ do+      putMVar inside ()+      _ <- timeout 2000000 (takeMVar release)+      pure ()+  pure (inside, release)++-- Keys whose comparison makes a marked call when the left key is 1. The+-- comparison results are the ordinary ones.+newtype Key = Key Int deriving (Eq, Show)++instance Ord Key where+  compare (Key a) (Key b) = unsafePerformIO (when (a == 1) markedCall) `seq` compare a b++-- Two threads query a memoized stochastic function at two new keys at once.+-- The first thread pauses at its second comparison, after it has read the+-- table and before it has stored its entry, and the second thread runs in+-- between. Each key must still get its own draw, and a repeated query must+-- return the draw made for its key.+memoInterleaving :: IO Bool+memoInterleaving = do+  let f = runProb (generalmemoize (const uniform :: Key -> Prob Double)) (randomTree (mkStdGen 7))+  _ <- evaluate (f (Key 0))+  (inside, release) <- pauseAt 2+  slot <- newEmptyMVar+  _ <- forkFinally (evaluate (f (Key 1))) (putMVar slot)+  takeMVar inside+  second <- evaluate (f (Key 2))+  putMVar release ()+  first <- takeMVar slot >>= either throwIO pure+  writeIORef pauseHook (pure ())+  repeated1 <- evaluate (f (Key 1))+  repeated2 <- evaluate (f (Key 2))+  report "memoize under an interleaving" (first /= second && repeated1 == first && repeated2 == second)+    ("(first1,first2,repeat1,repeat2) = " ++ show (first, second, repeated1, repeated2))++-- The same interleaving for a white-noise Gaussian process, whose values at+-- distinct points are independent. The covariance function makes the marked+-- call at (1,1), inside the first thread's query.+gpInterleaving :: IO Bool+gpInterleaving = do+  let cov x y = unsafePerformIO (when (x == 1 && y == 1) markedCall) `seq` (if x == y then 1 else 0)+      f = runProb (gp cov) (randomTree (mkStdGen 8))+  _ <- evaluate (f 0)+  (inside, release) <- pauseAt 1+  slot <- newEmptyMVar+  _ <- forkFinally (evaluate (f 1)) (putMVar slot)+  takeMVar inside+  second <- evaluate (f 2)+  putMVar release ()+  first <- takeMVar slot >>= either throwIO pure+  writeIORef pauseHook (pure ())+  repeated1 <- evaluate (f 1)+  repeated2 <- evaluate (f 2)+  report "white-noise gp under an interleaving" (first /= second && repeated1 == first && repeated2 == second)+    ("(f1,f2,repeat f1,repeat f2) = " ++ show (first, second, repeated1, repeated2))++-- A counter hands out each number once, even to two threads at once. The+-- counter is bound by evaluate, as a Prob bind would bind it: a let-bound+-- counter can be inlined into the lambda below, with one IORef per call.+counterUnique :: IO Bool+counterUnique = do+  counter <- evaluate (runProb newCounter (randomTree (mkStdGen 9)))+  let next k = runProb (readAndIncrement counter) (randomTree (mkStdGen k))+  seeds <- newIORef (0 :: Int)+  numbers <- concurrentDraws 20000 $ do+    k <- atomicModifyIORef' seeds (\s -> (s + 1, s))+    evaluate (next k)+  let distinct = Set.size (Set.fromList numbers)+  report "counter shared by two threads" (distinct == length numbers)+    (show distinct ++ " distinct numbers from " ++ show (length numbers) ++ " increments")++main :: IO ()+main = do+  results <- sequence+    [ distinctFirstDraws "weightedsamples" (fst . head <$> weightedsamples (sample uniform))+    , distinctFirstDraws "mh" (fst . head <$> mh 1 (sample uniform))+    , memoInterleaving+    , gpInterleaving+    , counterUnique+    ]+  unless (and results) exitFailure

@@ -0,0 +1,237 @@+module Main where++import Control.Exception (SomeException, displayException, evaluate, try)+import Control.Monad (forM, forM_, unless)+import Data.List (isPrefixOf, transpose)+import LazyPPL (Tree(..), randomTree, runProb)+import LazyPPL.Distributions (normal)+import LazyPPL.Distributions.GP (gp, wiener)+import LazyPPL.Distributions.GP.Pure (emptyGPState, sampleGP)+import System.Environment (getArgs)+import System.Random (mkStdGen)+import Test.QuickCheck+import Test.QuickCheck.Random (mkQCGen)++finite :: Double -> Bool+finite x = not (isNaN x || isInfinite x)++close :: Double -> Double -> Double -> Bool+close tolerance actual expected =+  finite actual && abs (actual - expected) <= tolerance++-- 2^-33 * cov x x is added to every variance, the first one included.+nugget :: Double+nugget = 2 ** (-33)++-- Runs gp's own sampler, sampleGP, on the draws zs we choose.+samplePath :: (Double -> Double -> Double) -> [Double] -> [Double] -> [Double]+samplePath cov = go emptyGPState+  where+    go _ [] [] = []+    go state (x:xs) (z:zs) =+      let (value,next) = sampleGP cov state x z+      in value : go next xs zs+    go _ _ _ = error "test point/innovation lengths differ"++-- Unit vectors give the columns of L (f = L z). L L^T should equal the kernel plus nugget, so no+-- second Cholesky is needed.+covarianceResidual :: [[Double]] -> Double+covarianceResidual kernel =+  let n = length kernel+      xs = map fromIntegral [0..n-1]+      cov x y = kernel !! round x !! round y+      columns = [samplePath cov xs [if j == k then 1 else 0 | j <- [0..n-1]]+                | k <- [0..n-1]]+      rows = transpose columns+      target = [[v + (if i == j then nugget*v else 0)+                | (j,v) <- zip [0::Int ..] row] | (i,row) <- zip [0..] kernel]+      actual = [[sum (zipWith (*) a b) | b <- rows] | a <- rows]+      norm values = sqrt (sum [v*v | row <- values, v <- row])+  in norm (zipWith (zipWith (-)) target actual) / norm target++-- Closed-form 2-by-2 Cholesky factor, so the reference needs no solver.+twoByTwo :: Property+twoByTwo = forAll (vectorOf 5 (choose (-0.5,0.5))) $ \values ->+  case values of+    [x,b,y,z,w] ->+      let a = 1+abs x; c = 1+abs y; q = 1+nugget+          cov i j = if i == j then (if i == 0 then a else c) else b+          actual = samplePath cov [0,1] [z,w]+          expected = [sqrt (a*q)*z, b/sqrt (a*q)*z + sqrt (c*q-b*b/(a*q))*w]+      in counterexample (show (actual,expected)) $+           and (zipWith (close 1e-12) actual expected)+    _ -> property False++-- A Householder reflection (https://en.wikipedia.org/wiki/Householder_transformation) gives+-- kernels with known eigenvalues, down to 1e-17 and 0. L L^T must reproduce+-- kernel + nugget*diag(kernel), dropping no mode.+knownSpectrum :: Property+knownSpectrum = forAll (chooseInt (3,10)) $ \n ->+  forAll (vectorOf n (choose (0.2,1))) $ \axis ->+  forAll (elements [0,1,2::Int]) $ \kind ->+    let norm = sum (map (\x -> x*x) axis)+        q = [[(if i == j then 1 else 0)-2*a*b/norm+             | (j,b) <- zip [0..] axis] | (i,a) <- zip [0::Int ..] axis]+        eigenvalues = 2 : replicate (n-1) (case kind of 0 -> 0.5; 1 -> 1e-17; _ -> 0)+        kernel = [[sum (zipWith3 (\d a b -> d*a*b) eigenvalues row col)+                  | col <- q] | row <- q]+        residual = covarianceResidual kernel+        -- 64*n*eps covers dot-product rounding and Cholesky's usual n-dependent error constant.+        bound = 64 * fromIntegral n * encodeFloat 1 (-52)+    in counterexample (show (kind,residual,bound)) (finite residual && residual <= bound)++residual :: Property+residual = forAll (chooseInt (2,10)) $ \n ->+  forAll (vectorOf n (vectorOf n (choose (-1,1)))) $ \a ->+    let kernel = [[sum (zipWith (*) row col)+(if i == j then 1 else 0)+                  | (j,col) <- zip [0..] a] | (i,row) <- zip [0::Int ..] a]+        r = covarianceResidual kernel+    in counterexample (show r) (finite r && r <= 64*fromIntegral n*encodeFloat 1 (-52))++-- A diagonal covariance s gives independent draws with standard deviation sqrt(s*(1+nugget)).+scaleCase :: [Double] -> Property+scaleCase scales = forAll (elements scales) $ \s ->+  forAll (vectorOf 4 (choose (-2,2))) $ \zs ->+    let cov x y = if x == y then s else 0+        actual = map (/sqrt s) (samplePath cov [0..3] zs)+        expected = map (*(sqrt (1+nugget))) zs+    in counterexample (show (s,actual,expected)) $+         and (zipWith (close 1e-12) actual expected)++smallScale, largeScale :: Property+smallScale = scaleCase [1e-160,1e-200,1e-250]+largeScale = scaleCase [1e8,1e160,1e200,1e250]++-- gp uses its normal draws in order of first query, so we force each value before asking for+-- the next.+draw :: (Double -> Double -> Double) -> [Double] -> Int -> IO [Double]+draw cov order seed = do+  let f = runProb (gp cov) (randomTree (mkStdGen seed))+  mapM (evaluate . f) order++-- Both query orders must match the specified law and each other. For zero-mean Gaussians the+-- standard error of a mean of X_i X_j is sqrt((Cii*Cjj+Cij^2)/N), and six of them keep false+-- failures rare.+orderIndependence :: Property+orderIndependence = forAll (choose (-0.8,0.8)) $ \rho ->+  forAll (chooseInt (1,1000000)) $ \firstSeed -> ioProperty $ do+    let cov x y = if x == y then (if x == 0 then 1 else 2) else rho+        count = 8192+        q = 1+nugget+        expected = [q,rho,2*q]+        errors = [sqrt (2*q*q/fromIntegral count),+                  sqrt ((2*q*q+rho*rho)/fromIntegral count),+                  sqrt (8*q*q/fromIntegral count)]+        moments pairs = [sum [a*a | [a,_] <- pairs]/fromIntegral count,+                         sum [a*b | [a,b] <- pairs]/fromIntegral count,+                         sum [b*b | [_,b] <- pairs]/fromIntegral count]+    forward <- mapM (draw cov [0,1]) [firstSeed..firstSeed+count-1]+    backward <- mapM (fmap reverse . draw cov [1,0]) [firstSeed+count..firstSeed+2*count-1]+    let a = moments forward; b = moments backward+        matches xs = and (zipWith3 (\v target se -> close (6*se) v target) xs expected errors)+        agrees = and (zipWith3 (\v w se -> close (6*sqrt 2*se) v w) a b errors)+    pure $ counterexample (show (rho,a,b,expected,errors)) (matches a && matches b && agrees)++expectError :: String -> (Double -> Double -> Double) -> [Double] -> IO ()+expectError name cov order = do+  outcome <- try (draw cov order 1) :: IO (Either SomeException [Double])+  case outcome of+    Left err | name `isPrefixOf` displayException err -> pure ()+    _ -> fail ("Expected " ++ name ++ ", got " ++ show outcome)++invalidCovariance :: IO ()+invalidCovariance = expectError "GPCovarianceNotPositiveDefinite"+  (\x y -> if x == y then 100 else 100.3) [0,1]++namedErrors :: IO ()+namedErrors = do+  invalidCovariance+  expectError "GPCovarianceNotPositiveDefinite" (\_ _ -> -1) [0]+  forM_ [0/0,1/0,negate (1/0)] $ \bad -> do+    expectError "GPCovarianceNotFinite" (\_ _ -> bad) [0]+    -- Only cov 0 1 is bad, an entry the factor never reads, yet the error must fire.+    expectError "GPCovarianceNotFinite" (\x y -> if x == 0 && y == 1 then bad else 1) [0,1]+  let zeroCross x y = if x == y then (if x == 0 then 0 else 1) else 0.5+      twoZeros x y = if x == y then 0 else 0.5+  forM_ [[0,1],[1,0]] $ \order -> do+    expectError "GPCovarianceNotPositiveDefinite" zeroCross order+    expectError "GPCovarianceNotPositiveDefinite" twoZeros order++wienerBridge :: IO ()+wienerBridge = forM_ (zip [20261029,20261030] [[2,1,3],[-2,-1,-3]]) $ \(seed,order) -> do+  -- Each path gets its own tree, so the compiler cannot share one runProb wiener expression+  -- and its memo table.+  let tree = randomTree (mkStdGen seed)+      f = runProb wiener tree+      -- The table starts at (0,0). A new point takes its normal draw from child 1 + table size+      -- of the input tree.+      z i = case tree of+        Tree _ children -> runProb (normal 0 1) (children !! i)+      go _ [] = pure ()+      go table (x:xs) = do+        let lower = filter ((<x) . fst) table+            upper = filter ((>x) . fst) table+            (mu,sd) = case (lower,upper) of+              ([],_) -> let (b,v) = minimum upper in (v,sqrt (b-x))+              (_,[]) -> let (a,v) = maximum lower in (v,sqrt (x-a))+              _ -> let (a,v) = maximum lower; (b,w) = minimum upper+                   in (v+(x-a)*(w-v)/(b-a),sqrt ((b-x)*(x-a)/(b-a)))+            expected = mu+sd*z (1+length table)+        actual <- evaluate (f x)+        unless (close 1e-12 actual expected) (fail (show (x,actual,expected)))+        go ((x,actual):table) xs+  go [(0,0)] order++runProperty :: Int -> Property -> IO ()+runProperty count prop = do+  result <- quickCheckWithResult stdArgs {maxSuccess=count,replay=Just(mkQCGen 20261029,0)} prop+  unless (isSuccess result) (fail "Gaussian process property failed")++allChecks :: IO ()+allChecks = do+  namedErrors+  zeros <- draw (\_ _ -> 0) [0,1,-1,2] 1+  unless (zeros == replicate 4 0) (fail ("Zero covariance: " ++ show zeros))+  -- Covariance 1 - 2^-53 leaves the kernel nearly singular, yet the second pivot squared stays+  -- at q - a^2/q, about 2^-32.+  let a = 1-encodeFloat 1 (-53)+      tinyMode x y = if x == y then 1 else a+      [_,innovation] = samplePath tinyMode [0,1] [0,1]+      q = toRational (1+nugget)+      expectedV = fromRational (q-toRational a*toRational a/q)+  unless (close (1e-5*expectedV) (innovation*innovation) expectedV) $+    fail ("Small spectral component: " ++ show (innovation*innovation,expectedV))+  -- Close points of the constant kernel have conditional variance q - 1/q, but the same key+  -- gives its stored value.+  let [_,near] = samplePath (\_ _ -> 1) [0,1e-6] [0,1]+      nearVariance = fromRational (q-1/q)+  unless (close (1e-5*nearVariance) (near*near) nearVariance) $+    fail ("Nearby re-observation: " ++ show (near*near,nearVariance))+  let f = runProb (gp (\_ _ -> 1)) (randomTree (mkStdGen 9))+  first <- evaluate (f 0)+  _ <- evaluate (f 1e-6)+  again <- evaluate (f 0)+  unless (first == again) (fail "Repeated key changed")+  grid <- draw (\x y -> exp (negate ((x-y)^ (2::Int))/0.1))+               [fromIntegral i*0.02 | i <- [0..150::Int]] 7+  unless (length grid == 151 && all finite grid) (fail "RBF grid failed")+  wienerBridge+  runProperty 512 twoByTwo+  runProperty 100 knownSpectrum+  runProperty 100 residual+  runProperty 100 smallScale+  runProperty 100 largeScale+  runProperty 24 orderIndependence++main :: IO ()+main = do+  args <- getArgs+  case args of+    [] -> allChecks+    ["invalid"] -> invalidCovariance+    ["spectrum"] -> runProperty 512 knownSpectrum+    ["small"] -> runProperty 100 smallScale+    ["large"] -> runProperty 100 largeScale+    ["order"] -> runProperty 24 orderIndependence+    ["bridge"] -> wienerBridge+    _ -> fail "Expected no arguments, invalid, spectrum, small, large or order"

@@ -0,0 +1,101 @@+module Main where++-- The documented use of the values with hidden state that runProb returns:+-- counters, IBP restaurants, generalmemoize functions, and the random functions+-- of wiener and gp. Each check binds the value with evaluate before an IO action+-- that runs once per query, and requires the answers that the same queries give+-- inside one Prob computation on the same tree. This module takes the compiler's+-- default options; hidden-state-o2-test compiles it at -O2.++import Control.Exception (evaluate)+import Control.Monad (replicateM, unless)+import Data.IORef (atomicModifyIORef', newIORef)+import Data.List (nub)+import LazyPPL (Prob, Tree(..), randomTree, runProb, uniform)+import LazyPPL.Distributions.Counter (newCounter, readAndIncrement)+import LazyPPL.Distributions.GP (gp, wiener)+import LazyPPL.Distributions.IBP (Dish, newCustomer, newRestaurant)+import LazyPPL.Distributions.Memoization (generalmemoize)+import System.Environment (getArgs)+import System.Exit (exitFailure)+import System.Random (mkStdGen)+import qualified KnownLimit++-- A bind in Prob runs its first computation on the tree's first child.+firstChild :: Tree -> Tree+firstChild (Tree _ (t : _)) = t+firstChild (Tree _ []) = error "a random tree has children"++-- The answers of one IO action that runs once per key: it takes the next key+-- and queries at it.+queried :: (k -> IO a) -> [k] -> IO [a]+queried query keys = do+  remaining <- newIORef keys+  replicateM (length keys) $ do+    key <- atomicModifyIORef' remaining (\ks -> (drop 1 ks, head ks))+    query key++report :: String -> Bool -> String -> IO Bool+report name ok detail = do+  putStrLn ((if ok then "PASS " else "FAIL ") ++ name ++ ": " ++ detail)+  pure ok++-- Inside one Prob computation, the counter gives 0, 1, 2, ... in the order of+-- the reads; read by an IO action after evaluate, it gives the same numbers.+counterCheck :: IO [Bool]+counterCheck = do+  let tree = randomTree (mkStdGen 9)+      n = 2000+  inside <- mapM evaluate (runProb (do c <- newCounter; replicateM n (readAndIncrement c)) tree)+  counter <- evaluate (runProb newCounter (firstChild tree))+  outside <- queried (\k -> evaluate (runProb (readAndIncrement counter) (randomTree (mkStdGen k)))) [1 .. n]+  sequence+    [ report "counter read inside one Prob computation" (inside == [0 .. n - 1]) ("first reads " ++ show (take 5 inside))+    , report "counter evaluated once, then read by an IO action" (outside == [0 .. n - 1]) ("first reads " ++ show (take 5 outside))+    ]++-- Six customers of one IBP restaurant take the first six rows of its matrix.+restaurantCheck :: IO [Bool]+restaurantCheck = do+  let tree = randomTree (mkStdGen 9)+      dishes = map show :: [Dish] -> [String]+  inside <- mapM (evaluate . dishes) (runProb (do r <- newRestaurant 2; replicateM 6 (newCustomer r)) tree)+  restaurant <- evaluate (runProb (newRestaurant 2) (firstChild tree))+  outside <- queried (\k -> evaluate (dishes (runProb (newCustomer restaurant) (randomTree (mkStdGen k))))) [1 .. 6 :: Int]+  sequence+    [ report "restaurant customers inside one Prob computation differ" (length (nub inside) > 1) (show inside)+    , report "restaurant evaluated once, then visited by an IO action" (outside == inside) (show outside)+    ]++-- A memoized, a Wiener and a Gaussian-process function queried at keys with+-- a repeat. Inside one Prob computation, a repeated key gives its first answer+-- and new keys give new answers; queried by an IO action after evaluate, the+-- function gives the same answers.+functionCheck :: String -> Prob (Double -> Double) -> Int -> IO [Bool]+functionCheck name process seed = do+  let tree = randomTree (mkStdGen seed)+      keys = [1, 2, 1, 3, 2]+  inside <- mapM evaluate (runProb (do f <- process; pure (map f keys)) tree)+  f <- evaluate (runProb process (firstChild tree))+  outside <- queried (evaluate . f) keys+  sequence+    [ report (name ++ " queried inside one Prob computation") (length (nub inside) == 3 && inside !! 2 == head inside)+        ("at " ++ show keys ++ ": " ++ show inside)+    , report (name ++ " evaluated once, then queried by an IO action") (outside == inside)+        ("at " ++ show keys ++ ": " ++ show outside)+    ]++main :: IO ()+main = do+  args <- getArgs+  results <- case args of+    ["known-limit"] -> KnownLimit.checks+    _ -> concat <$> sequence+      [ counterCheck+      , restaurantCheck+      , functionCheck "generalmemoize" (generalmemoize (const uniform)) 10+      , functionCheck "wiener" wiener 11+      , functionCheck "white-noise gp" (gp (\x y -> if x == y then 1 else 0)) 12+      ]+  putStrLn (show (length (filter id results)) ++ "/" ++ show (length results) ++ " checks passed.")+  unless (and results) exitFailure

@@ -0,0 +1,76 @@+{-# OPTIONS_GHC -fno-cse -fno-full-laziness #-}+-- The known limit of the values with hidden state that runProb returns: bound+-- by let, or returned inside a function from a Prob computation, and then used+-- by an IO action that runs more than once, such a value can lose its state.+-- GHC treats an IO action's lambda as entered once (the state hack), so it may+-- copy the binding into the action, and every run starts from new state. The+-- options above are those of the concurrency test, where the copy happens.+-- These checks fail while the limit holds; hidden-state-test runs them with+-- the argument known-limit, and its default run does not.+module KnownLimit (checks) where++import Control.Concurrent (forkFinally, newEmptyMVar, putMVar, takeMVar)+import Control.Exception (evaluate, throwIO)+import Control.Monad (forM, replicateM)+import Data.IORef (atomicModifyIORef', newIORef)+import qualified Data.Set as Set+import LazyPPL (randomTree, runProb)+import LazyPPL.Distributions.Counter (newCounter, readAndIncrement)+import LazyPPL.Distributions.GP (wiener)+import System.Random (mkStdGen)++-- Runs the action the given number of times in each of two threads at once.+twoThreads :: Int -> IO a -> IO [a]+twoThreads runs action = do+  slots <- forM [1, 2 :: Int] $ \_ -> do+    slot <- newEmptyMVar+    _ <- forkFinally (replicateM runs (action >>= evaluate)) (putMVar slot)+    pure slot+  concat <$> mapM (\slot -> takeMVar slot >>= either throwIO pure) slots++report :: String -> Bool -> String -> IO Bool+report name ok detail = do+  putStrLn ((if ok then "PASS " else "FAIL ") ++ "known limit, " ++ name ++ ": " ++ detail)+  pure ok++-- A counter bound by let, read 20000 times by each of two threads.+counterLet :: IO Bool+counterLet = do+  let counter = runProb newCounter (randomTree (mkStdGen 9))+      next k = runProb (readAndIncrement counter) (randomTree (mkStdGen k))+  seeds <- newIORef (0 :: Int)+  numbers <- twoThreads 20000 $ do+    k <- atomicModifyIORef' seeds (\s -> (s + 1, s))+    evaluate (next k)+  let distinct = Set.size (Set.fromList numbers)+  report "counter bound by let" (distinct == length numbers)+    (show distinct ++ " distinct numbers from " ++ show (length numbers) ++ " reads")++-- A function that reads a counter, returned from a Prob computation and bound by let.+counterReturned :: IO Bool+counterReturned = do+  let next = runProb (do c <- newCounter+                         pure (\k -> runProb (readAndIncrement c) (randomTree (mkStdGen k))))+                     (randomTree (mkStdGen 9))+  seeds <- newIORef (0 :: Int)+  numbers <- twoThreads 20000 $ do+    k <- atomicModifyIORef' seeds (\s -> (s + 1, s))+    evaluate (next k)+  let distinct = Set.size (Set.fromList numbers)+  report "counter inside a function returned from Prob" (distinct == length numbers)+    (show distinct ++ " distinct numbers from " ++ show (length numbers) ++ " reads")++-- A Wiener function bound by let and queried at 1, then at 2, by two runs of one+-- action. With one table, W(2) is drawn given W(1), and W(1) and W(2)/sqrt 2 differ.+wienerLet :: IO Bool+wienerLet = do+  let f = runProb wiener (randomTree (mkStdGen 9))+  times <- newIORef [1, 2 :: Double]+  values <- replicateM 2 $ do+    t <- atomicModifyIORef' times (\ts -> (drop 1 ts, head ts))+    v <- evaluate (f t)+    pure (v / sqrt t)+  report "wiener bound by let" (values !! 0 /= values !! 1) ("W(1), W(2)/sqrt 2 = " ++ show values)++checks :: IO [Bool]+checks = sequence [counterLet, counterReturned, wienerLet]

@@ -0,0 +1,123 @@+module Main where++import Control.Concurrent (forkFinally, newEmptyMVar, putMVar, takeMVar)+import Control.Exception (evaluate, throwIO)+import Control.Monad (forM, forM_, unless)+import Data.List (nub)+import Data.Word (Word64)+import GHC.Float (castDoubleToWord64)+import LazyPPL.Internal.History (History, appendMutation, labelAt, newHistory)+import LazyPPL.Internal.Sampler (randomLabel)+import System.Exit (exitFailure)+import System.Random (StdGen, mkStdGen, split)+import Test.QuickCheck++-- Reference randomTree and mutateTree, as in LazyPPL 1.0.1, built without replay or the seed+-- store.+data RefTree = RefTree Double [RefTree]++randomRefTree :: StdGen -> RefTree+randomRefTree generator =+  let (value, next) = randomLabel generator+  in RefTree value (randomRefTrees next)++randomRefTrees :: StdGen -> [RefTree]+randomRefTrees generator =+  let (left, right) = split generator+  in randomRefTree left : randomRefTrees right++mutateRefTree :: Double -> StdGen -> RefTree -> RefTree+mutateRefTree rate generator (RefTree value children) =+  let (decision, next) = randomLabel generator+      (replacement, rest) = randomLabel next+  in RefTree (if decision < rate then replacement else value)+       (mutateRefTrees rate rest children)++mutateRefTrees :: Double -> StdGen -> [RefTree] -> [RefTree]+mutateRefTrees rate generator (tree:trees) =+  let (left, right) = split generator+  in mutateRefTree rate left tree : mutateRefTrees rate right trees+mutateRefTrees _ _ [] = []++referenceLabel :: [Int] -> RefTree -> Double+referenceLabel [] (RefTree value _) = value+referenceLabel (index:path) (RefTree _ children) =+  referenceLabel path (children !! index)++data ReplayCase = ReplayCase+  { initialSeed :: Int+  , proposalSeed :: Int+  , prefixSteps :: Int+  , laterSteps :: Int+  , intermediateRate :: Double+  , demandedPaths :: [[Int]]+  } deriving Show++instance Arbitrary ReplayCase where+  arbitrary = do+    initial <- chooseInt (minBound, maxBound)+    proposals <- chooseInt (minBound, maxBound)+    prefix <- oneof [chooseInt (0, 256), elements+      [31,32,33,63,64,65,95,96,97,127,128,129,4095,4096,4097]]+    later <- chooseInt (1, 96)+    rate <- choose (0.0001, 0.9999)+    paths <- listOf1 $ do+      depth <- chooseInt (1, 4)+      vectorOf depth (chooseInt (0, 5))+    pure (ReplayCase initial proposals prefix later rate (nub ([] : take 6 paths)))+  shrink input =+    [input {prefixSteps=n} | n <- shrink (prefixSteps input), n >= 0] +++    [input {initialSeed=n} | n <- shrink (initialSeed input)] +++    [input {proposalSeed=n} | n <- shrink (proposalSeed input)] +++    [input {demandedPaths=[]} | not (null (demandedPaths input))]++proposalGenerators :: StdGen -> [StdGen]+proposalGenerators generator =+  let (left, right) = split generator+  in left : proposalGenerators right++appendAll :: History -> Int -> [StdGen] -> IO ()+appendAll history start generators =+  forM_ (zip [start..] generators) $ \(index, generator) ->+    appendMutation history index generator++replayedLabels :: Double -> History -> Int -> [[Int]] -> IO [Word64]+replayedLabels rate history count paths = do+  -- Each path gets its own thread, so with -N2 labels replay concurrently.+  slots <- forM paths $ \path -> do+    slot <- newEmptyMVar+    _ <- forkFinally+      (evaluate (castDoubleToWord64 (labelAt rate history count (reverse path))))+      (putMVar slot)+    pure slot+  answers <- mapM takeMVar slots+  mapM (either throwIO pure) answers++prop_replayPreservesDelayedLabels :: ReplayCase -> Property+prop_replayPreservesDelayedLabels input =+  classify (prefixSteps input >= 96) "at least 96 appended records" $+  classify (prefixSteps input >= 4095) "near the 4096-record growth" $+  classify (any ((> 1) . length) (demandedPaths input)) "nested paths" $+  ioProperty $ do+    let initial = mkStdGen (initialSeed input)+        generators = proposalGenerators (mkStdGen (proposalSeed input))+        (prefix, suffix) = splitAt (prefixSteps input) generators+        paths = nub ([] : demandedPaths input)+    results <- forM [0, intermediateRate input, 1] $ \rate -> do+      history <- newHistory initial+      appendAll history 0 prefix+      -- Later records are appended before any label is demanded, and replay must ignore them.+      appendAll history (prefixSteps input) (take (laterSteps input) suffix)+      actual <- replayedLabels rate history (prefixSteps input) paths+      let reference = foldl (\tree generator -> mutateRefTree rate generator tree)+                        (randomRefTree initial) prefix+          expected = map (castDoubleToWord64 . (`referenceLabel` reference)) paths+      pure $ counterexample ("rate=" ++ show rate ++ ", paths=" ++ show paths +++                             ", actual=" ++ show actual ++ ", expected=" ++ show expected)+              (actual === expected)+    pure $ conjoin results++main :: IO ()+main = do+  result <- quickCheckResult prop_replayPreservesDelayedLabels+  unless (isSuccess result) exitFailure

@@ -0,0 +1,50 @@+module Main where++import Control.Exception (evaluate)+import Control.Monad (unless)+import LazyPPL (Tree(..), mh, mhirreducible, runProb)+import LazyPPL.Distributions (categorical)+import System.Environment (getArgs)+import System.Exit (exitFailure)++check :: String -> Bool -> IO ()+check name result = do+  putStrLn $ (if result then "PASS " else "FAIL ") ++ name+  unless result exitFailure++categoricalRetention :: Bool -> IO ()+categoricalRetention infinite = do+  let n = 8000000+      remaining = if infinite then 1 : repeat 0 else [1]+      weights = replicate n 0 ++ remaining+      label = let t = Tree 0.5 (repeat t) in t+  selected <- evaluate (runProb (categorical weights) label)+  check (if infinite then "categorical infinite stream fits the 256 MiB heap budget"+                     else "categorical finite stream fits the 256 MiB heap budget")+    (selected == n)++main :: IO ()+main = do+  args <- getArgs+  case args of+    ["categorical-finite"] -> categoricalRetention False+    ["categorical-infinite"] -> categoricalRetention True+    [] -> streamChecks >> categoricalRetention False >> categoricalRetention True+    _ -> error "unknown stream test"++streamChecks :: IO ()+streamChecks = do+  lazyDraws <- mh 0.1 (pure (repeat True))+  check "infinite sample payload remains lazy"+    (take 3 (fst (head lazyDraws)) == [True, True, True])+  lazyIrreducibleDraws <- mhirreducible 0.1 1 (pure (repeat True))+  check "irreducible infinite sample payload remains lazy"+    (take 3 (fst (head lazyIrreducibleDraws)) == [True, True, True])+  draws <- mhirreducible 0.2 1 (pure True)+  result <- evaluate (fst (head (drop 4000000 draws)))+  check "discarding a long chain preserves the constant model" result+  partialDraws <- mh 0.2 (pure True)+  -- Four million accepted mutations require about 61 MiB of packed seeds.+  -- The RTS budget also allows collector growth and the sampler's live tree.+  partialResult <- evaluate (fst (head (drop 4000000 partialDraws)))+  check "partial-mutation seed records fit the 256 MiB heap budget" partialResult

@@ -0,0 +1,24 @@+{-# OPTIONS_GHC -fdefer-type-errors -Wno-deferred-type-errors #-}+-- This module defers type errors to run time, so that it compiles whether or+-- not a strict monad has a MonadMemo instance.+module StrictMemo (strictMemoChecks) where++import Control.Exception (SomeException, TypeError(..), evaluate, fromException, try)+import Data.Maybe (isJust)+import LazyPPL.Distributions.Memoization (memoize)+import System.Timeout (timeout)++-- Memoization at Int builds an infinite trie inside the monad, which only a+-- lazy monad such as Prob can return. A strict monad such as Maybe would+-- never return the memo function, so it has no instance and the call below+-- does not type-check.+strictMemoChecks :: IO [Bool]+strictMemoChecks = do+  result <- try (timeout 200000 (evaluate (isJust (memoize (Just :: Int -> Maybe Int)))))+  let (ok, outcome) = case result of+        Left err | Just (TypeError _) <- fromException err -> (True, "rejected by the type checker")+                 | otherwise -> (False, "an exception: " ++ show (err :: SomeException))+        Right Nothing -> (False, "no result within 0.2 s")+        Right (Just _) -> (False, "a result")+  putStrLn $ (if ok then "PASS" else "FAIL") ++ " memoize at Maybe Int gives " ++ outcome+  pure [ok]

@@ -0,0 +1,45 @@+module Main where++import LazyPPL.Distributions+import System.Exit (exitFailure, exitSuccess)++approxEq :: String -> Double -> Double -> Double -> IO Bool+approxEq name expected actual tol = do+  let ok = expected == actual || abs (expected - actual) < tol+  if ok+    then putStrLn $ "  PASS: " ++ name+    else putStrLn $ "  FAIL: " ++ name ++ " expected=" ++ show expected ++ " actual=" ++ show actual+  return ok++main :: IO ()+main = do+  putStrLn "Testing log-PDF/PMF functions..."+  results <- sequence+    [ approxEq "normalLogPdf 0 1 0" (-0.9189385) (normalLogPdf 0 1 0) 1e-4+    , approxEq "normalLogPdf 0 1 1" (-1.4189385) (normalLogPdf 0 1 1) 1e-4+    , approxEq "exponentialLogPdf 1 1" (-1.0) (exponentialLogPdf 1 1) 1e-6+    , approxEq "exponentialLogPdf 2 0.5" (log 2 - 1.0) (exponentialLogPdf 2 0.5) 1e-6+      -- (k-1)*log(1) - 1/1 - 2*log(1) - logGamma(2) = -1+    , approxEq "gammaLogPdf 2 1 1" (-1.0) (gammaLogPdf 2 1 1) 1e-6+    , approxEq "betaLogPdf 1 1 0.5" 0.0 (betaLogPdf 1 1 0.5) 1e-6+      -- B(2,2) = 1/6, pdf at 0.5 = 6*0.25 = 1.5+    , approxEq "betaLogPdf 2 2 0.5" (log 1.5) (betaLogPdf 2 2 0.5) 1e-6+    , approxEq "bernoulliLogPmf 0.3 True" (log 0.3) (bernoulliLogPmf 0.3 True) 1e-6+    , approxEq "bernoulliLogPmf 0.3 False" (log 0.7) (bernoulliLogPmf 0.3 False) 1e-6+    , approxEq "categoricalLogPmf [0.2,0.3,0.5] 2" (log 0.5) (categoricalLogPmf [0.2, 0.3, 0.5] 2) 1e-6+    , approxEq "categoricalLogPmf OOB" (-1/0) (categoricalLogPmf [0.5, 0.5] 3) 1e-6 -- Outside the support: probability 0, log 0 = -Infinity.+    , approxEq "uniformBoundedLogPdf 0 10 5" (-log 10) (uniformBoundedLogPdf 0 10 5) 1e-6+    , approxEq "uniformBoundedLogPdf OOB" (-1/0) (uniformBoundedLogPdf 0 1 2) 1e-6 -- Outside the support: probability 0, log 0 = -Infinity.+      -- Dirichlet(1,1,1) normalizer = Gamma(3)/(Gamma(1)^3) = 2+    , approxEq "dirichletLogPdf [1,1,1] uniform" (log 2) (dirichletLogPdf [1,1,1] [1/3, 1/3, 1/3]) 1e-4+    , approxEq "logNormalLogPdf 0 1 1" (-0.9189385) (logNormalLogPdf 0 1 1) 1e-4+    , approxEq "logNormalLogPdf negative" (-1/0) (logNormalLogPdf 0 1 (-1)) 1e-6 -- Outside the support: probability 0, log 0 = -Infinity.+      -- C(10,5) = 252+    , approxEq "binomialLogPmf 10 0.5 5" (log 252 - 10 * log 2) (binomialLogPmf 10 0.5 5) 1e-4+    , approxEq "binomialLogPmf OOB" (-1/0) (binomialLogPmf 5 0.3 6) 1e-6 -- Outside the support: probability 0, log 0 = -Infinity.+    , approxEq "poissonLogPmf 3 2" (2 * log 3 - 3 - log 2) (poissonLogPmf 3 2) 1e-6+    ]+  let nPass  = length (filter id results)+      nTotal = length results+  putStrLn $ show nPass ++ "/" ++ show nTotal ++ " tests passed."+  if nPass == nTotal then exitSuccess else exitFailure

@@ -0,0 +1,4274 @@+# Written by test/reference/reference.py with mpmath 1.4.1.+# a	b	x	P(X <= x), nearest double+2.5	3.5	0.024735511935950617	0.001+2.5	3.5	0.03283142434080519	0.0020000000000000005+2.5	3.5	0.03878049240662356	0.0030000000000000005+2.5	3.5	0.04366604159651973	0.004+2.5	3.5	0.04789128932392727	0.005000000000000001+2.5	3.5	0.05165760451749903	0.005999999999999999+2.5	3.5	0.05508220932156212	0.007+2.5	3.5	0.05824023346830339	0.008000000000000002+2.5	3.5	0.06118314256625607	0.009+2.5	3.5	0.06394796069062224	0.01+2.5	3.5	0.06656234329733389	0.011+2.5	3.5	0.06904757312926366	0.011999999999999999+2.5	3.5	0.07142043063430868	0.013+2.5	3.5	0.07369441515898603	0.014+2.5	3.5	0.07588057219558705	0.015000000000000001+2.5	3.5	0.07798807135447577	0.016+2.5	3.5	0.08002462093794613	0.017+2.5	3.5	0.08199677212798966	0.018000000000000002+2.5	3.5	0.08391014663077564	0.019000000000000003+2.5	3.5	0.08576961002160641	0.019999999999999997+2.5	3.5	0.08757940578842681	0.020999999999999998+2.5	3.5	0.08934326041719004	0.022+2.5	3.5	0.09106446679696091	0.023000000000000003+2.5	3.5	0.09274595115872164	0.024+2.5	3.5	0.09439032734434406	0.025+2.5	3.5	0.09599994121101994	0.025999999999999995+2.5	3.5	0.09757690727199335	0.026999999999999996+2.5	3.5	0.09912313916627749	0.028000000000000004+2.5	3.5	0.10064037517845022	0.029000000000000005+2.5	3.5	0.10213019975448094	0.03+2.5	3.5	0.1035940617534504	0.030999999999999996+2.5	3.5	0.10503329001899721	0.031999999999999994+2.5	3.5	0.10644910673502296	0.033+2.5	3.5	0.10784263893812121	0.034+2.5	3.5	0.10921492848753295	0.035+2.5	3.5	0.11056694073719951	0.03599999999999999+2.5	3.5	0.11189957211002574	0.037+2.5	3.5	0.11321365673906711	0.038+2.5	3.5	0.11450997231197947	0.039+2.5	3.5	0.11578924523218148	0.04+2.5	3.5	0.11705215519160758	0.041+2.5	3.5	0.11829933923477379	0.04200000000000001+2.5	3.5	0.11953139538144572	0.043+2.5	3.5	0.120748885864947	0.04399999999999999+2.5	3.5	0.12195234003465266	0.045+2.5	3.5	0.12314225696414395	0.04599999999999999+2.5	3.5	0.124319107800592	0.047+2.5	3.5	0.12548333788597762	0.048+2.5	3.5	0.12663536867657577	0.049000000000000016+2.5	3.5	0.12777559948359626	0.05+2.5	3.5	0.1289044090548724	0.05100000000000001+2.5	3.5	0.13002215701493	0.05200000000000001+2.5	3.5	0.13112918517858535	0.05299999999999999+2.5	3.5	0.1322258187513454	0.054000000000000006+2.5	3.5	0.13331236742827104	0.055+2.5	3.5	0.13438912640157527	0.055999999999999994+2.5	3.5	0.13545637728602283	0.056999999999999995+2.5	3.5	0.1365143889701562	0.05800000000000001+2.5	3.5	0.13756341840046407	0.059+2.5	3.5	0.13860371130481786	0.06+2.5	3.5	0.13963550286080903	0.06100000000000001+2.5	3.5	0.14065901831401537	0.06199999999999999+2.5	3.5	0.14167447355069132	0.06300000000000001+2.5	3.5	0.14268207562890906	0.064+2.5	3.5	0.14368202327176516	0.06499999999999999+2.5	3.5	0.14467450732590137	0.066+2.5	3.5	0.1456597111882657	0.067+2.5	3.5	0.1466378112037534	0.068+2.5	3.5	0.14760897703611217	0.06900000000000002+2.5	3.5	0.1485733720142691	0.07+2.5	3.5	0.14953115345603532	0.071+2.5	3.5	0.15048247297096204	0.07199999999999998+2.5	3.5	0.15142747674396118	0.073+2.5	3.5	0.15236630580115804	0.074+2.5	3.5	0.15329909625931357	0.07500000000000001+2.5	3.5	0.15422597956003617	0.07600000000000001+2.5	3.5	0.1551470826898983	0.07700000000000001+2.5	3.5	0.15606252838747728	0.078+2.5	3.5	0.1569724353382545	0.07900000000000001+2.5	3.5	0.15787691835822895	0.07999999999999999+2.5	3.5	0.1587760885670315	0.081+2.5	3.5	0.15967005355126146	0.082+2.5	3.5	0.1605589175187106	0.083+2.5	3.5	0.16144278144408572	0.08400000000000002+2.5	3.5	0.16232174320679407	0.08500000000000002+2.5	3.5	0.16319589772131188	0.086+2.5	3.5	0.16406533706061674	0.087+2.5	3.5	0.1649301505731277	0.08799999999999998+2.5	3.5	0.16579042499356497	0.08900000000000001+2.5	3.5	0.16664624454810908	0.09+2.5	3.5	0.16749769105421355	0.091+2.5	3.5	0.16834484401539748	0.09200000000000001+2.5	3.5	0.169187780711323	0.09299999999999999+2.5	3.5	0.1700265762834399	0.09399999999999999+2.5	3.5	0.17086130381646	0.095+2.5	3.5	0.17169203441590664	0.096+2.5	3.5	0.1725188372819665	0.09699999999999999+2.5	3.5	0.1733417797798567	0.09799999999999999+2.5	3.5	0.17416092750690534	0.09900000000000002+2.5	3.5	0.17497634435653014	0.1+2.5	3.5	0.17578809257928898	0.101+2.5	3.5	0.176596232841163	0.10199999999999998+2.5	3.5	0.1774008242792242	0.10299999999999998+2.5	3.5	0.17820192455482892	0.10400000000000001+2.5	3.5	0.17899958990446949	0.10499999999999998+2.5	3.5	0.1797938751884094	0.106+2.5	3.5	0.1805848339372171	0.10699999999999998+2.5	3.5	0.18137251839630958	0.108+2.5	3.5	0.18215697956860688	0.10899999999999999+2.5	3.5	0.1829382672553957	0.11000000000000001+2.5	3.5	0.1837164300954915	0.11100000000000002+2.5	3.5	0.18449151560278576	0.112+2.5	3.5	0.18526357020225806	0.113+2.5	3.5	0.18603263926452882	0.114+2.5	3.5	0.1867987671390245	0.11500000000000002+2.5	3.5	0.18756199718582212	0.11599999999999999+2.5	3.5	0.18832237180623693	0.117+2.5	3.5	0.1890799324722127	0.118+2.5	3.5	0.18983471975457178	0.119+2.5	3.5	0.19058677335017798	0.12000000000000001+2.5	3.5	0.19133613210806288	0.121+2.5	3.5	0.1920828340545635	0.12200000000000001+2.5	3.5	0.19282691641751626	0.12299999999999998+2.5	3.5	0.19356841564955043	0.124+2.5	3.5	0.1943073674505211	0.125+2.5	3.5	0.1950438067891204	0.126+2.5	3.5	0.1957777679237033	0.127+2.5	3.5	0.19650928442236248	0.128+2.5	3.5	0.19723838918228478	0.129+2.5	3.5	0.19796511444842066	0.13+2.5	3.5	0.1986894918314959	0.13100000000000003+2.5	3.5	0.19941155232539365	0.132+2.5	3.5	0.2001313263239335	0.133+2.5	3.5	0.2008488436370725	0.134+2.5	3.5	0.20156413350655286	0.135+2.5	3.5	0.20227722462101855	0.13600000000000004+2.5	3.5	0.20298814513062285	0.13699999999999998+2.5	3.5	0.20369692266114792	0.138+2.5	3.5	0.2044035843276552	0.13899999999999998+2.5	3.5	0.2051081567476867	0.14+2.5	3.5	0.20581066605403392	0.14100000000000001+2.5	3.5	0.20651113790709233	0.142+2.5	3.5	0.20720959750681733	0.143+2.5	3.5	0.20790606960429722	0.144+2.5	3.5	0.20860057851295818	0.145+2.5	3.5	0.20929314811941535	0.146+2.5	3.5	0.20998380189398352	0.147+2.5	3.5	0.21067256290086034	0.148+2.5	3.5	0.21135945380799442	0.149+2.5	3.5	0.21204449689665028	0.15+2.5	3.5	0.21272771407068097	0.15100000000000002+2.5	3.5	0.21340912686551977	0.152+2.5	3.5	0.21408875645690087	0.15300000000000002+2.5	3.5	0.21476662366931887	0.15399999999999997+2.5	3.5	0.21544274898423713	0.15500000000000003+2.5	3.5	0.21611715254805303	0.156+2.5	3.5	0.21678985417983004	0.157+2.5	3.5	0.2174608733788038	0.158+2.5	3.5	0.21813022933167084	0.15900000000000003+2.5	3.5	0.21879794091966737	0.16+2.5	3.5	0.21946402672544554	0.161+2.5	3.5	0.22012850503975404	0.162+2.5	3.5	0.22079139386792984	0.16299999999999998+2.5	3.5	0.22145271093620783	0.164+2.5	3.5	0.2221124736978539	0.165+2.5	3.5	0.22277069933912821	0.16600000000000004+2.5	3.5	0.22342740478508374	0.16699999999999998+2.5	3.5	0.2240826067052061	0.16800000000000004+2.5	3.5	0.22473632151889925	0.16899999999999998+2.5	3.5	0.22538856540082317	0.17+2.5	3.5	0.22603935428608726	0.171+2.5	3.5	0.22668870387530507	0.172+2.5	3.5	0.22733662963951434	0.173+2.5	3.5	0.2279831468249669	0.174+2.5	3.5	0.2286282704577925	0.175+2.5	3.5	0.22927201534854075	0.176+2.5	3.5	0.2299143960966049	0.177+2.5	3.5	0.23055542709453125	0.17800000000000002+2.5	3.5	0.23119512253221763	0.179+2.5	3.5	0.2318334964010048	0.18+2.5	3.5	0.23247056249766357	0.181+2.5	3.5	0.23310633442828121	0.182+2.5	3.5	0.23374082561205023	0.18300000000000002+2.5	3.5	0.23437404928496225	0.184+2.5	3.5	0.23500601850341027	0.185+2.5	3.5	0.23563674614770155	0.18599999999999997+2.5	3.5	0.23626624492548434	0.187+2.5	3.5	0.23689452737509062	0.18800000000000003+2.5	3.5	0.23752160586879753	0.189+2.5	3.5	0.23814749261601006	0.19000000000000003+2.5	3.5	0.23877219966636676	0.191+2.5	3.5	0.2393957389127714	0.192+2.5	3.5	0.24001812209435233	0.193+2.5	3.5	0.24063936079935155	0.19400000000000003+2.5	3.5	0.24125946646794597	0.19500000000000003+2.5	3.5	0.24187845039500228	0.19599999999999998+2.5	3.5	0.24249632373276778	0.197+2.5	3.5	0.24311309749349844	0.198+2.5	3.5	0.24372878255202665	0.19899999999999998+2.5	3.5	0.2443433896482698	0.2+2.5	3.5	0.24495692938968142	0.201+2.5	3.5	0.24556941225364695	0.202+2.5	3.5	0.2461808485898249	0.20299999999999999+2.5	3.5	0.2467912486224356	0.20400000000000001+2.5	3.5	0.24740062245249853	0.20499999999999996+2.5	3.5	0.24800898006001998	0.20600000000000002+2.5	3.5	0.24861633130613187	0.207+2.5	3.5	0.24922268593518376	0.208+2.5	3.5	0.2498280535767887	0.20899999999999996+2.5	3.5	0.2504324437478244	0.21000000000000002+2.5	3.5	0.2510358658543909	0.21100000000000002+2.5	3.5	0.251638329193726	0.21199999999999997+2.5	3.5	0.25223984295607926	0.21300000000000002+2.5	3.5	0.2528404162265459	0.21400000000000002+2.5	3.5	0.25344005798686176	0.21499999999999994+2.5	3.5	0.25403877711716	0.21600000000000005+2.5	3.5	0.25463658239769044	0.217+2.5	3.5	0.2552334825105035	0.218+2.5	3.5	0.25582948604109806	0.21899999999999997+2.5	3.5	0.25642460148003565	0.21999999999999997+2.5	3.5	0.25701883722452074	0.22099999999999997+2.5	3.5	0.2576122015799487	0.222+2.5	3.5	0.25820470276142177	0.22299999999999998+2.5	3.5	0.25879634889523445	0.224+2.5	3.5	0.2593871480203281	0.22500000000000006+2.5	3.5	0.25997710808971636	0.22600000000000003+2.5	3.5	0.26056623697188214	0.22700000000000004+2.5	3.5	0.26115454245214603	0.22800000000000004+2.5	3.5	0.2617420322340077	0.22899999999999998+2.5	3.5	0.2623287139404607	0.22999999999999998+2.5	3.5	0.2629145951152809	0.23100000000000004+2.5	3.5	0.2634996832242899	0.23199999999999998+2.5	3.5	0.2640839856565935	0.23299999999999998+2.5	3.5	0.26466750972579606	0.23399999999999996+2.5	3.5	0.26525026267119106	0.235+2.5	3.5	0.26583225165892893	0.23600000000000002+2.5	3.5	0.2664134837831621	0.23699999999999996+2.5	3.5	0.2669939660671684	0.23800000000000002+2.5	3.5	0.26757370546445264	0.23900000000000002+2.5	3.5	0.26815270885982745	0.23999999999999996+2.5	3.5	0.26873098307047394	0.24099999999999996+2.5	3.5	0.26930853484698175	0.24200000000000005+2.5	3.5	0.26988537087437003	0.24300000000000002+2.5	3.5	0.2704614977730895	0.24400000000000005+2.5	3.5	0.27103692210000524	0.24500000000000005+2.5	3.5	0.2716116503493621	0.24599999999999997+2.5	3.5	0.2721856889537317	0.24700000000000003+2.5	3.5	0.27275904428494213	0.24799999999999997+2.5	3.5	0.27333172265499117	0.24900000000000005+2.5	3.5	0.27390373031694204	0.25+2.5	3.5	0.27447507346580385	0.251+2.5	3.5	0.2750457582393955	0.252+2.5	3.5	0.2756157907191942	0.253+2.5	3.5	0.2761851769311691	0.254+2.5	3.5	0.2767539228465999	0.255+2.5	3.5	0.2773220343828807	0.25599999999999995+2.5	3.5	0.2778895174043103	0.25700000000000006+2.5	3.5	0.2784563777228675	0.258+2.5	3.5	0.2790226210989745	0.25899999999999995+2.5	3.5	0.2795882532422453	0.26+2.5	3.5	0.28015327981222193	0.26099999999999995+2.5	3.5	0.2807177064190983	0.262+2.5	3.5	0.2812815386244308	0.26300000000000007+2.5	3.5	0.281844781941837	0.264+2.5	3.5	0.28240744183768296	0.26499999999999996+2.5	3.5	0.2829695237317582	0.26600000000000007+2.5	3.5	0.2835310329979393	0.26700000000000007+2.5	3.5	0.2840919749648428	0.268+2.5	3.5	0.28465235491646673	0.26899999999999996+2.5	3.5	0.2852121780928216	0.26999999999999996+2.5	3.5	0.28577144969055085	0.271+2.5	3.5	0.28633017486354084	0.2720000000000001+2.5	3.5	0.2868883587235212	0.273+2.5	3.5	0.287446006340655	0.27399999999999997+2.5	3.5	0.2880031227441194	0.27499999999999997+2.5	3.5	0.2885597129226766	0.276+2.5	3.5	0.2891157818252361	0.277+2.5	3.5	0.2896713343614071	0.2780000000000001+2.5	3.5	0.29022637540204255	0.279+2.5	3.5	0.2907809097797747	0.2800000000000001+2.5	3.5	0.2913349422895413	0.281+2.5	3.5	0.29188847768910425	0.282+2.5	3.5	0.2924415206995596	0.283+2.5	3.5	0.2929940760058396	0.284+2.5	3.5	0.29354614825720704	0.2849999999999999+2.5	3.5	0.2940977420677418	0.28600000000000003+2.5	3.5	0.29464886201681956	0.28700000000000003+2.5	3.5	0.29519951264958355	0.2879999999999999+2.5	3.5	0.2957496984774089	0.2889999999999999+2.5	3.5	0.2962994239783595	0.29+2.5	3.5	0.29684869359763844	0.291+2.5	3.5	0.29739751174803103	0.2919999999999999+2.5	3.5	0.2979458828103418	0.2929999999999999+2.5	3.5	0.2984938111338241	0.294+2.5	3.5	0.2990413010366039	0.29499999999999993+2.5	3.5	0.299588356806097	0.29599999999999993+2.5	3.5	0.3001349826994199	0.297+2.5	3.5	0.30068118294379476	0.29799999999999993+2.5	3.5	0.30122696173694846	0.29900000000000004+2.5	3.5	0.3017723232475054	0.3+2.5	3.5	0.30231727161537525	0.301+2.5	3.5	0.30286181095213405	0.30199999999999994+2.5	3.5	0.3034059453414009	0.303+2.5	3.5	0.3039496788392083	0.304+2.5	3.5	0.30449301547436763	0.30500000000000005+2.5	3.5	0.30503595924882926	0.30599999999999994+2.5	3.5	0.30557851413803777	0.30700000000000005+2.5	3.5	0.30612068409128146	0.30799999999999994+2.5	3.5	0.3066624730320379	0.30899999999999994+2.5	3.5	0.3072038848583138	0.30999999999999994+2.5	3.5	0.3077449234429804	0.311+2.5	3.5	0.3082855926341042	0.312+2.5	3.5	0.30882589625527346	0.313+2.5	3.5	0.30936583810591933	0.31400000000000006+2.5	3.5	0.30990542196163334	0.31500000000000006+2.5	3.5	0.31044465157448026	0.316+2.5	3.5	0.31098353067330675	0.31700000000000006+2.5	3.5	0.31152206296404555	0.31799999999999995+2.5	3.5	0.3120602521300162	0.319+2.5	3.5	0.3125981018322209	0.32000000000000006+2.5	3.5	0.3131356157096371	0.32100000000000006+2.5	3.5	0.3136727973795058	0.322+2.5	3.5	0.3142096504376164	0.323+2.5	3.5	0.3147461784585871	0.324+2.5	3.5	0.31528238499614253	0.32500000000000007+2.5	3.5	0.315818273583387	0.32599999999999996+2.5	3.5	0.31635384773307473	0.32699999999999996+2.5	3.5	0.3168891109378762	0.328+2.5	3.5	0.31742406667064116	0.329+2.5	3.5	0.31795871838465856	0.33+2.5	3.5	0.3184930695139128	0.33100000000000007+2.5	3.5	0.3190271234733369	0.332+2.5	3.5	0.3195608836590624	0.33299999999999996+2.5	3.5	0.3200943534486664	0.334+2.5	3.5	0.32062753620141493	0.335+2.5	3.5	0.321160435258504	0.336+2.5	3.5	0.32169305394329717	0.33699999999999997+2.5	3.5	0.3222253955615604	0.33799999999999997+2.5	3.5	0.32275746340169387	0.339+2.5	3.5	0.3232892607349613	0.34+2.5	3.5	0.3238207908157161	0.3410000000000001+2.5	3.5	0.32435205688162505	0.34199999999999997+2.5	3.5	0.32488306215388946	0.34299999999999997+2.5	3.5	0.3254138098374631	0.34400000000000003+2.5	3.5	0.3259443031212681	0.345+2.5	3.5	0.3264745451784082	0.3459999999999999+2.5	3.5	0.3270045391663794	0.347+2.5	3.5	0.3275342882272781	0.348+2.5	3.5	0.32806379548800696	0.349+2.5	3.5	0.3285930640604785	0.35+2.5	3.5	0.329122097041816	0.351+2.5	3.5	0.32965089751455245	0.352+2.5	3.5	0.33017946854682717	0.353+2.5	3.5	0.33070781319258	0.3539999999999999+2.5	3.5	0.33123593449174393	0.35500000000000004+2.5	3.5	0.3317638354704345	0.35600000000000004+2.5	3.5	0.33229151914113825	0.357+2.5	3.5	0.3328189885028987	0.35800000000000004+2.5	3.5	0.33334624654149975	0.359+2.5	3.5	0.333873296229648	0.36+2.5	3.5	0.3344001405271525	0.361+2.5	3.5	0.3349267823811027	0.36200000000000004+2.5	3.5	0.33545322472604455	0.363+2.5	3.5	0.335979470484155	0.36400000000000005+2.5	3.5	0.336505522565414	0.365+2.5	3.5	0.3370313838677755	0.366+2.5	3.5	0.337557057277336	0.367+2.5	3.5	0.3380825456685019	0.36799999999999994+2.5	3.5	0.3386078519041548	0.36900000000000005+2.5	3.5	0.3391329788358152	0.37+2.5	3.5	0.3396579293038047	0.371+2.5	3.5	0.34018270613740625	0.372+2.5	3.5	0.34070731215502337	0.373+2.5	3.5	0.3412317501643371	0.37400000000000005+2.5	3.5	0.34175602296246194	0.375+2.5	3.5	0.34228013333610013	0.37599999999999995+2.5	3.5	0.3428040840616945	0.37700000000000006+2.5	3.5	0.3433278779055794	0.37800000000000006+2.5	3.5	0.343851517624131	0.379+2.5	3.5	0.34437500596391574	0.38+2.5	3.5	0.344898345661837	0.381+2.5	3.5	0.34542153944528137	0.38199999999999995+2.5	3.5	0.34594459003226263	0.383+2.5	3.5	0.34646750013156496	0.38399999999999995+2.5	3.5	0.34699027244288483	0.38499999999999995+2.5	3.5	0.34751290965697135	0.386+2.5	3.5	0.3480354144557656	0.38700000000000007+2.5	3.5	0.3485577895125386	0.388+2.5	3.5	0.34908003749202837	0.389+2.5	3.5	0.3496021610505754	0.39+2.5	3.5	0.3501241628362572	0.39100000000000007+2.5	3.5	0.3506460454890215	0.392+2.5	3.5	0.3511678116408189	0.393+2.5	3.5	0.3516894639157336	0.394+2.5	3.5	0.3522110049301136	0.3950000000000001+2.5	3.5	0.35273243729269976	0.396+2.5	3.5	0.3532537636047538	0.39699999999999996+2.5	3.5	0.3537749864601851	0.3980000000000001+2.5	3.5	0.3542961084456766	0.39899999999999997+2.5	3.5	0.35481713214081007	0.4+2.5	3.5	0.35533806011818964	0.4010000000000001+2.5	3.5	0.3558588949435652	0.402+2.5	3.5	0.3563796391759544	0.403+2.5	3.5	0.35690029536776374	0.40399999999999997+2.5	3.5	0.3574208660649091	0.405+2.5	3.5	0.3579413538069348	0.4060000000000001+2.5	3.5	0.35846176112713246	0.407+2.5	3.5	0.35898209055265873	0.40800000000000003+2.5	3.5	0.3595023446046518	0.409+2.5	3.5	0.360022525798348	0.41+2.5	3.5	0.36054263664319647	0.411+2.5	3.5	0.3610626796429743	0.41200000000000003+2.5	3.5	0.36158265729589956	0.413+2.5	3.5	0.3621025720947449	0.414+2.5	3.5	0.36262242652694937	0.41500000000000004+2.5	3.5	0.36314222307473004	0.416+2.5	3.5	0.36366196421519287	0.417+2.5	3.5	0.3641816524204428	0.41800000000000004+2.5	3.5	0.3647012901576931	0.41900000000000004+2.5	3.5	0.3652208798893742	0.42+2.5	3.5	0.36574042407324187	0.42100000000000004+2.5	3.5	0.3662599251624846	0.422+2.5	3.5	0.3667793856058306	0.423+2.5	3.5	0.36729880784765395	0.42399999999999993+2.5	3.5	0.3678181943280803	0.42500000000000004+2.5	3.5	0.3683375474830917	0.42599999999999993+2.5	3.5	0.3688568697446319	0.42699999999999994+2.5	3.5	0.3693761635407093	0.428+2.5	3.5	0.369895431295501	0.42899999999999994+2.5	3.5	0.37041467542945566	0.43+2.5	3.5	0.37093389835939555	0.431+2.5	3.5	0.37145310249861874	0.432+2.5	3.5	0.3719722902570003	0.43299999999999994+2.5	3.5	0.37249146404109335	0.43399999999999994+2.5	3.5	0.3730106262542293	0.435+2.5	3.5	0.373529779296618	0.436+2.5	3.5	0.37404892556544717	0.437+2.5	3.5	0.3745680674549815	0.43799999999999994+2.5	3.5	0.37508720735666146	0.43899999999999995+2.5	3.5	0.3756063476592014	0.43999999999999995+2.5	3.5	0.3761254907486877	0.44100000000000006+2.5	3.5	0.3766446390086757	0.442+2.5	3.5	0.37716379482028756	0.443+2.5	3.5	0.37768296056230855	0.444+2.5	3.5	0.37820213861128354	0.44500000000000006+2.5	3.5	0.3787213313416131	0.44600000000000006+2.5	3.5	0.37924054112564926	0.447+2.5	3.5	0.3797597703337909	0.44799999999999995+2.5	3.5	0.38027902133457886	0.449+2.5	3.5	0.38079829649479074	0.45+2.5	3.5	0.3813175981795355	0.451+2.5	3.5	0.3818369287523476	0.452+2.5	3.5	0.3823562905752811	0.45299999999999996+2.5	3.5	0.3828756860090034	0.45399999999999996+2.5	3.5	0.3833951174128887	0.45500000000000007+2.5	3.5	0.383914587145111	0.45600000000000007+2.5	3.5	0.38443409756273766	0.45699999999999996+2.5	3.5	0.3849536510218219	0.45799999999999996+2.5	3.5	0.3854732498774953	0.45899999999999996+2.5	3.5	0.3859928964840606	0.46+2.5	3.5	0.3865125931950835	0.461+2.5	3.5	0.38703234236348516	0.462+2.5	3.5	0.38755214634163376	0.463+2.5	3.5	0.3880720074814365	0.46399999999999997+2.5	3.5	0.3885919281344311	0.465+2.5	3.5	0.3891119106518773	0.4660000000000001+2.5	3.5	0.38963195738484807	0.467+2.5	3.5	0.39015207068432123	0.46799999999999997+2.5	3.5	0.3906722529012702	0.469+2.5	3.5	0.3911925063867553	0.4699999999999999+2.5	3.5	0.3917128334920145	0.471+2.5	3.5	0.3922332365685545	0.472+2.5	3.5	0.3927537179682415	0.47300000000000003+2.5	3.5	0.39327428004339177	0.47400000000000003+2.5	3.5	0.3937949251468627	0.47500000000000003+2.5	3.5	0.39431565563214327	0.476+2.5	3.5	0.3948364738534448	0.4769999999999999+2.5	3.5	0.3953573821657915	0.478+2.5	3.5	0.3958783829251112	0.479+2.5	3.5	0.39639947848832585	0.4799999999999999+2.5	3.5	0.39692067121344243	0.48100000000000004+2.5	3.5	0.3974419634596432	0.482+2.5	3.5	0.39796335758737694	0.483+2.5	3.5	0.3984848559584491	0.484+2.5	3.5	0.3990064609361132	0.48500000000000004+2.5	3.5	0.399528174885161	0.48600000000000004+2.5	3.5	0.40005000017201403	0.487+2.5	3.5	0.40057193916481426	0.48799999999999993+2.5	3.5	0.40109399423351527	0.489+2.5	3.5	0.40161616774997333	0.48999999999999994+2.5	3.5	0.4021384620880389	0.49100000000000005+2.5	3.5	0.40266087962364766	0.49200000000000005+2.5	3.5	0.4031834227349125	0.493+2.5	3.5	0.40370609380221484	0.494+2.5	3.5	0.4042288952082966	0.49500000000000005+2.5	3.5	0.40475182933835196	0.496+2.5	3.5	0.40527489858011984	0.497+2.5	3.5	0.4057981053239758	0.498+2.5	3.5	0.4063214519630249	0.49900000000000005+2.5	3.5	0.4068449408931939	0.5+2.5	3.5	0.4073685745133247	0.501+2.5	3.5	0.40789235522526696	0.502+2.5	3.5	0.4084162854339716	0.503+2.5	3.5	0.40894036754758456	0.504+2.5	3.5	0.4094646039775401	0.505+2.5	3.5	0.40998899713865516	0.506+2.5	3.5	0.41051354944922364	0.507+2.5	3.5	0.41103826333111093	0.508+2.5	3.5	0.41156314120984877	0.509+2.5	3.5	0.4120881855147303	0.51+2.5	3.5	0.41261339867890573	0.511+2.5	3.5	0.41313878313947777	0.512+2.5	3.5	0.413664341337598	0.513+2.5	3.5	0.41419007571856314	0.514+2.5	3.5	0.41471598873191196	0.515+2.5	3.5	0.4152420828315221	0.516+2.5	3.5	0.4157683604757078	0.517+2.5	3.5	0.41629482412731766	0.518+2.5	3.5	0.4168214762538328	0.519+2.5	3.5	0.41734831932746563	0.52+2.5	3.5	0.41787535582525887	0.521+2.5	3.5	0.4184025882291848	0.522+2.5	3.5	0.4189300190262453	0.523+2.5	3.5	0.4194576507085723	0.524+2.5	3.5	0.41998548577352834	0.525+2.5	3.5	0.42051352672380765	0.526+2.5	3.5	0.4210417760675384	0.527+2.5	3.5	0.4215702363183843	0.528+2.5	3.5	0.42209890999564764	0.529+2.5	3.5	0.4226277996243723	0.53+2.5	3.5	0.4231569077354476	0.531+2.5	3.5	0.42368623686571244	0.532+2.5	3.5	0.42421578955806005	0.533+2.5	3.5	0.42474556836154337	0.534+2.5	3.5	0.425275575831481	0.535+2.5	3.5	0.4258058145295636	0.536+2.5	3.5	0.4263362870239609	0.537+2.5	3.5	0.4268669958894296	0.538+2.5	3.5	0.4273979437074213	0.539+2.5	3.5	0.42792913306619174	0.54+2.5	3.5	0.4284605665609102	0.541+2.5	3.5	0.42899224679376957	0.542+2.5	3.5	0.4295241763740975	0.543+2.5	3.5	0.4300563579184678	0.544+2.5	3.5	0.4305887940508126	0.545+2.5	3.5	0.4311214874025353	0.546+2.5	3.5	0.4316544406126246	0.547+2.5	3.5	0.4321876563277683	0.548+2.5	3.5	0.4327211372024688	0.549+2.5	3.5	0.433254885899159	0.55+2.5	3.5	0.43378890508831885	0.551+2.5	3.5	0.4343231974485929	0.552+2.5	3.5	0.4348577656669084	0.553+2.5	3.5	0.43539261243859484	0.554+2.5	3.5	0.43592774046750316	0.555+2.5	3.5	0.4364631524661271	0.556+2.5	3.5	0.4369988511557245	0.557+2.5	3.5	0.43753483926644	0.558+2.5	3.5	0.4380711195374283	0.559+2.5	3.5	0.4386076947169784	0.56+2.5	3.5	0.4391445675626391	0.561+2.5	3.5	0.43968174084134476	0.562+2.5	3.5	0.44021921732954283	0.563+2.5	3.5	0.4407569998133217	0.564+2.5	3.5	0.4412950910885398	0.565+2.5	3.5	0.44183349396095567	0.566+2.5	3.5	0.4423722112463592	0.567+2.5	3.5	0.4429112457707037	0.568+2.5	3.5	0.443450600370239	0.569+2.5	3.5	0.4439902778916459	0.57+2.5	3.5	0.4445302811921715	0.571+2.5	3.5	0.44507061313976554	0.572+2.5	3.5	0.4456112766132183	0.573+2.5	3.5	0.44615227450229905	0.574+2.5	3.5	0.4466936097078963	0.575+2.5	3.5	0.4472352851421585	0.576+2.5	3.5	0.4477773037286369	0.577+2.5	3.5	0.4483196684024285	0.578+2.5	3.5	0.44886238211032137	0.579+2.5	3.5	0.44940544781094044	0.58+2.5	3.5	0.44994886847489485	0.581+2.5	3.5	0.45049264708492665	0.582+2.5	3.5	0.4510367866360608	0.583+2.5	3.5	0.45158129013575643	0.584+2.5	3.5	0.4521261606040596	0.585+2.5	3.5	0.4526714010737573	0.586+2.5	3.5	0.4532170145905332	0.587+2.5	3.5	0.45376300421312427	0.588+2.5	3.5	0.4543093730134794	0.589+2.5	3.5	0.4548561240769193	0.59+2.5	3.5	0.4554032605022976	0.591+2.5	3.5	0.45595078540216427	0.592+2.5	3.5	0.4564987019029298	0.593+2.5	3.5	0.45704701314503116	0.594+2.5	3.5	0.4575957222830998	0.595+2.5	3.5	0.4581448324861307	0.596+2.5	3.5	0.45869434693765354	0.597+2.5	3.5	0.45924426883590513	0.598+2.5	3.5	0.45979460139400385	0.599+2.5	3.5	0.46034534784012593	0.6+2.5	3.5	0.46089651141768295	0.601+2.5	3.5	0.46144809538550186	0.602+2.5	3.5	0.4620001030180063	0.603+2.5	3.5	0.4625525376053998	0.604+2.5	3.5	0.4631054024538513	0.605+2.5	3.5	0.4636587008856819	0.606+2.5	3.5	0.46421243623955427	0.607+2.5	3.5	0.4647666118706634	0.608+2.5	3.5	0.46532123115092994	0.609+2.5	3.5	0.4658762974691948	0.61+2.5	3.5	0.4664318142314168	0.611+2.5	3.5	0.46698778486087156	0.612+2.5	3.5	0.46754421279835295	0.613+2.5	3.5	0.4681011015023767	0.614+2.5	3.5	0.4686584544493861	0.615+2.5	3.5	0.4692162751339598	0.616+2.5	3.5	0.4697745670690223	0.617+2.5	3.5	0.4703333337860564	0.618+2.5	3.5	0.470892578835318	0.619+2.5	3.5	0.47145230578605335	0.62+2.5	3.5	0.47201251822671914	0.621+2.5	3.5	0.47257321976520406	0.622+2.5	3.5	0.4731344140290539	0.623+2.5	3.5	0.4736961046656984	0.624+2.5	3.5	0.4742582953426814	0.625+2.5	3.5	0.4748209897478929	0.626+2.5	3.5	0.4753841915898042	0.627+2.5	3.5	0.47594790459770575	0.628+2.5	3.5	0.4765121325219475	0.629+2.5	3.5	0.47707687913418234	0.63+2.5	3.5	0.4776421482276122	0.631+2.5	3.5	0.4782079436172369	0.632+2.5	3.5	0.47877426914010635	0.633+2.5	3.5	0.4793411286555754	0.634+2.5	3.5	0.47990852604556156	0.635+2.5	3.5	0.4804764652148066	0.636+2.5	3.5	0.4810449500911399	0.637+2.5	3.5	0.48161398462574667	0.638+2.5	3.5	0.48218357279343776	0.639+2.5	3.5	0.48275371859292404	0.64+2.5	3.5	0.4833244260470933	0.641+2.5	3.5	0.48389569920329095	0.642+2.5	3.5	0.484467542133604	0.643+2.5	3.5	0.48503995893514856	0.644+2.5	3.5	0.4856129537303612	0.645+2.5	3.5	0.4861865306672931	0.646+2.5	3.5	0.4867606939199093	0.647+2.5	3.5	0.4873354476883899	0.648+2.5	3.5	0.4879107961994368	0.649+2.5	3.5	0.48848674370658296	0.65+2.5	3.5	0.4890632944905066	0.651+2.5	3.5	0.48964045285934865	0.652+2.5	3.5	0.4902182231490346	0.653+2.5	3.5	0.49079660972360084	0.654+2.5	3.5	0.4913756169755242	0.655+2.5	3.5	0.49195524932605683	0.656+2.5	3.5	0.49253551122556505	0.657+2.5	3.5	0.4931164071538723	0.658+2.5	3.5	0.493697941620607	0.659+2.5	3.5	0.4942801191655548	0.66+2.5	3.5	0.49486294435901573	0.661+2.5	3.5	0.4954464218021656	0.662+2.5	3.5	0.49603055612742264	0.663+2.5	3.5	0.49661535199881907	0.664+2.5	3.5	0.49720081411237715	0.665+2.5	3.5	0.497786947196491	0.666+2.5	3.5	0.498373756012313	0.667+2.5	3.5	0.49896124535414565	0.668+2.5	3.5	0.4995494200498389	0.669+2.5	3.5	0.5001382849611928	0.67+2.5	3.5	0.5007278449843657	0.671+2.5	3.5	0.5013181050502878	0.672+2.5	3.5	0.5019090701250811	0.673+2.5	3.5	0.5025007452104844	0.674+2.5	3.5	0.5030931353442846	0.675+2.5	3.5	0.5036862456007545	0.676+2.5	3.5	0.5042800810910956	0.677+2.5	3.5	0.5048746469638884	0.6779999999999999+2.5	3.5	0.5054699484055483	0.679+2.5	3.5	0.5060659906407877	0.68+2.5	3.5	0.5066627789330862	0.6810000000000002+2.5	3.5	0.5072603185851654	0.682+2.5	3.5	0.5078586149394725	0.683+2.5	3.5	0.5084576733786694	0.684+2.5	3.5	0.5090574993261299	0.6850000000000002+2.5	3.5	0.5096580982464431	0.686+2.5	3.5	0.5102594756459259	0.687+2.5	3.5	0.5108616370731404	0.688+2.5	3.5	0.5114645881194214	0.6889999999999998+2.5	3.5	0.5120683344194106	0.6899999999999998+2.5	3.5	0.512672881651598	0.6909999999999998+2.5	3.5	0.5132782355388729	0.692+2.5	3.5	0.5138844018490818	0.6930000000000001+2.5	3.5	0.5144913863955956	0.694+2.5	3.5	0.5150991950378845	0.695+2.5	3.5	0.5157078336821025	0.6960000000000001+2.5	3.5	0.5163173082816794	0.697+2.5	3.5	0.5169276248379236	0.6979999999999998+2.5	3.5	0.5175387894006326	0.699+2.5	3.5	0.5181508080687129	0.7+2.5	3.5	0.5187636869908113	0.7010000000000001+2.5	3.5	0.5193774323659529	0.702+2.5	3.5	0.5199920504441919	0.703+2.5	3.5	0.5206075475272707	0.704+2.5	3.5	0.5212239299692897	0.7049999999999998+2.5	3.5	0.5218412041773882	0.706+2.5	3.5	0.5224593766124351	0.7069999999999999+2.5	3.5	0.5230784537897313	0.708+2.5	3.5	0.5236984422797224	0.709+2.5	3.5	0.5243193487087235	0.71+2.5	3.5	0.5249411797596556	0.711+2.5	3.5	0.5255639421727927	0.712+2.5	3.5	0.5261876427465222	0.713+2.5	3.5	0.5268122883381171	0.714+2.5	3.5	0.5274378858645203	0.715+2.5	3.5	0.5280644423031425	0.716+2.5	3.5	0.5286919646926723	0.717+2.5	3.5	0.5293204601339001	0.718+2.5	3.5	0.5299499357905554	0.719+2.5	3.5	0.530580398890158	0.72+2.5	3.5	0.531211856724883	0.721+2.5	3.5	0.531844316652441	0.722+2.5	3.5	0.5324777860969718	0.723+2.5	3.5	0.5331122725499542	0.724+2.5	3.5	0.5337477835711306	0.7250000000000001+2.5	3.5	0.5343843267894471	0.726+2.5	3.5	0.5350219099040102	0.7270000000000001+2.5	3.5	0.5356605406850589	0.728+2.5	3.5	0.536300226974954	0.7289999999999999+2.5	3.5	0.5369409766891851	0.7300000000000001+2.5	3.5	0.5375827978173924	0.7310000000000001+2.5	3.5	0.5382256984244097	0.7320000000000001+2.5	3.5	0.5388696866513228	0.733+2.5	3.5	0.5395147707165479	0.7340000000000001+2.5	3.5	0.5401609589169272	0.735+2.5	3.5	0.5408082596288464	0.736+2.5	3.5	0.5414566813093693	0.737+2.5	3.5	0.542106232497394	0.7380000000000001+2.5	3.5	0.542756921814829	0.7390000000000001+2.5	3.5	0.5434087579677912	0.74+2.5	3.5	0.5440617497478243	0.7410000000000001+2.5	3.5	0.5447159060331394	0.742+2.5	3.5	0.5453712357898789	0.743+2.5	3.5	0.5460277480734025	0.744+2.5	3.5	0.5466854520295964	0.7449999999999999+2.5	3.5	0.5473443568962076	0.7459999999999999+2.5	3.5	0.5480044720042018	0.7470000000000001+2.5	3.5	0.5486658067791459	0.748+2.5	3.5	0.5493283707426182	0.749+2.5	3.5	0.5499921735136423	0.7499999999999999+2.5	3.5	0.5506572248101497	0.7510000000000001+2.5	3.5	0.5513235344504682	0.7519999999999999+2.5	3.5	0.5519911123548408	0.7530000000000001+2.5	3.5	0.5526599685469697	0.7540000000000001+2.5	3.5	0.5533301131555937	0.755+2.5	3.5	0.5540015564160924	0.756+2.5	3.5	0.5546743086721231	0.7569999999999999+2.5	3.5	0.5553483803772883	0.758+2.5	3.5	0.5560237820968356	0.7590000000000001+2.5	3.5	0.5567005245093908	0.7600000000000001+2.5	3.5	0.5573786184087246	0.761+2.5	3.5	0.5580580747055542	0.7620000000000001+2.5	3.5	0.5587389044293792	0.763+2.5	3.5	0.5594211187303556	0.764+2.5	3.5	0.560104728881205	0.765+2.5	3.5	0.5607897462791633	0.766+2.5	3.5	0.5614761824479679	0.7670000000000001+2.5	3.5	0.5621640490398842	0.768+2.5	3.5	0.5628533578377738	0.769+2.5	3.5	0.5635441207572048	0.77+2.5	3.5	0.5642363498486039	0.771+2.5	3.5	0.5649300572994536	0.772+2.5	3.5	0.5656252554365337	0.7729999999999999+2.5	3.5	0.5663219567282105	0.7739999999999999+2.5	3.5	0.5670201737867712	0.7749999999999999+2.5	3.5	0.5677199193708088	0.776+2.5	3.5	0.5684212063876561	0.777+2.5	3.5	0.5691240478958711	0.7780000000000001+2.5	3.5	0.5698284571077741	0.779+2.5	3.5	0.57053444739204	0.7800000000000001+2.5	3.5	0.5712420322763441	0.781+2.5	3.5	0.5719512254500663	0.7819999999999999+2.5	3.5	0.5726620407670521	0.7830000000000001+2.5	3.5	0.5733744922484333	0.784+2.5	3.5	0.5740885940855112	0.785+2.5	3.5	0.574804360642701	0.786+2.5	3.5	0.5755218064605417	0.787+2.5	3.5	0.5762409462587722	0.788+2.5	3.5	0.576961794939475	0.7890000000000001+2.5	3.5	0.5776843675902897	0.79+2.5	3.5	0.5784086794876996	0.7909999999999999+2.5	3.5	0.5791347461003898	0.792+2.5	3.5	0.5798625830926829	0.793+2.5	3.5	0.5805922063280516	0.794+2.5	3.5	0.5813236318727117	0.795+2.5	3.5	0.582056875999297	0.796+2.5	3.5	0.5827919551906188	0.7969999999999999+2.5	3.5	0.5835288861435126	0.798+2.5	3.5	0.5842676857727735	0.799+2.5	3.5	0.5850083712151842	0.8+2.5	3.5	0.5857509598336366	0.801+2.5	3.5	0.5864954692213513	0.802+2.5	3.5	0.5872419172061975	0.803+2.5	3.5	0.5879903218551139	0.804+2.5	3.5	0.5887407014786381	0.805+2.5	3.5	0.5894930746355425	0.8059999999999999+2.5	3.5	0.5902474601375843	0.807+2.5	3.5	0.5910038770543696	0.808+2.5	3.5	0.5917623447183377	0.809+2.5	3.5	0.592522882729866	0.8099999999999999+2.5	3.5	0.5932855109625038	0.811+2.5	3.5	0.5940502495683331	0.812+2.5	3.5	0.5948171189834655	0.8130000000000001+2.5	3.5	0.5955861399336758	0.814+2.5	3.5	0.5963573334401787	0.815+2.5	3.5	0.5971307208255513	0.816+2.5	3.5	0.5979063237198071	0.817+2.5	3.5	0.5986841640666262	0.818+2.5	3.5	0.5994642641297451	0.819+2.5	3.5	0.6002466464995142	0.82+2.5	3.5	0.6010313340996241	0.821+2.5	3.5	0.6018183501940102	0.822+2.5	3.5	0.6026077183939385	0.823+2.5	3.5	0.6033994626652796	0.8240000000000001+2.5	3.5	0.6041936073359768	0.825+2.5	3.5	0.6049901771037155	0.826+2.5	3.5	0.6057891970437974	0.827+2.5	3.5	0.6065906926172313	0.828+2.5	3.5	0.6073946896790428	0.829+2.5	3.5	0.6082012144868129	0.83+2.5	3.5	0.6090102937094528	0.831+2.5	3.5	0.6098219544362229	0.832+2.5	3.5	0.6106362241860024	0.833+2.5	3.5	0.6114531309168219	0.834+2.5	3.5	0.612272703035664	0.835+2.5	3.5	0.6130949694085439	0.8359999999999999+2.5	3.5	0.6139199593708791	0.8370000000000001+2.5	3.5	0.6147477027381562	0.838+2.5	3.5	0.6155782298169098	0.839+2.5	3.5	0.6164115714160197	0.84+2.5	3.5	0.6172477588583406	0.841+2.5	3.5	0.6180868239926759	0.842+2.5	3.5	0.6189287992061069	0.843+2.5	3.5	0.6197737174366917	0.8439999999999999+2.5	3.5	0.6206216121865465	0.845+2.5	3.5	0.6214725175353231	0.846+2.5	3.5	0.6223264681540981	0.847+2.5	3.5	0.6231834993196895	0.848+2.5	3.5	0.6240436469294144	0.8490000000000001+2.5	3.5	0.6249069475163073	0.85+2.5	3.5	0.6257734382648154	0.851+2.5	3.5	0.6266431570269878	0.852+2.5	3.5	0.6275161423391806	0.853+2.5	3.5	0.6283924334392951	0.854+2.5	3.5	0.6292720702845709	0.855+2.5	3.5	0.6301550935699571	0.856+2.5	3.5	0.6310415447470823	0.857+2.5	3.5	0.6319314660438483	0.858+2.5	3.5	0.6328249004846733	0.859+2.5	3.5	0.6337218919114096	0.86+2.5	3.5	0.6346224850049637	0.861+2.5	3.5	0.6355267253076481	0.862+2.5	3.5	0.636434659246295	0.863+2.5	3.5	0.637346334156163	0.864+2.5	3.5	0.6382617983056701	0.865+2.5	3.5	0.639181100921989	0.866+2.5	3.5	0.6401042922175405	0.867+2.5	3.5	0.6410314234174237	0.868+2.5	3.5	0.6419625467878233	0.8690000000000001+2.5	3.5	0.642897715665437	0.87+2.5	3.5	0.6438369844879683	0.871+2.5	3.5	0.6447804088257303	0.872+2.5	3.5	0.6457280454144104	0.873+2.5	3.5	0.6466799521890509	0.874+2.5	3.5	0.6476361883192954	0.875+2.5	3.5	0.6485968142459636	0.876+2.5	3.5	0.6495618917190136	0.877+2.5	3.5	0.6505314838369549	0.878+2.5	3.5	0.6515056550877832	0.879+2.5	3.5	0.6524844713915056	0.88+2.5	3.5	0.6534680001443334	0.881+2.5	3.5	0.6544563102646231	0.882+2.5	3.5	0.6554494722406489	0.883+2.5	3.5	0.6564475581802957	0.884+2.5	3.5	0.6574506418627702	0.885+2.5	3.5	0.658458798792426	0.886+2.5	3.5	0.659472106254812	0.887+2.5	3.5	0.6604906433750547	0.888+2.5	3.5	0.6615144911786944	0.889+2.5	3.5	0.6625437326551005	0.89+2.5	3.5	0.6635784528236008	0.891+2.5	3.5	0.6646187388024667	0.892+2.5	3.5	0.6656646798809046	0.893+2.5	3.5	0.6667163675942155	0.894+2.5	3.5	0.6677738958022924	0.895+2.5	3.5	0.668837360771637	0.896+2.5	3.5	0.6699068612610899	0.897+2.5	3.5	0.6709824986114794	0.898+2.5	3.5	0.672064376839409	0.899+2.5	3.5	0.6731526027354174	0.9+2.5	3.5	0.6742472859667613	0.901+2.5	3.5	0.6753485391850865	0.902+2.5	3.5	0.6764564781392743	0.903+2.5	3.5	0.6775712217937668	0.904+2.5	3.5	0.6786928924526967	0.905+2.5	3.5	0.6798216158901733	0.906+2.5	3.5	0.6809575214870958	0.907+2.5	3.5	0.6821007423748972	0.908+2.5	3.5	0.6832514155866489	0.909+2.5	3.5	0.6844096822159889	0.91+2.5	3.5	0.6855756875843675	0.911+2.5	3.5	0.6867495814171509	0.912+2.5	3.5	0.6879315180291512	0.913+2.5	3.5	0.6891216565202073	0.914+2.5	3.5	0.6903201609814832	0.915+2.5	3.5	0.6915272007132037	0.916+2.5	3.5	0.6927429504546084	0.917+2.5	3.5	0.6939675906269642	0.918+2.5	3.5	0.6952013075905469	0.919+2.5	3.5	0.69644429391658	0.92+2.5	3.5	0.6976967486751992	0.921+2.5	3.5	0.6989588777406011	0.922+2.5	3.5	0.7002308941146413	0.923+2.5	3.5	0.7015130182702469	0.924+2.5	3.5	0.70280547851614	0.925+2.5	3.5	0.7041085113844947	0.926+2.5	3.5	0.7054223620433043	0.927+2.5	3.5	0.706747284735392	0.928+2.5	3.5	0.7080835432461864	0.929+2.5	3.5	0.7094314114025799	0.93+2.5	3.5	0.7107911736054112	0.931+2.5	3.5	0.7121631253983615	0.932+2.5	3.5	0.7135475740763321	0.933+2.5	3.5	0.7149448393366759	0.934+2.5	3.5	0.7163552539770014	0.935+2.5	3.5	0.717779164643654	0.936+2.5	3.5	0.7192169326354076	0.937+2.5	3.5	0.7206689347673888	0.938+2.5	3.5	0.7221355643007987	0.939+2.5	3.5	0.7236172319446109	0.94+2.5	3.5	0.7251143669361225	0.941+2.5	3.5	0.7266274182080154	0.942+2.5	3.5	0.728156855650477	0.943+2.5	3.5	0.7297031714779427	0.944+2.5	3.5	0.7312668817111663	0.945+2.5	3.5	0.7328485277866464	0.946+2.5	3.5	0.7344486783069318	0.947+2.5	3.5	0.7360679309470503	0.948+2.5	3.5	0.7377069145342855	0.949+2.5	3.5	0.7393662913208028	0.95+2.5	3.5	0.7410467594712553	0.951+2.5	3.5	0.7427490557905511	0.952+2.5	3.5	0.7444739587205018	0.953+2.5	3.5	0.7462222916381986	0.954+2.5	3.5	0.7479949264937811	0.955+2.5	3.5	0.7497927878309176	0.956+2.5	3.5	0.7516168572399661	0.957+2.5	3.5	0.7534681783016328	0.958+2.5	3.5	0.7553478620882519	0.959+2.5	3.5	0.757257093300869	0.96+2.5	3.5	0.7591971371335339	0.961+2.5	3.5	0.7611693469720471	0.962+2.5	3.5	0.7631751730535044	0.963+2.5	3.5	0.765216172236084	0.964+2.5	3.5	0.7672940190566176	0.965+2.5	3.5	0.7694105182878377	0.966+2.5	3.5	0.7715676192493947	0.967+2.5	3.5	0.7737674321789196	0.968+2.5	3.5	0.7760122470342761	0.969+2.5	3.5	0.7783045551793343	0.97+2.5	3.5	0.7806470745078942	0.971+2.5	3.5	0.783042778690194	0.972+2.5	3.5	0.7854949313924454	0.973+2.5	3.5	0.7880071265338603	0.974+2.5	3.5	0.7905833359239951	0.975+2.5	3.5	0.7932279659886734	0.976+2.5	3.5	0.7959459257773533	0.977+2.5	3.5	0.7987427090944373	0.978+2.5	3.5	0.8016244944781329	0.979+2.5	3.5	0.8045982679608165	0.98+2.5	3.5	0.8076719752305533	0.981+2.5	3.5	0.8108547121968444	0.982+2.5	3.5	0.8141569663897786	0.983+2.5	3.5	0.8175909266372244	0.984+2.5	3.5	0.821170885956372	0.985+2.5	3.5	0.8249137740406043	0.986+2.5	3.5	0.8288398736518926	0.987+2.5	3.5	0.8329738041268693	0.988+2.5	3.5	0.8373459033144117	0.989+2.5	3.5	0.8419942223937258	0.99+2.5	3.5	0.8469674980236408	0.991+2.5	3.5	0.8523297511092095	0.992+2.5	3.5	0.8581677364607236	0.993+2.5	3.5	0.8646037185584075	0.994+2.5	3.5	0.8718190386426733	0.995+2.5	3.5	0.8801020172521	0.996+2.5	3.5	0.8899595673323942	0.997+2.5	3.5	0.9024381036828013	0.998+2.5	3.5	0.9204807521237621	0.999+1000.0	1000.0	0.43293472489851564	8.455771776076388e-10+1000.0	1000.0	0.4330465003570181	9.002038924842756e-10+1000.0	1000.0	0.4331582758155206	9.58260861413111e-10+1000.0	1000.0	0.433270051274023	1.0199569836826873e-09+1000.0	1000.0	0.4333818267325255	1.0855134631978023e-09+1000.0	1000.0	0.433493602191028	1.1551645071928435e-09+1000.0	1000.0	0.43360537764953044	1.2291580630401776e-09+1000.0	1000.0	0.43371715310803294	1.3077565951371996e-09+1000.0	1000.0	0.43382892856653543	1.3912379039535333e-09+1000.0	1000.0	0.43394070402503787	1.4798959894234642e-09+1000.0	1000.0	0.43405247948354037	1.5740419609750502e-09+1000.0	1000.0	0.4341642549420428	1.6740049965993806e-09+1000.0	1000.0	0.4342760304005453	1.7801333534825056e-09+1000.0	1000.0	0.4343878058590478	1.892795432842798e-09+1000.0	1000.0	0.43449958131755023	2.0123809017474897e-09+1000.0	1000.0	0.4346113567760527	2.1393018748147604e-09+1000.0	1000.0	0.4347231322345552	2.2739941588478816e-09+1000.0	1000.0	0.43483490769305766	2.416918563596261e-09+1000.0	1000.0	0.43494668315156015	2.5685622819899946e-09+1000.0	1000.0	0.43505845861006265	2.7294403433543976e-09+1000.0	1000.0	0.4351702340685651	2.900097143280314e-09+1000.0	1000.0	0.4352820095270676	3.0811080539989725e-09+1000.0	1000.0	0.43539378498557	3.273081119292255e-09+1000.0	1000.0	0.4355055604440725	3.476658838163066e-09+1000.0	1000.0	0.435617335902575	3.692520041684596e-09+1000.0	1000.0	0.43572911136107745	3.9213818676600556e-09+1000.0	1000.0	0.43584088681957994	4.164001837938251e-09+1000.0	1000.0	0.43595266227808244	4.421180043456379e-09+1000.0	1000.0	0.4360644377365849	4.693761442320807e-09+1000.0	1000.0	0.43617621319508737	4.982638276480159e-09+1000.0	1000.0	0.4362879886535898	5.288752612801408e-09+1000.0	1000.0	0.4363997641120923	5.613099014632907e-09+1000.0	1000.0	0.4365115395705948	5.956727350210059e-09+1000.0	1000.0	0.43662331502909724	6.3207457445589035e-09+1000.0	1000.0	0.43673509048759973	6.706323681851887e-09+1000.0	1000.0	0.4368468659461022	7.114695265486478e-09+1000.0	1000.0	0.43695864140460466	7.547162643492553e-09+1000.0	1000.0	0.43707041686310716	8.005099607214072e-09+1000.0	1000.0	0.43718219232160965	8.489955371568261e-09+1000.0	1000.0	0.4372939677801121	9.003258545565177e-09+1000.0	1000.0	0.4374057432386146	9.546621302154046e-09+1000.0	1000.0	0.437517518697117	1.0121743756866783e-08+1000.0	1000.0	0.4376292941556195	1.073041856516051e-08+1000.0	1000.0	0.437741069614122	1.137453574878541e-08+1000.0	1000.0	0.43785284507262445	1.2056087761976528e-08+1000.0	1000.0	0.43796462053112695	1.2777174808734697e-08+1000.0	1000.0	0.43807639598962944	1.3540010422956043e-08+1000.0	1000.0	0.4381881714481319	1.4346927323694274e-08+1000.0	1000.0	0.4382999469066344	1.5200383558367365e-08+1000.0	1000.0	0.43841172236513687	1.6102968947276317e-08+1000.0	1000.0	0.4385234978236393	1.7057411843394756e-08+1000.0	1000.0	0.4386352732821418	1.806658622198081e-08+1000.0	1000.0	0.43874704874064424	1.913351911518721e-08+1000.0	1000.0	0.43885882419914674	2.026139840751482e-08+1000.0	1000.0	0.43897059965764923	2.145358100860455e-08+1000.0	1000.0	0.43908237511615167	2.27136014205941e-08+1000.0	1000.0	0.43919415057465416	2.404518071797979e-08+1000.0	1000.0	0.43930592603315666	2.5452235958681446e-08+1000.0	1000.0	0.4394177014916591	2.693889004581453e-08+1000.0	1000.0	0.4395294769501616	2.8509482060477016e-08+1000.0	1000.0	0.43964125240866403	3.016857808670537e-08+1000.0	1000.0	0.4397530278671665	3.192098255066443e-08+1000.0	1000.0	0.439864803325669	3.377175009701e-08+1000.0	1000.0	0.43997657878417146	3.572619802635705e-08+1000.0	1000.0	0.44008835424267395	3.7789919318746664e-08+1000.0	1000.0	0.44020012970117645	3.9968796269025896e-08+1000.0	1000.0	0.4403119051596789	4.226901476114366e-08+1000.0	1000.0	0.4404236806181814	4.469707920944264e-08+1000.0	1000.0	0.4405354560766839	4.7259828196165034e-08+1000.0	1000.0	0.4406472315351863	4.9964450835605786e-08+1000.0	1000.0	0.4407590069936888	5.28185038965455e-08+1000.0	1000.0	0.44087078245219125	5.582992971586042e-08+1000.0	1000.0	0.44098255791069374	5.9007074937575254e-08+1000.0	1000.0	0.44109433336919623	6.235871011291271e-08+1000.0	1000.0	0.4412061088276987	6.589405019838658e-08+1000.0	1000.0	0.44131788428620117	6.9622775990402e-08+1000.0	1000.0	0.44142965974470366	7.35550565363381e-08+1000.0	1000.0	0.4415414352032061	7.770157256370875e-08+1000.0	1000.0	0.4416532106617086	8.207354097057774e-08+1000.0	1000.0	0.44176498612021103	8.668274042207824e-08+1000.0	1000.0	0.44187676157871353	9.154153809970398e-08+1000.0	1000.0	0.441988537037216	9.666291765172295e-08+1000.0	1000.0	0.44210031249571846	1.020605083950485e-07+1000.0	1000.0	0.44221208795422096	1.0774861582075503e-07+1000.0	1000.0	0.44232386341272345	1.1374225345741206e-07+1000.0	1000.0	0.4424356388712259	1.2005717614854555e-07+1000.0	1000.0	0.4425474143297284	1.267099148025974e-07+1000.0	1000.0	0.4426591897882309	1.337178126759441e-07+1000.0	1000.0	0.4427709652467333	1.4109906325189918e-07+1000.0	1000.0	0.4428827407052358	1.4887274978089243e-07+1000.0	1000.0	0.44299451616373825	1.5705888654943161e-07+1000.0	1000.0	0.44310629162224074	1.6567846194809394e-07+1000.0	1000.0	0.44321806708074324	1.747534834111739e-07+1000.0	1000.0	0.4433298425392457	1.843070243034995e-07+1000.0	1000.0	0.44344161799774817	1.943632728325559e-07+1000.0	1000.0	0.44355339345625067	2.0494758306688825e-07+1000.0	1000.0	0.4436651689147531	2.1608652814482705e-07+1000.0	1000.0	0.4437769443732556	2.2780795576048724e-07+1000.0	1000.0	0.4438887198317581	2.401410460170985e-07+1000.0	1000.0	0.44400049529026053	2.5311637174110226e-07+1000.0	1000.0	0.444112270748763	2.667659613536326e-07+1000.0	1000.0	0.44422404620726547	2.8112336439939133e-07+1000.0	1000.0	0.44433582166576796	2.962237198367006e-07+1000.0	1000.0	0.44444759712427045	3.1210382719577966e-07+1000.0	1000.0	0.4445593725827729	3.288022207164168e-07+1000.0	1000.0	0.4446711480412754	3.463592465798339e-07+1000.0	1000.0	0.4447829234997779	3.6481714335349217e-07+1000.0	1000.0	0.4448946989582803	3.842201257719133e-07+1000.0	1000.0	0.4450064744167828	4.046144719805893e-07+1000.0	1000.0	0.44511824987528525	4.2604861437434466e-07+1000.0	1000.0	0.44523002533378775	4.4857323416633454e-07+1000.0	1000.0	0.44534180079229024	4.7224135982789567e-07+1000.0	1000.0	0.4454535762507927	4.97108469544749e-07+1000.0	1000.0	0.4455653517092952	5.232325978395571e-07+1000.0	1000.0	0.44567712716779767	5.506744465157972e-07+1000.0	1000.0	0.4457889026263001	5.794975000833844e-07+1000.0	1000.0	0.4459006780848026	6.09768145831435e-07+1000.0	1000.0	0.4460124535433051	6.415557987189249e-07+1000.0	1000.0	0.44612422900180754	6.749330312599641e-07+1000.0	1000.0	0.44623600446031003	7.099757085857494e-07+1000.0	1000.0	0.44634777991881247	7.467631288710538e-07+1000.0	1000.0	0.44645955537731496	7.853781693197444e-07+1000.0	1000.0	0.44657133083581746	8.259074379090472e-07+1000.0	1000.0	0.4466831062943199	8.684414310995874e-07+1000.0	1000.0	0.4467948817528224	9.130746977241336e-07+1000.0	1000.0	0.4469066572113249	9.599060092745956e-07+1000.0	1000.0	0.4470184326698273	1.0090385368142594e-06+1000.0	1000.0	0.4471302081283298	1.0605800347487056e-06+1000.0	1000.0	0.44724198358683226	1.1146430316959473e-06+1000.0	1000.0	0.44735375904533475	1.1713450287045798e-06+1000.0	1000.0	0.44746553450383725	1.230808705074892e-06+1000.0	1000.0	0.4475773099623397	1.2931621320470233e-06+1000.0	1000.0	0.4476890854208422	1.3585389946272836e-06+1000.0	1000.0	0.4478008608793447	1.4270788218317762e-06+1000.0	1000.0	0.4479126363378471	1.498927225635621e-06+1000.0	1000.0	0.4480244117963496	1.574236148923747e-06+1000.0	1000.0	0.4481361872548521	1.6531641227477965e-06+1000.0	1000.0	0.44824796271335454	1.7358765332035427e-06+1000.0	1000.0	0.44835973817185704	1.822545898251373e-06+1000.0	1000.0	0.4484715136303595	1.913352154811483e-06+1000.0	1000.0	0.44858328908886197	2.0084829564764183e-06+1000.0	1000.0	0.44869506454736446	2.1081339821909183e-06+1000.0	1000.0	0.4488068400058669	2.212509256261285e-06+1000.0	1000.0	0.4489186154643694	2.321821480065104e-06+1000.0	1000.0	0.4490303909228719	2.436292375842314e-06+1000.0	1000.0	0.44914216638137433	2.5561530429605878e-06+1000.0	1000.0	0.4492539418398768	2.681644327057357e-06+1000.0	1000.0	0.44936571729837926	2.8130172024715068e-06+1000.0	1000.0	0.44947749275688176	2.9505331683910908e-06+1000.0	1000.0	0.44958926821538425	3.094464659151392e-06+1000.0	1000.0	0.4497010436738867	3.245095469132658e-06+1000.0	1000.0	0.4498128191323892	3.402721192716499e-06+1000.0	1000.0	0.4499245945908917	3.567649679771752e-06+1000.0	1000.0	0.4500363700493941	3.7402015071549425e-06+1000.0	1000.0	0.4501481455078966	3.92071046672097e-06+1000.0	1000.0	0.4502599209663991	4.1095240703520325e-06+1000.0	1000.0	0.45037169642490155	4.307004072528079e-06+1000.0	1000.0	0.45048347188340404	4.513527010972932e-06+1000.0	1000.0	0.4505952473419065	4.729484765923115e-06+1000.0	1000.0	0.450707022800409	4.955285138583547e-06+1000.0	1000.0	0.45081879825891147	5.1913524493423815e-06+1000.0	1000.0	0.4509305737174139	5.438128156336842e-06+1000.0	1000.0	0.4510423491759164	5.696071494972558e-06+1000.0	1000.0	0.4511541246344189	5.9656601390128925e-06+1000.0	1000.0	0.45126590009292133	6.247390883872805e-06+1000.0	1000.0	0.45137767555142383	6.541780352763275e-06+1000.0	1000.0	0.4514894510099263	6.849365726346889e-06+1000.0	1000.0	0.45160122646842876	7.1707054965842234e-06+1000.0	1000.0	0.45171300192693126	7.506380245462302e-06+1000.0	1000.0	0.4518247773854337	7.85699344931133e-06+1000.0	1000.0	0.4519365528439362	8.223172309437403e-06+1000.0	1000.0	0.4520483283024387	8.605568609805687e-06+1000.0	1000.0	0.4521601037609411	9.004859602533753e-06+1000.0	1000.0	0.4522718792194436	9.421748921965262e-06+1000.0	1000.0	0.4523836546779461	9.856967528110056e-06+1000.0	1000.0	0.45249543013644855	1.0311274680258768e-05+1000.0	1000.0	0.45260720559495105	1.07854589415916e-05+1000.0	1000.0	0.4527189810534535	1.1280339215616734e-05+1000.0	1000.0	0.452830756511956	1.1796765815299123e-05+1000.0	1000.0	0.4529425319704585	1.2335621565744507e-05+1000.0	1000.0	0.4530543074289609	1.2897822941333729e-05+1000.0	1000.0	0.4531660828874634	1.3484321238211514e-05+1000.0	1000.0	0.4532778583459659	1.4096103783050625e-05+1000.0	1000.0	0.45338963380446834	1.473419517903757e-05+1000.0	1000.0	0.45350140926297083	1.5399658590036245e-05+1000.0	1000.0	0.45361318472147333	1.60935970639026e-05+1000.0	1000.0	0.45372496017997577	1.6817154895949825e-05+1000.0	1000.0	0.45383673563847826	1.7571519033572993e-05+1000.0	1000.0	0.4539485110969807	1.835792052305842e-05+1000.0	1000.0	0.4540602865554832	1.9177635999633776e-05+1000.0	1000.0	0.4541720620139857	2.0031989221812258e-05+1000.0	1000.0	0.45428383747248813	2.0922352651122558e-05+1000.0	1000.0	0.4543956129309906	2.1850149078320966e-05+1000.0	1000.0	0.4545073883894931	2.281685329719867e-05+1000.0	1000.0	0.45461916384799556	2.382399382712726e-05+1000.0	1000.0	0.45473093930649805	2.487315468549121e-05+1000.0	1000.0	0.4548427147650005	2.5965977211171692e-05+1000.0	1000.0	0.454954490223503	2.710416194028158e-05+1000.0	1000.0	0.4550662656820055	2.8289470535340068e-05+1000.0	1000.0	0.4551780411405079	2.952372776912117e-05+1000.0	1000.0	0.4552898165990104	3.080882356440898e-05+1000.0	1000.0	0.4554015920575129	3.214671509090809e-05+1000.0	1000.0	0.45551336751601534	3.353942892059084e-05+1000.0	1000.0	0.45562514297451784	3.498906324276327e-05+1000.0	1000.0	0.45573691843302033	3.649779014014564e-05+1000.0	1000.0	0.45584869389152277	3.806785792729817e-05+1000.0	1000.0	0.45596046935002527	3.970159355271981e-05+1000.0	1000.0	0.4560722448085277	4.140140506595981e-05+1000.0	1000.0	0.4561840202670302	4.31697841511252e-05+1000.0	1000.0	0.4562957957255327	4.500930872813656e-05+1000.0	1000.0	0.45640757118403513	4.6922645623143586e-05+1000.0	1000.0	0.4565193466425376	4.8912553309496774e-05+1000.0	1000.0	0.4566311221010401	5.0981884720682794e-05+1000.0	1000.0	0.45674289755954256	5.3133590136669716e-05+1000.0	1000.0	0.45685467301804505	5.5370720145095426e-05+1000.0	1000.0	0.4569664484765475	5.7696428678739564e-05+1000.0	1000.0	0.45707822393505	6.011397613076929e-05+1000.0	1000.0	0.4571899993935525	6.262673254919815e-05+1000.0	1000.0	0.4573017748520549	6.523818091206816e-05+1000.0	1000.0	0.4574135503105574	6.795192048483561e-05+1000.0	1000.0	0.4575253257690599	7.077167026144605e-05+1000.0	1000.0	0.45763710122756235	7.370127249062786e-05+1000.0	1000.0	0.45774887668606484	7.674469628890561e-05+1000.0	1000.0	0.45786065214456734	7.99060413418368e-05+1000.0	1000.0	0.4579724276030698	8.318954169502001e-05+1000.0	1000.0	0.45808420306157227	8.659956963638865e-05+1000.0	1000.0	0.4581959785200747	9.014063967130133e-05+1000.0	1000.0	0.4583077539785772	9.381741259200089e-05+1000.0	1000.0	0.4584195294370797	9.763469964292257e-05+1000.0	1000.0	0.45853130489558214	0.00010159746678342499+1000.0	1000.0	0.45864308035408463	0.00010571083904945763+1000.0	1000.0	0.4587548558125871	0.00010998010501567208+1000.0	1000.0	0.45886663127108956	0.00011441072135953612+1000.0	1000.0	0.45897840672959206	0.00011900831752895232+1000.0	1000.0	0.45909018218809455	0.00012377870051487176+1000.0	1000.0	0.459201957646597	0.0001287278597304468+1000.0	1000.0	0.4593137331050995	0.00013386197199820017+1000.0	1000.0	0.4594255085636019	0.00013918740664666906+1000.0	1000.0	0.4595372840221044	0.00014471073071806032+1000.0	1000.0	0.4596490594806069	0.00015043871428830003+1000.0	1000.0	0.45976083493910935	0.00015637833590098776+1000.0	1000.0	0.45987261039761185	0.00016253678811665968+1000.0	1000.0	0.45998438585611434	0.00016892148317874307+1000.0	1000.0	0.4600961613146168	0.000175540058797646+1000.0	1000.0	0.4602079367731193	0.00018240038405432823+1000.0	1000.0	0.4603197122316217	0.00018951056542466422+1000.0	1000.0	0.4604314876901242	0.0001968789529260039+1000.0	1000.0	0.4605432631486267	0.00020451414638712406+1000.0	1000.0	0.46065503860712914	0.00021242500184292257+1000.0	1000.0	0.46076681406563164	0.00022062063805505886+1000.0	1000.0	0.46087858952413413	0.00022911044315970507+1000.0	1000.0	0.46099036498263657	0.00023790408144364725+1000.0	1000.0	0.46110214044113906	0.00024701150024983553+1000.0	1000.0	0.46121391589964156	0.00025644293701343354+1000.0	1000.0	0.461325691358144	0.0002662089264294975+1000.0	1000.0	0.4614374668166465	0.00027632030775325743+1000.0	1000.0	0.46154924227514893	0.0002867882322339112+1000.0	1000.0	0.4616610177336514	0.00029762417068297044+1000.0	1000.0	0.4617727931921539	0.0003088399211778738+1000.0	1000.0	0.46188456865065636	0.00032044761690180575+1000.0	1000.0	0.46199634410915885	0.00033245973412043215+1000.0	1000.0	0.46210811956766135	0.00034488910029619786+1000.0	1000.0	0.4622198950261638	0.0003577489023409287+1000.0	1000.0	0.4623316704846663	0.0003710526950072712+1000.0	1000.0	0.4624434459431687	0.00038481440941941296+1000.0	1000.0	0.4625552214016712	0.0003990483617436934+1000.0	1000.0	0.4626669968601737	0.0004137692619992836+1000.0	1000.0	0.46277877231867615	0.00042899222300940326+1000.0	1000.0	0.46289054777717864	0.00044473276949324087+1000.0	1000.0	0.46300232323568113	0.00046100684729865373+1000.0	1000.0	0.4631140986941836	0.00047783083277585045+1000.0	1000.0	0.46322587415268607	0.0004952215422919792+1000.0	1000.0	0.46333764961118856	0.0005131962418864389+1000.0	1000.0	0.463449425069691	0.0005317726570668704+1000.0	1000.0	0.4635612005281935	0.0005509689827454784+1000.0	1000.0	0.46367297598669593	0.0005708038933152004+1000.0	1000.0	0.46378475144519843	0.0005912965528654877+1000.0	1000.0	0.4638965269037009	0.0006124666255368289+1000.0	1000.0	0.46400830236220336	0.0006343342860135778+1000.0	1000.0	0.46412007782070586	0.0006569202301542061+1000.0	1000.0	0.46423185327920835	0.0006802456857579751+1000.0	1000.0	0.4643436287377108	0.0007043324234672137+1000.0	1000.0	0.4644554041962133	0.0007292027678039825+1000.0	1000.0	0.4645671796547158	0.0007548796083397644+1000.0	1000.0	0.4646789551132182	0.0007813864109970293+1000.0	1000.0	0.4647907305717207	0.0008087472294810808+1000.0	1000.0	0.46490250603022315	0.0008369867168404247+1000.0	1000.0	0.46501428148872564	0.0008661301371542445+1000.0	1000.0	0.46512605694722814	0.000896203377344694+1000.0	1000.0	0.4652378324057306	0.0009272329591123275+1000.0	1000.0	0.4653496078642331	0.0009592460509923809+1000.0	1000.0	0.46546138332273557	0.0009922704805294536+1000.0	1000.0	0.465573158781238	0.0010263347465684198+1000.0	1000.0	0.4656849342397405	0.0010614680316588486+1000.0	1000.0	0.46579670969824294	0.0010977002145700068+1000.0	1000.0	0.46590848515674543	0.0011350618829139747+1000.0	1000.0	0.4660202606152479	0.0011735843458733012+1000.0	1000.0	0.46613203607375037	0.00121329964703042+1000.0	1000.0	0.46624381153225286	0.00125424057729529+1000.0	1000.0	0.46635558699075536	0.001296440687927527+1000.0	1000.0	0.4664673624492578	0.00133993430364968+1000.0	1000.0	0.4665791379077603	0.001384756535847607+1000.0	1000.0	0.4666909133662628	0.0014309432958537031+1000.0	1000.0	0.4668026888247652	0.0014785313083091442+1000.0	1000.0	0.4669144642832677	0.0015275581246005708+1000.0	1000.0	0.46702623974177015	0.001578062136366386+1000.0	1000.0	0.46713801520027265	0.0016300825890685146+1000.0	1000.0	0.46724979065877514	0.0016836595956239543+1000.0	1000.0	0.4673615661172776	0.00173883415009161+1000.0	1000.0	0.4674733415757801	0.0017956481414088735+1000.0	1000.0	0.46758511703428257	0.001854144367172185+1000.0	1000.0	0.467696892492785	0.0019143665474563839+1000.0	1000.0	0.4678086679512875	0.0019763593386667422+1000.0	1000.0	0.46792044340978994	0.0020401683474172904+1000.0	1000.0	0.46803221886829244	0.0021058401444299323+1000.0	1000.0	0.46814399432679493	0.0021734222784469655+1000.0	1000.0	0.46825576978529737	0.002242963290151126+1000.0	1000.0	0.46836754524379987	0.0023145127260860063+1000.0	1000.0	0.46847932070230236	0.0023881211525694688+1000.0	1000.0	0.4685910961608048	0.002463840169593413+1000.0	1000.0	0.4687028716193073	0.0025417224247021673+1000.0	1000.0	0.4688146470778098	0.002621821626841485+1000.0	1000.0	0.4689264225363122	0.0027041925601709684+1000.0	1000.0	0.4690381979948147	0.0027888910978315605+1000.0	1000.0	0.46914997345331716	0.002875974215659406+1000.0	1000.0	0.46926174891181965	0.002965500005838677+1000.0	1000.0	0.46937352437032215	0.003057527690483445+1000.0	1000.0	0.4694852998288246	0.0031521176351408044+1000.0	1000.0	0.4695970752873271	0.0032493313622057973+1000.0	1000.0	0.4697088507458296	0.0033492315642384405+1000.0	1000.0	0.469820626204332	0.0034518821171742476+1000.0	1000.0	0.4699324016628345	0.0035573480934182203+1000.0	1000.0	0.47004417712133695	0.0036656957748119213+1000.0	1000.0	0.47015595257983944	0.003776992665464907+1000.0	1000.0	0.47026772803834194	0.0038913075044387266+1000.0	1000.0	0.4703795034968444	0.0040087102782744004+1000.0	1000.0	0.47049127895534687	0.004129272233352289+1000.0	1000.0	0.47060305441384936	0.004253065888073007+1000.0	1000.0	0.4707148298723518	0.004380165044849491+1000.0	1000.0	0.4708266053308543	0.004510644801898582+1000.0	1000.0	0.4709383807893568	0.004644581564820211+1000.0	1000.0	0.47105015624785923	0.004782053057953852+1000.0	1000.0	0.4711619317063617	0.004923138335500059+1000.0	1000.0	0.47127370716486416	0.005067917792394533+1000.0	1000.0	0.47138548262336666	0.005216473174924509+1000.0	1000.0	0.47149725808186915	0.005368887591073216+1000.0	1000.0	0.4716090335403716	0.005525245520581894+1000.0	1000.0	0.4717208089988741	0.005685632824716283+1000.0	1000.0	0.4718325844573766	0.005850136755724253+1000.0	1000.0	0.471944359915879	0.006018845965973298+1000.0	1000.0	0.4720561353743815	0.006191850516754376+1000.0	1000.0	0.472167910832884	0.006369241886738353+1000.0	1000.0	0.47227968629138645	0.006551112980073482+1000.0	1000.0	0.47239146174988894	0.00673755813411001+1000.0	1000.0	0.4725032372083914	0.006928673126737668+1000.0	1000.0	0.4726150126668939	0.007124555183324939+1000.0	1000.0	0.47272678812539637	0.007325302983243842+1000.0	1000.0	0.4728385635838988	0.007531016665968988+1000.0	1000.0	0.4729503390424013	0.007741797836736367+1000.0	1000.0	0.4730621145009038	0.00795774957174717+1000.0	1000.0	0.47317388995940624	0.00817897642290471+1000.0	1000.0	0.47328566541790873	0.00840558442206973+1000.0	1000.0	0.47339744087641117	0.00863768108481906+1000.0	1000.0	0.47350921633491366	0.008875375413696502+1000.0	1000.0	0.47362099179341616	0.009118777900938612+1000.0	1000.0	0.4737327672519186	0.009368000530664255+1000.0	1000.0	0.4738445427104211	0.009623156780512869+1000.0	1000.0	0.4739563181689236	0.009884361622716302+1000.0	1000.0	0.474068093627426	0.010151731524592478+1000.0	1000.0	0.4741798690859285	0.010425384448445896+1000.0	1000.0	0.474291644544431	0.01070543985085982+1000.0	1000.0	0.47440342000293345	0.010992018681368643+1000.0	1000.0	0.47451519546143595	0.01128524338049551+1000.0	1000.0	0.4746269709199384	0.01158523787713995+1000.0	1000.0	0.4747387463784409	0.011892127585305423+1000.0	1000.0	0.4748505218369434	0.012206039400148872+1000.0	1000.0	0.4749622972954458	0.012527101693342398+1000.0	1000.0	0.4750740727539483	0.012855444307732307+1000.0	1000.0	0.4751858482124508	0.013191198551280748+1000.0	1000.0	0.47529762367095324	0.013534497190279658+1000.0	1000.0	0.47540939912945573	0.013885474441822716+1000.0	1000.0	0.4755211745879582	0.014244265965520647+1000.0	1000.0	0.47563295004646067	0.014611008854451579+1000.0	1000.0	0.47574472550496316	0.014985841625328733+1000.0	1000.0	0.4758565009634656	0.015368904207877562+1000.0	1000.0	0.4759682764219681	0.015760337933408596+1000.0	1000.0	0.4760800518804706	0.01616028552257236+1000.0	1000.0	0.47619182733897303	0.016568891072288146+1000.0	1000.0	0.4763036027974755	0.01698630004183369+1000.0	1000.0	0.476415378255978	0.017412659238082745+1000.0	1000.0	0.47652715371448046	0.01784811679988338+1000.0	1000.0	0.47663892917298295	0.018292822181564684+1000.0	1000.0	0.4767507046314854	0.018746926135559505+1000.0	1000.0	0.4768624800899879	0.01921058069413867+1000.0	1000.0	0.4769742555484904	0.019683939150240564+1000.0	1000.0	0.4770860310069928	0.020167156037392327+1000.0	1000.0	0.4771978064654953	0.020660387108711736+1000.0	1000.0	0.4773095819239978	0.021163789314978913+1000.0	1000.0	0.47742135738250024	0.02167752078177411+1000.0	1000.0	0.47753313284100274	0.02220174078567179+1000.0	1000.0	0.47764490829950523	0.022736609729481272+1000.0	1000.0	0.47775668375800767	0.023282289116531688+1000.0	1000.0	0.47786845921651017	0.023838941523992567+1000.0	1000.0	0.4779802346750126	0.02440673057522129+1000.0	1000.0	0.4780920101335151	0.02498582091113869+1000.0	1000.0	0.4782037855920176	0.025576378160619532+1000.0	1000.0	0.47831556105052003	0.02617856890990019+1000.0	1000.0	0.4784273365090225	0.026792560670996883+1000.0	1000.0	0.478539111967525	0.027418521849128078+1000.0	1000.0	0.47865088742602746	0.028056621709143453+1000.0	1000.0	0.47876266288452995	0.028707030340954395+1000.0	1000.0	0.4788744383430324	0.02936991862396087+1000.0	1000.0	0.4789862138015349	0.030045458190481326+1000.0	1000.0	0.4790979892600374	0.030733821388175188+1000.0	1000.0	0.4792097647185398	0.031435181241465944+1000.0	1000.0	0.4793215401770423	0.0321497114119621+1000.0	1000.0	0.4794333156355448	0.03287758615787382+1000.0	1000.0	0.47954509109404725	0.033618980292433175+1000.0	1000.0	0.47965686655254974	0.03437406914131738+1000.0	1000.0	0.47976864201105224	0.035143028499074455+1000.0	1000.0	0.4798804174695547	0.035926034584561624+1000.0	1000.0	0.47999219292805717	0.036723263995397576+1000.0	1000.0	0.4801039683865596	0.03753489366142945+1000.0	1000.0	0.4802157438450621	0.038361100797230034+1000.0	1000.0	0.4803275193035646	0.039202062853619664+1000.0	1000.0	0.48043929476206704	0.040057957468229856+1000.0	1000.0	0.48055107022056953	0.04092896241511239+1000.0	1000.0	0.480662845679072	0.041815255553398545+1000.0	1000.0	0.48077462113757446	0.04271701477502511+1000.0	1000.0	0.48088639659607696	0.04363441795153352+1000.0	1000.0	0.4809981720545794	0.044567642879948084+1000.0	1000.0	0.4811099475130819	0.045516867227755976+1000.0	1000.0	0.4812217229715844	0.04648226847698741+1000.0	1000.0	0.4813334984300868	0.04746402386742037+1000.0	1000.0	0.4814452738885893	0.04846231033891908+1000.0	1000.0	0.4815570493470918	0.04947730447291589+1000.0	1000.0	0.48166882480559425	0.050509182433060594+1000.0	1000.0	0.48178060026409675	0.05155811990504887+1000.0	1000.0	0.48189237572259924	0.052624292035641534+1000.0	1000.0	0.4820041511811017	0.053707873370901375+1000.0	1000.0	0.4821159266396042	0.05480903779366128+1000.0	1000.0	0.4822277020981066	0.0559279584602369+1000.0	1000.0	0.4823394775566091	0.05706480773641722+1000.0	1000.0	0.4824512530151116	0.05821975713273705+1000.0	1000.0	0.48256302847361404	0.059392977239066294+1000.0	1000.0	0.48267480393211654	0.060584637658532746+1000.0	1000.0	0.48278657939061903	0.0617949069407955+1000.0	1000.0	0.48289835484912147	0.06302395251470305+1000.0	1000.0	0.48301013030762396	0.06427194062035509+1000.0	1000.0	0.4831219057661264	0.06553903624058667+1000.0	1000.0	0.4832336812246289	0.06682540303191586+1000.0	1000.0	0.4833454566831314	0.0681312032549624+1000.0	1000.0	0.48345723214163383	0.06945659770438034+1000.0	1000.0	0.4835690076001363	0.07080174563832618+1000.0	1000.0	0.4836807830586388	0.07216680470748489+1000.0	1000.0	0.48379255851714126	0.07355193088369483+1000.0	1000.0	0.48390433397564375	0.07495727838819582+1000.0	1000.0	0.48401610943414625	0.07638299961952426+1000.0	1000.0	0.4841278848926487	0.07782924508109901+1000.0	1000.0	0.4842396603511512	0.07929616330852383+1000.0	1000.0	0.4843514358096536	0.08078390079663142+1000.0	1000.0	0.4844632112681561	0.08229260192632006+1000.0	1000.0	0.4845749867266586	0.08382240889119498+1000.0	1000.0	0.48468676218516105	0.08537346162406677+1000.0	1000.0	0.48479853764366354	0.0869458977233344+1000.0	1000.0	0.48491031310216604	0.0885398523792815+1000.0	1000.0	0.4850220885606685	0.09015545830033503+1000.0	1000.0	0.48513386401917097	0.09179284563931675+1000.0	1000.0	0.48524563947767346	0.09345214191971632+1000.0	1000.0	0.4853574149361759	0.09513347196203835+1000.0	1000.0	0.4854691903946784	0.09683695781025403+1000.0	1000.0	0.48558096585318083	0.09856271865838706+1000.0	1000.0	0.48569274131168333	0.10031087077729381+1000.0	1000.0	0.4858045167701858	0.10208152744165118+1000.0	1000.0	0.48591629222868826	0.10387479885721321+1000.0	1000.0	0.48602806768719076	0.10569079208836768+1000.0	1000.0	0.48613984314569325	0.10752961098602495+1000.0	1000.0	0.4862516186041957	0.1093913561158952+1000.0	1000.0	0.4863633940626982	0.11127612468718735+1000.0	1000.0	0.4864751695212006	0.113184010481761+1000.0	1000.0	0.4865869449797031	0.11511510378379598+1000.0	1000.0	0.4866987204382056	0.1170694913099931+1000.0	1000.0	0.48681049589670805	0.11904725614037066+1000.0	1000.0	0.48692227135521055	0.12104847764969005+1000.0	1000.0	0.48703404681371304	0.12307323143954264+1000.0	1000.0	0.4871458222722155	0.12512158927115796+1000.0	1000.0	0.487257597730718	0.12719361899896656+1000.0	1000.0	0.48736937318922047	0.1292893845049503+1000.0	1000.0	0.4874811486477229	0.1314089456338404+1000.0	1000.0	0.4875929241062254	0.13355235812919633+1000.0	1000.0	0.48770469956472784	0.13571967357039738+1000.0	1000.0	0.48781647502323033	0.1379109393106144+1000.0	1000.0	0.48792825048173283	0.1401261984157726+1000.0	1000.0	0.48804002594023527	0.1423654896045731+1000.0	1000.0	0.48815180139873776	0.14462884718960478+1000.0	1000.0	0.48826357685724026	0.14691630101957723+1000.0	1000.0	0.4883753523157427	0.14922787642273594+1000.0	1000.0	0.4884871277742452	0.15156359415149093+1000.0	1000.0	0.48859890323274763	0.15392347032828754+1000.0	1000.0	0.4887106786912501	0.15630751639278845+1000.0	1000.0	0.4888224541497526	0.15871573905037192+1000.0	1000.0	0.48893422960825506	0.16114814022201512+1000.0	1000.0	0.48904600506675755	0.16360471699558968+1000.0	1000.0	0.48915778052526004	0.16608546157859722+1000.0	1000.0	0.4892695559837625	0.1685903612524047+1000.0	1000.0	0.489381331442265	0.17111939832800654+1000.0	1000.0	0.48949310690076747	0.17367255010333862+1000.0	1000.0	0.4896048823592699	0.17624978882220393+1000.0	1000.0	0.4897166578177724	0.17885108163483388+1000.0	1000.0	0.48982843327627484	0.18147639056010767+1000.0	1000.0	0.48994020873477734	0.18412567244949685+1000.0	1000.0	0.49005198419327983	0.18679887895272926+1000.0	1000.0	0.49016375965178227	0.18949595648523873+1000.0	1000.0	0.49027553511028477	0.1922168461974194+1000.0	1000.0	0.49038731056878726	0.19496148394570445+1000.0	1000.0	0.4904990860272897	0.1977298002655232+1000.0	1000.0	0.4906108614857922	0.20052172034615592+1000.0	1000.0	0.4907226369442947	0.20333716400750126+1000.0	1000.0	0.4908344124027971	0.2061760456788102+1000.0	1000.0	0.4909461878612996	0.20903827437940084+1000.0	1000.0	0.49105796331980206	0.21192375370136626+1000.0	1000.0	0.49116973877830455	0.2148323817943358+1000.0	1000.0	0.49128151423680705	0.21776405135227134+1000.0	1000.0	0.4913932896953095	0.2207186496023572+1000.0	1000.0	0.491505065153812	0.2236960582959922+1000.0	1000.0	0.4916168406123145	0.2266961537018905+1000.0	1000.0	0.4917286160708169	0.2297188066013383+1000.0	1000.0	0.4918403915293194	0.23276388228561176+1000.0	1000.0	0.49195216698782185	0.23583124055555846+1000.0	1000.0	0.49206394244632434	0.2389207357233966+1000.0	1000.0	0.49217571790482684	0.2420322166167017+1000.0	1000.0	0.4922874933633293	0.24516552658463295+1000.0	1000.0	0.49239926882183177	0.24832050350639615+1000.0	1000.0	0.49251104428033426	0.25149697980194174+1000.0	1000.0	0.4926228197388367	0.2546947824449353+1000.0	1000.0	0.4927345951973392	0.25791373297799547+1000.0	1000.0	0.4928463706558417	0.2611536475301937+1000.0	1000.0	0.49295814611434413	0.264414336836849+1000.0	1000.0	0.4930699215728466	0.2676956062616101+1000.0	1000.0	0.49318169703134906	0.27099725582081147+1000.0	1000.0	0.49329347248985156	0.2743190802101463+1000.0	1000.0	0.49340524794835405	0.27766086883360863+1000.0	1000.0	0.4935170234068565	0.28102240583474514+1000.0	1000.0	0.493628798865359	0.28440347013019807+1000.0	1000.0	0.4937405743238615	0.28780383544552185+1000.0	1000.0	0.4938523497823639	0.2912232703532978+1000.0	1000.0	0.4939641252408664	0.294661538313527+1000.0	1000.0	0.49407590069936885	0.29811839771627646+1000.0	1000.0	0.49418767615787135	0.3015936019266123+1000.0	1000.0	0.49429945161637384	0.30508689933175853+1000.0	1000.0	0.4944112270748763	0.3085980333905133+1000.0	1000.0	0.4945230025333788	0.3121267426848914+1000.0	1000.0	0.49463477799188127	0.31567276097396424+1000.0	1000.0	0.4947465534503837	0.3192358172499118+1000.0	1000.0	0.4948583289088862	0.32281563579625444+1000.0	1000.0	0.4949701043673887	0.3264119362482295+1000.0	1000.0	0.49508187982589114	0.3300244336553255+1000.0	1000.0	0.49519365528439363	0.3336528385459353+1000.0	1000.0	0.49530543074289607	0.3372968569940891+1000.0	1000.0	0.49541720620139856	0.34095619068828736+1000.0	1000.0	0.49552898165990106	0.34463053700235374+1000.0	1000.0	0.4956407571184035	0.3483195890683279+1000.0	1000.0	0.495752532576906	0.3520230358513505+1000.0	1000.0	0.4958643080354085	0.3557405622264958+1000.0	1000.0	0.4959760834939109	0.3594718490575535+1000.0	1000.0	0.4960878589524134	0.36321657327771134+1000.0	1000.0	0.49619963441091586	0.3669744079720874+1000.0	1000.0	0.49631140986941835	0.37074502246212343+1000.0	1000.0	0.49642318532792085	0.37452808239174806+1000.0	1000.0	0.4965349607864233	0.3783232498153195+1000.0	1000.0	0.4966467362449258	0.3821301832872915+1000.0	1000.0	0.4967585117034283	0.3859485379535454+1000.0	1000.0	0.4968702871619307	0.3897779656443837+1000.0	1000.0	0.4969820626204332	0.39361811496912447+1000.0	1000.0	0.4970938380789357	0.3974686314122391+1000.0	1000.0	0.49720561353743814	0.40132915743102265+1000.0	1000.0	0.49731738899594063	0.4051993325547357+1000.0	1000.0	0.4974291644544431	0.4090787934851533+1000.0	1000.0	0.49754093991294557	0.41296717419852114+1000.0	1000.0	0.49765271537144806	0.4168641060488157+1000.0	1000.0	0.4977644908299505	0.42076921787230587+1000.0	1000.0	0.497876266288453	0.42468213609335004+1000.0	1000.0	0.4979880417469555	0.4286024848313584+1000.0	1000.0	0.49809981720545793	0.4325298860089068+1000.0	1000.0	0.4982115926639604	0.4364639594609313+1000.0	1000.0	0.4983233681224629	0.4404043230449345+1000.0	1000.0	0.49843514358096536	0.44435059275218464+1000.0	1000.0	0.49854691903946785	0.44830238281983714+1000.0	1000.0	0.4986586944979703	0.4522593058439035+1000.0	1000.0	0.4987704699564728	0.4562209728930621+1000.0	1000.0	0.4988822454149753	0.46018699362319626+1000.0	1000.0	0.4989940208734777	0.4641569763926523+1000.0	1000.0	0.4991057963319802	0.46813052837814145+1000.0	1000.0	0.4992175717904827	0.47210725569121115+1000.0	1000.0	0.49932934724898514	0.47608676349526274+1000.0	1000.0	0.49944112270748764	0.48006865612304095+1000.0	1000.0	0.4995528981659901	0.48405253719451674+1000.0	1000.0	0.4996646736244926	0.4880380097351544+1000.0	1000.0	0.49977644908299507	0.49202467629444474+1000.0	1000.0	0.4998882245414975	0.4960121390646957+1000.0	1000.0	0.5	0.5+1000.0	1000.0	0.5001117754585025	0.5039878609353043+1000.0	1000.0	0.500223550917005	0.5079753237055572+1000.0	1000.0	0.5003353263755074	0.5119619902648437+1000.0	1000.0	0.5004471018340099	0.5159474628054813+1000.0	1000.0	0.5005588772925124	0.519931343876959+1000.0	1000.0	0.5006706527510149	0.5239132365047373+1000.0	1000.0	0.5007824282095173	0.5278927443087909+1000.0	1000.0	0.5008942036680198	0.5318694716218605+1000.0	1000.0	0.5010059791265222	0.5358430236073457+1000.0	1000.0	0.5011177545850247	0.5398130063768037+1000.0	1000.0	0.5012295300435272	0.5437790271069379+1000.0	1000.0	0.5013413055020297	0.5477406941560965+1000.0	1000.0	0.5014530809605322	0.5516976171801649+1000.0	1000.0	0.5015648564190346	0.5556494072478134+1000.0	1000.0	0.5016766318775371	0.5595956769550655+1000.0	1000.0	0.5017884073360396	0.5635360405390687+1000.0	1000.0	0.5019001827945421	0.5674701139910932+1000.0	1000.0	0.5020119582530446	0.5713975151686436+1000.0	1000.0	0.5021237337115471	0.575317863906652+1000.0	1000.0	0.5022355091700494	0.5792307821276922+1000.0	1000.0	0.5023472846285519	0.5831358939511844+1000.0	1000.0	0.5024590600870544	0.5870328258014789+1000.0	1000.0	0.5025708355455569	0.5909212065148467+1000.0	1000.0	0.5026826110040594	0.5948006674452662+1000.0	1000.0	0.5027943864625618	0.5986708425689754+1000.0	1000.0	0.5029061619210643	0.6025313685877609+1000.0	1000.0	0.5030179373795668	0.6063818850308755+1000.0	1000.0	0.5031297128380693	0.6102220343556163+1000.0	1000.0	0.5032414882965718	0.6140514620464564+1000.0	1000.0	0.5033532637550742	0.6178698167127067+1000.0	1000.0	0.5034650392135767	0.6216767501846786+1000.0	1000.0	0.5035768146720792	0.6254719176082519+1000.0	1000.0	0.5036885901305816	0.6292549775378765+1000.0	1000.0	0.5038003655890841	0.6330255920279126+1000.0	1000.0	0.5039121410475866	0.6367834267222905+1000.0	1000.0	0.504023916506089	0.6405281509424446+1000.0	1000.0	0.5041356919645915	0.6442594377735041+1000.0	1000.0	0.504247467423094	0.6479769641486496+1000.0	1000.0	0.5043592428815965	0.6516804109316722+1000.0	1000.0	0.504471018340099	0.6553694629976481+1000.0	1000.0	0.5045827937986014	0.6590438093117108+1000.0	1000.0	0.5046945692571039	0.662703143005909+1000.0	1000.0	0.5048063447156064	0.6663471614540647+1000.0	1000.0	0.5049181201741089	0.6699755663446745+1000.0	1000.0	0.5050298956326114	0.6735880637517723+1000.0	1000.0	0.5051416710911139	0.6771843642037474+1000.0	1000.0	0.5052534465496162	0.6807641827500864+1000.0	1000.0	0.5053652220081187	0.6843272390260358+1000.0	1000.0	0.5054769974666212	0.6878732573151086+1000.0	1000.0	0.5055887729251237	0.6914019666094867+1000.0	1000.0	0.5057005483836262	0.6949131006682432+1000.0	1000.0	0.5058123238421286	0.698406398073386+1000.0	1000.0	0.5059240993006311	0.7018816022837218+1000.0	1000.0	0.5060358747591336	0.705338461686473+1000.0	1000.0	0.5061476502176361	0.7087767296467022+1000.0	1000.0	0.5062594256761386	0.7121961645544799+1000.0	1000.0	0.5063712011346411	0.7155965298698036+1000.0	1000.0	0.5064829765931435	0.7189775941652532+1000.0	1000.0	0.506594752051646	0.7223391311663914+1000.0	1000.0	0.5067065275101484	0.7256809197898537+1000.0	1000.0	0.5068183029686509	0.7290027441791885+1000.0	1000.0	0.5069300784271534	0.7323043937383915+1000.0	1000.0	0.5070418538856558	0.7355856631631493+1000.0	1000.0	0.5071536293441583	0.7388463524698063+1000.0	1000.0	0.5072654048026608	0.7420862670220045+1000.0	1000.0	0.5073771802611633	0.7453052175550647+1000.0	1000.0	0.5074889557196658	0.7485030201980598+1000.0	1000.0	0.5076007311781683	0.7516794964936054+1000.0	1000.0	0.5077125066366707	0.7548344734153655+1000.0	1000.0	0.5078242820951732	0.7579677833832983+1000.0	1000.0	0.5079360575536757	0.7610792642766034+1000.0	1000.0	0.5080478330121782	0.7641687594444415+1000.0	1000.0	0.5081596084706806	0.7672361177143897+1000.0	1000.0	0.508271383929183	0.7702811933986602+1000.0	1000.0	0.5083831593876855	0.7733038462981094+1000.0	1000.0	0.508494934846188	0.7763039417040077+1000.0	1000.0	0.5086067103046905	0.7792813503976428+1000.0	1000.0	0.508718485763193	0.7822359486477302+1000.0	1000.0	0.5088302612216954	0.7851676182056627+1000.0	1000.0	0.5089420366801979	0.7880762462986323+1000.0	1000.0	0.5090538121387004	0.7909617256205992+1000.0	1000.0	0.5091655875972029	0.7938239543211898+1000.0	1000.0	0.5092773630557054	0.7966628359925001+1000.0	1000.0	0.5093891385142079	0.7994782796538454+1000.0	1000.0	0.5095009139727102	0.8022701997344754+1000.0	1000.0	0.5096126894312127	0.8050385160542955+1000.0	1000.0	0.5097244648897152	0.8077831538025806+1000.0	1000.0	0.5098362403482177	0.8105040435147612+1000.0	1000.0	0.5099480158067202	0.813201121047272+1000.0	1000.0	0.5100597912652226	0.8158743275505018+1000.0	1000.0	0.5101715667237251	0.818523609439891+1000.0	1000.0	0.5102833421822276	0.8211489183651661+1000.0	1000.0	0.5103951176407301	0.823750211177796+1000.0	1000.0	0.5105068930992326	0.8263274498966626+1000.0	1000.0	0.5106186685577351	0.8288806016719947+1000.0	1000.0	0.5107304440162375	0.831409638747594+1000.0	1000.0	0.51084221947474	0.8339145384214028+1000.0	1000.0	0.5109539949332425	0.8363952830044104+1000.0	1000.0	0.511065770391745	0.8388518597779848+1000.0	1000.0	0.5111775458502474	0.8412842609496293+1000.0	1000.0	0.5112893213087498	0.8436924836072104+1000.0	1000.0	0.5114010967672523	0.8460765296717113+1000.0	1000.0	0.5115128722257548	0.8484364058485091+1000.0	1000.0	0.5116246476842573	0.8507721235772641+1000.0	1000.0	0.5117364231427598	0.8530836989804239+1000.0	1000.0	0.5118481986012623	0.8553711528103963+1000.0	1000.0	0.5119599740597647	0.8576345103954258+1000.0	1000.0	0.5120717495182672	0.8598738015842274+1000.0	1000.0	0.5121835249767697	0.8620890606893856+1000.0	1000.0	0.5122953004352722	0.8642803264296026+1000.0	1000.0	0.5124070758937747	0.8664476418708047+1000.0	1000.0	0.512518851352277	0.8685910543661586+1000.0	1000.0	0.5126306268107795	0.8707106154950497+1000.0	1000.0	0.512742402269282	0.8728063810010335+1000.0	1000.0	0.5128541777277845	0.8748784107288421+1000.0	1000.0	0.512965953186287	0.8769267685604584+1000.0	1000.0	0.5130777286447894	0.8789515223503089+1000.0	1000.0	0.5131895041032919	0.8809527438596283+1000.0	1000.0	0.5133012795617944	0.8829305086900069+1000.0	1000.0	0.5134130550202969	0.884884896216204+1000.0	1000.0	0.5135248304787994	0.886815989518239+1000.0	1000.0	0.5136366059373019	0.8887238753128136+1000.0	1000.0	0.5137483813958043	0.8906086438841039+1000.0	1000.0	0.5138601568543067	0.892470389013975+1000.0	1000.0	0.5139719323128092	0.8943092079116323+1000.0	1000.0	0.5140837077713117	0.8961252011427868+1000.0	1000.0	0.5141954832298142	0.8979184725583497+1000.0	1000.0	0.5143072586883166	0.8996891292227053+1000.0	1000.0	0.5144190341468191	0.901437281341612+1000.0	1000.0	0.5145308096053216	0.903163042189746+1000.0	1000.0	0.5146425850638241	0.9048665280379616+1000.0	1000.0	0.5147543605223266	0.9065478580802845+1000.0	1000.0	0.5148661359808291	0.9082071543606841+1000.0	1000.0	0.5149779114393315	0.9098445416996641+1000.0	1000.0	0.515089686897834	0.9114601476207185+1000.0	1000.0	0.5152014623563365	0.9130541022766656+1000.0	1000.0	0.515313237814839	0.9146265383759332+1000.0	1000.0	0.5154250132733414	0.9161775911088058+1000.0	1000.0	0.5155367887318438	0.9177073980736792+1000.0	1000.0	0.5156485641903463	0.9192160992033679+1000.0	1000.0	0.5157603396488488	0.9207038366914762+1000.0	1000.0	0.5158721151073513	0.922170754918901+1000.0	1000.0	0.5159838905658538	0.9236170003804764+1000.0	1000.0	0.5160956660243563	0.9250427216118049+1000.0	1000.0	0.5162074414828587	0.9264480691163045+1000.0	1000.0	0.5163192169413612	0.9278331952925151+1000.0	1000.0	0.5164309923998637	0.9291982543616738+1000.0	1000.0	0.5165427678583662	0.9305434022956196+1000.0	1000.0	0.5166545433168687	0.9318687967450383+1000.0	1000.0	0.516766318775371	0.9331745969680835+1000.0	1000.0	0.5168780942338735	0.9344609637594127+1000.0	1000.0	0.516989869692376	0.9357280593796449+1000.0	1000.0	0.5171016451508785	0.936976047485297+1000.0	1000.0	0.517213420609381	0.9382050930592051+1000.0	1000.0	0.5173251960678835	0.9394153623414678+1000.0	1000.0	0.5174369715263859	0.9406070227609331+1000.0	1000.0	0.5175487469848884	0.9417802428672629+1000.0	1000.0	0.5176605224433909	0.9429351922635828+1000.0	1000.0	0.5177722979018934	0.9440720415397631+1000.0	1000.0	0.5178840733603959	0.9451909622063392+1000.0	1000.0	0.5179958488188983	0.9462921266290981+1000.0	1000.0	0.5181076242774008	0.9473757079643584+1000.0	1000.0	0.5182193997359033	0.9484418800949511+1000.0	1000.0	0.5183311751944057	0.9494908175669394+1000.0	1000.0	0.5184429506529082	0.9505226955270846+1000.0	1000.0	0.5185547261114106	0.9515376896610804+1000.0	1000.0	0.5186665015699131	0.9525359761325791+1000.0	1000.0	0.5187782770284156	0.9535177315230126+1000.0	1000.0	0.5188900524869181	0.9544831327722441+1000.0	1000.0	0.5190018279454206	0.9554323571200519+1000.0	1000.0	0.5191136034039231	0.9563655820484669+1000.0	1000.0	0.5192253788624255	0.9572829852249745+1000.0	1000.0	0.519337154320928	0.9581847444466014+1000.0	1000.0	0.5194489297794305	0.9590710375848877+1000.0	1000.0	0.519560705237933	0.9599420425317702+1000.0	1000.0	0.5196724806964355	0.9607979371463807+1000.0	1000.0	0.5197842561549378	0.9616388992027696+1000.0	1000.0	0.5198960316134403	0.9624651063385702+1000.0	1000.0	0.5200078070719428	0.9632767360046024+1000.0	1000.0	0.5201195825304453	0.9640739654154383+1000.0	1000.0	0.5202313579889478	0.964856971500926+1000.0	1000.0	0.5203431334474503	0.965625930858683+1000.0	1000.0	0.5204549089059527	0.9663810197075664+1000.0	1000.0	0.5205666843644552	0.9671224138421262+1000.0	1000.0	0.5206784598229577	0.9678502885880379+1000.0	1000.0	0.5207902352814602	0.9685648187585341+1000.0	1000.0	0.5209020107399627	0.9692661786118252+1000.0	1000.0	0.5210137861984651	0.9699545418095183+1000.0	1000.0	0.5211255616569676	0.9706300813760388+1000.0	1000.0	0.52123733711547	0.9712929696590457+1000.0	1000.0	0.5213491125739725	0.9719433782908565+1000.0	1000.0	0.521460888032475	0.9725814781508723+1000.0	1000.0	0.5215726634909775	0.9732074393290034+1000.0	1000.0	0.5216844389494799	0.9738214310900996+1000.0	1000.0	0.5217962144079824	0.9744236218393805+1000.0	1000.0	0.5219079898664849	0.9750141790888613+1000.0	1000.0	0.5220197653249874	0.9755932694247788+1000.0	1000.0	0.5221315407834899	0.9761610584760078+1000.0	1000.0	0.5222433162419923	0.976717710883468+1000.0	1000.0	0.5223550917004948	0.9772633902705187+1000.0	1000.0	0.5224668671589973	0.9777982592143282+1000.0	1000.0	0.5225786426174998	0.9783224792182259+1000.0	1000.0	0.5226904180760023	0.9788362106850214+1000.0	1000.0	0.5228021935345047	0.9793396128912885+1000.0	1000.0	0.5229139689930071	0.9798328439626074+1000.0	1000.0	0.5230257444515096	0.9803160608497594+1000.0	1000.0	0.5231375199100121	0.9807894193058613+1000.0	1000.0	0.5232492953685146	0.9812530738644405+1000.0	1000.0	0.5233610708270171	0.9817071778184355+1000.0	1000.0	0.5234728462855195	0.9821518832001164+1000.0	1000.0	0.523584621744022	0.9825873407619172+1000.0	1000.0	0.5236963972025245	0.9830136999581663+1000.0	1000.0	0.523808172661027	0.9834311089277119+1000.0	1000.0	0.5239199481195295	0.9838397144774278+1000.0	1000.0	0.5240317235780318	0.9842396620665912+1000.0	1000.0	0.5241434990365343	0.9846310957921223+1000.0	1000.0	0.5242552744950368	0.9850141583746712+1000.0	1000.0	0.5243670499535393	0.9853889911455485+1000.0	1000.0	0.5244788254120418	0.9857557340344794+1000.0	1000.0	0.5245906008705443	0.9861145255581775+1000.0	1000.0	0.5247023763290467	0.9864655028097201+1000.0	1000.0	0.5248141517875492	0.9868088014487193+1000.0	1000.0	0.5249259272460517	0.9871445556922677+1000.0	1000.0	0.5250377027045542	0.9874728983066576+1000.0	1000.0	0.5251494781630567	0.9877939605998513+1000.0	1000.0	0.5252612536215591	0.9881078724146944+1000.0	1000.0	0.5253730290800616	0.98841476212286+1000.0	1000.0	0.525484804538564	0.9887147566195045+1000.0	1000.0	0.5255965799970665	0.9890079813186313+1000.0	1000.0	0.525708355455569	0.9892945601491403+1000.0	1000.0	0.5258201309140715	0.9895746155515542+1000.0	1000.0	0.5259319063725739	0.9898482684754074+1000.0	1000.0	0.5260436818310764	0.9901156383772837+1000.0	1000.0	0.5261554572895789	0.9903768432194872+1000.0	1000.0	0.5262672327480814	0.9906319994693358+1000.0	1000.0	0.5263790082065839	0.9908812220990615+1000.0	1000.0	0.5264907836650863	0.9911246245863033+1000.0	1000.0	0.5266025591235888	0.9913623189151808+1000.0	1000.0	0.5267143345820913	0.9915944155779303+1000.0	1000.0	0.5268261100405938	0.9918210235770953+1000.0	1000.0	0.5269378854990963	0.9920422504282529+1000.0	1000.0	0.5270496609575988	0.9922582021632638+1000.0	1000.0	0.5271614364161011	0.9924689833340309+1000.0	1000.0	0.5272732118746036	0.9926746970167561+1000.0	1000.0	0.5273849873331061	0.9928754448166751+1000.0	1000.0	0.5274967627916086	0.9930713268732624+1000.0	1000.0	0.5276085382501111	0.9932624418658901+1000.0	1000.0	0.5277203137086135	0.9934488870199264+1000.0	1000.0	0.527832089167116	0.9936307581132616+1000.0	1000.0	0.5279438646256185	0.9938081494832456+1000.0	1000.0	0.528055640084121	0.9939811540340266+1000.0	1000.0	0.5281674155426235	0.9941498632442758+1000.0	1000.0	0.528279191001126	0.9943143671752838+1000.0	1000.0	0.5283909664596284	0.994474754479418+1000.0	1000.0	0.5285027419181308	0.9946311124089268+1000.0	1000.0	0.5286145173766333	0.9947835268250755+1000.0	1000.0	0.5287262928351358	0.9949320822076054+1000.0	1000.0	0.5288380682936383	0.9950768616645+1000.0	1000.0	0.5289498437521407	0.9952179469420461+1000.0	1000.0	0.5290616192106432	0.9953554184351798+1000.0	1000.0	0.5291733946691457	0.9954893551981014+1000.0	1000.0	0.5292851701276482	0.9956198349551505+1000.0	1000.0	0.5293969455861507	0.995746934111927+1000.0	1000.0	0.5295087210446531	0.9958707277666476+1000.0	1000.0	0.5296204965031556	0.9959912897217256+1000.0	1000.0	0.5297322719616581	0.9961086924955612+1000.0	1000.0	0.5298440474201606	0.9962230073345351+1000.0	1000.0	0.529955822878663	0.9963343042251881+1000.0	1000.0	0.5300675983371655	0.9964426519065819+1000.0	1000.0	0.5301793737956679	0.9965481178828257+1000.0	1000.0	0.5302911492541704	0.9966507684357616+1000.0	1000.0	0.5304029247126729	0.9967506686377942+1000.0	1000.0	0.5305147001711754	0.9968478823648592+1000.0	1000.0	0.5306264756296779	0.9969424723095166+1000.0	1000.0	0.5307382510881803	0.9970344999941613+1000.0	1000.0	0.5308500265466828	0.9971240257843406+1000.0	1000.0	0.5309618020051853	0.9972111089021685+1000.0	1000.0	0.5310735774636878	0.997295807439829+1000.0	1000.0	0.5311853529221903	0.9973781783731586+1000.0	1000.0	0.5312971283806928	0.9974582775752979+1000.0	1000.0	0.5314089038391951	0.9975361598304066+1000.0	1000.0	0.5315206792976976	0.9976118788474305+1000.0	1000.0	0.5316324547562001	0.997685487273914+1000.0	1000.0	0.5317442302147026	0.9977570367098488+1000.0	1000.0	0.5318560056732051	0.997826577721553+1000.0	1000.0	0.5319677811317075	0.9978941598555701+1000.0	1000.0	0.53207955659021	0.9979598316525827+1000.0	1000.0	0.5321913320487125	0.9980236406613333+1000.0	1000.0	0.532303107507215	0.9980856334525436+1000.0	1000.0	0.5324148829657175	0.9981458556328279+1000.0	1000.0	0.53252665842422	0.9982043518585911+1000.0	1000.0	0.5326384338827224	0.9982611658499083+1000.0	1000.0	0.5327502093412249	0.998316340404376+1000.0	1000.0	0.5328619847997274	0.9983699174109315+1000.0	1000.0	0.5329737602582298	0.9984219378636336+1000.0	1000.0	0.5330855357167323	0.9984724418753994+1000.0	1000.0	0.5331973111752347	0.9985214686916908+1000.0	1000.0	0.5333090866337372	0.9985690567041463+1000.0	1000.0	0.5334208620922397	0.9986152434641524+1000.0	1000.0	0.5335326375507422	0.9986600656963504+1000.0	1000.0	0.5336444130092447	0.9987035593120724+1000.0	1000.0	0.5337561884677472	0.9987457594227047+1000.0	1000.0	0.5338679639262496	0.9987867003529696+1000.0	1000.0	0.5339797393847521	0.9988264156541267+1000.0	1000.0	0.5340915148432546	0.998864938117086+1000.0	1000.0	0.5342032903017571	0.99890229978543+1000.0	1000.0	0.5343150657602596	0.9989385319683411+1000.0	1000.0	0.5344268412187619	0.9989736652534316+1000.0	1000.0	0.5345386166772644	0.9990077295194706+1000.0	1000.0	0.5346503921357669	0.9990407539490076+1000.0	1000.0	0.5347621675942694	0.9990727670408877+1000.0	1000.0	0.5348739430527719	0.9991037966226554+1000.0	1000.0	0.5349857185112743	0.9991338698628457+1000.0	1000.0	0.5350974939697768	0.9991630132831596+1000.0	1000.0	0.5352092694282793	0.9991912527705189+1000.0	1000.0	0.5353210448867818	0.999218613589003+1000.0	1000.0	0.5354328203452843	0.9992451203916602+1000.0	1000.0	0.5355445958037868	0.999270797232196+1000.0	1000.0	0.5356563712622892	0.9992956675765328+1000.0	1000.0	0.5357681467207916	0.999319754314242+1000.0	1000.0	0.5358799221792941	0.9993430797698458+1000.0	1000.0	0.5359916976377966	0.9993656657139864+1000.0	1000.0	0.5361034730962991	0.9993875333744632+1000.0	1000.0	0.5362152485548015	0.9994087034471345+1000.0	1000.0	0.536327024013304	0.9994291961066848+1000.0	1000.0	0.5364387994718065	0.9994490310172546+1000.0	1000.0	0.536550574930309	0.9994682273429332+1000.0	1000.0	0.5366623503888115	0.9994868037581136+1000.0	1000.0	0.536774125847314	0.999504778457708+1000.0	1000.0	0.5368859013058164	0.9995221691672241+1000.0	1000.0	0.5369976767643189	0.9995389931527013+1000.0	1000.0	0.5371094522228214	0.9995552672305068+1000.0	1000.0	0.5372212276813239	0.9995710077769906+1000.0	1000.0	0.5373330031398263	0.9995862307380007+1000.0	1000.0	0.5374447785983287	0.9996009516382564+1000.0	1000.0	0.5375565540568312	0.9996151855905806+1000.0	1000.0	0.5376683295153337	0.9996289473049927+1000.0	1000.0	0.5377801049738362	0.999642251097659+1000.0	1000.0	0.5378918804323387	0.9996551108997038+1000.0	1000.0	0.5380036558908412	0.9996675402658796+1000.0	1000.0	0.5381154313493436	0.9996795523830981+1000.0	1000.0	0.5382272068078461	0.9996911600788221+1000.0	1000.0	0.5383389822663486	0.999702375829317+1000.0	1000.0	0.5384507577248511	0.9997132117677661+1000.0	1000.0	0.5385625331833536	0.9997236796922467+1000.0	1000.0	0.538674308641856	0.9997337910735705+1000.0	1000.0	0.5387860841003584	0.9997435570629866+1000.0	1000.0	0.5388978595588609	0.9997529884997501+1000.0	1000.0	0.5390096350173634	0.9997620959185564+1000.0	1000.0	0.5391214104758659	0.9997708895568403+1000.0	1000.0	0.5392331859343683	0.9997793793619449+1000.0	1000.0	0.5393449613928708	0.9997875749981571+1000.0	1000.0	0.5394567368513733	0.9997954858536129+1000.0	1000.0	0.5395685123098758	0.999803121047074+1000.0	1000.0	0.5396802877683783	0.9998104894345753+1000.0	1000.0	0.5397920632268808	0.9998175996159456+1000.0	1000.0	0.5399038386853832	0.9998244599412024+1000.0	1000.0	0.5400156141438857	0.9998310785168213+1000.0	1000.0	0.5401273896023882	0.9998374632118834+1000.0	1000.0	0.5402391650608906	0.999843621664099+1000.0	1000.0	0.5403509405193931	0.9998495612857117+1000.0	1000.0	0.5404627159778955	0.9998552892692819+1000.0	1000.0	0.540574491436398	0.9998608125933534+1000.0	1000.0	0.5406862668949005	0.9998661380280018+1000.0	1000.0	0.540798042353403	0.9998712721402696+1000.0	1000.0	0.5409098178119055	0.9998762212994852+1000.0	1000.0	0.541021593270408	0.999880991682471+1000.0	1000.0	0.5411333687289104	0.9998855892786405+1000.0	1000.0	0.5412451441874129	0.9998900198949844+1000.0	1000.0	0.5413569196459154	0.9998942891609506+1000.0	1000.0	0.5414686951044179	0.9998984025332166+1000.0	1000.0	0.5415804705629204	0.999902365300357+1000.0	1000.0	0.5416922460214227	0.999906182587408+1000.0	1000.0	0.5418040214799252	0.9999098593603287+1000.0	1000.0	0.5419157969384277	0.9999134004303636+1000.0	1000.0	0.5420275723969302	0.9999168104583049+1000.0	1000.0	0.5421393478554327	0.9999200939586582+1000.0	1000.0	0.5422511233139352	0.9999232553037111+1000.0	1000.0	0.5423628987724376	0.9999262987275094+1000.0	1000.0	0.5424746742309401	0.9999292283297385+1000.0	1000.0	0.5425864496894426	0.9999320480795152+1000.0	1000.0	0.5426982251479451	0.9999347618190879+1000.0	1000.0	0.5428100006064476	0.9999373732674508+1000.0	1000.0	0.54292177606495	0.9999398860238692+1000.0	1000.0	0.5430335515234525	0.9999423035713213+1000.0	1000.0	0.543145326981955	0.999944629279855+1000.0	1000.0	0.5432571024404574	0.9999468664098633+1000.0	1000.0	0.5433688778989599	0.9999490181152794+1000.0	1000.0	0.5434806533574624	0.9999510874466905+1000.0	1000.0	0.5435924288159648	0.9999530773543769+1000.0	1000.0	0.5437042042744673	0.9999549906912719+1000.0	1000.0	0.5438159797329698	0.9999568302158489+1000.0	1000.0	0.5439277551914723	0.9999585985949341+1000.0	1000.0	0.5440395306499748	0.9999602984064473+1000.0	1000.0	0.5441513061084772	0.9999619321420727+1000.0	1000.0	0.5442630815669797	0.9999635022098599+1000.0	1000.0	0.5443748570254822	0.9999650109367573+1000.0	1000.0	0.5444866324839847	0.9999664605710794+1000.0	1000.0	0.5445984079424872	0.9999678532849091+1000.0	1000.0	0.5447101834009895	0.9999691911764356+1000.0	1000.0	0.544821958859492	0.9999704762722309+1000.0	1000.0	0.5449337343179945	0.9999717105294647+1000.0	1000.0	0.545045509776497	0.9999728958380597+1000.0	1000.0	0.5451572852349995	0.9999740340227888+1000.0	1000.0	0.545269060693502	0.9999751268453145+1000.0	1000.0	0.5453808361520044	0.9999761760061728+1000.0	1000.0	0.5454926116105069	0.9999771831467028+1000.0	1000.0	0.5456043870690094	0.9999781498509217+1000.0	1000.0	0.5457161625275119	0.9999790776473488+1000.0	1000.0	0.5458279379860144	0.9999799680107782+1000.0	1000.0	0.5459397134445167	0.9999808223640003+1000.0	1000.0	0.5460514889030192	0.9999816420794769+1000.0	1000.0	0.5461632643615217	0.9999824284809664+1000.0	1000.0	0.5462750398200242	0.999983182845104+1000.0	1000.0	0.5463868152785267	0.9999839064029361+1000.0	1000.0	0.5464985907370292	0.99998460034141+1000.0	1000.0	0.5466103661955316	0.9999852658048209+1000.0	1000.0	0.5467221416540341	0.999985903896217+1000.0	1000.0	0.5468339171125366	0.9999865156787617+1000.0	1000.0	0.5469456925710391	0.9999871021770587+1000.0	1000.0	0.5470574680295416	0.9999876643784342+1000.0	1000.0	0.547169243488044	0.9999882032341847+1000.0	1000.0	0.5472810189465465	0.9999887196607844+1000.0	1000.0	0.547392794405049	0.9999892145410584+1000.0	1000.0	0.5475045698635514	0.9999896887253198+1000.0	1000.0	0.5476163453220539	0.9999901430324719+1000.0	1000.0	0.5477281207805564	0.9999905782510781+1000.0	1000.0	0.5478398962390588	0.9999909951403975+1000.0	1000.0	0.5479516716975613	0.9999913944313902+1000.0	1000.0	0.5480634471560638	0.9999917768276906+1000.0	1000.0	0.5481752226145663	0.9999921430065507+1000.0	1000.0	0.5482869980730688	0.9999924936197545+1000.0	1000.0	0.5483987735315712	0.9999928292945034+1000.0	1000.0	0.5485105489900737	0.9999931506342736+1000.0	1000.0	0.5486223244485762	0.9999934582196472+1000.0	1000.0	0.5487340999070787	0.9999937526091162+1000.0	1000.0	0.5488458753655812	0.999994034339861+1000.0	1000.0	0.5489576508240837	0.9999943039285051+1000.0	1000.0	0.549069426282586	0.9999945618718437+1000.0	1000.0	0.5491812017410885	0.9999948086475506+1000.0	1000.0	0.549292977199591	0.9999950447148614+1000.0	1000.0	0.5494047526580935	0.9999952705152341+1000.0	1000.0	0.549516528116596	0.9999954864729891+1000.0	1000.0	0.5496283035750984	0.9999956929959275+1000.0	1000.0	0.5497400790336009	0.9999958904759296+1000.0	1000.0	0.5498518544921034	0.9999960792895333+1000.0	1000.0	0.5499636299506059	0.9999962597984928+1000.0	1000.0	0.5500754054091084	0.9999964323503202+1000.0	1000.0	0.5501871808676108	0.9999965972788073+1000.0	1000.0	0.5502989563261133	0.9999967549045309+1000.0	1000.0	0.5504107317846157	0.9999969055353408+1000.0	1000.0	0.5505225072431182	0.9999970494668317+1000.0	1000.0	0.5506342827016207	0.9999971869827975+1000.0	1000.0	0.5507460581601232	0.999997318355673+1000.0	1000.0	0.5508578336186256	0.999997443846957+1000.0	1000.0	0.5509696090771281	0.9999975637076242+1000.0	1000.0	0.5510813845356306	0.9999976781785199+1000.0	1000.0	0.5511931599941331	0.9999977874907438+1000.0	1000.0	0.5513049354526356	0.9999978918660178+1000.0	1000.0	0.551416710911138	0.9999979915170435+1000.0	1000.0	0.5515284863696405	0.9999980866478452+1000.0	1000.0	0.551640261828143	0.9999981774541018+1000.0	1000.0	0.5517520372866455	0.9999982641234668+1000.0	1000.0	0.551863812745148	0.9999983468358773+1000.0	1000.0	0.5519755882036504	0.9999984257638511+1000.0	1000.0	0.5520873636621528	0.9999985010727743+1000.0	1000.0	0.5521991391206553	0.9999985729211782+1000.0	1000.0	0.5523109145791578	0.9999986414610054+1000.0	1000.0	0.5524226900376603	0.999998706837868+1000.0	1000.0	0.5525344654961628	0.9999987691912949+1000.0	1000.0	0.5526462409546652	0.9999988286549713+1000.0	1000.0	0.5527580164131677	0.9999988853569683+1000.0	1000.0	0.5528697918716702	0.9999989394199652+1000.0	1000.0	0.5529815673301727	0.9999989909614632+1000.0	1000.0	0.5530933427886752	0.9999990400939908+1000.0	1000.0	0.5532051182471777	0.9999990869253023+1000.0	1000.0	0.55331689370568	0.9999991315585689+1000.0	1000.0	0.5534286691641825	0.9999991740925621+1000.0	1000.0	0.553540444622685	0.9999992146218307+1000.0	1000.0	0.5536522200811875	0.9999992532368711+1000.0	1000.0	0.55376399553969	0.9999992900242914+1000.0	1000.0	0.5538757709981924	0.9999993250669688+1000.0	1000.0	0.5539875464566949	0.9999993584442013+1000.0	1000.0	0.5540993219151974	0.9999993902318541+1000.0	1000.0	0.5542110973736999	0.9999994205024999+1000.0	1000.0	0.5543228728322024	0.9999994493255535+1000.0	1000.0	0.5544346482907049	0.9999994767674022+1000.0	1000.0	0.5545464237492073	0.9999995028915305+1000.0	1000.0	0.5546581992077098	0.9999995277586402+1000.0	1000.0	0.5547699746662123	0.9999995514267659+1000.0	1000.0	0.5548817501247147	0.9999995739513856+1000.0	1000.0	0.5549935255832172	0.9999995953855281+1000.0	1000.0	0.5551053010417196	0.9999996157798742+1000.0	1000.0	0.5552170765002221	0.9999996351828566+1000.0	1000.0	0.5553288519587246	0.9999996536407534+1000.0	1000.0	0.5554406274172271	0.9999996711977793+1000.0	1000.0	0.5555524028757296	0.9999996878961728+1000.0	1000.0	0.555664178334232	0.9999997037762801+1000.0	1000.0	0.5557759537927345	0.9999997188766356+1000.0	1000.0	0.555887729251237	0.9999997332340387+1000.0	1000.0	0.5559995047097395	0.9999997468836282+1000.0	1000.0	0.556111280168242	0.999999759858954+1000.0	1000.0	0.5562230556267445	0.9999997721920443+1000.0	1000.0	0.5563348310852468	0.9999997839134719+1000.0	1000.0	0.5564466065437493	0.999999795052417+1000.0	1000.0	0.5565583820022518	0.9999998056367272+1000.0	1000.0	0.5566701574607543	0.9999998156929757+1000.0	1000.0	0.5567819329192568	0.9999998252465165+1000.0	1000.0	0.5568937083777592	0.999999834321538+1000.0	1000.0	0.5570054838362617	0.9999998429411134+1000.0	1000.0	0.5571172592947642	0.9999998511272502+1000.0	1000.0	0.5572290347532667	0.9999998589009368+1000.0	1000.0	0.5573408102117692	0.9999998662821873+1000.0	1000.0	0.5574525856702717	0.9999998732900852+1000.0	1000.0	0.557564361128774	0.9999998799428238+1000.0	1000.0	0.5576761365872765	0.9999998862577465+1000.0	1000.0	0.557787912045779	0.9999998922513842+1000.0	1000.0	0.5578996875042815	0.9999998979394916+1000.0	1000.0	0.558011462962784	0.9999999033370823+1000.0	1000.0	0.5581232384212864	0.9999999084584619+1000.0	1000.0	0.5582350138797889	0.9999999133172596+1000.0	1000.0	0.5583467893382914	0.999999917926459+1000.0	1000.0	0.5584585647967939	0.9999999222984275+1000.0	1000.0	0.5585703402552964	0.9999999264449435+1000.0	1000.0	0.5586821157137989	0.999999930377224+1000.0	1000.0	0.5587938911723013	0.9999999341059498+1000.0	1000.0	0.5589056666308038	0.9999999376412899+1000.0	1000.0	0.5590174420893063	0.9999999409929251+1000.0	1000.0	0.5591292175478088	0.9999999441700703+1000.0	1000.0	0.5592409930063112	0.9999999471814961+1000.0	1000.0	0.5593527684648136	0.9999999500355492+1000.0	1000.0	0.5594645439233161	0.9999999527401718+1000.0	1000.0	0.5595763193818186	0.9999999553029207+1000.0	1000.0	0.5596880948403211	0.9999999577309853+1000.0	1000.0	0.5597998702988236	0.9999999600312037+1000.0	1000.0	0.5599116457573261	0.9999999622100807+1000.0	1000.0	0.5600234212158285	0.999999964273802+1000.0	1000.0	0.560135196674331	0.9999999662282499+1000.0	1000.0	0.5602469721328335	0.9999999680790175+1000.0	1000.0	0.560358747591336	0.9999999698314219+1000.0	1000.0	0.5604705230498385	0.9999999714905179+1000.0	1000.0	0.5605822985083408	0.99999997306111+1000.0	1000.0	0.5606940739668433	0.999999974547764+1000.0	1000.0	0.5608058494253458	0.9999999759548193+1000.0	1000.0	0.5609176248838483	0.9999999772863986+1000.0	1000.0	0.5610294003423508	0.999999978546419+1000.0	1000.0	0.5611411758008532	0.9999999797386016+1000.0	1000.0	0.5612529512593557	0.9999999808664809+1000.0	1000.0	0.5613647267178582	0.9999999819334138+1000.0	1000.0	0.5614765021763607	0.9999999829425882+1000.0	1000.0	0.5615882776348632	0.999999983897031+1000.0	1000.0	0.5617000530933657	0.9999999847996165+1000.0	1000.0	0.5618118285518681	0.9999999856530727+1000.0	1000.0	0.5619236040103706	0.9999999864599896+1000.0	1000.0	0.562035379468873	0.9999999872228252+1000.0	1000.0	0.5621471549273755	0.9999999879439122+1000.0	1000.0	0.562258930385878	0.9999999886254642+1000.0	1000.0	0.5623707058443804	0.9999999892695814+1000.0	1000.0	0.5624824813028829	0.9999999898782562+1000.0	1000.0	0.5625942567613854	0.9999999904533787+1000.0	1000.0	0.5627060322198879	0.9999999909967414+1000.0	1000.0	0.5628178076783904	0.9999999915100446+1000.0	1000.0	0.5629295831368929	0.9999999919949004+1000.0	1000.0	0.5630413585953953	0.9999999924528373+1000.0	1000.0	0.5631531340538978	0.9999999928853047+1000.0	1000.0	0.5632649095124003	0.9999999932936763+1000.0	1000.0	0.5633766849709028	0.9999999936792543+1000.0	1000.0	0.5634884604294053	0.9999999940432727+1000.0	1000.0	0.5636002358879076	0.999999994386901+1000.0	1000.0	0.5637120113464101	0.9999999947112473+1000.0	1000.0	0.5638237868049126	0.9999999950173617+1000.0	1000.0	0.5639355622634151	0.9999999953062385+1000.0	1000.0	0.5640473377219176	0.9999999955788199+1000.0	1000.0	0.5641591131804201	0.9999999958359982+1000.0	1000.0	0.5642708886389225	0.9999999960786181+1000.0	1000.0	0.564382664097425	0.9999999963074799+1000.0	1000.0	0.5644944395559275	0.9999999965233412+1000.0	1000.0	0.56460621501443	0.9999999967269189+1000.0	1000.0	0.5647179904729325	0.999999996918892+1000.0	1000.0	0.5648297659314349	0.9999999970999028+1000.0	1000.0	0.5649415413899374	0.9999999972705597+1000.0	1000.0	0.5650533168484398	0.9999999974314377+1000.0	1000.0	0.5651650923069423	0.9999999975830814+1000.0	1000.0	0.5652768677654448	0.9999999977260059+1000.0	1000.0	0.5653886432239472	0.9999999978606982+1000.0	1000.0	0.5655004186824497	0.9999999979876191+1000.0	1000.0	0.5656121941409522	0.9999999981072045+1000.0	1000.0	0.5657239695994547	0.9999999982198666+1000.0	1000.0	0.5658357450579572	0.999999998325995+1000.0	1000.0	0.5659475205164597	0.999999998425958+1000.0	1000.0	0.5660592959749621	0.999999998520104+1000.0	1000.0	0.5661710714334646	0.9999999986087621+1000.0	1000.0	0.5662828468919671	0.9999999986922434+1000.0	1000.0	0.5663946223504696	0.9999999987708419+1000.0	1000.0	0.566506397808972	0.9999999988448355+1000.0	1000.0	0.5666181732674744	0.9999999989144865+1000.0	1000.0	0.5667299487259769	0.999999998980043+1000.0	1000.0	0.5668417241844794	0.9999999990417391+1000.0	1000.0	0.5669534996429819	0.9999999990997961+1000.0	1000.0	0.5670652751014844	0.9999999991544228+1.0	0.0001	0.9999548266540229	0.0009999999999999092+1.0	0.0001	0.999999997979714	0.0020000000020092583+1.0	0.0001	0.9999999999999105	0.002999962774430376+1.0	10000.0	1.0005002835334893e-07	0.001+1.0	10000.0	2.002002470272356e-07	0.002+1.0	10000.0	3.0045085689450443e-07	0.003+1.0	10000.0	4.00802059432715e-07	0.004+1.0	10000.0	5.012540567265716e-07	0.005000000000000001+1.0	10000.0	6.018070514703654e-07	0.006+1.0	10000.0	7.024612469704287e-07	0.007+1.0	10000.0	8.032168471476014e-07	0.008+1.0	10000.0	9.0407405653971e-07	0.009+1.0	10000.0	1.0050330803040595e-06	0.01+1.0	10000.0	1.1060941242199367e-06	0.010999999999999998+1.0	10000.0	1.2072573946911288e-06	0.012000000000000002+1.0	10000.0	1.30852309874845e-06	0.013+1.0	10000.0	1.4098914440522873e-06	0.014+1.0	10000.0	1.5113626388951531e-06	0.015+1.0	10000.0	1.612936892204255e-06	0.016+1.0	10000.0	1.7146144135440763e-06	0.017+1.0	10000.0	1.8163954131189703e-06	0.018+1.0	10000.0	1.9182801017757696e-06	0.019+1.0	10000.0	2.0202686910064044e-06	0.02+1.0	10000.0	2.1223613929505394e-06	0.021+1.0	10000.0	2.224558420398219e-06	0.022+1.0	10000.0	2.3268599867925328e-06	0.023+1.0	10000.0	2.4292663062322845e-06	0.024+1.0	10000.0	2.531777593474687e-06	0.025000000000000005+1.0	10000.0	2.634394063938059e-06	0.026+1.0	10000.0	2.7371159337045464e-06	0.027+1.0	10000.0	2.8399434195228495e-06	0.028+1.0	10000.0	2.9428767388109687e-06	0.029+1.0	10000.0	3.0459161096589614e-06	0.03+1.0	10000.0	3.1490617508317173e-06	0.031+1.0	10000.0	3.2523138817717436e-06	0.032+1.0	10000.0	3.3556727226019667e-06	0.033+1.0	10000.0	3.459138494128548e-06	0.034+1.0	10000.0	3.5627114178437156e-06	0.035+1.0	10000.0	3.6663917159286076e-06	0.036+1.0	10000.0	3.7701796112561353e-06	0.037+1.0	10000.0	3.874075327393853e-06	0.03799999999999999+1.0	10000.0	3.978079088606854e-06	0.039+1.0	10000.0	4.082191119860668e-06	0.04+1.0	10000.0	4.186411646824188e-06	0.041+1.0	10000.0	4.2907408958726016e-06	0.042+1.0	10000.0	4.3951790940903425e-06	0.043+1.0	10000.0	4.499726469274058e-06	0.044+1.0	10000.0	4.604383249935586e-06	0.045+1.0	10000.0	4.709149665304957e-06	0.046+1.0	10000.0	4.8140259453334044e-06	0.047+1.0	10000.0	4.919012320696394e-06	0.048+1.0	10000.0	5.024109022796666e-06	0.049+1.0	10000.0	5.1293162837673e-06	0.05+1.0	10000.0	5.234634336474788e-06	0.051+1.0	10000.0	5.340063414522129e-06	0.052+1.0	10000.0	5.445603752251938e-06	0.053+1.0	10000.0	5.55125558474957e-06	0.054+1.0	10000.0	5.657019147846265e-06	0.055+1.0	10000.0	5.762894678122304e-06	0.056+1.0	10000.0	5.868882412910185e-06	0.057+1.0	10000.0	5.9749825902978195e-06	0.05800000000000001+1.0	10000.0	6.081195449131733e-06	0.059+1.0	10000.0	6.187521229020303e-06	0.06+1.0	10000.0	6.2939601703369916e-06	0.061+1.0	10000.0	6.400512514223613e-06	0.062+1.0	10000.0	6.507178502593609e-06	0.063+1.0	10000.0	6.613958378135345e-06	0.064+1.0	10000.0	6.720852384315426e-06	0.065+1.0	10000.0	6.827860765382023e-06	0.066+1.0	10000.0	6.934983766368226e-06	0.067+1.0	10000.0	7.042221633095407e-06	0.068+1.0	10000.0	7.1495746121766105e-06	0.069+1.0	10000.0	7.25704295101995e-06	0.07+1.0	10000.0	7.364626897832033e-06	0.071+1.0	10000.0	7.472326701621404e-06	0.072+1.0	10000.0	7.580142612201997e-06	0.073+1.0	10000.0	7.688074880196612e-06	0.074+1.0	10000.0	7.796123757040419e-06	0.075+1.0	10000.0	7.904289494984463e-06	0.076+1.0	10000.0	8.012572347099208e-06	0.07700000000000001+1.0	10000.0	8.120972567278076e-06	0.078+1.0	10000.0	8.229490410241034e-06	0.079+1.0	10000.0	8.338126131538177e-06	0.08+1.0	10000.0	8.446879987553343e-06	0.081+1.0	10000.0	8.555752235507737e-06	0.082+1.0	10000.0	8.664743133463591e-06	0.083+1.0	10000.0	8.77385294032783e-06	0.084+1.0	10000.0	8.88308191585576e-06	0.085+1.0	10000.0	8.992430320654779e-06	0.086+1.0	10000.0	9.101898416188119e-06	0.087+1.0	10000.0	9.211486464778582e-06	0.088+1.0	10000.0	9.32119472961232e-06	0.089+1.0	10000.0	9.431023474742629e-06	0.09+1.0	10000.0	9.540972965093754e-06	0.091+1.0	10000.0	9.651043466464736e-06	0.092+1.0	10000.0	9.76123524553326e-06	0.093+1.0	10000.0	9.871548569859533e-06	0.094+1.0	10000.0	9.981983707890183e-06	0.095+1.0	10000.0	1.0092540928962178e-05	0.096+1.0	10000.0	1.020322050330677e-05	0.097+1.0	10000.0	1.0314022702053456e-05	0.098+1.0	10000.0	1.0424947797233964e-05	0.099+1.0	10000.0	1.0535996061786264e-05	0.1+1.0	10000.0	1.0647167769558591e-05	0.101+1.0	10000.0	1.0758463195313501e-05	0.102+1.0	10000.0	1.0869882614731952e-05	0.103+1.0	10000.0	1.0981426304417391e-05	0.104+1.0	10000.0	1.109309454189988e-05	0.105+1.0	10000.0	1.1204887605640237e-05	0.10600000000000001+1.0	10000.0	1.1316805775034207e-05	0.107+1.0	10000.0	1.1428849330416647e-05	0.10799999999999998+1.0	10000.0	1.1541018553065746e-05	0.109+1.0	10000.0	1.1653313725207254e-05	0.11+1.0	10000.0	1.1765735130018748e-05	0.111+1.0	10000.0	1.1878283051633917e-05	0.112+1.0	10000.0	1.199095777514687e-05	0.113+1.0	10000.0	1.210375958661647e-05	0.114+1.0	10000.0	1.221668877307069e-05	0.115+1.0	10000.0	1.2329745622511e-05	0.116+1.0	10000.0	1.2442930423916766e-05	0.117+1.0	10000.0	1.2556243467249694e-05	0.118+1.0	10000.0	1.2669685043458281e-05	0.119+1.0	10000.0	1.2783255444482293e-05	0.12+1.0	10000.0	1.289695496325728e-05	0.121+1.0	10000.0	1.301078389371911e-05	0.122+1.0	10000.0	1.3124742530808526e-05	0.123+1.0	10000.0	1.3238831170475732e-05	0.124+1.0	10000.0	1.3353050109685004e-05	0.125+1.0	10000.0	1.3467399646419334e-05	0.126+1.0	10000.0	1.3581880079685088e-05	0.127+1.0	10000.0	1.36964917095167e-05	0.128+1.0	10000.0	1.381123483698139e-05	0.129+1.0	10000.0	1.3926109764183913e-05	0.13+1.0	10000.0	1.4041116794271328e-05	0.131+1.0	10000.0	1.4156256231437797e-05	0.132+1.0	10000.0	1.4271528380929421e-05	0.133+1.0	10000.0	1.4386933549049087e-05	0.134+1.0	10000.0	1.4502472043161354e-05	0.135+1.0	10000.0	1.461814417169737e-05	0.136+1.0	10000.0	1.4733950244159807e-05	0.137+1.0	10000.0	1.4849890571127836e-05	0.138+1.0	10000.0	1.4965965464262116e-05	0.139+1.0	10000.0	1.5082175236309832e-05	0.14+1.0	10000.0	1.519852020110974e-05	0.141+1.0	10000.0	1.531500067359727e-05	0.142+1.0	10000.0	1.5431616969809624e-05	0.143+1.0	10000.0	1.5548369406890925e-05	0.144+1.0	10000.0	1.5665258303097397e-05	0.145+1.0	10000.0	1.5782283977802572e-05	0.146+1.0	10000.0	1.589944675150252e-05	0.14699999999999996+1.0	10000.0	1.6016746945821127e-05	0.148+1.0	10000.0	1.6134184883515373e-05	0.149+1.0	10000.0	1.625176088848068e-05	0.15+1.0	10000.0	1.636947528575627e-05	0.151+1.0	10000.0	1.648732840153054e-05	0.152+1.0	10000.0	1.660532056314651e-05	0.153+1.0	10000.0	1.6723452099107265e-05	0.154+1.0	10000.0	1.6841723339081436e-05	0.155+1.0	10000.0	1.6960134613908738e-05	0.156+1.0	10000.0	1.707868625560552e-05	0.157+1.0	10000.0	1.7197378597370327e-05	0.158+1.0	10000.0	1.7316211973589558e-05	0.159+1.0	10000.0	1.7435186719843094e-05	0.16+1.0	10000.0	1.755430317290998e-05	0.161+1.0	10000.0	1.7673561670774173e-05	0.162+1.0	10000.0	1.779296255263026e-05	0.163+1.0	10000.0	1.791250615888928e-05	0.164+1.0	10000.0	1.8032192831184532e-05	0.165+1.0	10000.0	1.8152022912377424e-05	0.166+1.0	10000.0	1.827199674656339e-05	0.167+1.0	10000.0	1.83921146790778e-05	0.168+1.0	10000.0	1.8512377056501935e-05	0.169+1.0	10000.0	1.8632784226668983e-05	0.17+1.0	10000.0	1.8753336538670075e-05	0.171+1.0	10000.0	1.8874034342860355e-05	0.172+1.0	10000.0	1.8994877990865112e-05	0.173+1.0	10000.0	1.9115867835585894e-05	0.174+1.0	10000.0	1.9237004231206713e-05	0.175+1.0	10000.0	1.935828753320026e-05	0.176+1.0	10000.0	1.947971809833417e-05	0.177+1.0	10000.0	1.960129628467731e-05	0.178+1.0	10000.0	1.9723022451606127e-05	0.179+1.0	10000.0	1.9844896959811006e-05	0.18+1.0	10000.0	1.99669201713027e-05	0.181+1.0	10000.0	2.0089092449418774e-05	0.182+1.0	10000.0	2.0211414158830095e-05	0.183+1.0	10000.0	2.0333885665547367e-05	0.184+1.0	10000.0	2.045650733692771e-05	0.185+1.0	10000.0	2.057927954168125e-05	0.186+1.0	10000.0	2.0702202649877798e-05	0.187+1.0	10000.0	2.0825277032953528e-05	0.188+1.0	10000.0	2.0948503063717714e-05	0.189+1.0	10000.0	2.1071881116359513e-05	0.19+1.0	10000.0	2.1195411566454768e-05	0.191+1.0	10000.0	2.1319094790972892e-05	0.192+1.0	10000.0	2.1442931168283748e-05	0.193+1.0	10000.0	2.1566921078164603e-05	0.194+1.0	10000.0	2.1691064901807127e-05	0.195+1.0	10000.0	2.1815363021824407e-05	0.196+1.0	10000.0	2.193981582225804e-05	0.197+1.0	10000.0	2.2064423688585237e-05	0.198+1.0	10000.0	2.2189187007726003e-05	0.199+1.0	10000.0	2.2314106168050334e-05	0.2+1.0	10000.0	2.243918155938548e-05	0.201+1.0	10000.0	2.256441357302324e-05	0.202+1.0	10000.0	2.268980260172731e-05	0.203+1.0	10000.0	2.2815349039740664e-05	0.204+1.0	10000.0	2.294105328279302e-05	0.205+1.0	10000.0	2.306691572810829e-05	0.206+1.0	10000.0	2.3192936774412135e-05	0.207+1.0	10000.0	2.3319116821939534e-05	0.208+1.0	10000.0	2.3445456272442422e-05	0.209+1.0	10000.0	2.3571955529197357e-05	0.21+1.0	10000.0	2.369861499701324e-05	0.211+1.0	10000.0	2.3825435082239103e-05	0.212+1.0	10000.0	2.3952416192771917e-05	0.213+1.0	10000.0	2.4079558738064477e-05	0.214+1.0	10000.0	2.4206863129133316e-05	0.215+1.0	10000.0	2.433432977856668e-05	0.216+1.0	10000.0	2.4461959100532543e-05	0.217+1.0	10000.0	2.458975151078671e-05	0.218+1.0	10000.0	2.4717707426680916e-05	0.219+1.0	10000.0	2.4845827267171006e-05	0.22+1.0	10000.0	2.4974111452825185e-05	0.221+1.0	10000.0	2.5102560405832286e-05	0.222+1.0	10000.0	2.5231174550010116e-05	0.223+1.0	10000.0	2.5359954310813846e-05	0.224+1.0	10000.0	2.5488900115344446e-05	0.225+1.0	10000.0	2.56180123923572e-05	0.226+1.0	10000.0	2.5747291572270257e-05	0.227+1.0	10000.0	2.5876738087173226e-05	0.228+1.0	10000.0	2.6006352370835864e-05	0.229+1.0	10000.0	2.6136134858716783e-05	0.23+1.0	10000.0	2.6266085987972238e-05	0.231+1.0	10000.0	2.6396206197464955e-05	0.232+1.0	10000.0	2.6526495927773026e-05	0.233+1.0	10000.0	2.6656955621198872e-05	0.234+1.0	10000.0	2.6787585721778224e-05	0.235+1.0	10000.0	2.691838667528924e-05	0.236+1.0	10000.0	2.704935892926158e-05	0.237+1.0	10000.0	2.7180502932985626e-05	0.238+1.0	10000.0	2.731181913752173e-05	0.239+1.0	10000.0	2.7443307995709504e-05	0.24+1.0	10000.0	2.7574969962177218e-05	0.241+1.0	10000.0	2.77068054933512e-05	0.242+1.0	10000.0	2.7838815047465364e-05	0.243+1.0	10000.0	2.797099908457074e-05	0.244+1.0	10000.0	2.8103358066545105e-05	0.245+1.0	10000.0	2.823589245710268e-05	0.246+1.0	10000.0	2.8368602721803853e-05	0.247+1.0	10000.0	2.8501489328065013e-05	0.248+1.0	10000.0	2.8634552745168423e-05	0.249+1.0	10000.0	2.8767793444272156e-05	0.25+1.0	10000.0	2.8901211898420116e-05	0.251+1.0	10000.0	2.9034808582552114e-05	0.252+1.0	10000.0	2.916858397351401e-05	0.253+1.0	10000.0	2.9302538550067933e-05	0.254+1.0	10000.0	2.9436672792902543e-05	0.255+1.0	10000.0	2.9570987184643407e-05	0.256+1.0	10000.0	2.9705482209863407e-05	0.257+1.0	10000.0	2.9840158355093228e-05	0.258+1.0	10000.0	2.997501610883192e-05	0.259+1.0	10000.0	3.011005596155754e-05	0.26+1.0	10000.0	3.0245278405737838e-05	0.261+1.0	10000.0	3.0380683935841057e-05	0.262+1.0	10000.0	3.051627304834676e-05	0.263+1.0	10000.0	3.065204624175677e-05	0.264+1.0	10000.0	3.078800401660616e-05	0.265+1.0	10000.0	3.092414687547432e-05	0.266+1.0	10000.0	3.1060475322996116e-05	0.267+1.0	10000.0	3.1196989865873125e-05	0.268+1.0	10000.0	3.13336910128849e-05	0.269+1.0	10000.0	3.1470579274900375e-05	0.27+1.0	10000.0	3.160765516488932e-05	0.271+1.0	10000.0	3.174491919793387e-05	0.272+1.0	10000.0	3.188237189124015e-05	0.273+1.0	10000.0	3.202001376414994e-05	0.274+1.0	10000.0	3.215784533815247e-05	0.275+1.0	10000.0	3.229586713689629e-05	0.276+1.0	10000.0	3.2434079686201195e-05	0.277+1.0	10000.0	3.2572483514070225e-05	0.278+1.0	10000.0	3.27110791507018e-05	0.279+1.0	10000.0	3.284986712850189e-05	0.28+1.0	10000.0	3.2988847982096295e-05	0.281+1.0	10000.0	3.312802224834298e-05	0.282+1.0	10000.0	3.3267390466344586e-05	0.283+1.0	10000.0	3.340695317746087e-05	0.284+1.0	10000.0	3.354671092532138e-05	0.285+1.0	10000.0	3.3686664255838155e-05	0.286+1.0	10000.0	3.38268137172185e-05	0.287+1.0	10000.0	3.396715985997791e-05	0.288+1.0	10000.0	3.4107703236952995e-05	0.289+1.0	10000.0	3.4248444403314585e-05	0.29+1.0	10000.0	3.4389383916580884e-05	0.291+1.0	10000.0	3.453052233663071e-05	0.292+1.0	10000.0	3.467186022571686e-05	0.293+1.0	10000.0	3.481339814847951e-05	0.294+1.0	10000.0	3.495513667195981e-05	0.295+1.0	10000.0	3.5097076365613465e-05	0.296+1.0	10000.0	3.523921780132449e-05	0.297+1.0	10000.0	3.5381561553419025e-05	0.298+1.0	10000.0	3.552410819867928e-05	0.299+1.0	10000.0	3.566685831635752e-05	0.3+1.0	10000.0	3.580981248819023e-05	0.301+1.0	10000.0	3.5952971298412316e-05	0.302+1.0	10000.0	3.609633533377144e-05	0.303+1.0	10000.0	3.623990518354246e-05	0.304+1.0	10000.0	3.6383681439541934e-05	0.305+1.0	10000.0	3.6527664696142804e-05	0.306+1.0	10000.0	3.6671855550289104e-05	0.307+1.0	10000.0	3.6816254601510814e-05	0.308+1.0	10000.0	3.6960862451938844e-05	0.309+1.0	10000.0	3.710567970632007e-05	0.31+1.0	10000.0	3.725070697203253e-05	0.311+1.0	10000.0	3.739594485910069e-05	0.312+1.0	10000.0	3.754139398021089e-05	0.313+1.0	10000.0	3.768705495072676e-05	0.314+1.0	10000.0	3.7832928388704956e-05	0.315+1.0	10000.0	3.79790149149108e-05	0.316+1.0	10000.0	3.81253151528342e-05	0.317+1.0	10000.0	3.827182972870556e-05	0.318+1.0	10000.0	3.8418559271511925e-05	0.319+1.0	10000.0	3.856550441301313e-05	0.32+1.0	10000.0	3.871266578775816e-05	0.321+1.0	10000.0	3.8860044033101564e-05	0.322+1.0	10000.0	3.9007639789220044e-05	0.323+1.0	10000.0	3.915545369912913e-05	0.324+1.0	10000.0	3.9303486408699986e-05	0.325+1.0	10000.0	3.945173856667635e-05	0.326+1.0	10000.0	3.960021082469159e-05	0.327+1.0	10000.0	3.97489038372859e-05	0.328+1.0	10000.0	3.9897818261923606e-05	0.329+1.0	10000.0	4.0046954759010604e-05	0.33+1.0	10000.0	4.0196313991911965e-05	0.331+1.0	10000.0	4.034589662696959e-05	0.332+1.0	10000.0	4.04957033335201e-05	0.333+1.0	10000.0	4.064573478391277e-05	0.334+1.0	10000.0	4.079599165352767e-05	0.335+1.0	10000.0	4.0946474620793876e-05	0.336+1.0	10000.0	4.1097184367207864e-05	0.337+1.0	10000.0	4.124812157735204e-05	0.338+1.0	10000.0	4.139928693891339e-05	0.339+1.0	10000.0	4.155068114270228e-05	0.34+1.0	10000.0	4.170230488267139e-05	0.341+1.0	10000.0	4.1854158855934825e-05	0.342+1.0	10000.0	4.200624376278734e-05	0.343+1.0	10000.0	4.215856030672368e-05	0.344+1.0	10000.0	4.2311109194458206e-05	0.345+1.0	10000.0	4.246389113594444e-05	0.346+1.0	10000.0	4.2616906844394994e-05	0.347+1.0	10000.0	4.2770157036301496e-05	0.348+1.0	10000.0	4.2923642431454736e-05	0.349+1.0	10000.0	4.307736375296497e-05	0.35+1.0	10000.0	4.3231321727282334e-05	0.351+1.0	10000.0	4.338551708421745e-05	0.352+1.0	10000.0	4.353995055696221e-05	0.353+1.0	10000.0	4.369462288211065e-05	0.354+1.0	10000.0	4.384953479968009e-05	0.355+1.0	10000.0	4.400468705313234e-05	0.356+1.0	10000.0	4.41600803893951e-05	0.357+1.0	10000.0	4.431571555888359e-05	0.358+1.0	10000.0	4.4471593315522216e-05	0.359+1.0	10000.0	4.462771441676658e-05	0.36+1.0	10000.0	4.478407962362549e-05	0.361+1.0	10000.0	4.494068970068326e-05	0.362+1.0	10000.0	4.5097545416122096e-05	0.363+1.0	10000.0	4.525464754174479e-05	0.364+1.0	10000.0	4.5411996852997435e-05	0.365+1.0	10000.0	4.5569594128992406e-05	0.366+1.0	10000.0	4.572744015253153e-05	0.367+1.0	10000.0	4.588553571012942e-05	0.368+1.0	10000.0	4.604388159203696e-05	0.369+1.0	10000.0	4.620247859226503e-05	0.37+1.0	10000.0	4.6361327508608394e-05	0.371+1.0	10000.0	4.6520429142669786e-05	0.372+1.0	10000.0	4.667978429988419e-05	0.373+1.0	10000.0	4.683939378954327e-05	0.374+1.0	10000.0	4.6999258424820065e-05	0.375+1.0	10000.0	4.7159379022793864e-05	0.376+1.0	10000.0	4.731975640447522e-05	0.377+1.0	10000.0	4.748039139483125e-05	0.378+1.0	10000.0	4.7641284822811075e-05	0.379+1.0	10000.0	4.7802437521371496e-05	0.38+1.0	10000.0	4.7963850327502877e-05	0.381+1.0	10000.0	4.8125524082255225e-05	0.382+1.0	10000.0	4.8287459630764486e-05	0.383+1.0	10000.0	4.8449657822279053e-05	0.384+1.0	10000.0	4.8612119510186505e-05	0.385+1.0	10000.0	4.877484555204051e-05	0.386+1.0	10000.0	4.893783680958806e-05	0.387+1.0	10000.0	4.910109414879678e-05	0.388+1.0	10000.0	4.926461843988257e-05	0.389+1.0	10000.0	4.9428410557337444e-05	0.39+1.0	10000.0	4.959247137995758e-05	0.391+1.0	10000.0	4.975680179087162e-05	0.392+1.0	10000.0	4.99214026775692e-05	0.393+1.0	10000.0	5.008627493192968e-05	0.394+1.0	10000.0	5.02514194502512e-05	0.395+1.0	10000.0	5.0416837133279865e-05	0.396+1.0	10000.0	5.0582528886239245e-05	0.397+1.0	10000.0	5.07484956188601e-05	0.398+1.0	10000.0	5.091473824541037e-05	0.399+1.0	10000.0	5.108125768472535e-05	0.4+1.0	10000.0	5.1248054860238234e-05	0.401+1.0	10000.0	5.141513070001077e-05	0.402+1.0	10000.0	5.158248613676429e-05	0.403+1.0	10000.0	5.1750122107910945e-05	0.404+1.0	10000.0	5.191803955558521e-05	0.405+1.0	10000.0	5.208623942667566e-05	0.406+1.0	10000.0	5.2254722672857e-05	0.407+1.0	10000.0	5.2423490250622416e-05	0.408+1.0	10000.0	5.259254312131609e-05	0.409+1.0	10000.0	5.276188225116611e-05	0.41+1.0	10000.0	5.293150861131761e-05	0.411+1.0	10000.0	5.3101423177866154e-05	0.412+1.0	10000.0	5.327162693189148e-05	0.413+1.0	10000.0	5.344212085949145e-05	0.414+1.0	10000.0	5.361290595181638e-05	0.415+1.0	10000.0	5.378398320510359e-05	0.416+1.0	10000.0	5.3955353620712256e-05	0.417+1.0	10000.0	5.4127018205158625e-05	0.418+1.0	10000.0	5.429897797015145e-05	0.419+1.0	10000.0	5.4471233932627786e-05	0.42+1.0	10000.0	5.4643787114789076e-05	0.421+1.0	10000.0	5.481663854413755e-05	0.422+1.0	10000.0	5.498978925351294e-05	0.423+1.0	10000.0	5.516324028112951e-05	0.424+1.0	10000.0	5.5336992670613406e-05	0.425+1.0	10000.0	5.551104747104036e-05	0.426+1.0	10000.0	5.568540573697367e-05	0.427+1.0	10000.0	5.586006852850254e-05	0.428+1.0	10000.0	5.603503691128078e-05	0.429+1.0	10000.0	5.621031195656581e-05	0.43+1.0	10000.0	5.638589474125801e-05	0.431+1.0	10000.0	5.656178634794041e-05	0.432+1.0	10000.0	5.6737987864918796e-05	0.433+1.0	10000.0	5.691450038626205e-05	0.434+1.0	10000.0	5.709132501184296e-05	0.435+1.0	10000.0	5.726846284737933e-05	0.436+1.0	10000.0	5.744591500447545e-05	0.437+1.0	10000.0	5.7623682600664015e-05	0.438+1.0	10000.0	5.780176675944827e-05	0.439+1.0	10000.0	5.7980168610344705e-05	0.44+1.0	10000.0	5.815888928892597e-05	0.441+1.0	10000.0	5.833792993686429e-05	0.442+1.0	10000.0	5.851729170197523e-05	0.443+1.0	10000.0	5.8696975738261844e-05	0.444+1.0	10000.0	5.88769832059592e-05	0.445+1.0	10000.0	5.9057315271579365e-05	0.446+1.0	10000.0	5.923797310795678e-05	0.447+1.0	10000.0	5.9418957894293995e-05	0.448+1.0	10000.0	5.960027081620788e-05	0.449+1.0	10000.0	5.9781913065776215e-05	0.45+1.0	10000.0	5.996388584158473e-05	0.451+1.0	10000.0	6.014619034877458e-05	0.452+1.0	10000.0	6.032882779909019e-05	0.453+1.0	10000.0	6.0511799410927636e-05	0.454+1.0	10000.0	6.06951064093834e-05	0.455+1.0	10000.0	6.087875002630357e-05	0.456+1.0	10000.0	6.106273150033355e-05	0.457+1.0	10000.0	6.124705207696816e-05	0.458+1.0	10000.0	6.143171300860228e-05	0.459+1.0	10000.0	6.161671555458182e-05	0.45999999999999996+1.0	10000.0	6.180206098125537e-05	0.461+1.0	10000.0	6.198775056202612e-05	0.462+1.0	10000.0	6.217378557740441e-05	0.46299999999999997+1.0	10000.0	6.236016731506074e-05	0.464+1.0	10000.0	6.254689706987916e-05	0.4650000000000001+1.0	10000.0	6.273397614401134e-05	0.466+1.0	10000.0	6.292140584693102e-05	0.467+1.0	10000.0	6.310918749548902e-05	0.468+1.0	10000.0	6.329732241396876e-05	0.469+1.0	10000.0	6.348581193414232e-05	0.47+1.0	10000.0	6.367465739532695e-05	0.471+1.0	10000.0	6.386386014444223e-05	0.472+1.0	10000.0	6.40534215360677e-05	0.473+1.0	10000.0	6.424334293250103e-05	0.474+1.0	10000.0	6.443362570381683e-05	0.47500000000000003+1.0	10000.0	6.462427122792588e-05	0.476+1.0	10000.0	6.481528089063509e-05	0.477+1.0	10000.0	6.500665608570785e-05	0.478+1.0	10000.0	6.519839821492514e-05	0.479+1.0	10000.0	6.539050868814707e-05	0.4799999999999999+1.0	10000.0	6.558298892337519e-05	0.4809999999999999+1.0	10000.0	6.577584034681518e-05	0.482+1.0	10000.0	6.596906439294031e-05	0.48300000000000004+1.0	10000.0	6.616266250455548e-05	0.484+1.0	10000.0	6.63566361328619e-05	0.485+1.0	10000.0	6.655098673752223e-05	0.48600000000000004+1.0	10000.0	6.674571578672667e-05	0.487+1.0	10000.0	6.694082475725939e-05	0.488+1.0	10000.0	6.713631513456576e-05	0.489+1.0	10000.0	6.733218841282026e-05	0.49+1.0	10000.0	6.752844609499494e-05	0.491+1.0	10000.0	6.772508969292868e-05	0.492+1.0	10000.0	6.792212072739705e-05	0.493+1.0	10000.0	6.811954072818287e-05	0.49400000000000005+1.0	10000.0	6.83173512341475e-05	0.495+1.0	10000.0	6.851555379330282e-05	0.49599999999999994+1.0	10000.0	6.871414996288395e-05	0.497+1.0	10000.0	6.89131413094226e-05	0.49800000000000005+1.0	10000.0	6.911252940882125e-05	0.499+1.0	10000.0	6.931231584642809e-05	0.5+1.0	10000.0	6.951250221711256e-05	0.501+1.0	10000.0	6.971309012534187e-05	0.502+1.0	10000.0	6.991408118525803e-05	0.503+1.0	10000.0	7.011547702075589e-05	0.504+1.0	10000.0	7.031727926556185e-05	0.505+1.0	10000.0	7.051948956331337e-05	0.506+1.0	10000.0	7.07221095676393e-05	0.507+1.0	10000.0	7.092514094224104e-05	0.508+1.0	10000.0	7.112858536097448e-05	0.509+1.0	10000.0	7.133244450793289e-05	0.51+1.0	10000.0	7.153672007753045e-05	0.511+1.0	10000.0	7.174141377458688e-05	0.512+1.0	10000.0	7.194652731441272e-05	0.513+1.0	10000.0	7.215206242289567e-05	0.514+1.0	10000.0	7.235802083658761e-05	0.515+1.0	10000.0	7.256440430279283e-05	0.516+1.0	10000.0	7.277121457965678e-05	0.517+1.0	10000.0	7.297845343625606e-05	0.518+1.0	10000.0	7.318612265268923e-05	0.519+1.0	10000.0	7.339422402016851e-05	0.52+1.0	10000.0	7.360275934111251e-05	0.521+1.0	10000.0	7.38117304292399e-05	0.522+1.0	10000.0	7.402113910966408e-05	0.523+1.0	10000.0	7.423098721898886e-05	0.524+1.0	10000.0	7.444127660540506e-05	0.525+1.0	10000.0	7.46520091287882e-05	0.526+1.0	10000.0	7.486318666079723e-05	0.527+1.0	10000.0	7.507481108497428e-05	0.528+1.0	10000.0	7.528688429684544e-05	0.529+1.0	10000.0	7.549940820402266e-05	0.53+1.0	10000.0	7.571238472630674e-05	0.531+1.0	10000.0	7.592581579579134e-05	0.532+1.0	10000.0	7.613970335696819e-05	0.533+1.0	10000.0	7.635404936683341e-05	0.534+1.0	10000.0	7.656885579499492e-05	0.535+1.0	10000.0	7.678412462378105e-05	0.536+1.0	10000.0	7.699985784835036e-05	0.537+1.0	10000.0	7.721605747680253e-05	0.538+1.0	10000.0	7.74327255302906e-05	0.539+1.0	10000.0	7.764986404313427e-05	0.54+1.0	10000.0	7.786747506293458e-05	0.541+1.0	10000.0	7.808556065068973e-05	0.542+1.0	10000.0	7.830412288091226e-05	0.543+1.0	10000.0	7.852316384174741e-05	0.544+1.0	10000.0	7.874268563509289e-05	0.545+1.0	10000.0	7.896269037671984e-05	0.546+1.0	10000.0	7.918318019639526e-05	0.547+1.0	10000.0	7.940415723800571e-05	0.548+1.0	10000.0	7.962562365968238e-05	0.549+1.0	10000.0	7.984758163392757e-05	0.55+1.0	10000.0	8.007003334774259e-05	0.551+1.0	10000.0	8.029298100275704e-05	0.552+1.0	10000.0	8.051642681535958e-05	0.553+1.0	10000.0	8.07403730168302e-05	0.554+1.0	10000.0	8.096482185347378e-05	0.555+1.0	10000.0	8.118977558675542e-05	0.556+1.0	10000.0	8.141523649343717e-05	0.557+1.0	10000.0	8.164120686571621e-05	0.558+1.0	10000.0	8.186768901136475e-05	0.559+1.0	10000.0	8.209468525387147e-05	0.56+1.0	10000.0	8.232219793258456e-05	0.561+1.0	10000.0	8.25502294028563e-05	0.562+1.0	10000.0	8.277878203618948e-05	0.563+1.0	10000.0	8.300785822038541e-05	0.564+1.0	10000.0	8.32374603596935e-05	0.565+1.0	10000.0	8.346759087496276e-05	0.566+1.0	10000.0	8.369825220379494e-05	0.567+1.0	10000.0	8.392944680069952e-05	0.568+1.0	10000.0	8.416117713725035e-05	0.569+1.0	10000.0	8.439344570224428e-05	0.57+1.0	10000.0	8.462625500186148e-05	0.571+1.0	10000.0	8.485960755982777e-05	0.572+1.0	10000.0	8.50935059175788e-05	0.573+1.0	10000.0	8.532795263442606e-05	0.574+1.0	10000.0	8.5562950287725e-05	0.575+1.0	10000.0	8.579850147304498e-05	0.576+1.0	10000.0	8.603460880434141e-05	0.577+1.0	10000.0	8.627127491412966e-05	0.578+1.0	10000.0	8.650850245366132e-05	0.579+1.0	10000.0	8.67462940931024e-05	0.58+1.0	10000.0	8.698465252171359e-05	0.581+1.0	10000.0	8.722358044803287e-05	0.582+1.0	10000.0	8.746308060006015e-05	0.583+1.0	10000.0	8.770315572544416e-05	0.584+1.0	10000.0	8.794380859167164e-05	0.585+1.0	10000.0	8.818504198625876e-05	0.586+1.0	10000.0	8.842685871694488e-05	0.587+1.0	10000.0	8.866926161188859e-05	0.588+1.0	10000.0	8.89122535198663e-05	0.589+1.0	10000.0	8.915583731047307e-05	0.59+1.0	10000.0	8.940001587432593e-05	0.591+1.0	10000.0	8.96447921232698e-05	0.592+1.0	10000.0	8.989016899058584e-05	0.593+1.0	10000.0	9.013614943120231e-05	0.594+1.0	10000.0	9.038273642190817e-05	0.595+1.0	10000.0	9.062993296156926e-05	0.596+1.0	10000.0	9.087774207134708e-05	0.597+1.0	10000.0	9.112616679492041e-05	0.598+1.0	10000.0	9.137521019870965e-05	0.599+1.0	10000.0	9.162487537210387e-05	0.6+1.0	10000.0	9.187516542769086e-05	0.601+1.0	10000.0	9.212608350148996e-05	0.602+1.0	10000.0	9.237763275318782e-05	0.603+1.0	10000.0	9.262981636637719e-05	0.604+1.0	10000.0	9.288263754879868e-05	0.605+1.0	10000.0	9.313609953258558e-05	0.606+1.0	10000.0	9.339020557451187e-05	0.607+1.0	10000.0	9.364495895624324e-05	0.608+1.0	10000.0	9.39003629845915e-05	0.609+1.0	10000.0	9.415642099177208e-05	0.61+1.0	10000.0	9.441313633566497e-05	0.611+1.0	10000.0	9.467051240007888e-05	0.612+1.0	10000.0	9.492855259501899e-05	0.613+1.0	10000.0	9.518726035695794e-05	0.614+1.0	10000.0	9.544663914911053e-05	0.615+1.0	10000.0	9.570669246171193e-05	0.616+1.0	10000.0	9.596742381229943e-05	0.617+1.0	10000.0	9.622883674599801e-05	0.618+1.0	10000.0	9.649093483580956e-05	0.619+1.0	10000.0	9.675372168290593e-05	0.62+1.0	10000.0	9.701720091692583e-05	0.621+1.0	10000.0	9.728137619627573e-05	0.622+1.0	10000.0	9.754625120843458e-05	0.623+1.0	10000.0	9.78118296702627e-05	0.624+1.0	10000.0	9.807811532831483e-05	0.625+1.0	10000.0	9.834511195915719e-05	0.626+1.0	10000.0	9.861282336968894e-05	0.627+1.0	10000.0	9.888125339746786e-05	0.628+1.0	10000.0	9.91504059110405e-05	0.629+1.0	10000.0	9.942028481027666e-05	0.63+1.0	10000.0	9.969089402670856e-05	0.631+1.0	10000.0	9.99622375238744e-05	0.632+1.0	10000.0	0.00010023431929766688	0.633+1.0	10000.0	0.00010050714337668617	0.634+1.0	10000.0	0.00010078071382259788	0.635+1.0	10000.0	0.000101055034730496	0.636+1.0	10000.0	0.0001013301102292706	0.637+1.0	10000.0	0.00010160594448198085	0.638+1.0	10000.0	0.00010188254168623308	0.639+1.0	10000.0	0.00010215990607456405	0.64+1.0	10000.0	0.00010243804191482964	0.641+1.0	10000.0	0.00010271695351059886	0.642+1.0	10000.0	0.00010299664520155354	0.643+1.0	10000.0	0.00010327712136389339	0.644+1.0	10000.0	0.00010355838641074695	0.645+1.0	10000.0	0.00010384044479258827	0.646+1.0	10000.0	0.00010412330099765945	0.647+1.0	10000.0	0.00010440695955239921	0.648+1.0	10000.0	0.00010469142502187759	0.649+1.0	10000.0	0.00010497670201023675	0.65+1.0	10000.0	0.00010526279516113816	0.651+1.0	10000.0	0.00010554970915821623	0.652+1.0	10000.0	0.00010583744872553834	0.653+1.0	10000.0	0.00010612601862807168	0.654+1.0	10000.0	0.00010641542367215666	0.655+1.0	10000.0	0.00010670566870598734	0.656+1.0	10000.0	0.00010699675862009879	0.657+1.0	10000.0	0.00010728869834786159	0.658+1.0	10000.0	0.00010758149286598348	0.659+1.0	10000.0	0.00010787514719501858	0.66+1.0	10000.0	0.00010816966639988383	0.661+1.0	10000.0	0.00010846505559038332	0.662+1.0	10000.0	0.00010876131992174022	0.663+1.0	10000.0	0.00010905846459513668	0.664+1.0	10000.0	0.00010935649485826175	0.665+1.0	10000.0	0.00010965541600586756	0.666+1.0	10000.0	0.00010995523338033371	0.667+1.0	10000.0	0.00011025595237224025	0.668+1.0	10000.0	0.0001105575784209493	0.669+1.0	10000.0	0.00011086011701519538	0.67+1.0	10000.0	0.00011116357369368481	0.671+1.0	10000.0	0.00011146795404570418	0.672+1.0	10000.0	0.00011177326371173806	0.673+1.0	10000.0	0.00011207950838409629	0.674+1.0	10000.0	0.0001123866938075508	0.675+1.0	10000.0	0.00011269482577998225	0.676+1.0	10000.0	0.00011300391015303676	0.677+1.0	10000.0	0.00011331395283279269	0.678+1.0	10000.0	0.00011362495978043791	0.679+1.0	10000.0	0.00011393693701295753	0.68+1.0	10000.0	0.00011424989060383247	0.681+1.0	10000.0	0.00011456382668374897	0.682+1.0	10000.0	0.00011487875144131925	0.683+1.0	10000.0	0.00011519467112381361	0.684+1.0	10000.0	0.00011551159203790405	0.685+1.0	10000.0	0.00011582952055041985	0.686+1.0	10000.0	0.00011614846308911497	0.687+1.0	10000.0	0.00011646842614344799	0.688+1.0	10000.0	0.00011678941626537445	0.689+1.0	10000.0	0.00011711144007015185	0.69+1.0	10000.0	0.00011743450423715783	0.691+1.0	10000.0	0.0001177586155107215	0.692+1.0	10000.0	0.00011808378070096832	0.693+1.0	10000.0	0.0001184100066846788	0.694+1.0	10000.0	0.00011873730040616118	0.695+1.0	10000.0	0.00011906566887813845	0.696+1.0	10000.0	0.0001193951191826501	0.697+1.0	10000.0	0.00011972565847196857	0.698+1.0	10000.0	0.0001200572939695311	0.699+1.0	10000.0	0.00012039003297088694	0.7+1.0	10000.0	0.00012072388284466047	0.701+1.0	10000.0	0.00012105885103353038	0.702+1.0	10000.0	0.00012139494505522533	0.703+1.0	10000.0	0.00012173217250353647	0.704+1.0	10000.0	0.00012207054104934696	0.705+1.0	10000.0	0.00012241005844167903	0.706+1.0	10000.0	0.000122750732508759	0.707+1.0	10000.0	0.0001230925711591003	0.708+1.0	10000.0	0.00012343558238260503	0.709+1.0	10000.0	0.00012377977425168498	0.71+1.0	10000.0	0.0001241251549224012	0.711+1.0	10000.0	0.0001244717326356241	0.712+1.0	10000.0	0.00012481951571821317	0.713+1.0	10000.0	0.00012516851258421758	0.714+1.0	10000.0	0.00012551873173609774	0.715+1.0	10000.0	0.0001258701817659683	0.716+1.0	10000.0	0.0001262228713568631	0.717+1.0	10000.0	0.0001265768092840225	0.718+1.0	10000.0	0.00012693200441620344	0.719+1.0	10000.0	0.00012728846571701322	0.72+1.0	10000.0	0.00012764620224626675	0.721+1.0	10000.0	0.0001280052231613683	0.722+1.0	10000.0	0.0001283655377187183	0.723+1.0	10000.0	0.00012872715527514523	0.724+1.0	10000.0	0.00012909008528936398	0.725+1.0	10000.0	0.00012945433732346025	0.726+1.0	10000.0	0.00012981992104440252	0.727+1.0	10000.0	0.0001301868462255815	0.728+1.0	10000.0	0.00013055512274837803	0.729+1.0	10000.0	0.00013092476060375997	0.73+1.0	10000.0	0.00013129576989390863	0.731+1.0	10000.0	0.00013166816083387564	0.732+1.0	10000.0	0.0001320419437532706	0.733+1.0	10000.0	0.00013241712909798059	0.734+1.0	10000.0	0.00013279372743192186	0.735+1.0	10000.0	0.00013317174943882482	0.736+1.0	10000.0	0.00013355120592405275	0.737+1.0	10000.0	0.00013393210781645527	0.738+1.0	10000.0	0.0001343144661702572	0.739+1.0	10000.0	0.00013469829216698363	0.74+1.0	10000.0	0.00013508359711742208	0.741+1.0	10000.0	0.00013547039246362283	0.742+1.0	10000.0	0.00013585868978093768	0.743+1.0	10000.0	0.00013624850078009887	0.744+1.0	10000.0	0.00013663983730933843	0.745+1.0	10000.0	0.00013703271135654925	0.746+1.0	10000.0	0.00013742713505148868	0.747+1.0	10000.0	0.000137823120668026	0.748+1.0	10000.0	0.00013822068062643427	0.749+1.0	10000.0	0.00013861982749572817	0.75+1.0	10000.0	0.00013902057399604856	0.751+1.0	10000.0	0.000139422933001095	0.752+1.0	10000.0	0.00013982691754060752	0.753+1.0	10000.0	0.00014023254080289867	0.754+1.0	10000.0	0.00014063981613743714	0.755+1.0	10000.0	0.00014104875705748415	0.756+1.0	10000.0	0.00014145937724278413	0.757+1.0	10000.0	0.00014187169054231068	0.758+1.0	10000.0	0.0001422857109770696	0.759+1.0	10000.0	0.00014270145274296	0.76+1.0	10000.0	0.0001431189302136953	0.761+1.0	10000.0	0.00014353815794378524	0.762+1.0	10000.0	0.000143959150671581	0.763+1.0	10000.0	0.00014438192332238433	0.764+1.0	10000.0	0.0001448064910116231	0.765+1.0	10000.0	0.0001452328690480941	0.766+1.0	10000.0	0.00014566107293727563	0.767+1.0	10000.0	0.0001460911183847112	0.768+1.0	10000.0	0.0001465230212994664	0.769+1.0	10000.0	0.00014695679779766038	0.77+1.0	10000.0	0.00014739246420607496	0.771+1.0	10000.0	0.00014783003706584204	0.772+1.0	10000.0	0.00014826953313621245	0.773+1.0	10000.0	0.0001487109693984078	0.774+1.0	10000.0	0.00014915436305955787	0.775+1.0	10000.0	0.0001495997315567256	0.776+1.0	10000.0	0.00015004709256102218	0.777+1.0	10000.0	0.00015049646398181455	0.778+1.0	10000.0	0.00015094786397102793	0.779+1.0	10000.0	0.00015140131092754593	0.78+1.0	10000.0	0.00015185682350171082	0.781+1.0	10000.0	0.00015231442059992681	0.782+1.0	10000.0	0.0001527741213893692	0.783+1.0	10000.0	0.00015323594530280217	0.784+1.0	10000.0	0.0001536999120435085	0.785+1.0	10000.0	0.00015416604159033408	0.786+1.0	10000.0	0.00015463435420285065	0.787+1.0	10000.0	0.00015510487042663985	0.788+1.0	10000.0	0.00015557761109870246	0.789+1.0	10000.0	0.00015605259735299582	0.79+1.0	10000.0	0.0001565298506261037	0.791+1.0	10000.0	0.0001570093926630419	0.792+1.0	10000.0	0.0001574912455232038	0.793+1.0	10000.0	0.00015797543158645	0.794+1.0	10000.0	0.00015846197355934597	0.795+1.0	10000.0	0.0001589508944815523	0.796+1.0	10000.0	0.00015944221773237186	0.797+1.0	10000.0	0.0001599359670374588	0.798+1.0	10000.0	0.00016043216647569376	0.799+1.0	10000.0	0.0001609308404862308	0.8+1.0	10000.0	0.00016143201387572073	0.801+1.0	10000.0	0.00016193571182571652	0.802+1.0	10000.0	0.0001624419599002662	0.803+1.0	10000.0	0.00016295078405369876	0.804+1.0	10000.0	0.00016346221063860943	0.805+1.0	10000.0	0.0001639762664140502	0.806+1.0	10000.0	0.00016449297855393162	0.807+1.0	10000.0	0.00016501237465564353	0.808+1.0	10000.0	0.00016553448274890005	0.809+1.0	10000.0	0.00016605933130481743	0.81+1.0	10000.0	0.00016658694924523096	0.811+1.0	10000.0	0.00016711736595225918	0.812+1.0	10000.0	0.00016765061127812318	0.813+1.0	10000.0	0.00016818671555522913	0.814+1.0	10000.0	0.0001687257096065224	0.815+1.0	10000.0	0.0001692676247561225	0.816+1.0	10000.0	0.00016981249284024768	0.817+1.0	10000.0	0.00017036034621843894	0.818+1.0	10000.0	0.0001709112177850931	0.819+1.0	10000.0	0.00017146514098131542	0.82+1.0	10000.0	0.0001720221498071024	0.821+1.0	10000.0	0.00017258227883386575	0.822+1.0	10000.0	0.00017314556321730923	0.823+1.0	10000.0	0.00017371203871067021	0.824+1.0	10000.0	0.00017428174167833854	0.825+1.0	10000.0	0.0001748547091098656	0.826+1.0	10000.0	0.00017543097863437706	0.827+1.0	10000.0	0.00017601058853540345	0.828+1.0	10000.0	0.000176593577766143	0.829+1.0	10000.0	0.00017717998596517226	0.83+1.0	10000.0	0.00017776985347261998	0.831+1.0	10000.0	0.00017836322134682099	0.832+1.0	10000.0	0.00017896013138146716	0.833+1.0	10000.0	0.00017956062612327353	0.834+1.0	10000.0	0.0001801647488901781	0.835+1.0	10000.0	0.00018077254379009466	0.836+1.0	10000.0	0.0001813840557402393	0.837+1.0	10000.0	0.0001819993304870516	0.838+1.0	10000.0	0.0001826184146267322	0.839+1.0	10000.0	0.00018324135562642076	0.84+1.0	10000.0	0.00018386820184603734	0.841+1.0	10000.0	0.00018449900256081289	0.842+1.0	10000.0	0.00018513380798453516	0.843+1.0	10000.0	0.00018577266929353708	0.844+1.0	10000.0	0.00018641563865145656	0.845+1.0	10000.0	0.00018706276923479768	0.846+1.0	10000.0	0.00018771411525932418	0.847+1.0	10000.0	0.0001883697320073188	0.848+1.0	10000.0	0.00018902967585574174	0.849+1.0	10000.0	0.000189694004305325	0.85+1.0	10000.0	0.0001903627760106396	0.851+1.0	10000.0	0.00019103605081117531	0.852+1.0	10000.0	0.00019171388976347363	0.853+1.0	10000.0	0.00019239635517435784	0.854+1.0	10000.0	0.00019308351063530475	0.855+1.0	10000.0	0.0001937754210580058	0.856+1.0	10000.0	0.0001944721527111674	0.857+1.0	10000.0	0.00019517377325860218	0.858+1.0	10000.0	0.00019588035179866664	0.859+1.0	10000.0	0.00019659195890510194	0.86+1.0	10000.0	0.00019730866666933865	0.861+1.0	10000.0	0.0001980305487443289	0.862+1.0	10000.0	0.00019875768038997207	0.863+1.0	10000.0	0.00019949013852020494	0.864+1.0	10000.0	0.00020022800175182896	0.865+1.0	10000.0	0.00020097135045515308	0.866+1.0	10000.0	0.00020172026680653318	0.867+1.0	10000.0	0.00020247483484289397	0.868+1.0	10000.0	0.00020323514051832442	0.869+1.0	10000.0	0.00020400127176284106	0.87+1.0	10000.0	0.00020477331854342062	0.871+1.0	10000.0	0.0002055513729274073	0.872+1.0	10000.0	0.0002063355291484065	0.873+1.0	10000.0	0.0002071258836747834	0.874+1.0	10000.0	0.0002079225352808903	0.875+1.0	10000.0	0.00020872558512115467	0.876+1.0	10000.0	0.00020953513680716667	0.877+1.0	10000.0	0.00021035129648791333	0.878+1.0	10000.0	0.00021117417293331422	0.879+1.0	10000.0	0.00021200387762122385	0.88+1.0	10000.0	0.0002128405248280739	0.881+1.0	10000.0	0.00021368423172334058	0.882+1.0	10000.0	0.00021453511846803144	0.883+1.0	10000.0	0.00021539330831739947	0.884+1.0	10000.0	0.00021625892772810372	0.885+1.0	10000.0	0.00021713210647004945	0.886+1.0	10000.0	0.00021801297774315512	0.887+1.0	10000.0	0.00021890167829930948	0.888+1.0	10000.0	0.00021979834856979735	0.889+1.0	10000.0	0.00022070313279849146	0.89+1.0	10000.0	0.00022161617918112632	0.891+1.0	10000.0	0.0002225376400109904	0.892+1.0	10000.0	0.00022346767183139467	0.893+1.0	10000.0	0.0002244064355952996	0.894+1.0	10000.0	0.00022535409683250752	0.895+1.0	10000.0	0.00022631082582485486	0.896+1.0	10000.0	0.000227276797789868	0.897+1.0	10000.0	0.000228252193073378	0.898+1.0	10000.0	0.00022923719735162438	0.899+1.0	10000.0	0.00023023200184341366	0.9+1.0	10000.0	0.00023123680353293974	0.901+1.0	10000.0	0.00023225180540391508	0.902+1.0	10000.0	0.00023327721668570928	0.903+1.0	10000.0	0.0002343132531122409	0.904+1.0	10000.0	0.00023536013719442487	0.905+1.0	10000.0	0.00023641809850703498	0.906+1.0	10000.0	0.00023748737399090756	0.907+1.0	10000.0	0.00023856820827148096	0.908+1.0	10000.0	0.0002396608539947423	0.909+1.0	10000.0	0.00024076557218173522	0.91+1.0	10000.0	0.00024188263260287388	0.911+1.0	10000.0	0.00024301231417340493	0.912+1.0	10000.0	0.0002441549053714687	0.913+1.0	10000.0	0.0002453107046803267	0.914+1.0	10000.0	0.00024648002105645154	0.915+1.0	10000.0	0.00024766317442531573	0.916+1.0	10000.0	0.00024886049620687024	0.917+1.0	10000.0	0.00025007232987287126	0.918+1.0	10000.0	0.0002512990315384012	0.919+1.0	10000.0	0.00025254097059013164	0.92+1.0	10000.0	0.0002537985303541022	0.921+1.0	10000.0	0.00025507210880603554	0.922+1.0	10000.0	0.00025636211932748154	0.923+1.0	10000.0	0.00025766899151138614	0.924+1.0	10000.0	0.00025899317202101257	0.925+1.0	10000.0	0.0002603351255065116	0.926+1.0	10000.0	0.00026169533558384846	0.927+1.0	10000.0	0.0002630743058812463	0.928+1.0	10000.0	0.0002644725611588133	0.929+1.0	10000.0	0.0002658906485075816	0.93+1.0	10000.0	0.00026732913863481523	0.931+1.0	10000.0	0.00026878862724314377	0.932+1.0	10000.0	0.0002702697365118624	0.933+1.0	10000.0	0.0002717731166896186	0.934+1.0	10000.0	0.0002732994478086877	0.935+1.0	10000.0	0.00027484944153215244	0.936+1.0	10000.0	0.00027642384314654236	0.937+1.0	10000.0	0.0002780234337139015	0.938+1.0	10000.0	0.0002796490323988404	0.939+1.0	10000.0	0.0002813014989879255	0.94+1.0	10000.0	0.000282981736620808	0.941+1.0	10000.0	0.00028469069475481015	0.942+1.0	10000.0	0.00028642937238733336	0.943+1.0	10000.0	0.0002881988215634719	0.944+1.0	10000.0	0.0002900001511996706	0.945+1.0	10000.0	0.00029183453125822625	0.946+1.0	10000.0	0.00029370319731198563	0.947+1.0	10000.0	0.00029560745554384383	0.948+1.0	10000.0	0.00029754868823170705	0.949+1.0	10000.0	0.0002995283597766119	0.95+1.0	10000.0	0.000301548023339851	0.951+1.0	10000.0	0.00030360932816446217	0.952+1.0	10000.0	0.0003057140276675474	0.953+1.0	10000.0	0.0003078639884029063	0.954+1.0	10000.0	0.00031006120000877916	0.955+1.0	10000.0	0.0003123077862735583	0.956+1.0	10000.0	0.0003146060174737114	0.957+1.0	10000.0	0.0003169583241635744	0.958+1.0	10000.0	0.0003193673126269866	0.959+1.0	10000.0	0.0003218357822370419	0.96+1.0	10000.0	0.0003243667450138943	0.961+1.0	10000.0	0.0003269634477232906	0.962+1.0	10000.0	0.0003296293969224945	0.963+1.0	10000.0	0.00033236838743829394	0.964+1.0	10000.0	0.00033518453485739956	0.965+1.0	10000.0	0.00033808231272734426	0.966+1.0	10000.0	0.0003410665953119439	0.967+1.0	10000.0	0.0003441427069272787	0.968+1.0	10000.0	0.00034731647911227933	0.969+1.0	10000.0	0.00035059431717600765	0.97+1.0	10000.0	0.00035398327802988486	0.971+1.0	10000.0	0.0003574911616821532	0.972+1.0	10000.0	0.0003611266193774489	0.973+1.0	10000.0	0.0003648992821529467	0.974+1.0	10000.0	0.00036881991461876223	0.975+1.0	10000.0	0.0003729006001452426	0.976+1.0	10000.0	0.0003771549654833325	0.977+1.0	10000.0	0.00038159845534406294	0.978+1.0	10000.0	0.0003862486708945279	0.979+1.0	10000.0	0.00039112579090008087	0.98+1.0	10000.0	0.00039625310097595986	0.981+1.0	10000.0	0.0004016576660620745	0.982+1.0	10000.0	0.0004073711953057305	0.983+1.0	10000.0	0.000413431169445708	0.984+1.0	10000.0	0.00041988233251838967	0.985+1.0	10000.0	0.0004267786989882705	0.986+1.0	10000.0	0.0004341863059850781	0.987+1.0	10000.0	0.0004421870693874941	0.988+1.0	10000.0	0.0004508843217178277	0.989+1.0	10000.0	0.0004604109969121544	0.99+1.0	10000.0	0.00047094214208549036	0.991+1.0	10000.0	0.0004827148294203387	0.992+1.0	10000.0	0.0004960614338147178	0.993+1.0	10000.0	0.0005114687362241056	0.994+1.0	10000.0	0.0005296914006061454	0.995+1.0	10000.0	0.0005519936871840462	0.996+1.0	10000.0	0.0005807456009879429	0.997+1.0	10000.0	0.0006212677430696829	0.998+1.0	10000.0	0.0006905369974100782	0.999+0.0001	1.0	1.436e-320	0.9289999872360728+0.0001	1.0	6.7528257e-316	0.9300000000003071+0.0001	1.0	3.1391337009846e-311	0.931+0.0001	1.0	1.4425256353565396e-306	0.932+0.0001	1.0	6.552959215649117e-302	0.933+0.0001	1.0	2.942810527037945e-297	0.934+0.0001	1.0	1.3064979276089767e-292	0.935+0.0001	1.0	5.73439204339534e-288	0.936+0.0001	1.0	2.4883351693354845e-283	0.937+0.0001	1.0	1.067539100145012e-278	0.938+0.0001	1.0	4.528169353652694e-274	0.939+0.0001	1.0	1.899048240326503e-269	0.94+0.0001	1.0	7.874704623233369e-265	0.941+0.0001	1.0	3.2287017476071874e-260	0.942+0.0001	1.0	1.3089630078656712e-255	0.943+0.0001	1.0	5.247385618491116e-251	0.944+0.0001	1.0	2.0801042082777007e-246	0.945+0.0001	1.0	8.15387440958366e-242	0.946+0.0001	1.0	3.160749171221132e-237	0.947+0.0001	1.0	1.21163938163189e-232	0.948+0.0001	1.0	4.5932943068519563e-228	0.949+0.0001	1.0	1.7220782772880994e-223	0.95+0.0001	1.0	6.385124564542957e-219	0.951+0.0001	1.0	2.341444374503419e-214	0.952+0.0001	1.0	8.4919295638972e-210	0.953+0.0001	1.0	3.046120230431811e-205	0.954+0.0001	1.0	1.0807265913977428e-200	0.955+0.0001	1.0	3.792475300416719e-196	0.956+0.0001	1.0	1.316369370291913e-191	0.957+0.0001	1.0	4.519504107616196e-187	0.958+0.0001	1.0	1.5348703852964253e-182	0.959+0.0001	1.0	5.15620761212608e-178	0.96+0.0001	1.0	1.7134707117228317e-173	0.961+0.0001	1.0	5.632748727474093e-169	0.962+0.0001	1.0	1.831771279289087e-164	0.963+0.0001	1.0	5.893035106336996e-160	0.964+0.0001	1.0	1.8755706927264964e-155	0.965+0.0001	1.0	5.905601728075971e-151	0.966+0.0001	1.0	1.8396738283444808e-146	0.967+0.0001	1.0	5.669869771882024e-142	0.968+0.0001	1.0	1.7289025213382995e-137	0.969+0.0001	1.0	5.21606100929039e-133	0.97+0.0001	1.0	1.5570375276778067e-128	0.971+0.0001	1.0	4.598850193344028e-124	0.972+0.0001	1.0	1.3440105677392934e-119	0.973+0.0001	1.0	3.8865905122265024e-115	0.974+0.0001	1.0	1.1121336599990214e-110	0.975+0.0001	1.0	3.149029049042071e-106	0.976+0.0001	1.0	8.823424781290704e-102	0.977+0.0001	1.0	2.446514961562747e-97	0.978+0.0001	1.0	6.713021395860465e-93	0.979+0.0001	1.0	1.8228751517174776e-88	0.98+0.0001	1.0	4.89862054356261e-84	0.981+0.0001	1.0	1.3028003587884003e-79	0.982+0.0001	1.0	3.4290855593065557e-75	0.983+0.0001	1.0	8.932732183930374e-71	0.984+0.0001	1.0	2.303058534451975e-66	0.985+0.0001	1.0	5.876915036730232e-62	0.986+0.0001	1.0	1.4843171192092788e-57	0.987+0.0001	1.0	3.7106148327127247e-53	0.988+0.0001	1.0	9.18154932757296e-49	0.989+0.0001	1.0	2.2487748498162915e-44	0.99+0.0001	1.0	5.451861981296685e-40	0.991+0.0001	1.0	1.3083427789881014e-35	0.992+0.0001	1.0	3.108028222836071e-31	0.993+0.0001	1.0	7.308764578787502e-27	0.994+0.0001	1.0	1.7014040303942292e-22	0.995+0.0001	1.0	3.9208861953888696e-18	0.996+0.0001	1.0	8.945057086892307e-14	0.997+0.0001	1.0	2.0202860902384654e-09	0.998+0.0001	1.0	4.517334597704827e-05	0.999+10000.0	1.0	0.9993094630025899	0.0010000000000001553+10000.0	1.0	0.9993787322569303	0.0019999999999997056+10000.0	1.0	0.999419254399012	0.002999999999999033+10000.0	1.0	0.9994480063128159	0.0039999999999994415+10000.0	1.0	0.9994703085993939	0.00500000000000214+10000.0	1.0	0.9994885312637759	0.006000000000000544+10000.0	1.0	0.9995039385661852	0.006999999999997468+10000.0	1.0	0.9995172851705797	0.00800000000000267+10000.0	1.0	0.9995290578579145	0.008999999999995562+10000.0	1.0	0.9995395890030878	0.009999999999996834+10000.0	1.0	0.9995491156782822	0.01100000000000264+10000.0	1.0	0.9995578129306125	0.012000000000001877+10000.0	1.0	0.9995658136940149	0.01299999999999336+10000.0	1.0	0.9995732213010118	0.014000000000007473+10000.0	1.0	0.9995801176674816	0.014999999999992495+10000.0	1.0	0.9995865688305543	0.016000000000001395+10000.0	1.0	0.9995926288046942	0.016999999999994783+10000.0	1.0	0.999598342333938	0.01800000000000649+10000.0	1.0	0.9996037468990241	0.019000000000009995+10000.0	1.0	0.9996088742090999	0.020000000000004223+10000.0	1.0	0.9996137513291055	0.02099999999999747+10000.0	1.0	0.999618401544656	0.02200000000000875+10000.0	1.0	0.9996228450345167	0.023000000000008625+10000.0	1.0	0.9996270993998547	0.023999999999991965+10000.0	1.0	0.9996311800853812	0.024999999999993115+10000.0	1.0	0.999635100717847	0.025999999999986867+10000.0	1.0	0.9996388733806225	0.026999999999990764+10000.0	1.0	0.9996425088383178	0.027999999999999456+10000.0	1.0	0.9996460167219701	0.028999999999985607+10000.0	1.0	0.999649405682824	0.030000000000005456+10000.0	1.0	0.9996526835208878	0.03100000000001337+10000.0	1.0	0.9996558572930727	0.031999999999997614+10000.0	1.0	0.999658933404688	0.032999999999992286+10000.0	1.0	0.9996619176872726	0.03399999999999422+10000.0	1.0	0.9996648154651426	0.03500000000001074+10000.0	1.0	0.9996676316125617	0.035999999999986695+10000.0	1.0	0.9996703706030775	0.03700000000000066+10000.0	1.0	0.9996730365522767	0.03799999999999027+10000.0	1.0	0.9996756332549861	0.03899999999999614+10000.0	1.0	0.999678164217763	0.04000000000002081+10000.0	1.0	0.9996806326873731	0.04100000000001949+10000.0	1.0	0.9996830416758364	0.04200000000000476+10000.0	1.0	0.9996853939825263	0.04300000000001171+10000.0	1.0	0.9996876922137264	0.04400000000000021+10000.0	1.0	0.9996899387999912	0.045000000000008714+10000.0	1.0	0.9996921360115971	0.04600000000001266+10000.0	1.0	0.9996942859723325	0.04700000000000004+10000.0	1.0	0.9996963906718356	0.04800000000001906+10000.0	1.0	0.9996984519766602	0.04900000000001774+10000.0	1.0	0.9997004716402234	0.049999999999982864+10000.0	1.0	0.9997024513117683	0.05099999999998823+10000.0	1.0	0.9997043925444562	0.05200000000002262+10000.0	1.0	0.999706296802688	0.05299999999997436+10000.0	1.0	0.9997081654687417	0.053999999999986975+10000.0	1.0	0.9997099998488004	0.05500000000002742+10000.0	1.0	0.9997118011784365	0.05599999999998364+10000.0	1.0	0.9997135706276127	0.05700000000000759+10000.0	1.0	0.9997153093052452	0.058000000000014304+10000.0	1.0	0.9997170182633792	0.05900000000001664+10000.0	1.0	0.999718698501012	0.05999999999997115+10000.0	1.0	0.9997203509676011	0.0609999999999916+10000.0	1.0	0.9997219765662861	0.06200000000002659+10000.0	1.0	0.9997235761568535	0.06300000000001232+10000.0	1.0	0.9997251505584679	0.06400000000002705+10000.0	1.0	0.9997267005521913	0.0649999999999665+10000.0	1.0	0.9997282268833104	0.066000000000027+10000.0	1.0	0.9997297302634881	0.066999999999977+10000.0	1.0	0.9997312113727569	0.0680000000000225+10000.0	1.0	0.9997326708613652	0.06900000000000449+10000.0	1.0	0.9997341093514924	0.07000000000000679+10000.0	1.0	0.9997355274388412	0.07100000000000821+10000.0	1.0	0.9997369256941188	0.0720000000000052+10000.0	1.0	0.9997383046644162	0.07300000000000267+10000.0	1.0	0.9997396648744935	0.0740000000000126+10000.0	1.0	0.999741006827979	0.07499999999997246+10000.0	1.0	0.9997423310084886	0.07600000000001855+10000.0	1.0	0.9997436378806726	0.0770000000000267+10000.0	1.0	0.999744927891194	0.07799999999999445+10000.0	1.0	0.9997462014696459	0.0789999999999849+10000.0	1.0	0.9997474590294099	0.07999999999999999+10000.0	1.0	0.9997487009684616	0.08099999999998378+10000.0	1.0	0.9997499276701272	0.08200000000002451+10000.0	1.0	0.9997511395037931	0.08299999999999952+10000.0	1.0	0.9997523368255747	0.0840000000000459+10000.0	1.0	0.9997535199789436	0.0850000000000202+10000.0	1.0	0.9997546892953196	0.08599999999996427+10000.0	1.0	0.9997558450946286	0.08700000000002416+10000.0	1.0	0.9997569876858265	0.0879999999999586+10000.0	1.0	0.9997581173673972	0.08900000000003898+10000.0	1.0	0.9997592344278182	0.08999999999996139+10000.0	1.0	0.9997603391460053	0.09100000000000324+10000.0	1.0	0.9997614317917285	0.09199999999997721+10000.0	1.0	0.9997625126260091	0.09300000000003072+10000.0	1.0	0.999763581901493	0.0939999999999841+10000.0	1.0	0.9997646398628056	0.0950000000000381+10000.0	1.0	0.9997656867468877	0.09599999999994814+10000.0	1.0	0.9997667227833142	0.09699999999995403+10000.0	1.0	0.9997677481945961	0.09800000000004284+10000.0	1.0	0.999768763196467	0.09899999999995424+10000.0	1.0	0.9997697679981565	0.09999999999994694+10000.0	1.0	0.9997707628026484	0.10100000000005212+10000.0	1.0	0.9997717478069266	0.10199999999994644+10000.0	1.0	0.9997727232022101	0.10299999999997916+10000.0	1.0	0.9997736891741752	0.10400000000005209+10000.0	1.0	0.9997746459031674	0.10499999999995231+10000.0	1.0	0.9997755935644047	0.10600000000000764+10000.0	1.0	0.9997765323281687	0.1070000000000526+10000.0	1.0	0.999777462359989	0.10799999999997743+10000.0	1.0	0.9997783838208189	0.10899999999997421+10000.0	1.0	0.9997792968672015	0.109999999999948+10000.0	1.0	0.9997802016514302	0.11100000000002211+10000.0	1.0	0.9997810983217007	0.11200000000001153+10000.0	1.0	0.9997819870222568	0.1130000000000014+10000.0	1.0	0.99978286789353	0.11400000000004695+10000.0	1.0	0.9997837410722719	0.11499999999995682+10000.0	1.0	0.9997846066916826	0.11599999999995841+10000.0	1.0	0.999785464881532	0.117000000000003+10000.0	1.0	0.9997863157682767	0.1180000000000491+10000.0	1.0	0.9997871594751719	0.11900000000001439+10000.0	1.0	0.9997879961223788	0.1200000000000117+10000.0	1.0	0.9997888258270667	0.12100000000006628+10000.0	1.0	0.9997896487035121	0.12199999999996401+10000.0	1.0	0.9997904648631928	0.1229999999999404+10000.0	1.0	0.9997912744148788	0.12399999999995068+10000.0	1.0	0.9997920774647191	0.12499999999998511+10000.0	1.0	0.9997928741163252	0.12600000000001096+10000.0	1.0	0.9997936644708516	0.12700000000001624+10000.0	1.0	0.9997944486270726	0.12799999999996173+10000.0	1.0	0.9997952266814566	0.12900000000001977+10000.0	1.0	0.9997959987282372	0.13000000000006878+10000.0	1.0	0.9997967648594817	0.13099999999997564+10000.0	1.0	0.9997975251651571	0.13199999999995898+10000.0	1.0	0.9997982797331935	0.13299999999999404+10000.0	1.0	0.9997990286495448	0.13399999999999193+10000.0	1.0	0.9997997719982482	0.13500000000005274+10000.0	1.0	0.9998005098614798	0.13600000000005547+10000.0	1.0	0.99980124231961	0.1369999999999399+10000.0	1.0	0.9998019694512557	0.13799999999997992+10000.0	1.0	0.9998026913333307	0.13900000000000093+10000.0	1.0	0.9998034080410949	0.14000000000001647+10000.0	1.0	0.9998041196482014	0.14100000000005536+10000.0	1.0	0.9998048262267414	0.14199999999997934+10000.0	1.0	0.9998055278472888	0.14299999999998467+10000.0	1.0	0.999806224578942	0.1439999999999667+10000.0	1.0	0.9998069164893647	0.1450000000000147+10000.0	1.0	0.9998076036448257	0.14600000000005822+10000.0	1.0	0.9998082861102365	0.14700000000000923+10000.0	1.0	0.9998089639491888	0.14800000000001326+10000.0	1.0	0.9998096372239894	0.14900000000007607+10000.0	1.0	0.9998103059956946	0.1499999999999597+10000.0	1.0	0.9998109703241442	0.15099999999996117+10000.0	1.0	0.9998116302679927	0.1520000000000139+10000.0	1.0	0.9998122858847407	0.15300000000002542+10000.0	1.0	0.9998129372307653	0.15400000000007796+10000.0	1.0	0.9998135843613486	0.15500000000008018+10000.0	1.0	0.9998142273307065	0.15599999999999864+10000.0	1.0	0.9998148661920154	0.156999999999942+10000.0	1.0	0.9998155009974392	0.15800000000003692+10000.0	1.0	0.999816131798154	0.15900000000003425+10000.0	1.0	0.9998167586443736	0.1600000000000047+10000.0	1.0	0.9998173815853733	0.16099999999997924+10000.0	1.0	0.999818000669513	0.16200000000005949+10000.0	1.0	0.9998186159442598	0.1630000000000692+10000.0	1.0	0.9998192274562099	0.16399999999995887+10000.0	1.0	0.9998198352511098	0.16500000000003853+10000.0	1.0	0.9998204393738768	0.1660000000000692+10000.0	1.0	0.9998210398686185	0.1669999999999075+10000.0	1.0	0.9998216367786532	0.16800000000005388+10000.0	1.0	0.9998222301465274	0.16899999999999968+10000.0	1.0	0.9998228200140348	0.16999999999990925+10000.0	1.0	0.9998234064222339	0.17100000000004725+10000.0	1.0	0.9998239894114646	0.1719999999999254+10000.0	1.0	0.9998245690213656	0.1730000000000085+10000.0	1.0	0.9998251452908902	0.17400000000006466+10000.0	1.0	0.9998257182583217	0.1750000000000181+10000.0	1.0	0.9998262879612894	0.17600000000005614+10000.0	1.0	0.9998268544367827	0.17699999999993263+10000.0	1.0	0.9998274177211661	0.17799999999997984+10000.0	1.0	0.999827977850193	0.17900000000009467+10000.0	1.0	0.9998285348590187	0.17999999999999275+10000.0	1.0	0.9998290887822149	0.180999999999953+10000.0	1.0	0.9998296396537816	0.18200000000002847+10000.0	1.0	0.9998301875071598	0.18300000000007907+10000.0	1.0	0.9998307323752439	0.18400000000002548+10000.0	1.0	0.9998312742903934	0.18499999999990016+10000.0	1.0	0.9998318132844448	0.18600000000008166+10000.0	1.0	0.9998323493887219	0.18700000000003802+10000.0	1.0	0.9998328826340478	0.18800000000009737+10000.0	1.0	0.9998334130507548	0.18900000000004608+10000.0	1.0	0.9998339406686951	0.18999999999990272+10000.0	1.0	0.9998344655172511	0.1909999999999635+10000.0	1.0	0.9998349876253444	0.19200000000007536+10000.0	1.0	0.9998355070214461	0.19300000000002115+10000.0	1.0	0.9998360237335859	0.19399999999996617+10000.0	1.0	0.9998365377893614	0.19500000000008613+10000.0	1.0	0.9998370492159463	0.19599999999998596+10000.0	1.0	0.9998375580400998	0.19700000000007514+10000.0	1.0	0.9998380642881742	0.19799999999990012+10000.0	1.0	0.9998385679861242	0.1989999999999401+10000.0	1.0	0.9998390691595138	0.20000000000009313+10000.0	1.0	0.9998395678335243	0.20100000000007845+10000.0	1.0	0.9998400640329625	0.20200000000001764+10000.0	1.0	0.9998405577822677	0.20300000000006102+10000.0	1.0	0.9998410491055184	0.2039999999999211+10000.0	1.0	0.9998415380264406	0.20499999999989246+10000.0	1.0	0.9998420245684135	0.2059999999999677+10000.0	1.0	0.9998425087544768	0.2069999999999613+10000.0	1.0	0.999842990607337	0.20799999999997382+10000.0	1.0	0.9998434701493739	0.20900000000003713+10000.0	1.0	0.999843947402647	0.21000000000006994+10000.0	1.0	0.9998444223889013	0.21099999999991015+10000.0	1.0	0.9998448951295733	0.211999999999889+10000.0	1.0	0.9998453656457972	0.21300000000006405+10000.0	1.0	0.9998458339584096	0.21399999999995864+10000.0	1.0	0.9998463000879565	0.2150000000000109+10000.0	1.0	0.9998467640546972	0.21599999999998346+10000.0	1.0	0.9998472258786106	0.21700000000000869+10000.0	1.0	0.9998476855794001	0.21799999999999195+10000.0	1.0	0.9998481431764983	0.21899999999998637+10000.0	1.0	0.9998485986890725	0.22000000000009792+10000.0	1.0	0.9998490521360289	0.22099999999989872+10000.0	1.0	0.9998495035360182	0.22200000000003806+10000.0	1.0	0.9998499529074389	0.22299999999992362+10000.0	1.0	0.9998504002684433	0.22400000000004758+10000.0	1.0	0.9998508456369405	0.22500000000010417+10000.0	1.0	0.9998512890306016	0.22600000000003742+10000.0	1.0	0.9998517304668638	0.22700000000007248+10000.0	1.0	0.9998521699629341	0.22799999999994572+10000.0	1.0	0.999852607535794	0.22900000000010085+10000.0	1.0	0.9998530432022024	0.2300000000000534+10000.0	1.0	0.9998534769787005	0.2309999999999877+10000.0	1.0	0.9998539088816153	0.23200000000001259+10000.0	1.0	0.9998543389270628	0.23300000000010407+10000.0	1.0	0.9998547671309519	0.23399999999997118+10000.0	1.0	0.9998551935089883	0.23499999999988247+10000.0	1.0	0.9998556180766777	0.23600000000011823+10000.0	1.0	0.9998560408493284	0.23700000000005533+10000.0	1.0	0.9998564618420562	0.2380000000000415+10000.0	1.0	0.9998568810697863	0.23900000000003296+10000.0	1.0	0.999857298547257	0.23999999999995902+10000.0	1.0	0.9998577142890229	0.24099999999998753+10000.0	1.0	0.9998581283094576	0.2419999999998778+10000.0	1.0	0.9998585406227573	0.24300000000008903+10000.0	1.0	0.9998589512429426	0.2440000000000852+10000.0	1.0	0.9998593601838626	0.24500000000000924+10000.0	1.0	0.9998597674591971	0.24600000000008324+10000.0	1.0	0.9998601730824594	0.24700000000013442+10000.0	1.0	0.9998605770669989	0.24800000000007325+10000.0	1.0	0.9998609794260039	0.24899999999996503+10000.0	1.0	0.9998613801725043	0.24999999999995773+10000.0	1.0	0.9998617793193736	0.25100000000005895+10000.0	1.0	0.999862176879332	0.25200000000003403+10000.0	1.0	0.9998625728649485	0.25299999999998835+10000.0	1.0	0.9998629672886434	0.2539999999999571+10000.0	1.0	0.9998633601626906	0.25499999999988776+10000.0	1.0	0.9998637514992199	0.2560000000000239+10000.0	1.0	0.9998641413102191	0.2569999999999856+10000.0	1.0	0.9998645296075364	0.2579999999999465+10000.0	1.0	0.9998649164028826	0.2589999999999358+10000.0	1.0	0.999865301707833	0.26000000000009793+10000.0	1.0	0.9998656855338297	0.2609999999999058+10000.0	1.0	0.9998660678921836	0.26200000000007745+10000.0	1.0	0.9998664487940759	0.2629999999999675+10000.0	1.0	0.9998668282505612	0.2639999999999815+10000.0	1.0	0.9998672062725681	0.26500000000002505+10000.0	1.0	0.999867582870902	0.2660000000000805+10000.0	1.0	0.9998679580562467	0.2670000000000033+10000.0	1.0	0.9998683318391661	0.2680000000000005+10000.0	1.0	0.9998687042301061	0.2690000000000175+10000.0	1.0	0.9998690752393963	0.27000000000009877+10000.0	1.0	0.9998694448772516	0.2709999999999426+10000.0	1.0	0.9998698131537744	0.2720000000000416+10000.0	1.0	0.9998701800789556	0.27299999999990204+10000.0	1.0	0.9998705456626765	0.27399999999995744+10000.0	1.0	0.9998709099147106	0.27499999999994207+10000.0	1.0	0.9998712728447249	0.27600000000006625+10000.0	1.0	0.9998716344622813	0.2770000000001434+10000.0	1.0	0.9998719947768386	0.27800000000000963+10000.0	1.0	0.9998723537977537	0.2790000000000285+10000.0	1.0	0.999872711534283	0.2800000000001262+10000.0	1.0	0.9998730679955838	0.2810000000001273+10000.0	1.0	0.9998734231907159	0.281999999999856+10000.0	1.0	0.9998737771286431	0.2829999999999389+10000.0	1.0	0.999874129818234	0.28399999999985587+10000.0	1.0	0.999874481268264	0.28500000000013104+10000.0	1.0	0.9998748314874157	0.2859999999998739+10000.0	1.0	0.9998751804842818	0.2870000000000705+10000.0	1.0	0.9998755282673644	0.28799999999997233+10000.0	1.0	0.9998758748450776	0.2889999999998917+10000.0	1.0	0.9998762202257483	0.28999999999993087+10000.0	1.0	0.9998765644176174	0.29100000000003895+10000.0	1.0	0.9998769074288409	0.2920000000001299+10000.0	1.0	0.9998772492674912	0.2929999999999366+10000.0	1.0	0.9998775899415583	0.29399999999989973+10000.0	1.0	0.9998779294589506	0.2949999999998387+10000.0	1.0	0.9998782678274964	0.2959999999999526+10000.0	1.0	0.9998786050549447	0.2969999999999174+10000.0	1.0	0.9998789411489665	0.2980000000000052+10000.0	1.0	0.9998792761171553	0.29899999999995513+10000.0	1.0	0.9998796099670291	0.29999999999987226+10000.0	1.0	0.9998799427060304	0.30099999999985994+10000.0	1.0	0.999880274341528	0.3020000000000302+10000.0	1.0	0.9998806048808173	0.3029999999998917+10000.0	1.0	0.9998809343311219	0.3040000000001372+10000.0	1.0	0.9998812626995939	0.30500000000011257+10000.0	1.0	0.9998815899933153	0.3060000000000391+10000.0	1.0	0.999881916219299	0.30699999999991845+10000.0	1.0	0.9998822413844892	0.30799999999982963+10000.0	1.0	0.9998825654957628	0.30899999999993344+10000.0	1.0	0.9998828885599298	0.3099999999998335+10000.0	1.0	0.9998832105837346	0.3110000000000402+10000.0	1.0	0.9998835315738566	0.31200000000004197+10000.0	1.0	0.9998838515369108	0.3129999999998568+10000.0	1.0	0.9998841704794496	0.3139999999999121+10000.0	1.0	0.9998844884079621	0.31499999999991424+10000.0	1.0	0.9998848053288761	0.3159999999998412+10000.0	1.0	0.9998851212485587	0.31699999999993883+10000.0	1.0	0.9998854361733163	0.3180000000000493+10000.0	1.0	0.9998857501093962	0.31900000000002815+10000.0	1.0	0.999886063062987	0.31999999999985157+10000.0	1.0	0.9998863750402196	0.3210000000001195+10000.0	1.0	0.9998866860471672	0.3220000000001023+10000.0	1.0	0.999886996089847	0.3230000000000899+10000.0	1.0	0.99988730517422	0.323999999999864+10000.0	1.0	0.9998876133061925	0.32500000000006335+10000.0	1.0	0.9998879204916159	0.3259999999999887+10000.0	1.0	0.9998882267362883	0.32700000000012175+10000.0	1.0	0.9998885320459543	0.32799999999998897+10000.0	1.0	0.9998888364263063	0.32900000000003765+10000.0	1.0	0.9998891398829848	0.3299999999999146+10000.0	1.0	0.9998894424215791	0.33100000000005264+10000.0	1.0	0.9998897440476278	0.33200000000000024+10000.0	1.0	0.9998900447666197	0.33300000000007896+10000.0	1.0	0.9998903445839942	0.33400000000013486+10000.0	1.0	0.9998906435051418	0.3350000000001571+10000.0	1.0	0.9998909415354048	0.3359999999998179+10000.0	1.0	0.9998912386800782	0.3369999999999046+10000.0	1.0	0.9998915349444096	0.33800000000005287+10000.0	1.0	0.9998918303336001	0.3389999999998552+10000.0	1.0	0.999892124852805	0.3400000000000174+10000.0	1.0	0.9998924185071341	0.3410000000001551+10000.0	1.0	0.9998927113016521	0.34199999999999614+10000.0	1.0	0.9998930032413799	0.34299999999986885+10000.0	1.0	0.999893294331294	0.3440000000000796+10000.0	1.0	0.9998935845763278	0.344999999999922+10000.0	1.0	0.999893873981372	0.3460000000001521+10000.0	1.0	0.9998941625512745	0.34700000000013753+10000.0	1.0	0.9998944502908418	0.34800000000000525+10000.0	1.0	0.9998947372048389	0.3490000000001384+10000.0	1.0	0.9998950232979897	0.3499999999999112+10000.0	1.0	0.9998953085749781	0.35099999999993303+10000.0	1.0	0.9998955930404476	0.35200000000005593+10000.0	1.0	0.9998958766990024	0.3530000000001137+10000.0	1.0	0.9998961595552074	0.3539999999999112+10000.0	1.0	0.9998964416135893	0.35500000000002346+10000.0	1.0	0.9998967228786361	0.3559999999998638+10000.0	1.0	0.9998970033547985	0.35700000000010124+10000.0	1.0	0.9998972830464894	0.3579999999999726+10000.0	1.0	0.9998975619580852	0.3590000000001514+10000.0	1.0	0.9998978400939255	0.36000000000008914+10000.0	1.0	0.9998981174583138	0.361000000000143+10000.0	1.0	0.9998983940555181	0.3620000000001417+10000.0	1.0	0.9998986698897707	0.3629999999999632+10000.0	1.0	0.9998989449652695	0.3640000000001409+10000.0	1.0	0.9998992192861774	0.36500000000006844+10000.0	1.0	0.9998994928566233	0.3659999999998511+10000.0	1.0	0.9998997656807024	0.3670000000001694+10000.0	1.0	0.9999000377624762	0.3680000000001159+10000.0	1.0	0.9999003091059733	0.3689999999999314+10000.0	1.0	0.9999005797151898	0.3700000000001073+10000.0	1.0	0.999900849594089	0.3710000000000786+10000.0	1.0	0.9999011187466025	0.37199999999980493+10000.0	1.0	0.9999013871766304	0.3730000000001521+10000.0	1.0	0.9999016548880408	0.3739999999999751+10000.0	1.0	0.9999019218846716	0.3749999999998328+10000.0	1.0	0.9999021881703297	0.3759999999999345+10000.0	1.0	0.9999024537487916	0.37700000000016304+10000.0	1.0	0.9999027186238038	0.3780000000001145+10000.0	1.0	0.9999029827990831	0.3789999999999964+10000.0	1.0	0.9999032462783171	0.37999999999986817+10000.0	1.0	0.9999035090651642	0.3810000000001546+10000.0	1.0	0.999903771163254	0.3820000000000619+10000.0	1.0	0.9999040325761877	0.3829999999998114+10000.0	1.0	0.9999042933075383	0.38399999999993906+10000.0	1.0	0.9999045533608509	0.3850000000001713+10000.0	1.0	0.999904812739643	0.3860000000000121+10000.0	1.0	0.999905071447405	0.3870000000002132+10000.0	1.0	0.9999053294875999	0.3879999999999735+10000.0	1.0	0.9999055868636644	0.389000000000156+10000.0	1.0	0.9999058435790082	0.390000000000073+10000.0	1.0	0.9999060996370154	0.39099999999987894+10000.0	1.0	0.9999063550410437	0.3919999999999673+10000.0	1.0	0.9999066097944255	0.3929999999999351+10000.0	1.0	0.9999068639004675	0.39400000000016966+10000.0	1.0	0.9999071173624512	0.3949999999999637+10000.0	1.0	0.9999073701836336	0.3960000000000122+10000.0	1.0	0.9999076223672468	0.3970000000001104+10000.0	1.0	0.9999078739164985	0.39799999999992347+10000.0	1.0	0.9999081248345723	0.3989999999997796+10000.0	1.0	0.9999083751246279	0.40000000000015556+10000.0	1.0	0.9999086247898012	0.4009999999998304+10000.0	1.0	0.999908873833205	0.40199999999982616+10000.0	1.0	0.9999091222579286	0.4029999999998128+10000.0	1.0	0.9999093700670384	0.4039999999998532+10000.0	1.0	0.9999096172635781	0.40499999999993835+10000.0	1.0	0.9999098638505688	0.40599999999997666+10000.0	1.0	0.9999101098310094	0.40699999999979514+10000.0	1.0	0.9999103552078767	0.40800000000005754+10000.0	1.0	0.9999105999841257	0.4090000000000279+10000.0	1.0	0.9999108441626895	0.4099999999998645+10000.0	1.0	0.9999110877464802	0.41100000000021997+10000.0	1.0	0.9999113307383881	0.41200000000001696+10000.0	1.0	0.9999115731412831	0.41300000000016185+10000.0	1.0	0.9999118149580137	0.4139999999999693+10000.0	1.0	0.9999120561914083	0.4149999999999951+10000.0	1.0	0.9999122968442745	0.4159999999998401+10000.0	1.0	0.9999125369194	0.4170000000000987+10000.0	1.0	0.999912776419552	0.41800000000016835+10000.0	1.0	0.9999130153474783	0.41900000000022464+10000.0	1.0	0.9999132537059069	0.4199999999999697+10000.0	1.0	0.9999134914975464	0.42100000000017657+10000.0	1.0	0.9999137287250859	0.4220000000000487+10000.0	1.0	0.9999139653911957	0.42300000000018634+10000.0	1.0	0.9999142014985269	0.4239999999999552+10000.0	1.0	0.9999144370497123	0.4250000000000176+10000.0	1.0	0.9999146720473656	0.4260000000001784+10000.0	1.0	0.9999149064940824	0.42700000000005406+10000.0	1.0	0.9999151403924402	0.4280000000002347+10000.0	1.0	0.9999153737449982	0.42900000000013716+10000.0	1.0	0.9999156065542978	0.4300000000001333+10000.0	1.0	0.9999158388228627	0.4309999999998881+10000.0	1.0	0.9999160705531993	0.4320000000000406+10000.0	1.0	0.9999163017477962	0.43300000000007+10000.0	1.0	0.999916532409125	0.4339999999999955+10000.0	1.0	0.9999167625396403	0.4350000000001759+10000.0	1.0	0.9999169921417796	0.436000000000146+10000.0	1.0	0.9999172212179638	0.4369999999998705+10000.0	1.0	0.9999174497705972	0.4380000000000511+10000.0	1.0	0.9999176778020674	0.4390000000000078+10000.0	1.0	0.9999179053147461	0.4399999999999352+10000.0	1.0	0.9999181323109886	0.44099999999977074+10000.0	1.0	0.9999183587931343	0.4420000000000167+10000.0	1.0	0.9999185847635066	0.44300000000012235+10000.0	1.0	0.9999188102244132	0.4439999999998086+10000.0	1.0	0.9999190351781465	0.44499999999992396+10000.0	1.0	0.9999192596269831	0.4459999999998527+10000.0	1.0	0.9999194835731846	0.44699999999985823+10000.0	1.0	0.9999197070189972	0.4479999999999829+10000.0	1.0	0.9999199299666522	0.44899999999993684+10000.0	1.0	0.9999201524183661	0.4499999999999914+10000.0	1.0	0.9999203743763403	0.4509999999998887+10000.0	1.0	0.9999205958427619	0.4519999999997498+10000.0	1.0	0.9999208168198036	0.45299999999999646+10000.0	1.0	0.9999210373096232	0.45399999999976537+10000.0	1.0	0.9999212573143649	0.4549999999998549+10000.0	1.0	0.9999214768361583	0.4560000000001479+10000.0	1.0	0.9999216958771191	0.4570000000000507+10000.0	1.0	0.9999219144393493	0.4579999999999619+10000.0	1.0	0.9999221325249371	0.4590000000002176+10000.0	1.0	0.9999223501359569	0.4600000000000298+10000.0	1.0	0.9999225672744697	0.4609999999999986+10000.0	1.0	0.9999227839425232	0.4620000000000479+10000.0	1.0	0.9999230001421516	0.4629999999999093+10000.0	1.0	0.9999232158753762	0.46400000000013397+10000.0	1.0	0.999923431144205	0.46500000000002506+10000.0	1.0	0.9999236459506332	0.4660000000002063+10000.0	1.0	0.999923860296643	0.46700000000004355+10000.0	1.0	0.9999240741842043	0.46800000000023567+10000.0	1.0	0.9999242876152737	0.46900000000023606+10000.0	1.0	0.9999245005917959	0.46999999999982195+10000.0	1.0	0.9999247131157032	0.471000000000166+10000.0	1.0	0.9999249251889151	0.47200000000021014+10000.0	1.0	0.9999251368133392	0.4729999999998259+10000.0	1.0	0.9999253479908712	0.4739999999998625+10000.0	1.0	0.9999255587233946	0.4750000000000909+10000.0	1.0	0.999925769012781	0.47599999999976983+10000.0	1.0	0.9999259788608903	0.4769999999998123+10000.0	1.0	0.9999261882695708	0.4780000000002083+10000.0	1.0	0.9999263972406589	0.4790000000000793+10000.0	1.0	0.9999266057759798	0.47999999999986953+10000.0	1.0	0.9999268138773473	0.48099999999983195+10000.0	1.0	0.9999270215465638	0.4820000000001048+10000.0	1.0	0.9999272287854204	0.4830000000002581+10000.0	1.0	0.9999274355956972	0.4839999999999138+10000.0	1.0	0.9999276419791634	0.4849999999999166+10000.0	1.0	0.999927847937577	0.4859999999997471+10000.0	1.0	0.9999280534726855	0.4869999999997733+10000.0	1.0	0.9999282585862254	0.48799999999974214+10000.0	1.0	0.9999284632799225	0.48899999999997135+10000.0	1.0	0.999928667555492	0.4899999999998402+10000.0	1.0	0.999928871414639	0.491000000000082+10000.0	1.0	0.9999290748590578	0.49200000000019295+10000.0	1.0	0.9999292778904324	0.4930000000001924+10000.0	1.0	0.9999294805104367	0.49400000000021466+10000.0	1.0	0.9999296827207345	0.4950000000001005+10000.0	1.0	0.9999298845229793	0.496000000000089+10000.0	1.0	0.9999300859188147	0.49699999999986605+10000.0	1.0	0.9999302869098746	0.49799999999981803+10000.0	1.0	0.9999304874977829	0.4990000000000875+10000.0	1.0	0.9999306876841536	0.5000000000001773+10000.0	1.0	0.9999308874705912	0.5010000000002223+10000.0	1.0	0.9999310868586906	0.5020000000000512+10000.0	1.0	0.9999312858500371	0.5029999999999157+10000.0	1.0	0.9999314844462067	0.5040000000001139+10000.0	1.0	0.9999316826487659	0.5050000000000515+10000.0	1.0	0.9999318804592718	0.5060000000001053+10000.0	1.0	0.9999320778792726	0.5070000000001342+10000.0	1.0	0.9999322749103071	0.5080000000002336+10000.0	1.0	0.999932471553905	0.5089999999998054+10000.0	1.0	0.9999326678115872	0.5100000000000197+10000.0	1.0	0.9999328636848654	0.5110000000000664+10000.0	1.0	0.9999330591752428	0.51200000000019+10000.0	1.0	0.9999332542842133	0.5130000000002042+10000.0	1.0	0.9999334490132625	0.5140000000002785+10000.0	1.0	0.9999336433638671	0.5150000000000188+10000.0	1.0	0.9999338373374954	0.515999999999835+10000.0	1.0	0.9999340309356071	0.5170000000000301+10000.0	1.0	0.9999342241596532	0.5179999999998828+10000.0	1.0	0.9999344170110767	0.5190000000001797+10000.0	1.0	0.9999346094913119	0.5200000000000073+10000.0	1.0	0.999934801601785	0.5209999999998728+10000.0	1.0	0.9999349933439143	0.5220000000002271+10000.0	1.0	0.9999351847191094	0.5229999999999747+10000.0	1.0	0.9999353757287721	0.5240000000002059+10000.0	1.0	0.9999355663742961	0.5249999999998128+10000.0	1.0	0.9999357566570675	0.5259999999998208+10000.0	1.0	0.999935946578464	0.5270000000001636+10000.0	1.0	0.9999361361398555	0.5279999999999432+10000.0	1.0	0.9999363253426047	0.5290000000000452+10000.0	1.0	0.9999365141880658	0.5299999999999021+10000.0	1.0	0.999936702677586	0.5310000000001178+10000.0	1.0	0.9999368908125045	0.53199999999981+10000.0	1.0	0.999937078594153	0.5329999999998417+10000.0	1.0	0.999937266023856	0.5340000000001648+10000.0	1.0	0.9999374531029301	0.5350000000001052+10000.0	1.0	0.9999376398326849	0.5359999999998445+10000.0	1.0	0.9999378262144226	0.5370000000001331+10000.0	1.0	0.999938012249438	0.5380000000002103+10000.0	1.0	0.9999381979390187	0.5389999999998927+10000.0	1.0	0.9999383832844454	0.5399999999998903+10000.0	1.0	0.9999385682869913	0.5409999999997266+10000.0	1.0	0.999938752947923	0.5419999999998498+10000.0	1.0	0.9999389372684997	0.5430000000001529+10000.0	1.0	0.9999391212499738	0.5440000000002927+10000.0	1.0	0.9999393048935906	0.5450000000000141+10000.0	1.0	0.9999394882005891	0.5460000000000872+10000.0	1.0	0.999939671172201	0.5470000000002203+10000.0	1.0	0.9999398538096512	0.5479999999999953+10000.0	1.0	0.9999400361141584	0.5489999999998146+10000.0	1.0	0.9999402180869342	0.5500000000000307+10000.0	1.0	0.9999403997291838	0.5510000000000667+10000.0	1.0	0.9999405810421057	0.5519999999999824+10000.0	1.0	0.9999407620268921	0.5530000000002181+10000.0	1.0	0.9999409426847284	0.5540000000001067+10000.0	1.0	0.999941123016794	0.55500000000007+10000.0	1.0	0.9999413030242618	0.5560000000001372+10000.0	1.0	0.9999414827082981	0.5570000000003046+10000.0	1.0	0.9999416620700632	0.5580000000002827+10000.0	1.0	0.999941841110711	0.558999999999862+10000.0	1.0	0.9999420198313896	0.5599999999999091+10000.0	1.0	0.9999421982332406	0.5610000000002586+10000.0	1.0	0.9999423763173994	0.5620000000000825+10000.0	1.0	0.9999425540849955	0.563000000000141+10000.0	1.0	0.9999427315371526	0.5640000000000484+10000.0	1.0	0.9999429086749881	0.5649999999999045+10000.0	1.0	0.9999430854996137	0.56600000000006+10000.0	1.0	0.9999432620121351	0.5670000000002519+10000.0	1.0	0.9999434382136521	0.5679999999999925+10000.0	1.0	0.9999436141052588	0.5690000000002294+10000.0	1.0	0.9999437896880434	0.569999999999961+10000.0	1.0	0.9999439649630887	0.5709999999997922+10000.0	1.0	0.9999441399314715	0.5719999999998151+10000.0	1.0	0.999944314594263	0.573000000000012+10000.0	1.0	0.999944488952529	0.5740000000000297+10000.0	1.0	0.9999446630073294	0.5750000000002278+10000.0	1.0	0.9999448367597189	0.5760000000001841+10000.0	1.0	0.9999450102107464	0.5769999999997476+10000.0	1.0	0.9999451833614559	0.578000000000104+10000.0	1.0	0.9999453562128852	0.578999999999713+10000.0	1.0	0.9999455287660673	0.579999999999863+10000.0	1.0	0.9999457010220298	0.580999999999965+10000.0	1.0	0.9999458729817948	0.5819999999999076+10000.0	1.0	0.9999460446463793	0.5829999999998486+10000.0	1.0	0.9999462160167949	0.5840000000000072+10000.0	1.0	0.9999463870940482	0.5849999999998048+10000.0	1.0	0.9999465578791406	0.5860000000002533+10000.0	1.0	0.9999467283730681	0.5869999999998521+10000.0	1.0	0.9999468985768222	0.5880000000002364+10000.0	1.0	0.9999470684913887	0.5890000000001153+10000.0	1.0	0.9999472381177489	0.5900000000002852+10000.0	1.0	0.9999474074568787	0.5910000000001652+10000.0	1.0	0.9999475765097494	0.5920000000002156+10000.0	1.0	0.9999477452773271	0.5929999999997727+10000.0	1.0	0.9999479137605733	0.594000000000141+10000.0	1.0	0.9999480819604444	0.5949999999997888+10000.0	1.0	0.9999482498778921	0.5960000000000866+10000.0	1.0	0.9999484175138632	0.5969999999998349+10000.0	1.0	0.9999485848692999	0.5979999999996997+10000.0	1.0	0.9999487519451398	0.599000000000042+10000.0	1.0	0.9999489187423153	0.6000000000000605+10000.0	1.0	0.9999490852617546	0.6010000000002622+10000.0	1.0	0.9999492515043812	0.6020000000002829+10000.0	1.0	0.9999494174711138	0.6030000000000326+10000.0	1.0	0.9999495831628668	0.6040000000001853+10000.0	1.0	0.9999497485805497	0.6049999999999863+10000.0	1.0	0.999949913725068	0.6059999999997507+10000.0	1.0	0.9999500785973224	0.6070000000000176+10000.0	1.0	0.9999502431982091	0.6080000000000231+10000.0	1.0	0.9999504075286201	0.6090000000002149+10000.0	1.0	0.9999505715894427	0.6100000000000553+10000.0	1.0	0.9999507353815601	0.6109999999998709+10000.0	1.0	0.9999508989058512	0.6120000000000065+10000.0	1.0	0.9999510621631904	0.6129999999999713+10000.0	1.0	0.999951225154448	0.6140000000003043+10000.0	1.0	0.9999513878804898	0.6149999999996804+10000.0	1.0	0.9999515503421778	0.616000000000197+10000.0	1.0	0.9999517125403692	0.6169999999996969+10000.0	1.0	0.9999518744759177	0.6179999999998003+10000.0	1.0	0.9999520361496725	0.6190000000002697+10000.0	1.0	0.9999521975624787	0.6200000000002066+10000.0	1.0	0.9999523587151772	0.6209999999999498+10000.0	1.0	0.9999525196086052	0.6220000000002311+10000.0	1.0	0.9999526802435955	0.6229999999999408+10000.0	1.0	0.9999528406209772	0.6240000000001087+10000.0	1.0	0.9999530007415752	0.6250000000002904+10000.0	1.0	0.9999531606062104	0.6259999999997873+10000.0	1.0	0.9999533202157002	0.6270000000002695+10000.0	1.0	0.9999534795708573	0.6280000000000603+10000.0	1.0	0.9999536386724914	0.6290000000002378+10000.0	1.0	0.9999537975214078	0.6300000000003051+10000.0	1.0	0.999953956118408	0.6310000000001246+10000.0	1.0	0.9999541144642898	0.6319999999997676+10000.0	1.0	0.9999542725598475	0.6330000000000671+10000.0	1.0	0.999954430405871	0.633999999999655+10000.0	1.0	0.999954588003147	0.6349999999997312+10000.0	1.0	0.9999547453524582	0.6359999999996964+10000.0	1.0	0.9999549024545838	0.6369999999998183+10000.0	1.0	0.9999550593102993	0.6379999999996745+10000.0	1.0	0.9999552159203764	0.6390000000001282+10000.0	1.0	0.9999553722855833	0.6400000000003508+10000.0	1.0	0.9999555284066844	0.6409999999997981+10000.0	1.0	0.9999556842844411	0.6420000000002067+10000.0	1.0	0.9999558399196106	0.6429999999997587+10000.0	1.0	0.9999559953129469	0.6440000000000602+10000.0	1.0	0.9999561504652004	0.6450000000002992+10000.0	1.0	0.9999563053771179	0.6459999999999539+10000.0	1.0	0.999956460049443	0.6470000000000932+10000.0	1.0	0.9999566144829157	0.6479999999996517+10000.0	1.0	0.9999567686782728	0.649000000000321+10000.0	1.0	0.999956922636247	0.6499999999999371+10000.0	1.0	0.9999570763575686	0.6510000000002659+10000.0	1.0	0.9999572298429636	0.6519999999996599+10000.0	1.0	0.9999573830931556	0.6530000000001457+10000.0	1.0	0.9999575361088641	0.6540000000000591+10000.0	1.0	0.9999576888908055	0.6549999999997009+10000.0	1.0	0.9999578414396932	0.6559999999997631+10000.0	1.0	0.9999579937562372	0.6569999999997421+10000.0	1.0	0.9999581458411441	0.6579999999999907+10000.0	1.0	0.9999582976951173	0.6590000000001313+10000.0	1.0	0.9999584493188572	0.6599999999996545+10000.0	1.0	0.9999586007130611	0.660999999999992+10000.0	1.0	0.9999587518784226	0.6619999999999935+10000.0	1.0	0.9999589028156328	0.6630000000001971+10000.0	1.0	0.9999590535253792	0.6639999999997681+10000.0	1.0	0.9999592040083465	0.6650000000000548+10000.0	1.0	0.9999593542652161	0.6660000000000437+10000.0	1.0	0.9999595042966665	0.6669999999999219+10000.0	1.0	0.9999596541033731	0.6680000000002206+10000.0	1.0	0.9999598036860081	0.6690000000002112+10000.0	1.0	0.999959953045241	0.6700000000000048+10000.0	1.0	0.999960102181738	0.6709999999996944+10000.0	1.0	0.9999602510961627	0.6719999999999797+10000.0	1.0	0.9999603997891753	0.6729999999998119+10000.0	1.0	0.9999605482614333	0.6740000000000027+10000.0	1.0	0.9999606965135913	0.6750000000001238+10000.0	1.0	0.9999608445463009	0.6759999999998809+10000.0	1.0	0.9999609923602107	0.6769999999997504+10000.0	1.0	0.9999611399559669	0.6780000000001163+10000.0	1.0	0.9999612873342122	0.6789999999996474+10000.0	1.0	0.9999614344955869	0.6799999999996997+10000.0	1.0	0.9999615814407284	0.6809999999996861+10000.0	1.0	0.9999617281702713	0.681999999999978+10000.0	1.0	0.9999618746848472	0.6830000000002834+10000.0	1.0	0.9999620209850851	0.6840000000002898+10000.0	1.0	0.9999621670716113	0.6850000000003138+10000.0	1.0	0.9999623129450492	0.6859999999996717+10000.0	1.0	0.9999624586060198	0.6870000000003764+10000.0	1.0	0.9999626040551409	0.6880000000001734+10000.0	1.0	0.999962749293028	0.6890000000000508+10000.0	1.0	0.9999628943202937	0.6900000000003249+10000.0	1.0	0.999963039137548	0.6909999999997605+10000.0	1.0	0.9999631837453985	0.6920000000000637+10000.0	1.0	0.9999633281444497	0.6929999999996409+10000.0	1.0	0.9999634723353038	0.693999999999635+10000.0	1.0	0.9999636163185605	0.6949999999999813+10000.0	1.0	0.9999637600948165	0.6960000000000679+10000.0	1.0	0.9999639036646663	0.6970000000001757+10000.0	1.0	0.9999640470287016	0.6979999999998324+10000.0	1.0	0.9999641901875118	0.6990000000000324+10000.0	1.0	0.9999643331416836	0.7000000000000374+10000.0	1.0	0.9999644758918014	0.7010000000003754+10000.0	1.0	0.9999646184384465	0.7019999999996325+10000.0	1.0	0.9999647607821986	0.7029999999998013+10000.0	1.0	0.9999649029236344	0.7039999999999541+10000.0	1.0	0.999965044863328	0.7050000000000359+10000.0	1.0	0.9999651866018515	0.7059999999999891+10000.0	1.0	0.9999653281397742	0.7069999999996552+10000.0	1.0	0.9999654694776634	0.7080000000002494+10000.0	1.0	0.9999656106160835	0.7090000000003379+10000.0	1.0	0.9999657515555966	0.7099999999996638+10000.0	1.0	0.999965892296763	0.71099999999985+10000.0	1.0	0.9999660328401401	0.7120000000003646+10000.0	1.0	0.9999661731862828	0.7129999999999944+10000.0	1.0	0.9999663133357442	0.7139999999999171+10000.0	1.0	0.9999664532890746	0.7149999999996557+10000.0	1.0	0.9999665930468226	0.7160000000001533+10000.0	1.0	0.9999667326095336	0.7169999999997106+10000.0	1.0	0.9999668719777517	0.7180000000002534+10000.0	1.0	0.9999670111520179	0.719000000000081+10000.0	1.0	0.9999671501328715	0.7200000000001461+10000.0	1.0	0.9999672889208493	0.7209999999999799+10000.0	1.0	0.9999674275164859	0.7219999999999972+10000.0	1.0	0.9999675659203138	0.7229999999998085+10000.0	1.0	0.9999677041328631	0.7239999999997319+10000.0	1.0	0.9999678421546618	0.7249999999999058+10000.0	1.0	0.9999679799862359	0.7260000000001996+10000.0	1.0	0.9999681176281088	0.7270000000001238+10000.0	1.0	0.9999682550808021	0.728000000000358+10000.0	1.0	0.9999683923448351	0.7290000000002439+10000.0	1.0	0.9999685294207251	0.7300000000001216+10000.0	1.0	0.9999686663089871	0.7309999999996277+10000.0	1.0	0.9999688030101341	0.7320000000000444+10000.0	1.0	0.999968939524677	0.7330000000001559+10000.0	1.0	0.9999690758531246	0.7340000000002227+10000.0	1.0	0.9999692119959834	0.7350000000002755+10000.0	1.0	0.9999693479537582	0.7360000000000311+10000.0	1.0	0.9999694837269516	0.7369999999996248+10000.0	1.0	0.9999696193160642	0.7380000000003517+10000.0	1.0	0.9999697547215942	0.7389999999996683+10000.0	1.0	0.9999698899440385	0.7400000000001238+10000.0	1.0	0.9999700249838912	0.7410000000002633+10000.0	1.0	0.999970159841645	0.7420000000002874+10000.0	1.0	0.9999702945177902	0.7430000000003328+10000.0	1.0	0.9999704290128154	0.7440000000003928+10000.0	1.0	0.9999705633272071	0.7450000000002367+10000.0	1.0	0.99997069746145	0.7460000000001582+10000.0	1.0	0.9999708314160265	0.7470000000000708+10000.0	1.0	0.9999709651914175	0.7480000000002581+10000.0	1.0	0.9999710987881015	0.7489999999996351+10000.0	1.0	0.9999712322065557	0.7499999999998259+10000.0	1.0	0.9999713654472548	0.7510000000001023+10000.0	1.0	0.9999714985106719	0.7519999999997959+10000.0	1.0	0.9999716313972782	0.7529999999998954+10000.0	1.0	0.9999717641075428	0.7539999999996334+10000.0	1.0	0.9999718966419334	0.7549999999997521+10000.0	1.0	0.9999720290009154	0.7559999999999215+10000.0	1.0	0.9999721611849526	0.7570000000003382+10000.0	1.0	0.9999722931945066	0.7579999999999718+10000.0	1.0	0.9999724250300378	0.7589999999998549+10000.0	1.0	0.9999725566920042	0.7599999999996474+10000.0	1.0	0.9999726881808625	0.7610000000000899+10000.0	1.0	0.9999728194970671	0.7620000000004006+10000.0	1.0	0.9999729506410707	0.7629999999998869+10000.0	1.0	0.9999730816133248	0.76400000000042+10000.0	1.0	0.9999732124142782	0.7649999999995849+10000.0	1.0	0.9999733430443788	0.7659999999999372+10000.0	1.0	0.9999734735040722	0.7669999999995973+10000.0	1.0	0.9999736037938025	0.7679999999998259+10000.0	1.0	0.999973733914012	0.7689999999998517+10000.0	1.0	0.9999738638651413	0.7700000000002085+10000.0	1.0	0.9999739936476292	0.7710000000001065+10000.0	1.0	0.9999741232619128	0.7719999999999267+10000.0	1.0	0.9999742527084278	0.7730000000003018+10000.0	1.0	0.9999743819876077	0.7740000000003311+10000.0	1.0	0.9999745110998847	0.7750000000000844+10000.0	1.0	0.9999746400456891	0.7759999999996795+10000.0	1.0	0.99997476882545	0.7770000000000785+10000.0	1.0	0.9999748974395941	0.7779999999995709+10000.0	1.0	0.9999750258885471	0.7789999999997452+10000.0	1.0	0.9999751541727329	0.7800000000002477+10000.0	1.0	0.9999752822925733	0.7809999999995718+10000.0	1.0	0.9999754102484892	0.7820000000001947+10000.0	1.0	0.9999755380408994	0.7829999999998447+10000.0	1.0	0.9999756656702214	0.7840000000001108+10000.0	1.0	0.9999757931368709	0.7850000000003022+10000.0	1.0	0.9999759204412619	0.785999999999988+10000.0	1.0	0.9999760475838072	0.7869999999998089+10000.0	1.0	0.9999761745649177	0.787999999999671+10000.0	1.0	0.999976301385003	0.7890000000004305+10000.0	1.0	0.9999764280444708	0.7900000000003282+10000.0	1.0	0.9999765545437276	0.7910000000001809+10000.0	1.0	0.999976680883178	0.7919999999998171+10000.0	1.0	0.9999768070632256	0.792999999999771+10000.0	1.0	0.9999769330842719	0.794000000000344+10000.0	1.0	0.9999770589467172	0.7949999999997764+10000.0	1.0	0.9999771846509603	0.7960000000003603+10000.0	1.0	0.9999773101973983	0.797000000000441+10000.0	1.0	0.9999774355864269	0.7979999999996478+10000.0	1.0	0.9999775608184406	0.7989999999997291+10000.0	1.0	0.999977685893832	0.800000000000066+10000.0	1.0	0.9999778108129923	0.8010000000002652+10000.0	1.0	0.9999779355763114	0.8020000000001019+10000.0	1.0	0.9999780601841778	0.8030000000003541+10000.0	1.0	0.9999781846369782	0.8040000000000723+10000.0	1.0	0.9999783089350982	0.8050000000000855+10000.0	1.0	0.9999784330789219	0.8060000000002703+10000.0	1.0	0.9999785570688318	0.8070000000003833+10000.0	1.0	0.999978680905209	0.8080000000000033+10000.0	1.0	0.9999788045884336	0.8090000000002695+10000.0	1.0	0.9999789281188837	0.8100000000002375+10000.0	1.0	0.9999790514969363	0.8110000000004121+10000.0	1.0	0.999979174722967	0.8119999999999943+10000.0	1.0	0.9999792977973502	0.8130000000004299+10000.0	1.0	0.9999794207204583	0.8139999999999439+10000.0	1.0	0.999979543492663	0.8149999999997999+10000.0	1.0	0.9999796661143344	0.8159999999997344+10000.0	1.0	0.9999797885858411	0.8169999999997061+10000.0	1.0	0.9999799109075506	0.8179999999998435+10000.0	1.0	0.9999800330798287	0.8190000000003915+10000.0	1.0	0.9999801551030402	0.8199999999998375+10000.0	1.0	0.9999802769775484	0.8210000000003159+10000.0	1.0	0.9999803987037154	0.8220000000002764+10000.0	1.0	0.9999805202819017	0.8229999999998905+10000.0	1.0	0.9999806417124668	0.8240000000000944+10000.0	1.0	0.9999807629957688	0.8249999999997933+10000.0	1.0	0.9999808841321645	0.826000000000382+10000.0	1.0	0.9999810051220092	0.8270000000001986+10000.0	1.0	0.9999811259656571	0.8279999999999703+10000.0	1.0	0.9999812466634613	0.8290000000000148+10000.0	1.0	0.9999813672157734	0.8300000000001867+10000.0	1.0	0.9999814876229435	0.8309999999998233+10000.0	1.0	0.9999816078853209	0.8319999999995386+10000.0	1.0	0.9999817280032535	0.8330000000002549+10000.0	1.0	0.9999818479770877	0.8340000000003772+10000.0	1.0	0.9999819678071689	0.8350000000003627+10000.0	1.0	0.9999820874938411	0.8359999999998988+10000.0	1.0	0.9999822070374473	0.8369999999997079+10000.0	1.0	0.9999823264383292	0.8379999999996428+10000.0	1.0	0.999982445696827	0.8389999999995642+10000.0	1.0	0.9999825648132802	0.8400000000002245+10000.0	1.0	0.9999826837880265	0.8410000000004229+10000.0	1.0	0.9999828026214026	0.8419999999997501+10000.0	1.0	0.9999829213137444	0.8430000000004167+10000.0	1.0	0.9999830398653861	0.8439999999997242+10000.0	1.0	0.999983158276661	0.845000000000308+10000.0	1.0	0.9999832765479009	0.8459999999997893+10000.0	1.0	0.9999833946794369	0.8470000000001021+10000.0	1.0	0.9999835126715985	0.8479999999999397+10000.0	1.0	0.9999836305247143	0.849000000000344+10000.0	1.0	0.9999837482391115	0.8499999999999547+10000.0	1.0	0.9999838658151164	0.8509999999996722+10000.0	1.0	0.9999839832530542	0.8519999999997873+10000.0	1.0	0.9999841005532485	0.8529999999999315+10000.0	1.0	0.9999842177160222	0.8539999999999753+10000.0	1.0	0.9999843347416969	0.8549999999999848+10000.0	1.0	0.9999844516305931	0.8560000000001764+10000.0	1.0	0.9999845683830302	0.8569999999999217+10000.0	1.0	0.9999846849993264	0.857999999999603+10000.0	1.0	0.9999848014797988	0.8589999999996201+10000.0	1.0	0.9999849178247637	0.8600000000003466+10000.0	1.0	0.9999850340345358	0.8610000000001732+10000.0	1.0	0.9999851501094289	0.8620000000002372+10000.0	1.0	0.9999852660497558	0.8629999999996043+10000.0	1.0	0.9999853818558283	0.8640000000000085+10000.0	1.0	0.9999854975279568	0.8650000000000624+10000.0	1.0	0.999985613066451	0.866000000000003+10000.0	1.0	0.999985728471619	0.8669999999997328+10000.0	1.0	0.9999858437437685	0.8679999999997389+10000.0	1.0	0.9999859588832057	0.8690000000000887+10000.0	1.0	0.9999860738902359	0.870000000000385+10000.0	1.0	0.999986188765163	0.8709999999997211+10000.0	1.0	0.9999863035082904	0.8719999999995401+10000.0	1.0	0.9999864181199203	0.8729999999997297+10000.0	1.0	0.9999865326003535	0.8739999999995414+10000.0	1.0	0.9999866469498904	0.8750000000004604+10000.0	1.0	0.9999867611688296	0.8760000000003412+10000.0	1.0	0.9999868752574692	0.8770000000001615+10000.0	1.0	0.9999869892161063	0.8780000000001003+10000.0	1.0	0.9999871030450368	0.8790000000004675+10000.0	1.0	0.9999872167445555	0.8799999999997115+10000.0	1.0	0.9999873303149566	0.8810000000002353+10000.0	1.0	0.9999874437565327	0.8819999999995671+10000.0	1.0	0.9999875570695761	0.8830000000000952+10000.0	1.0	0.9999876702543775	0.8840000000002228+10000.0	1.0	0.999987783311227	0.8850000000002014+10000.0	1.0	0.9999878962404134	0.8860000000001338+10000.0	1.0	0.9999880090422248	0.8869999999999346+10000.0	1.0	0.9999881217169484	0.8880000000002756+10000.0	1.0	0.9999882342648699	0.8889999999995901+10000.0	1.0	0.9999883466862748	0.8899999999999504+10000.0	1.0	0.999988458981447	0.8910000000001244+10000.0	1.0	0.9999885711506696	0.8920000000004713+10000.0	1.0	0.999988683194225	0.8929999999999372+10000.0	1.0	0.9999887951123944	0.8939999999999795+10000.0	1.0	0.999988906905458	0.8949999999995747+10000.0	1.0	0.9999890185736956	0.8960000000001407+10000.0	1.0	0.9999891301173852	0.8969999999995376+10000.0	1.0	0.9999892415368047	0.8979999999999863+10000.0	1.0	0.9999893528322304	0.8990000000000703+10000.0	1.0	0.9999894640039382	0.8999999999996755+10000.0	1.0	0.9999895750522028	0.9009999999999595+10000.0	1.0	0.999989685977298	0.902000000000318+10000.0	1.0	0.9999897967794967	0.9030000000003436+10000.0	1.0	0.999989907459071	0.9039999999997903+10000.0	1.0	0.9999900180162921	0.9049999999995431+10000.0	1.0	0.9999901284514301	0.9059999999995758+10000.0	1.0	0.9999902387647545	0.9069999999999181+10000.0	1.0	0.9999903489565335	0.907999999999611+10000.0	1.0	0.9999904590270349	0.9089999999996936+10000.0	1.0	0.9999905689765253	0.9100000000001481+10000.0	1.0	0.9999906788052704	0.9109999999998581+10000.0	1.0	0.9999907885135352	0.9119999999996055+10000.0	1.0	0.9999908981015838	0.9130000000000181+10000.0	1.0	0.9999910075696793	0.9139999999995049+10000.0	1.0	0.9999911169180842	0.9150000000003039+10000.0	1.0	0.9999912261470597	0.9160000000003334+10000.0	1.0	0.9999913352568666	0.91700000000026+10000.0	1.0	0.9999914442477645	0.9180000000004218+10000.0	1.0	0.9999915531200124	0.9189999999997726+10000.0	1.0	0.9999916618738685	0.9199999999999255+10000.0	1.0	0.9999917705095898	0.9210000000000217+10000.0	1.0	0.9999918790274327	0.9219999999997535+10000.0	1.0	0.9999919874276529	0.923000000000359+10000.0	1.0	0.9999920957105051	0.9240000000004935+10000.0	1.0	0.999992203876243	0.9250000000002906+10000.0	1.0	0.9999923119251198	0.9260000000003082+10000.0	1.0	0.9999924198573878	0.9270000000004696+10000.0	1.0	0.9999925276732984	0.9280000000000264+10000.0	1.0	0.9999926353731021	0.9289999999995855+10000.0	1.0	0.999992742957049	0.9300000000000519+10000.0	1.0	0.9999928504253878	0.9309999999994959+10000.0	1.0	0.9999929577783669	0.9320000000003495+10000.0	1.0	0.9999930650162336	0.9330000000000753+10000.0	1.0	0.9999931721392347	0.9340000000004399+10000.0	1.0	0.9999932791476157	0.9350000000002389+10000.0	1.0	0.9999933860416219	0.9360000000004454+10000.0	1.0	0.9999934928214974	0.9370000000000289+10000.0	1.0	0.9999935994874858	0.9380000000000784+10000.0	1.0	0.9999937060398296	0.938999999999613+10000.0	1.0	0.999993812478771	0.9399999999997137+10000.0	1.0	0.9999939188045509	0.9410000000003694+10000.0	1.0	0.9999940250174097	0.9420000000004403+10000.0	1.0	0.9999941311175871	0.9429999999997143+10000.0	1.0	0.9999942371053219	0.9439999999999309+10000.0	1.0	0.9999943429808521	0.9449999999995136+10000.0	1.0	0.9999944487444152	0.9459999999998248+10000.0	1.0	0.9999945543962477	0.9469999999998931+10000.0	1.0	0.9999946599365854	0.9479999999995299+10000.0	1.0	0.9999947653656636	0.9490000000003554+10000.0	1.0	0.9999948706837163	0.9500000000005059+10000.0	1.0	0.9999949758909772	0.9509999999998645+10000.0	1.0	0.9999950809876793	0.9520000000000405+10000.0	1.0	0.9999951859740547	0.9530000000001173+10000.0	1.0	0.9999952908503347	0.9539999999997867+10000.0	1.0	0.9999953956167501	0.9550000000003835+10000.0	1.0	0.9999955002735307	0.9559999999995581+10000.0	1.0	0.9999956048209059	0.9569999999997277+10000.0	1.0	0.9999957092591041	0.9579999999995643+10000.0	1.0	0.9999958135883532	0.9590000000003341+10000.0	1.0	0.9999959178088801	0.9599999999994877+10000.0	1.0	0.9999960219209114	0.9610000000002138+10000.0	1.0	0.9999961259246726	0.9619999999996928+10000.0	1.0	0.9999962298203887	0.9629999999997284+10000.0	1.0	0.9999963336082841	0.9640000000003218+10000.0	1.0	0.9999964372885821	0.9649999999996348+10000.0	1.0	0.9999965408615059	0.9660000000003867+10000.0	1.0	0.9999966443272774	0.967000000000188+10000.0	1.0	0.9999967476861182	0.9680000000000841+10000.0	1.0	0.9999968509382492	0.9690000000002419+10000.0	1.0	0.9999969540838903	0.9699999999999179+10000.0	1.0	0.9999970571232611	0.9709999999995825+10000.0	1.0	0.9999971600565805	0.9719999999998196+10000.0	1.0	0.9999972628840663	0.9730000000002196+10000.0	1.0	0.9999973656059361	0.9740000000004293+10000.0	1.0	0.9999974682224065	0.9750000000001247+10000.0	1.0	0.9999975707336938	0.976000000000068+10000.0	1.0	0.9999976731400132	0.9769999999999179+10000.0	1.0	0.9999977754415796	0.9780000000003677+10000.0	1.0	0.999997877638607	0.9789999999998643+10000.0	1.0	0.999997979731309	0.9800000000000103+10000.0	1.0	0.9999980817198982	0.981000000000108+10000.0	1.0	0.9999981836045869	0.9820000000003927+10000.0	1.0	0.9999982853855864	0.982999999999828+10000.0	1.0	0.9999983870631078	0.984000000000444+10000.0	1.0	0.9999984886373611	0.9849999999996717+10000.0	1.0	0.999998590108556	0.9860000000001461+10000.0	1.0	0.9999986914769012	0.9869999999998449+10000.0	1.0	0.9999987927426053	0.9879999999997175+10000.0	1.0	0.9999988939058758	0.9890000000003812+10000.0	1.0	0.9999989949669197	0.9899999999998953+10000.0	1.0	0.9999990959259435	0.9910000000003247+10000.0	1.0	0.9999991967831529	0.9920000000000256+10000.0	1.0	0.999999297538753	0.9930000000002166+10000.0	1.0	0.9999993981929485	0.9939999999994501+10000.0	1.0	0.9999994987459433	0.9950000000002027+10000.0	1.0	0.9999995991979406	0.9960000000000232+10000.0	1.0	0.9999996995491431	0.9970000000003393+10000.0	1.0	0.999999799799753	0.9979999999998035+10000.0	1.0	0.9999998999499716	0.9989999999998016