armor 0.1 → 0.2

raw patch · 8 files changed

+203/−40 lines, 8 filesdep +hashabledep ~aesondep ~basedep ~bytestringPVP ok

version bump matches the API change (PVP)

Dependencies added: hashable

Dependency ranges changed: aeson, base, bytestring, containers, lens

API changes (from Hackage documentation)

Files

• ChangeLog.md +18/−1
• README.md +42/−0
• armor.cabal +20/−11
• default.nix +63/−0
• src/Armor.hs +55/−23
• test/AppA.lhs +2/−2
• test/AppB.lhs +2/−2
• test/TestAppA.lhs +1/−1

@@ -1,5 +1,22 @@ # Revision history for armor -## 0.1      -- YYYY-mm-dd+## 0.2      -- 2021-05-06++* Widen version bounds+* Support and test through GHC 9+* Expose more customizable test function+* Change default mechanics of FilePath generation++WARNING: Depending on how you use armor this is a potentially+backwards-incompatible change!  As a precaution we are doing a major version+bump even though the change would only technically need a C bump.++The best way to upgrade your app is to upgrade armor in a commit by itself+with no other changes to your app.  That way, if there armor has test case+failures, you know that they are innocuous because no part of the rest of your+app changed functionaly.  So you can simple delete the failing golds,+regenerate them, and check the new ones into source control.++## 0.1      -- 2018-03-14  * First version. Released on an unsuspecting world.

@@ -0,0 +1,42 @@+# Armor++[![Build Status](https://travis-ci.org/mightybyte/armor.svg?branch=master)](https://travis-ci.org/mightybyte/armor)++Armor yourself against backwards-incompatible serializations once and for all.++See the+[ChangeLog](https://github.com/mightybyte/armor/blob/master/ChangeLog.md) for+information about changes between versions.++## Motivation++As almost everyone with significant experience managing production software+systems knows, backwards compatibility is incredibly important for any data that+is persisted by an application. If you make a change to a data structure that is+not backwards compatible with the existing serialized formats, your app will+break as soon as it encounters the existing format. Even if you have 100% test+coverage, your tests still might not catch this problem because it's not a+problem with your app at any single point in time, but a problem with how your+app evolves over time.++More subtly, if you deploy a backwards incompatible migration, your app may+persist some data in the new format before it crashes on the old format. This+can leave your system in the horrible state where not only will it not work with+the new code, but rolling back to the old code will also break because the old+code doesn't support the new serialized format! You have two incomptable+serializations active at the same time!++Proper migration systems can reduce the chances of this problem occurring, but+if your system has any kind of queueing system or message bus, your migrations+might not be applied to in-flight messages. Clearly we need something to help us+protect against this problem.  Enter `armor`.++For an overview of how to use this package, check out the [literate Haskell+tutorial in the test suite](test/AppA.lhs).++## Credits++Inspiration for this package came from [Soostone's safecopy-hunit package](https://github.com/Soostone/safecopy-hunit).++Details were refined in production at [Formation](http://formation.ai/)+(previously Takt).

@@ -1,6 +1,6 @@ name:                armor-version:             0.1-synopsis:            Armor data structures against serialization backwards compatibility problems+version:             0.2+synopsis:            Prevent serialization backwards compatibility problems using golden tests description:         Tests the serialization backwards compatibility of data types by storing                      serialized representations in .test files to be checked into your project's                      version control.@@ -11,16 +11,24 @@ copyright:           Doug Beardsley, Formation Inc. homepage:            https://github.com/mightybyte/armor bug-reports:         https://github.com/mightybyte/armor/issues-category:            Data+category:            Data,Testing build-type:          Simple extra-source-files:  ChangeLog.md cabal-version:       >=1.10 tested-with:   GHC==7.10.3,   GHC==8.0.2,-  GHC==8.2.1,-  GHC==8.4.1+  GHC==8.2.2,+  GHC==8.4.4,+  GHC==8.6.3,+  GHC==8.8.3,+  GHC==8.10.1,+  GHC==9.0.1 +extra-source-files:+  default.nix+  README.md+ Source-repository head   Type:     git   Location: https://github.com/mightybyte/armor.git@@ -33,12 +41,13 @@   ghc-options: -Wall   build-depends:     HUnit      >= 1.5  && < 1.7,-    base       >= 4.6  && < 4.12,-    bytestring >= 0.10 && < 0.11,-    containers >= 0.5  && < 0.6,+    base       >= 4.6  && < 4.16,+    bytestring >= 0.10 && < 0.12,+    containers >= 0.5  && < 0.7,     directory  >= 1.2  && < 1.4,     filepath   >= 1.4  && < 1.5,-    lens       >= 4.16 && < 4.17+    hashable   >= 1.3  && < 1.4,+    lens       >= 4.16 && < 5.1    default-language:    Haskell2010 @@ -56,13 +65,13 @@   ghc-options: -Wall   build-depends:     HUnit,-    aeson        >= 1.0 && < 1.3,+    aeson        >= 1.0 && < 1.6,     armor,     base,     bytestring,     containers,     directory,-    hspec        >= 2.4 && < 2.5,+    hspec        >= 2.4 && < 2.8,     lens,     text         >= 1.2 && < 1.3 

@@ -0,0 +1,63 @@+{ compiler ? "ghc8104"+, rev      ? "282473158cf8fafb736ed2a09f328be8b7ed2efa"+, sha256   ? "1nz2h6dmb6s5zcilb39l116y4dmp1c6w9n110g4dilm6v01linj2"+, pkgs     ?+    import (builtins.fetchTarball {+      url    = "https://github.com/NixOS/nixpkgs/archive/${rev}.tar.gz";+      inherit sha256; }) {+      config.allowBroken = false;+      config.allowUnfree = true;+    }+}:+let gitignoreSrc = import (pkgs.fetchFromGitHub {+      owner = "hercules-ci";+      repo = "gitignore";+      rev = "2ced4519f865341adcb143c5d668f955a2cb997f";+      sha256 = "0fc5bgv9syfcblp23y05kkfnpgh3gssz6vn24frs8dzw39algk2z";+    }) {};++in+pkgs.haskell.packages.${compiler}.developPackage {+  name = builtins.baseNameOf ./.;+  root = gitignoreSrc.gitignoreSource ./.;++  overrides = self: super: with pkgs.haskell.lib; {+    # Don't run a package's test suite+    # foo = dontCheck super.foo;+    #+    # Don't enforce package's version constraints+    # bar = doJailbreak super.bar;+    #+    # Get a specific hackage version straight from hackage. Unlike the above+    # callHackage approach, this will always succeed if the version is on+    # hackage. The downside is that you have to specify the hash manually.+    # aeson = callHackageDirect {+    #   pkg = "aeson";+    #   ver = "1.4.2.0";+    #   sha256 = "0qcczw3l596knj9s4ha07wjspd9wkva0jv4734sv3z3vdad5piqh";+    # } {};+    #+    # To discover more functions that can be used to modify haskell+    # packages, run "nix-repl", type "pkgs.haskell.lib.", then hit+    # <TAB> to get a tab-completed list of functions.+  };+  source-overrides = {+    # Use a specific hackage version using callHackage. Only works if the+    # version you want is in the version of all-cabal-hashes that you have.+    # bytestring = "0.10.8.1";+    #+    # Use a particular commit from github+    # parsec = pkgs.fetchFromGitHub+    #   { owner = "hvr";+    #     repo = "parsec";+    #     rev = "c22d391c046ef075a6c771d05c612505ec2cd0c3";+    #     sha256 = "0phar79fky4yzv4hq28py18i4iw779gp5n327xx76mrj7yj87id3";+    #   };+  };+  modifier = drv: pkgs.haskell.lib.overrideCabal drv (attrs: {+    buildTools = (attrs.buildTools or []) ++ [+      pkgs.haskell.packages.${compiler}.cabal-install+      pkgs.haskell.packages.${compiler}.ghcid+    ];+  });+}

@@ -5,10 +5,14 @@ module Armor   ( Version(..)   , Armored(..)+  , ArmorMode(..)   , ArmorConfig(..)   , defArmorConfig   , testArmor   , testArmorMany+  , testSerialization+  , GoldenTest(..)+  , goldenFilePath   ) where  ------------------------------------------------------------------------------@@ -16,12 +20,15 @@ import           Control.Monad import           Data.ByteString (ByteString) import qualified Data.ByteString as B+import           Data.Char+import           Data.Hashable import           Data.Map        (Map) import qualified Data.Map        as M import           Data.Typeable #if !MIN_VERSION_base(4,8,0) import           Data.Word #endif+import           Numeric import           System.Directory import           System.FilePath import           Test.HUnit.Base@@ -103,11 +110,11 @@ testArmor ac valId val =     TestList [ testIt s | s <- M.toList serializations ]   where-    testIt s = test (testSerialization ac valId val s)+    testIt s = test (testSerialization ac goldenFilePath valId s val)   --------------------------------------------------------------------------------- Same as 'testArmor', but more convenient for testing several values of the+-- | Same as 'testArmor', but more convenient for testing several values of the -- same type. testArmorMany     :: (Eq a, Show a, Typeable a, Armored a)@@ -120,31 +127,37 @@   ------------------------------------------------------------------------------+-- | Lower level assertion function that works for a wider array of test+-- frameworks. testSerialization     :: forall a. (Eq a, Show a, Typeable a, Armored a)     => ArmorConfig+    -> (GoldenTest a -> FilePath)+    -- ^ Customizable location where the serializations will be stored. We+    -- recommend 'goldenFilePath' as a standard out-of-the-box scheme.     -> String-    -> a     -> (String, APrism' ByteString a)+    -> a     -> Assertion-testSerialization ac valId val s@(_,p) = do-    let d = getVersionDir ac val s-        f = getVersionFilename valId curVer-        fp = d </> f-    when (acArmorMode ac /= TestOnly) $ do-      createDirectoryIfMissing True d-      fileExists <- doesFileExist fp-      when (not fileExists) $-        B.writeFile fp (review (clonePrism p) val)-    when (acArmorMode ac /= SaveOnly) $ do-      mapM_ (assertVersionParses d . Version) vs+testSerialization ac makeFilePath valName (sname,p) val = do+    ensureTestFileExists+    when (acArmorMode ac /= SaveOnly) $+      mapM_ (assertVersionParses . Version) vs   where+    makeGT = GoldenTest val valName sname p     curVer :: Version a     curVer = version     vs = reverse [maybe 0 (unVersion curVer -) (acNumVersions ac) .. unVersion curVer]-    assertVersionParses d ver = do-        let f = getVersionFilename valId ver-            fp = d </> f+    ensureTestFileExists = do+      let fp = acStoreDir ac </> makeFilePath (makeGT curVer)+          d = dropFileName fp+      when (acArmorMode ac /= TestOnly) $ do+        createDirectoryIfMissing True d+        fileExists <- doesFileExist fp+        when (not fileExists) $+          B.writeFile fp (review (clonePrism p) val)+    assertVersionParses ver = do+        let fp = acStoreDir ac </> makeFilePath (makeGT ver)         exists <- doesFileExist fp         if exists           then do bs <- B.readFile fp@@ -155,12 +168,31 @@                     Just v -> assertEqual ("File parsed but values didn't match: " ++ fp) val v           else putStrLn $ "\nSkipping missing file " ++ fp - -------------------------------------------------------------------------------getVersionFilename :: String -> Version a -> String-getVersionFilename valId ver = printf "%s-%03d.test" valId (unVersion ver)-+-- | Data structure that holds all the values needed for a golden test+data GoldenTest a = GoldenTest+  { gtTestVal :: a+  , gtValName :: String+  , gtSerializationName :: String+  , gtPrism :: APrism' ByteString a+  , gtVersion :: Version a+  }  -------------------------------------------------------------------------------getVersionDir :: Typeable a => ArmorConfig -> a -> (FilePath, t) -> FilePath-getVersionDir ac val (nm,_) = acStoreDir ac </> show (typeOf val) </> nm+-- | Constructs the FilePath where the serialization will be stored (relative to+-- the base directory defined in ArmorConfig).+--+-- This function uses typeOf as a part of the directory hierarchy to+-- disambiguate tests for different data types. typeOf can contain single+-- quotes, spaces, and parenthesis in the case of type constructors that have+-- type variables so we only take the first alphanumeric characters so that the+-- paths will be meaningful to humans and then add four characters of the type's+-- hash for disambiguation.+goldenFilePath :: Typeable a => GoldenTest a -> FilePath+goldenFilePath gt =+    (concat [takeWhile isAlpha ty, "-", h]) </>+    gtSerializationName gt </>+    printf "%s-%03d.test" (gtValName gt) (unVersion $ gtVersion gt)+  where+    ty = show $ typeOf $ gtTestVal gt+    h = take 4 $ showHex (abs $ hash ty) ""

@@ -69,7 +69,7 @@  This tutorial is a part of the armor test suite, and since we don't want armor to depend on any specific serialization packages we're using Show as an example-of how armor supports any number of serialiaztions.+of how armor supports any number of serializations.  A serialization is simply a pair of a serialization function that converts your data type to ByteString and a deserialization function that converts a@@ -88,4 +88,4 @@ Once you have defined your `Armored` instances, the next step is to define your tests.  To see an example of that go here: -https://github.com/TaktInc/armor/blob/master/test/TestAppA.lhs+https://github.com/mightybyte/armor/blob/master/test/TestAppA.lhs

@@ -34,7 +34,7 @@ > instance FromJSON EmployeeLevel > instance ToJSON EmployeeLevel -We update the `Armored` instance to version 1. If you forget to updated the+We update the `Armored` instance to version 1. If you forget to update the version, the armor tests should still fail because the existing version 0 serialization files will not be overwritten. @@ -57,4 +57,4 @@  Now go to the TestAppB module to see how we update the tests for the new field. -https://github.com/TaktInc/armor/blob/master/test/TestAppB.lhs+https://github.com/mightybyte/armor/blob/master/test/TestAppB.lhs

@@ -24,4 +24,4 @@ that they can be correctly parsed.  Eventually you'll need to change the data type in some way.  See this file for a case study of how that might play out: -https://github.com/TaktInc/armor/blob/master/test/AppB.lhs+https://github.com/mightybyte/armor/blob/master/test/AppB.lhs