vflow-types (empty) → 0.1
raw patch · 10 files changed
+795/−0 lines, 10 filesdep +QuickCheckdep +aesondep +base
Dependencies added: QuickCheck, aeson, base, bytestring, ip, json-alt, json-autotype, neat-interpolation, quickcheck-classes, scientific, text, vflow-types
Files
- CHANGELOG.md +12/−0
- LICENSE +29/−0
- README.md +6/−0
- src/VFlow/Types/IpFix.hs +93/−0
- src/VFlow/Types/NetFlow5.hs +91/−0
- src/VFlow/Types/NetFlow9.hs +91/−0
- src/VFlow/Types/SFlow.hs +198/−0
- test/Laws.hs +101/−0
- test/UnitTests.hs +63/−0
- vflow-types.cabal +111/−0
+ CHANGELOG.md view
@@ -0,0 +1,12 @@+# Changelog++`vflow-types` uses [PVP Versioning][1].+The changelog is available [on GitHub][2].++0.0.0+=====++* Initially created.++[1]: https://pvp.haskell.org+[2]: https://github.com/chessai/vflow-types/releases
+ LICENSE view
@@ -0,0 +1,29 @@+BSD 3-Clause License++Copyright (c) 2019, chessai+All rights reserved.++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++1. Redistributions of source code must retain the above copyright notice, this+ list of conditions and the following disclaimer.++2. Redistributions in binary form must reproduce the above copyright notice,+ this list of conditions and the following disclaimer in the documentation+ and/or other materials provided with the distribution.++3. Neither the name of the copyright holder nor the names of its+ contributors may be used to endorse or promote products derived from+ this software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ README.md view
@@ -0,0 +1,6 @@+# vflow-types++[](https://hackage.haskell.org/package/vflow-types)+[](LICENSE)++types for ingesting [vflow](https://github.com/VerizonDigital/vflow)-generated data. All types have ToJSON/FromJSON instances. `vflow-types` can handle IPFIX, sFlow, Netflow v5, and Netflow v9.
+ src/VFlow/Types/IpFix.hs view
@@ -0,0 +1,93 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE DeriveGeneric #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE RecordWildCards #-}+{-# LANGUAGE TypeOperators #-}++module VFlow.Types.IpFix+ ( IpFix(..)+ , DataSetsEltElt(..)+ , Header(..)+ , HexInt(..)+ )+ where++import Control.Monad (mzero)+import Data.Aeson(Value(..), FromJSON(..), ToJSON(..), pairs, (.:), (.=), object)+import Data.Aeson.AutoType.Alternative+import Data.Int (Int64)+import Data.Monoid((<>))+import Net.IPv4 (IPv4)+import Text.Read (readMaybe)+import qualified Data.Scientific as Sci+import qualified Data.Text+import qualified GHC.Generics++newtype HexInt = HexInt Int64+ deriving (Eq, Show)++instance FromJSON HexInt where+ parseJSON (String s) = case readMaybe (Data.Text.unpack s) of+ Nothing -> mzero+ Just x -> pure (HexInt x)+ parseJSON (Number s) = if Sci.isInteger s+ then case Sci.toBoundedInteger s of+ Nothing -> mzero+ Just x -> pure (HexInt x)+ else mzero+ parseJSON _ = mzero++instance ToJSON HexInt where+ toJSON (HexInt x) = toJSON x+ toEncoding (HexInt x) = toEncoding x++data DataSetsEltElt = DataSetsEltElt { + dataSetsEltEltV :: !(IPv4 :|: HexInt),+ dataSetsEltEltI :: !Int+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON DataSetsEltElt where+ parseJSON (Object v) = DataSetsEltElt <$> v .: "V" <*> v .: "I"+ parseJSON _ = mzero+++instance ToJSON DataSetsEltElt where+ toJSON (DataSetsEltElt {..}) = object ["V" .= dataSetsEltEltV, "I" .= dataSetsEltEltI]+ toEncoding (DataSetsEltElt {..}) = pairs ("V" .= dataSetsEltEltV<>"I" .= dataSetsEltEltI)+++data Header = Header { + headerLength :: !Int,+ headerSequenceNo :: !Int,+ headerExportTime :: !Int,+ headerVersion :: !Int,+ headerDomainID :: !Int+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON Header where+ parseJSON (Object v) = Header <$> v .: "Length" <*> v .: "SequenceNo" <*> v .: "ExportTime" <*> v .: "Version" <*> v .: "DomainID"+ parseJSON _ = mzero+++instance ToJSON Header where+ toJSON (Header {..}) = object ["Length" .= headerLength, "SequenceNo" .= headerSequenceNo, "ExportTime" .= headerExportTime, "Version" .= headerVersion, "DomainID" .= headerDomainID]+ toEncoding (Header {..}) = pairs ("Length" .= headerLength<>"SequenceNo" .= headerSequenceNo<>"ExportTime" .= headerExportTime<>"Version" .= headerVersion<>"DomainID" .= headerDomainID)+++data IpFix = IpFix { + ipFixAgentID :: !IPv4,+ ipFixHeader :: !Header,+ ipFixDataSets :: [[DataSetsEltElt]]+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON IpFix where+ parseJSON (Object v) = IpFix <$> v .: "AgentID" <*> v .: "Header" <*> v .: "DataSets"+ parseJSON _ = mzero+++instance ToJSON IpFix where+ toJSON (IpFix {..}) = object ["AgentID" .= ipFixAgentID, "Header" .= ipFixHeader, "DataSets" .= ipFixDataSets]+ toEncoding (IpFix {..}) = pairs ("AgentID" .= ipFixAgentID<>"Header" .= ipFixHeader<>"DataSets" .= ipFixDataSets)
+ src/VFlow/Types/NetFlow5.hs view
@@ -0,0 +1,91 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE RecordWildCards #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE TypeOperators #-}+{-# LANGUAGE DeriveGeneric #-}++module VFlow.Types.NetFlow5+ ( NetFlow(..)+ , FlowsElt(..)+ , Header(..)+ ) where++import Control.Monad (mzero)+import Data.Aeson(Value(..), FromJSON(..), ToJSON(..), pairs, (.:), (.=), object)+import Data.Monoid((<>))+import qualified GHC.Generics+import Net.Types (IPv4)++data FlowsElt = FlowsElt { + flowsEltDstAsNum :: !Int,+ flowsEltStartTime :: !Int,+ flowsEltL3Octets :: !Int,+ flowsEltTCPFlags :: !Int,+ flowsEltDstAddr :: !IPv4,+ flowsEltTos :: !Int,+ flowsEltPadding1 :: !Int,+ flowsEltNextHop :: !IPv4,+ flowsEltSrcPort :: !Int,+ flowsEltInput :: !Int,+ flowsEltPktCount :: !Int,+ flowsEltOutput :: !Int,+ flowsEltDstMask :: !Int,+ flowsEltDstPort :: !Int,+ flowsEltEndTime :: !Int,+ flowsEltPadding2 :: !Int,+ flowsEltSrcMask :: !Int,+ flowsEltSrcAsNum :: !Int,+ flowsEltSrcAddr :: !IPv4,+ flowsEltProtType :: !Int+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON FlowsElt where+ parseJSON (Object v) = FlowsElt <$> v .: "DstAsNum" <*> v .: "StartTime" <*> v .: "L3Octets" <*> v .: "TCPFlags" <*> v .: "DstAddr" <*> v .: "Tos" <*> v .: "Padding1" <*> v .: "NextHop" <*> v .: "SrcPort" <*> v .: "Input" <*> v .: "PktCount" <*> v .: "Output" <*> v .: "DstMask" <*> v .: "DstPort" <*> v .: "EndTime" <*> v .: "Padding2" <*> v .: "SrcMask" <*> v .: "SrcAsNum" <*> v .: "SrcAddr" <*> v .: "ProtType"+ parseJSON _ = mzero+++instance ToJSON FlowsElt where+ toJSON (FlowsElt {..}) = object ["DstAsNum" .= flowsEltDstAsNum, "StartTime" .= flowsEltStartTime, "L3Octets" .= flowsEltL3Octets, "TCPFlags" .= flowsEltTCPFlags, "DstAddr" .= flowsEltDstAddr, "Tos" .= flowsEltTos, "Padding1" .= flowsEltPadding1, "NextHop" .= flowsEltNextHop, "SrcPort" .= flowsEltSrcPort, "Input" .= flowsEltInput, "PktCount" .= flowsEltPktCount, "Output" .= flowsEltOutput, "DstMask" .= flowsEltDstMask, "DstPort" .= flowsEltDstPort, "EndTime" .= flowsEltEndTime, "Padding2" .= flowsEltPadding2, "SrcMask" .= flowsEltSrcMask, "SrcAsNum" .= flowsEltSrcAsNum, "SrcAddr" .= flowsEltSrcAddr, "ProtType" .= flowsEltProtType]+ toEncoding (FlowsElt {..}) = pairs ("DstAsNum" .= flowsEltDstAsNum<>"StartTime" .= flowsEltStartTime<>"L3Octets" .= flowsEltL3Octets<>"TCPFlags" .= flowsEltTCPFlags<>"DstAddr" .= flowsEltDstAddr<>"Tos" .= flowsEltTos<>"Padding1" .= flowsEltPadding1<>"NextHop" .= flowsEltNextHop<>"SrcPort" .= flowsEltSrcPort<>"Input" .= flowsEltInput<>"PktCount" .= flowsEltPktCount<>"Output" .= flowsEltOutput<>"DstMask" .= flowsEltDstMask<>"DstPort" .= flowsEltDstPort<>"EndTime" .= flowsEltEndTime<>"Padding2" .= flowsEltPadding2<>"SrcMask" .= flowsEltSrcMask<>"SrcAsNum" .= flowsEltSrcAsNum<>"SrcAddr" .= flowsEltSrcAddr<>"ProtType" .= flowsEltProtType)+++data Header = Header { + headerUNIXSecs :: !Int,+ headerCount :: !Int,+ headerEngType :: !Int,+ headerVersion :: !Int,+ headerSysUpTimeMSecs :: !Int,+ headerUNIXNSecs :: !Int,+ headerSmpInt :: !Int,+ headerSeqNum :: !Int,+ headerEngID :: !Int+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON Header where+ parseJSON (Object v) = Header <$> v .: "UNIXSecs" <*> v .: "Count" <*> v .: "EngType" <*> v .: "Version" <*> v .: "SysUpTimeMSecs" <*> v .: "UNIXNSecs" <*> v .: "SmpInt" <*> v .: "SeqNum" <*> v .: "EngID"+ parseJSON _ = mzero+++instance ToJSON Header where+ toJSON (Header {..}) = object ["UNIXSecs" .= headerUNIXSecs, "Count" .= headerCount, "EngType" .= headerEngType, "Version" .= headerVersion, "SysUpTimeMSecs" .= headerSysUpTimeMSecs, "UNIXNSecs" .= headerUNIXNSecs, "SmpInt" .= headerSmpInt, "SeqNum" .= headerSeqNum, "EngID" .= headerEngID]+ toEncoding (Header {..}) = pairs ("UNIXSecs" .= headerUNIXSecs<>"Count" .= headerCount<>"EngType" .= headerEngType<>"Version" .= headerVersion<>"SysUpTimeMSecs" .= headerSysUpTimeMSecs<>"UNIXNSecs" .= headerUNIXNSecs<>"SmpInt" .= headerSmpInt<>"SeqNum" .= headerSeqNum<>"EngID" .= headerEngID)+++data NetFlow = NetFlow { + netflowAgentID :: !IPv4,+ netflowHeader :: !Header,+ netflowFlows :: [FlowsElt]+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON NetFlow where+ parseJSON (Object v) = NetFlow <$> v .: "AgentID" <*> v .: "Header" <*> v .: "Flows"+ parseJSON _ = mzero+++instance ToJSON NetFlow where+ toJSON (NetFlow {..}) = object ["AgentID" .= netflowAgentID, "Header" .= netflowHeader, "Flows" .= netflowFlows]+ toEncoding (NetFlow {..}) = pairs ("AgentID" .= netflowAgentID<>"Header" .= netflowHeader<>"Flows" .= netflowFlows)+
+ src/VFlow/Types/NetFlow9.hs view
@@ -0,0 +1,91 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE DeriveGeneric #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE RecordWildCards #-}+{-# LANGUAGE TypeOperators #-}++module VFlow.Types.NetFlow9+ ( NetFlow(..)+ , Header(..)+ , DataSetsEltElt(..)+ , HexInt(..)+ ) where++import Control.Monad (mzero)+import Data.Aeson(Value(..), FromJSON(..), ToJSON(..), pairs, (.:), (.=), object)+import Data.Aeson.AutoType.Alternative+import Data.Int (Int64)+import Data.Monoid((<>))+import Net.Types (IPv4)+import Text.Read (readMaybe)+import qualified Data.Scientific as Sci+import qualified Data.Text+import qualified GHC.Generics++newtype HexInt = HexInt Int64+ deriving (Eq, Show)++instance FromJSON HexInt where+ parseJSON (String s) = case readMaybe (Data.Text.unpack s) of+ Nothing -> mzero+ Just x -> pure (HexInt x)+ parseJSON (Number s) = if Sci.isInteger s+ then case Sci.toBoundedInteger s of+ Nothing -> mzero+ Just x -> pure (HexInt x)+ else mzero+ parseJSON _ = mzero++instance ToJSON HexInt where+ toJSON (HexInt x) = toJSON x+ toEncoding (HexInt x) = toEncoding x++data DataSetsEltElt = DataSetsEltElt { + dataSetsEltEltV :: !(IPv4 :|: HexInt),+ dataSetsEltEltI :: !Int+ } deriving (Show,Eq,GHC.Generics.Generic)++instance FromJSON DataSetsEltElt where+ parseJSON (Object v) = DataSetsEltElt <$> v .: "V" <*> v .: "I"+ parseJSON _ = mzero++instance ToJSON DataSetsEltElt where+ toJSON (DataSetsEltElt {..}) = object ["V" .= dataSetsEltEltV, "I" .= dataSetsEltEltI]+ toEncoding (DataSetsEltElt {..}) = pairs ("V" .= dataSetsEltEltV<>"I" .= dataSetsEltEltI)+++data Header = Header { + headerUNIXSecs :: !Int,+ headerSrcID :: !Int,+ headerCount :: !Int,+ headerSysUpTime :: !Int,+ headerVersion :: !Int,+ headerSeqNum :: !Int+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON Header where+ parseJSON (Object v) = Header <$> v .: "UNIXSecs" <*> v .: "SrcID" <*> v .: "Count" <*> v .: "SysUpTime" <*> v .: "Version" <*> v .: "SeqNum"+ parseJSON _ = mzero+++instance ToJSON Header where+ toJSON (Header {..}) = object ["UNIXSecs" .= headerUNIXSecs, "SrcID" .= headerSrcID, "Count" .= headerCount, "SysUpTime" .= headerSysUpTime, "Version" .= headerVersion, "SeqNum" .= headerSeqNum]+ toEncoding (Header {..}) = pairs ("UNIXSecs" .= headerUNIXSecs<>"SrcID" .= headerSrcID<>"Count" .= headerCount<>"SysUpTime" .= headerSysUpTime<>"Version" .= headerVersion<>"SeqNum" .= headerSeqNum)+++data NetFlow = NetFlow { + netflowAgentID :: !IPv4,+ netflowHeader :: !Header,+ netflowDataSets :: [[DataSetsEltElt]]+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON NetFlow where+ parseJSON (Object v) = NetFlow <$> v .: "AgentID" <*> v .: "Header" <*> v .: "DataSets"+ parseJSON _ = mzero+++instance ToJSON NetFlow where+ toJSON (NetFlow {..}) = object ["AgentID" .= netflowAgentID, "Header" .= netflowHeader, "DataSets" .= netflowDataSets]+ toEncoding (NetFlow {..}) = pairs ("AgentID" .= netflowAgentID<>"Header" .= netflowHeader<>"DataSets" .= netflowDataSets)
+ src/VFlow/Types/SFlow.hs view
@@ -0,0 +1,198 @@+{-# LANGUAGE BangPatterns #-}+{-# LANGUAGE DeriveGeneric #-}+{-# LANGUAGE OverloadedStrings #-}+{-# LANGUAGE RecordWildCards #-}+{-# LANGUAGE TypeOperators #-}++module VFlow.Types.SFlow+ ( SFlow(..)+ , ExtRouter(..)+ , ExtSwitch(..)+ , L2(..)+ , L3(..)+ , L4(..)+ , RawHeader(..)+ , Records(..)+ , SamplesElt(..)+ ) where++import Control.Monad (mzero)+import Data.Aeson (Value(..), FromJSON(..), ToJSON(..), pairs, (.:), (.=), object)+import Net.Types (IPv4, Mac)+import qualified GHC.Generics++data ExtRouter = ExtRouter { + extRouterNextHop :: !IPv4,+ extRouterDstMask :: !Int,+ extRouterSrcMask :: !Int+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON ExtRouter where+ parseJSON (Object v) = ExtRouter <$> v .: "NextHop" <*> v .: "DstMask" <*> v .: "SrcMask"+ parseJSON _ = mzero+++instance ToJSON ExtRouter where+ toJSON (ExtRouter {..}) = object ["NextHop" .= extRouterNextHop, "DstMask" .= extRouterDstMask, "SrcMask" .= extRouterSrcMask]+ toEncoding (ExtRouter {..}) = pairs ("NextHop" .= extRouterNextHop<>"DstMask" .= extRouterDstMask<>"SrcMask" .= extRouterSrcMask)+++data ExtSwitch = ExtSwitch { + extSwitchDstPriority :: !Int,+ extSwitchSrcVlan :: !Int,+ extSwitchSrcPriority :: !Int,+ extSwitchDstVlan :: !Int+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON ExtSwitch where+ parseJSON (Object v) = ExtSwitch <$> v .: "DstPriority" <*> v .: "SrcVlan" <*> v .: "SrcPriority" <*> v .: "DstVlan"+ parseJSON _ = mzero+++instance ToJSON ExtSwitch where+ toJSON (ExtSwitch {..}) = object ["DstPriority" .= extSwitchDstPriority, "SrcVlan" .= extSwitchSrcVlan, "SrcPriority" .= extSwitchSrcPriority, "DstVlan" .= extSwitchDstVlan]+ toEncoding (ExtSwitch {..}) = pairs ("DstPriority" .= extSwitchDstPriority<>"SrcVlan" .= extSwitchSrcVlan<>"SrcPriority" .= extSwitchSrcPriority<>"DstVlan" .= extSwitchDstVlan)+++data L2 = L2 { + l2Vlan :: !Int,+ l2EtherType :: !Int,+ l2DstMAC :: !Mac,+ l2SrcMAC :: !Mac+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON L2 where+ parseJSON (Object v) = L2 <$> v .: "Vlan" <*> v .: "EtherType" <*> v .: "DstMAC" <*> v .: "SrcMAC"+ parseJSON _ = mzero+++instance ToJSON L2 where+ toJSON (L2 {..}) = object ["Vlan" .= l2Vlan, "EtherType" .= l2EtherType, "DstMAC" .= l2DstMAC, "SrcMAC" .= l2SrcMAC]+ toEncoding (L2 {..}) = pairs ("Vlan" .= l2Vlan<>"EtherType" .= l2EtherType<>"DstMAC" .= l2DstMAC<>"SrcMAC" .= l2SrcMAC)+++data L3 = L3 { + l3TTL :: !Int,+ l3Flags :: !Int,+ l3TotalLen :: !Int,+ l3Checksum :: !Int,+ l3TOS :: !Int,+ l3Dst :: !IPv4,+ l3Protocol :: !Int,+ l3Src :: !IPv4,+ l3Version :: !Int,+ l3ID :: !Int,+ l3FragOff :: !Int+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON L3 where+ parseJSON (Object v) = L3 <$> v .: "TTL" <*> v .: "Flags" <*> v .: "TotalLen" <*> v .: "Checksum" <*> v .: "TOS" <*> v .: "Dst" <*> v .: "Protocol" <*> v .: "Src" <*> v .: "Version" <*> v .: "ID" <*> v .: "FragOff"+ parseJSON _ = mzero+++instance ToJSON L3 where+ toJSON (L3 {..}) = object ["TTL" .= l3TTL, "Flags" .= l3Flags, "TotalLen" .= l3TotalLen, "Checksum" .= l3Checksum, "TOS" .= l3TOS, "Dst" .= l3Dst, "Protocol" .= l3Protocol, "Src" .= l3Src, "Version" .= l3Version, "ID" .= l3ID, "FragOff" .= l3FragOff]+ toEncoding (L3 {..}) = pairs ("TTL" .= l3TTL<>"Flags" .= l3Flags<>"TotalLen" .= l3TotalLen<>"Checksum" .= l3Checksum<>"TOS" .= l3TOS<>"Dst" .= l3Dst<>"Protocol" .= l3Protocol<>"Src" .= l3Src<>"Version" .= l3Version<>"ID" .= l3ID<>"FragOff" .= l3FragOff)+++data L4 = L4 { + l4Flags :: !Int,+ l4DataOffset :: !Int,+ l4SrcPort :: !Int,+ l4Reserved :: !Int,+ l4DstPort :: !Int+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON L4 where+ parseJSON (Object v) = L4 <$> v .: "Flags" <*> v .: "DataOffset" <*> v .: "SrcPort" <*> v .: "Reserved" <*> v .: "DstPort"+ parseJSON _ = mzero+++instance ToJSON L4 where+ toJSON (L4 {..}) = object ["Flags" .= l4Flags, "DataOffset" .= l4DataOffset, "SrcPort" .= l4SrcPort, "Reserved" .= l4Reserved, "DstPort" .= l4DstPort]+ toEncoding (L4 {..}) = pairs ("Flags" .= l4Flags<>"DataOffset" .= l4DataOffset<>"SrcPort" .= l4SrcPort<>"Reserved" .= l4Reserved<>"DstPort" .= l4DstPort)+++data RawHeader = RawHeader { + rawHeaderL2 :: !L2,+ rawHeaderL3 :: !L3,+ rawHeaderL4 :: !L4+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON RawHeader where+ parseJSON (Object v) = RawHeader <$> v .: "L2" <*> v .: "L3" <*> v .: "L4"+ parseJSON _ = mzero+++instance ToJSON RawHeader where+ toJSON (RawHeader {..}) = object ["L2" .= rawHeaderL2, "L3" .= rawHeaderL3, "L4" .= rawHeaderL4]+ toEncoding (RawHeader {..}) = pairs ("L2" .= rawHeaderL2<>"L3" .= rawHeaderL3<>"L4" .= rawHeaderL4)+++data Records = Records { + recordsExtRouter :: !ExtRouter,+ recordsExtSwitch :: !ExtSwitch,+ recordsRawHeader :: !RawHeader+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON Records where+ parseJSON (Object v) = Records <$> v .: "ExtRouter" <*> v .: "ExtSwitch" <*> v .: "RawHeader"+ parseJSON _ = mzero+++instance ToJSON Records where+ toJSON (Records {..}) = object ["ExtRouter" .= recordsExtRouter, "ExtSwitch" .= recordsExtSwitch, "RawHeader" .= recordsRawHeader]+ toEncoding (Records {..}) = pairs ("ExtRouter" .= recordsExtRouter<>"ExtSwitch" .= recordsExtSwitch<>"RawHeader" .= recordsRawHeader)+++data SamplesElt = SamplesElt { + samplesEltDrops :: !Int,+ samplesEltSourceID :: !Int,+ samplesEltRecords :: !Records,+ samplesEltInput :: !Int,+ samplesEltSequenceNo :: !Int,+ samplesEltSamplingRate :: !Int,+ samplesEltOutput :: !Int,+ samplesEltRecordsNo :: !Int,+ samplesEltSamplePool :: !Int+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON SamplesElt where+ parseJSON (Object v) = SamplesElt <$> v .: "Drops" <*> v .: "SourceID" <*> v .: "Records" <*> v .: "Input" <*> v .: "SequenceNo" <*> v .: "SamplingRate" <*> v .: "Output" <*> v .: "RecordsNo" <*> v .: "SamplePool"+ parseJSON _ = mzero+++instance ToJSON SamplesElt where+ toJSON (SamplesElt {..}) = object ["Drops" .= samplesEltDrops, "SourceID" .= samplesEltSourceID, "Records" .= samplesEltRecords, "Input" .= samplesEltInput, "SequenceNo" .= samplesEltSequenceNo, "SamplingRate" .= samplesEltSamplingRate, "Output" .= samplesEltOutput, "RecordsNo" .= samplesEltRecordsNo, "SamplePool" .= samplesEltSamplePool]+ toEncoding (SamplesElt {..}) = pairs ("Drops" .= samplesEltDrops<>"SourceID" .= samplesEltSourceID<>"Records" .= samplesEltRecords<>"Input" .= samplesEltInput<>"SequenceNo" .= samplesEltSequenceNo<>"SamplingRate" .= samplesEltSamplingRate<>"Output" .= samplesEltOutput<>"RecordsNo" .= samplesEltRecordsNo<>"SamplePool" .= samplesEltSamplePool)+++data SFlow = SFlow { + sflowIPAddress :: !IPv4,+ sflowAgentSubID :: !Int,+ sflowIPVersion :: !Int,+ sflowSequenceNo :: !Int,+ sflowSysUpTime :: !Int,+ sflowSamplesNo :: !Int,+ sflowVersion :: !Int,+ sflowSamples :: [SamplesElt]+ } deriving (Show,Eq,GHC.Generics.Generic)+++instance FromJSON SFlow where+ parseJSON (Object v) = SFlow <$> v .: "IPAddress" <*> v .: "AgentSubID" <*> v .: "IPVersion" <*> v .: "SequenceNo" <*> v .: "SysUpTime" <*> v .: "SamplesNo" <*> v .: "Version" <*> v .: "Samples"+ parseJSON _ = mzero+++instance ToJSON SFlow where+ toJSON (SFlow {..}) = object ["IPAddress" .= sflowIPAddress, "AgentSubID" .= sflowAgentSubID, "IPVersion" .= sflowIPVersion, "SequenceNo" .= sflowSequenceNo, "SysUpTime" .= sflowSysUpTime, "SamplesNo" .= sflowSamplesNo, "Version" .= sflowVersion, "Samples" .= sflowSamples]+ toEncoding (SFlow {..}) = pairs ("IPAddress" .= sflowIPAddress<>"AgentSubID" .= sflowAgentSubID<>"IPVersion" .= sflowIPVersion<>"SequenceNo" .= sflowSequenceNo<>"SysUpTime" .= sflowSysUpTime<>"SamplesNo" .= sflowSamplesNo<>"Version" .= sflowVersion<>"Samples" .= sflowSamples)+
+ test/Laws.hs view
@@ -0,0 +1,101 @@+{-# language TypeOperators #-}++module Main (main) where++import Data.Aeson.AutoType.Alternative+import Data.Proxy (Proxy(..))+import Data.Word (Word8)+import Net.Types (IPv4,Mac)+import Test.QuickCheck+import Test.QuickCheck.Classes+import VFlow.Types.IpFix (IpFix(..))+import VFlow.Types.SFlow (SFlow(..))+import qualified Net.IPv4 as I4+import qualified Net.Mac as Mac+import qualified VFlow.Types.IpFix as I+import qualified VFlow.Types.NetFlow5 as N5+import qualified VFlow.Types.NetFlow9 as N9+import qualified VFlow.Types.SFlow as S++main :: IO ()+main = lawsCheckMany laws++laws :: [(String,[Laws])]+laws =+ [ ("IPFIX", [jsonLaws (Proxy :: Proxy IpFix)])+ , ("sflow", [jsonLaws (Proxy :: Proxy SFlow)])+ , ("Netflow v5", [jsonLaws (Proxy :: Proxy N5.NetFlow)])+ , ("Netflow v9", [jsonLaws (Proxy :: Proxy N9.NetFlow)])+ ]++instance (Arbitrary a, Arbitrary b) => Arbitrary (a :|: b) where+ arbitrary = frequency [ (1, AltLeft <$> arbitrary), (1, AltRight <$> arbitrary) ]++instance Arbitrary IpFix where+ arbitrary = IpFix <$> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary I.Header where+ arbitrary = I.Header <$> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary I.DataSetsEltElt where+ arbitrary = I.DataSetsEltElt <$> arbitrary <*> arbitrary++instance Arbitrary IPv4 where+ arbitrary = I4.ipv4 <$> word8 <*> word8 <*> word8 <*> word8++instance Arbitrary Mac where+ arbitrary = Mac.fromOctets <$> word8 <*> word8 <*> word8 <*> word8 <*> word8 <*> word8+ +instance Arbitrary I.HexInt where+ arbitrary = I.HexInt <$> arbitrary++instance Arbitrary S.ExtRouter where+ arbitrary = S.ExtRouter <$> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary S.ExtSwitch where+ arbitrary = S.ExtSwitch <$> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary S.L2 where+ arbitrary = S.L2 <$> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary S.L3 where+ arbitrary = S.L3 <$> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary S.L4 where+ arbitrary = S.L4 <$> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary S.RawHeader where+ arbitrary = S.RawHeader <$> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary S.Records where+ arbitrary = S.Records <$> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary S.SamplesElt where+ arbitrary = S.SamplesElt <$> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary S.SFlow where+ arbitrary = S.SFlow <$> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary++word8 :: Gen Word8+word8 = arbitrary `suchThat` (\x -> x >= 0 && x <= 255)++instance Arbitrary N5.FlowsElt where+ arbitrary = N5.FlowsElt <$> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary N5.Header where+ arbitrary = N5.Header <$> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary N5.NetFlow where+ arbitrary = N5.NetFlow <$> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary N9.HexInt where+ arbitrary = N9.HexInt <$> arbitrary++instance Arbitrary N9.DataSetsEltElt where+ arbitrary = N9.DataSetsEltElt <$> arbitrary <*> arbitrary++instance Arbitrary N9.Header where+ arbitrary = N9.Header <$> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary <*> arbitrary++instance Arbitrary N9.NetFlow where+ arbitrary = N9.NetFlow <$> arbitrary <*> arbitrary <*> arbitrary
+ test/UnitTests.hs view
@@ -0,0 +1,63 @@+{-# language QuasiQuotes #-}+{-# language ScopedTypeVariables #-}++module Main (main) where++import Data.ByteString (ByteString)+import NeatInterpolation (text)+import System.Exit (exitFailure, exitSuccess)+import VFlow.Types.IpFix+import VFlow.Types.SFlow+import qualified VFlow.Types.NetFlow5 as N5+import qualified VFlow.Types.NetFlow9 as N9+import Data.Foldable (fold)+import qualified Data.Aeson as Aeson+import qualified Data.Text.Encoding as TE++main :: IO ()+main = exit $ mconcat [ ipFix, sflow, netflow5 ]++ipFix :: Exit+ipFix = eitherToExit (Aeson.eitherDecodeStrict ipfixJson :: Either String IpFix)++sflow :: Exit+sflow = eitherToExit (Aeson.eitherDecodeStrict sflowJson :: Either String SFlow)++netflow5 :: Exit+netflow5 = eitherToExit (Aeson.eitherDecodeStrict netflow5Json :: Either String N5.NetFlow)++netflow9 :: Exit+netflow9 = eitherToExit (Aeson.eitherDecodeStrict netflow9Json :: Either String N9.NetFlow)++eitherToExit :: Either a b -> Exit+eitherToExit = either (const ExitFail) (const ExitSucc)++exit :: Exit -> IO ()+exit ExitFail = exitFailure+exit ExitSucc = exitSuccess+ +data Exit = ExitFail | ExitSucc++instance Semigroup Exit where+ ExitFail <> _ = ExitFail+ _ <> ExitFail = ExitFail+ _ <> _ = ExitSucc++instance Monoid Exit where+ mempty = ExitSucc++sflowJson :: ByteString+sflowJson = TE.encodeUtf8 $+ [text|{"Version":5,"IPVersion":1,"AgentSubID":5,"SequenceNo":37591,"SysUpTime":3287084017,"SamplesNo":1,"Samples":[{"SequenceNo":1530345639,"SourceID":0,"SamplingRate":4096,"SamplePool":1938456576,"Drops":0,"Input":536,"Output":728,"RecordsNo":3,"Records":{"ExtRouter":{"NextHop":"115.131.251.90","SrcMask":24,"DstMask":14},"ExtSwitch":{"SrcVlan":0,"SrcPriority":0,"DstVlan":0,"DstPriority":0},"RawHeader":{"L2":{"SrcMAC":"58:00:bb:e7:57:6f","DstMAC":"f4:a7:39:44:a8:27","Vlan":0,"EtherType":2048},"L3":{"Version":4,"TOS":0,"TotalLen":1452,"ID":13515,"Flags":0,"FragOff":0,"TTL":62,"Protocol":6,"Checksum":8564,"Src":"10.1.8.5","Dst":"161.140.24.181"},"L4":{"SrcPort":443,"DstPort":56521,"DataOffset":5,"Reserved":0,"Flags":16}}}}],"IPAddress":"192.168.10.0"}|]++ipfixJson :: ByteString+ipfixJson = TE.encodeUtf8 $+ [text|{"AgentID":"192.168.21.15","Header":{"Version":10,"Length":420,"ExportTime":1483484642,"SequenceNo":1434533677,"DomainID":32771},"DataSets":[[{"I":8,"V":"192.16.28.217"},{"I":12,"V":"180.10.210.240"},{"I":5,"V":2},{"I":4,"V":6},{"I":7,"V":443},{"I":11,"V":64381},{"I":32,"V":0},{"I":10,"V":811},{"I":58,"V":0},{"I":9,"V":24},{"I":13,"V":20},{"I":16,"V":4200000000},{"I":17,"V":27747},{"I":15,"V":"180.105.10.210"},{"I":6,"V":"0x10"},{"I":14,"V":1113},{"I":1,"V":22500},{"I":2,"V":15},{"I":52,"V":63},{"I":53,"V":63},{"I":152,"V":1483484581770},{"I":153,"V":1483484622384},{"I":136,"V":2},{"I":243,"V":0},{"I":245,"V":0}]]}|]++netflow5Json :: ByteString+netflow5Json = TE.encodeUtf8 $+ [text|{"AgentID":"114.23.3.231","Header":{"Version":5,"Count":3,"SysUpTimeMSecs":51469784,"UNIXSecs":1544476581,"UNIXNSecs":0,"SeqNum":873873830,"EngType":0,"EngID":0,"SmpInt":1000},"Flows":[{"SrcAddr":"125.238.46.48","DstAddr":"114.23.236.96","NextHop":"114.23.3.231","Input":791,"Output":817,"PktCount":4,"L3Octets":1708,"StartTime":51402145,"EndTime":51433264,"SrcPort":49233,"DstPort":443,"Padding1":0,"TCPFlags":16,"ProtType":6,"Tos":0,"SrcAsNum":4771,"DstAsNum":56030,"SrcMask":20,"DstMask":22,"Padding2":0},{"SrcAddr":"125.238.46.48","DstAddr":"114.23.236.96","NextHop":"114.23.3.231","Input":791,"Output":817,"PktCount":1,"L3Octets":441,"StartTime":51425137,"EndTime":51425137,"SrcPort":49233,"DstPort":443,"Padding1":0,"TCPFlags":24,"ProtType":6,"Tos":0,"SrcAsNum":4771,"DstAsNum":56030,"SrcMask":20,"DstMask":22,"Padding2":0},{"SrcAddr":"210.5.53.48","DstAddr":"103.22.200.210","NextHop":"122.56.118.157","Input":564,"Output":802,"PktCount":1,"L3Octets":1500,"StartTime":51420072,"EndTime":51420072,"SrcPort":80,"DstPort":56108,"Padding1":0,"TCPFlags":16,"ProtType":6,"Tos":0,"SrcAsNum":56030,"DstAsNum":13335,"SrcMask":24,"DstMask":23,"Padding2":0}]}|]++netflow9Json :: ByteString+netflow9Json = TE.encodeUtf8 $+ [text|{"AgentID":"10.81.70.56","Header":{"Version":9,"Count":1,"SysUpTime":357280,"UNIXSecs":1493918653,"SeqNum":14,"SrcID":87},"DataSets":[[{"I":1,"V":"0x00000050"},{"I":2,"V":"0x00000002"},{"I":4,"V":2},{"I":5,"V":192},{"I":6,"V":"0x00"},{"I":7,"V":0},{"I":8,"V":"10.81.70.56"},{"I":9,"V":0},{"I":10,"V":0},{"I":11,"V":0},{"I":12,"V":"224.0.0.22"},{"I":13,"V":0},{"I":14,"V":0},{"I":15,"V":"0.0.0.0"},{"I":16,"V":0},{"I":17,"V":0},{"I":21,"V":300044},{"I":22,"V":299144}]]}|]
+ vflow-types.cabal view
@@ -0,0 +1,111 @@+cabal-version: 2.2+name:+ vflow-types+version:+ 0.1+synopsis:+ types for ingesting vflow data with aeson+description:+ `vflow-types` provides types suitable for ingesting vflow data with aeson.+ .+ Verizon Digital's <https://github.com/VerizonDigital/vflow vflow> is a network flow collector. Features:+ .+ * IPFIX RFC7011 collector+ .+ * sFlow v5 raw header / counters collector+ .+ * Netflow v5 collector+ .+ * Netflow v9 collector+ .+ * Decoding sFlow raw header L2/L3/L4+ .+ * Producer to Apache Kafka, NSQ, NATS+ .+ * Replicate IPFIX to 3rd party collector+ .+ * Support for IPv4 and IPv6+ .+ * Monitoring with InfluxDB and OpenTSDB backend+ .+ * Easy integration with JUNOS+ .+ Note that this is not an official Verizon Digital product.+homepage:+ https://github.com/chessai/vflow-types+bug-reports:+ https://github.com/chessai/vflow-types/issues+license:+ BSD-3-Clause+license-file:+ LICENSE+author:+ chessai+maintainer:+ chessai <chessai1996@gmail.com>+copyright:+ © 2019 chessai+category:+ Utility,Data,Parsing,Text+build-type:+ Simple+extra-doc-files:+ README.md+ , CHANGELOG.md+tested-with:+ GHC == 8.4.4, GHC == 8.6.3++library+ hs-source-dirs:+ src+ exposed-modules:+ VFlow.Types.IpFix+ VFlow.Types.SFlow+ VFlow.Types.NetFlow5+ VFlow.Types.NetFlow9+ build-depends:+ , aeson >= 1 && < 1.5+ , base >= 4.10.1 && < 4.13+ , ip >= 1.5 && < 1.6+ , json-alt >= 1 && < 1.1+ , json-autotype >= 3 && < 4+ , scientific >= 0.3 && < 0.4+ , text >= 1.2 && < 1.3+ ghc-options:+ -Wall+ default-language:+ Haskell2010++test-suite unit-tests+ type: exitcode-stdio-1.0+ hs-source-dirs: test+ main-is: UnitTests.hs+ build-depends:+ , base+ , vflow-types+ , bytestring+ , text+ , aeson+ , neat-interpolation+ default-language:+ Haskell2010++test-suite laws+ type: exitcode-stdio-1.0+ hs-source-dirs: test+ main-is: Laws.hs+ build-depends:+ , base+ , vflow-types+ , QuickCheck+ , quickcheck-classes+ , json-alt+ , ip+ default-language:+ Haskell2010++source-repository head+ type:+ git+ location:+ https://github.com/chessai/vflow-types.git