Graph500 (empty) → 0.4.0
raw patch · 9 files changed
+916/−0 lines, 9 filesdep +Graph500dep +arraydep +basesetup-changed
Dependencies added: Graph500, array, base, containers, mersenne-random-pure64, mtl, random
Files
- Graph500.cabal +40/−0
- LICENSE +340/−0
- Setup.hs +2/−0
- src/G500.hs +14/−0
- src/G500/Generate.hs +238/−0
- src/G500/GenerateFile.hs +88/−0
- src/G500/Index.hs +46/−0
- src/G500/Read.hs +60/−0
- src/Gen.hs +88/−0
+ Graph500.cabal view
@@ -0,0 +1,40 @@+Name: Graph500+Version: 0.4.0+Synopsis: Graph500 benchmark-related definitions and data set generator.+Description:+ Graph generator library and standalone graph generator. The+ generator allows you to generate your data once and then use it+ for benchmarking and debugging. Graph generation algorithm follows+ Graph500 specification <http://www.graph500.org/specifications#sec-3_3>.+License: GPL-2+License-file: LICENSE+Author: serguey.zefirov@parsci.com+Maintainer: alexander.vershilov@parsci.com+Copyright: (C) 2013 Parallel Scientific Labs, LLC+Category: Concurrency+Build-type: Simple++Cabal-version: >=1.8++source-repository head+ type: git+ location: git://github.com/ps-labs/graph500.git++Library+ Exposed-modules: G500, G500.Index, G500.Generate, G500.Read, G500.GenerateFile+ hs-source-dirs: src+ Build-depends: array >= 0.3,+ base >= 3.0 && < 5.0,+ containers >= 0.3,+ mtl >= 1.1,+ mersenne-random-pure64 >= 0.2 && < 0.3,+ random >= 1.0+ ghc-options: -Wall++-- Stream generator program.+Executable graph500gen+ Main-Is: src/Gen.hs+ Build-depends: array >= 0.3,+ base >= 0.3 && < 5.0,+ Graph500,+ mtl >= 1.1
+ LICENSE view
@@ -0,0 +1,340 @@+ GNU GENERAL PUBLIC LICENSE+ Version 2, June 1991++ Copyright (C) 1989, 1991 Free Software Foundation, Inc.+ 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA+ Everyone is permitted to copy and distribute verbatim copies+ of this license document, but changing it is not allowed.++ Preamble++ The licenses for most software are designed to take away your+freedom to share and change it. By contrast, the GNU General Public+License is intended to guarantee your freedom to share and change free+software--to make sure the software is free for all its users. This+General Public License applies to most of the Free Software+Foundation's software and to any other program whose authors commit to+using it. (Some other Free Software Foundation software is covered by+the GNU Library General Public License instead.) You can apply it to+your programs, too.++ When we speak of free software, we are referring to freedom, not+price. Our General Public Licenses are designed to make sure that you+have the freedom to distribute copies of free software (and charge for+this service if you wish), that you receive source code or can get it+if you want it, that you can change the software or use pieces of it+in new free programs; and that you know you can do these things.++ To protect your rights, we need to make restrictions that forbid+anyone to deny you these rights or to ask you to surrender the rights.+These restrictions translate to certain responsibilities for you if you+distribute copies of the software, or if you modify it.++ For example, if you distribute copies of such a program, whether+gratis or for a fee, you must give the recipients all the rights that+you have. You must make sure that they, too, receive or can get the+source code. And you must show them these terms so they know their+rights.++ We protect your rights with two steps: (1) copyright the software, and+(2) offer you this license which gives you legal permission to copy,+distribute and/or modify the software.++ Also, for each author's protection and ours, we want to make certain+that everyone understands that there is no warranty for this free+software. If the software is modified by someone else and passed on, we+want its recipients to know that what they have is not the original, so+that any problems introduced by others will not reflect on the original+authors' reputations.++ Finally, any free program is threatened constantly by software+patents. We wish to avoid the danger that redistributors of a free+program will individually obtain patent licenses, in effect making the+program proprietary. To prevent this, we have made it clear that any+patent must be licensed for everyone's free use or not licensed at all.++ The precise terms and conditions for copying, distribution and+modification follow.++ GNU GENERAL PUBLIC LICENSE+ TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION++ 0. This License applies to any program or other work which contains+a notice placed by the copyright holder saying it may be distributed+under the terms of this General Public License. The "Program", below,+refers to any such program or work, and a "work based on the Program"+means either the Program or any derivative work under copyright law:+that is to say, a work containing the Program or a portion of it,+either verbatim or with modifications and/or translated into another+language. (Hereinafter, translation is included without limitation in+the term "modification".) Each licensee is addressed as "you".++Activities other than copying, distribution and modification are not+covered by this License; they are outside its scope. The act of+running the Program is not restricted, and the output from the Program+is covered only if its contents constitute a work based on the+Program (independent of having been made by running the Program).+Whether that is true depends on what the Program does.++ 1. You may copy and distribute verbatim copies of the Program's+source code as you receive it, in any medium, provided that you+conspicuously and appropriately publish on each copy an appropriate+copyright notice and disclaimer of warranty; keep intact all the+notices that refer to this License and to the absence of any warranty;+and give any other recipients of the Program a copy of this License+along with the Program.++You may charge a fee for the physical act of transferring a copy, and+you may at your option offer warranty protection in exchange for a fee.++ 2. You may modify your copy or copies of the Program or any portion+of it, thus forming a work based on the Program, and copy and+distribute such modifications or work under the terms of Section 1+above, provided that you also meet all of these conditions:++ a) You must cause the modified files to carry prominent notices+ stating that you changed the files and the date of any change.++ b) You must cause any work that you distribute or publish, that in+ whole or in part contains or is derived from the Program or any+ part thereof, to be licensed as a whole at no charge to all third+ parties under the terms of this License.++ c) If the modified program normally reads commands interactively+ when run, you must cause it, when started running for such+ interactive use in the most ordinary way, to print or display an+ announcement including an appropriate copyright notice and a+ notice that there is no warranty (or else, saying that you provide+ a warranty) and that users may redistribute the program under+ these conditions, and telling the user how to view a copy of this+ License. (Exception: if the Program itself is interactive but+ does not normally print such an announcement, your work based on+ the Program is not required to print an announcement.)++These requirements apply to the modified work as a whole. If+identifiable sections of that work are not derived from the Program,+and can be reasonably considered independent and separate works in+themselves, then this License, and its terms, do not apply to those+sections when you distribute them as separate works. But when you+distribute the same sections as part of a whole which is a work based+on the Program, the distribution of the whole must be on the terms of+this License, whose permissions for other licensees extend to the+entire whole, and thus to each and every part regardless of who wrote it.++Thus, it is not the intent of this section to claim rights or contest+your rights to work written entirely by you; rather, the intent is to+exercise the right to control the distribution of derivative or+collective works based on the Program.++In addition, mere aggregation of another work not based on the Program+with the Program (or with a work based on the Program) on a volume of+a storage or distribution medium does not bring the other work under+the scope of this License.++ 3. You may copy and distribute the Program (or a work based on it,+under Section 2) in object code or executable form under the terms of+Sections 1 and 2 above provided that you also do one of the following:++ a) Accompany it with the complete corresponding machine-readable+ source code, which must be distributed under the terms of Sections+ 1 and 2 above on a medium customarily used for software interchange; or,++ b) Accompany it with a written offer, valid for at least three+ years, to give any third party, for a charge no more than your+ cost of physically performing source distribution, a complete+ machine-readable copy of the corresponding source code, to be+ distributed under the terms of Sections 1 and 2 above on a medium+ customarily used for software interchange; or,++ c) Accompany it with the information you received as to the offer+ to distribute corresponding source code. (This alternative is+ allowed only for noncommercial distribution and only if you+ received the program in object code or executable form with such+ an offer, in accord with Subsection b above.)++The source code for a work means the preferred form of the work for+making modifications to it. For an executable work, complete source+code means all the source code for all modules it contains, plus any+associated interface definition files, plus the scripts used to+control compilation and installation of the executable. However, as a+special exception, the source code distributed need not include+anything that is normally distributed (in either source or binary+form) with the major components (compiler, kernel, and so on) of the+operating system on which the executable runs, unless that component+itself accompanies the executable.++If distribution of executable or object code is made by offering+access to copy from a designated place, then offering equivalent+access to copy the source code from the same place counts as+distribution of the source code, even though third parties are not+compelled to copy the source along with the object code.++ 4. You may not copy, modify, sublicense, or distribute the Program+except as expressly provided under this License. Any attempt+otherwise to copy, modify, sublicense or distribute the Program is+void, and will automatically terminate your rights under this License.+However, parties who have received copies, or rights, from you under+this License will not have their licenses terminated so long as such+parties remain in full compliance.++ 5. You are not required to accept this License, since you have not+signed it. However, nothing else grants you permission to modify or+distribute the Program or its derivative works. These actions are+prohibited by law if you do not accept this License. Therefore, by+modifying or distributing the Program (or any work based on the+Program), you indicate your acceptance of this License to do so, and+all its terms and conditions for copying, distributing or modifying+the Program or works based on it.++ 6. Each time you redistribute the Program (or any work based on the+Program), the recipient automatically receives a license from the+original licensor to copy, distribute or modify the Program subject to+these terms and conditions. You may not impose any further+restrictions on the recipients' exercise of the rights granted herein.+You are not responsible for enforcing compliance by third parties to+this License.++ 7. If, as a consequence of a court judgment or allegation of patent+infringement or for any other reason (not limited to patent issues),+conditions are imposed on you (whether by court order, agreement or+otherwise) that contradict the conditions of this License, they do not+excuse you from the conditions of this License. If you cannot+distribute so as to satisfy simultaneously your obligations under this+License and any other pertinent obligations, then as a consequence you+may not distribute the Program at all. For example, if a patent+license would not permit royalty-free redistribution of the Program by+all those who receive copies directly or indirectly through you, then+the only way you could satisfy both it and this License would be to+refrain entirely from distribution of the Program.++If any portion of this section is held invalid or unenforceable under+any particular circumstance, the balance of the section is intended to+apply and the section as a whole is intended to apply in other+circumstances.++It is not the purpose of this section to induce you to infringe any+patents or other property right claims or to contest validity of any+such claims; this section has the sole purpose of protecting the+integrity of the free software distribution system, which is+implemented by public license practices. Many people have made+generous contributions to the wide range of software distributed+through that system in reliance on consistent application of that+system; it is up to the author/donor to decide if he or she is willing+to distribute software through any other system and a licensee cannot+impose that choice.++This section is intended to make thoroughly clear what is believed to+be a consequence of the rest of this License.++ 8. If the distribution and/or use of the Program is restricted in+certain countries either by patents or by copyrighted interfaces, the+original copyright holder who places the Program under this License+may add an explicit geographical distribution limitation excluding+those countries, so that distribution is permitted only in or among+countries not thus excluded. In such case, this License incorporates+the limitation as if written in the body of this License.++ 9. The Free Software Foundation may publish revised and/or new versions+of the General Public License from time to time. Such new versions will+be similar in spirit to the present version, but may differ in detail to+address new problems or concerns.++Each version is given a distinguishing version number. If the Program+specifies a version number of this License which applies to it and "any+later version", you have the option of following the terms and conditions+either of that version or of any later version published by the Free+Software Foundation. If the Program does not specify a version number of+this License, you may choose any version ever published by the Free Software+Foundation.++ 10. If you wish to incorporate parts of the Program into other free+programs whose distribution conditions are different, write to the author+to ask for permission. For software which is copyrighted by the Free+Software Foundation, write to the Free Software Foundation; we sometimes+make exceptions for this. Our decision will be guided by the two goals+of preserving the free status of all derivatives of our free software and+of promoting the sharing and reuse of software generally.++ NO WARRANTY++ 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,+REPAIR OR CORRECTION.++ 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE+POSSIBILITY OF SUCH DAMAGES.++ END OF TERMS AND CONDITIONS++ How to Apply These Terms to Your New Programs++ If you develop a new program, and you want it to be of the greatest+possible use to the public, the best way to achieve this is to make it+free software which everyone can redistribute and change under these terms.++ To do so, attach the following notices to the program. It is safest+to attach them to the start of each source file to most effectively+convey the exclusion of warranty; and each file should have at least+the "copyright" line and a pointer to where the full notice is found.++ <one line to give the program's name and a brief idea of what it does.>+ Copyright (C) <year> <name of author>++ This program is free software; you can redistribute it and/or modify+ it under the terms of the GNU General Public License as published by+ the Free Software Foundation; either version 2 of the License, or+ (at your option) any later version.++ This program is distributed in the hope that it will be useful,+ but WITHOUT ANY WARRANTY; without even the implied warranty of+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+ GNU General Public License for more details.++ You should have received a copy of the GNU General Public License+ along with this program; if not, write to the Free Software+ Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA+++Also add information on how to contact you by electronic and paper mail.++If the program is interactive, make it output a short notice like this+when it starts in an interactive mode:++ Gnomovision version 69, Copyright (C) year name of author+ Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.+ This is free software, and you are welcome to redistribute it+ under certain conditions; type `show c' for details.++The hypothetical commands `show w' and `show c' should show the appropriate+parts of the General Public License. Of course, the commands you use may+be called something other than `show w' and `show c'; they could even be+mouse-clicks or menu items--whatever suits your program.++You should also get your employer (if you work as a programmer) or your+school, if any, to sign a "copyright disclaimer" for the program, if+necessary. Here is a sample; alter the names:++ Yoyodyne, Inc., hereby disclaims all copyright interest in the program+ `Gnomovision' (which makes passes at compilers) written by James Hacker.++ <signature of Ty Coon>, 1 April 1989+ Ty Coon, President of Vice++This General Public License does not permit incorporating your program into+proprietary programs. If your program is a subroutine library, you may+consider it more useful to permit linking proprietary applications with the+library. If this is what you want to do, use the GNU Library General+Public License instead of this License.
+ Setup.hs view
@@ -0,0 +1,2 @@+import Distribution.Simple+main = defaultMain
+ src/G500.hs view
@@ -0,0 +1,14 @@+-- |+-- Module : G500+-- Copyright : (C) 2013 Parallel Scientific Labs, LLC.+-- License : GPLv2+--+-- Top-level module for everything Graph500 related.++module G500+ ( module G500.Index+ , module G500.Generate+ ) where++import G500.Index+import G500.Generate
+ src/G500/Generate.hs view
@@ -0,0 +1,238 @@+-- |+-- Module : G500.Generate+-- Copyright : (C) 2013 Parallel Scientific Labs, LLC.+-- License : GPLv2+--+-- A generator for Graph500 benchmark. Translated from Graph500 specification in GNU Octave.++{-# LANGUAGE BangPatterns #-}++module G500.Generate+ ( generate+ -- $spec+ ) where++import Control.Concurrent+import Control.Monad+import Control.Monad.State+import Data.Array.IO+import Data.Bits+import System.Random.Mersenne.Pure64++import G500.Index++-------------------------------------------------------------------------------+-- Documentation.++{- $spec++@+%% Original specification in GNU Octave commented by sergueyz (wherever he feels original is too terse).+%%+%% GNU Octave expressions part of manual: <http://sunsite.univie.ac.at/textbooks/octave/octave_9.html>++function ij = kronecker_generator (SCALE, edgefactor)+%% Generate an edgelist according to the Graph500+%% parameters. In this sample, the edge list is+%% returned in an array with two rows, where StartVertex+%% is first row and EndVertex is the second. The vertex+%% labels start at zero.+%%+%% Example, creating a sparse matrix for viewing:+%% ij = kronecker_generator (10, 16);+%% G = sparse (ij(1,:)+1, ij(2,:)+1, ones (1, size (ij, 2)));+%% spy (G);+%% The spy plot should appear fairly dense. Any locality+%% is removed by the final permutations.++ %% Set number of vertices.+ N = 2^SCALE;++ %% Set number of edges.+ M = edgefactor * N;++ %% Set initiator probabilities.+ [A, B, C] = deal (0.57, 0.19, 0.19); %% Just a tuple assignment.++ %% Create index arrays.+ ij = ones (2, M); %% 2xM of ones.++ %% Probabilities.+ ab = A + B;+ c_norm = C/(1 - (A + B));+ a_norm = A/(A + B);++ %% Loop over each order of bit.+ for ib = 1:SCALE,+ %% Compare with probabilities and set bits of indices.+ ii_bit = rand (1, M) > ab; %% either 0 or 1+ jj_bit = rand (1, M) > ( c_norm * ii_bit + a_norm * not (ii_bit) ); %% either 0 or 1.++ %% please see that ij is one-based. We add current power of two to sums in ij.+ %% each ij(:,:) lies in range 1..2^SCALE.+ ij = ij + 2^(ib-1) * [ii_bit; jj_bit];+ end++ %% Permute vertex labels+ p = randperm (N); %% a column with numbers 1..N.+ ij = p(ij); %% the most appropriate meaning here is ij(a,b) = p(ij(a,b)).+ %% please correct me if I am wrong.++ %% Permute the edge list+ p = randperm (M);+ ij = ij(:, p); %% the most appropriate meaning here is ij(a,b) = ij(a,p(b)).+ %% please correct me if I am wrong.++ %% Adjust to zero-based labels.+ ij = ij - 1;+@+-}++-------------------------------------------------------------------------------+-- Monad definition and helpers.++type GenM a = StateT PureMT IO a++a, b, c, ab, c_norm, a_norm:: Float+(a, b, c) = (0.57, 0.19, 0.19 :: Float)+ab = a + b+c_norm = c / (1-ab)+a_norm = a / ab++-- |Generates index that is in range 0..maxIndex. indexMask should be power of two minus 1.+-- See @randomIndex@.+genRandomIndexMask :: Index -> GenM Index+genRandomIndexMask indexMask = do+ g <- get+ let (!r,!g') = randomIndex indexMask g+ put g'+ return r++genRandomIndex :: Index -> GenM Index+genRandomIndex maxIndex = do+ v <- genRandomIndexMask (mask 1)+ if v < maxIndex then return v else genRandomIndex maxIndex+ where+ mask n+ | n > maxIndex = n-1+ | otherwise = mask (2*n)++-- |Just an usual random float with randomness in full 24 bits.+getRandomFloat :: GenM Float+getRandomFloat = do+ g <- get+ let (!i,!g') = randomInt64 g+ put g'+ return $ fromIntegral (i .&. max') / fromIntegral (max' + 1)+ where+ max' = 0x3fffffff++-- |This is heart of data set generation code.+-- This generates either 0 or 1 for current (power of two) weights for+-- edge start and end indices.+ii_jj_bits :: GenM (Index, Index)+ii_jj_bits = do+ iiR <- getRandomFloat+ jjR <- getRandomFloat+ let iiBit = iiR > ab+ let jjThresh = if iiBit then c_norm else a_norm+ -- c_norm * iiBitFloat + a_norm * (1-iiBitFloat)+ let jjBit = jjR > jjThresh+ return (fromBool iiBit, fromBool jjBit)+ where+ fromBool = fromIntegral . fromEnum++-------------------------------------------------------------------------------+-- Main driver.++type GraphArr = IOUArray Index Index++generate :: Int -- ^ Scale+ -> Int -- ^ Edge Factor+ -> IO (GraphArr, GraphArr)+generate scale edgeFactor = do+ start <- newIndexArr+ end <- newIndexArr+ std <- newPureMT+ runGenM std $ go start end+ return (start,end)+ where+ runGenM g genM = runStateT genM g >> return ()+ newIndexArr = liftIO $ newArray (0,maxEdgeIndex) 0+ n = shiftL 1 scale+ m = n*fromIntegral edgeFactor+ maxEdgeIndex = m-1+ maxIndex = n-1+++ incrIndex arr i incr = do+ v <- readArray arr i+ writeArray arr i (v+incr)+ go start end = do+ gen (shiftL 1 (scale - 1)) start end+ p <- permutation maxIndex+ permute maxEdgeIndex p start+ permute maxEdgeIndex p end+ p1 <- permutation maxEdgeIndex+ permuteIndices maxEdgeIndex p1 end+ return ()++ permutation :: Index -> GenM GraphArr+ permutation maxIndex' = do+ p <- liftIO $ newArray (0,maxIndex') 0+ liftIO $ forM_ [0..maxIndex'] $ \i -> writeArray p i i+ forM_ (concat $ replicate 1 [0..maxIndex']) $ \i -> do+ j <- genRandomIndex maxIndex'+ liftIO $ do+ a1 <- readArray p i+ b1 <- readArray p j+ writeArray p i b1+ writeArray p j a1+ return p++ permute :: Index -> GraphArr -> GraphArr -> GenM ()+ permute n1 p arr = liftIO $ forM_ [0..n1] $ \i -> do+ a1 <- readArray arr i+ pa <- readArray p a1+ writeArray arr i pa++ permuteIndices :: Index -> GraphArr -> GraphArr -> GenM ()+ permuteIndices n1 p arr = liftIO $ forM_ [0..n1] $ \i -> do+ j <- readArray p i+ a1 <- readArray arr i+ b1 <- readArray arr j+ writeArray arr i b1+ writeArray arr j a1++ gen pow2 start end+ | pow2 < 1 = return ()+ | otherwise = do+ -- forM_ [0..maxEdgeIndex] $ genBit pow2 start end+ parallelGeneration pow2 start end+ gen (shiftR pow2 1) start end+ parallelPortion :: Index+ parallelPortion = fromIntegral edgeFactor*1024+ parallelGeneration pow2 start end = do+ let number = fromIntegral $ div (maxEdgeIndex+1) parallelPortion+ runningCount <- liftIO $ newMVar (number :: Int)+ forM_ [0..fromIntegral number - 1] $ \nt -> do+ threadG <- lift newPureMT+ let startI = parallelPortion * nt+ let endI = parallelPortion * (nt+1) - 1+ liftIO $ forkIO $ runGenM threadG $ do+ forM_ [startI..endI] $+ genBit pow2 start end+ liftIO $ modifyMVar_ runningCount $ return . (+(-1))+ let wait = do+ n1 <- takeMVar runningCount+ if n1 > 0 then do+ putMVar runningCount n1+ yield+ wait+ else return ()+ liftIO wait+ genBit pow2 start end i = do+ (startBit, endBit) <- ii_jj_bits+ liftIO $ do+ incrIndex start i (startBit * pow2)+ incrIndex end i (endBit * pow2)
+ src/G500/GenerateFile.hs view
@@ -0,0 +1,88 @@+-- |+-- Module : G500.GenerateFIle+-- Copyright : (C) 2013 Parallel Scientific Labs, LLC.+-- License : GPLv2+--+-- File generation utilities.++module G500.GenerateFile+ ( generateWriteFile+ , GraphType(..)+ ) where++import G500+import System.IO+import Data.Word (Word8)+import Control.Monad+import Data.Array.IO+import Data.Bits++-------------------------------------------------------------------------------+-- Graph edges generation.++data GraphType = Graph500 | Simple++generateGraph :: GraphType -> Int -> Int -> IO (IOUArray Index Index, IOUArray Index Index)+generateGraph Simple scale edgeFactor = do+ fromA <- newArray (0,maxEdgeIndex) 0+ toA <- newArray (0,maxEdgeIndex) 0+ forM_ [1..factor] $ \n -> do+ let ofs = ((n-1)*verticesCount)+ writeEdges fromA ofs 0 -- start is always the same index+ writeEdges toA ofs n -- end varies from 1 to edgeFactor+ return (fromA, toA)+ where+ verticesCount = (2 :: Index) ^ scale+ maxVertexIndex = verticesCount - 1+ maxEdgeIndex = verticesCount * factor - 1+ factor :: Index+ factor = fromIntegral edgeFactor+ writeEdges arr indexOfs indexInc = do+ forM_ [0..maxVertexIndex] $ \i -> do+ writeArray arr (i+indexOfs) (min (i+indexInc) maxVertexIndex)+generateGraph Graph500 scale edgeFactor = do+ when (edgeFactor /= 16) $ putStrLn "It is preferable for edgeFactor to be 16 in Graph500 benchmark data."+ generate scale edgeFactor+++-- | Generage file with graph+generateWriteFile :: String -- ^ Filename+ -> GraphType -- ^ Type of the graph+ -> Int -- ^ Scale+ -> Int -- ^ Edge factor+ -> IO ()+generateWriteFile fn ty scale edgeFactor = do+ (start,end) <- generateGraph ty scale edgeFactor+ buffer <- mkBuffer+ h <- openBinaryFile fn WriteMode+ write h 0 buffer start end+ hClose h+ where+ verticesExpected :: Index+ verticesExpected = shiftL 1 scale+ bufferPairs = min verticesExpected 8192+ bufferIndices = bufferPairs*2+ bufferBytes = fromIntegral (8*bufferIndices)++ mkBuffer :: IO (IOUArray Int Word8)+ mkBuffer = newArray (0,bufferBytes-1) 0++ write h start buffer edgeStart edgeEnd = do+ (_,top) <- getBounds edgeStart+ if start > top then return ()+ else do+ fill start buffer edgeStart edgeEnd+ -- !!! HACK !!!+ -- Int64 is not a Storable instance.+ hPutArray h buffer (8*fromIntegral bufferIndices)+ write h (start+bufferPairs) buffer edgeStart edgeEnd+ fill start buffer edgeStart edgeEnd = do+ forM_ [0..bufferPairs-1] $ \i -> do+ s <- readArray edgeStart (start + i)+ e <- readArray edgeEnd (start + i)+ writeIndexAsBytes buffer (fromIntegral i*2 ) s+ writeIndexAsBytes buffer (fromIntegral i*2+1) e+ writeIndexAsBytes :: IOUArray Int Word8 -> Int -> Index -> IO ()+ writeIndexAsBytes arr i ix = do+ forM_ [0..7] $ \s -> do+ writeArray arr (i*8 + s) (fromIntegral $ shiftR ix (s*8))
+ src/G500/Index.hs view
@@ -0,0 +1,46 @@+-- |+-- Module : G500.Index+-- Copyright : (C) 2013 Parallel Scientific Labs, LLC.+-- License : GPLv2+--+-- Definition for the type of indices.+--+-- I think index type can change over time, so I made that+-- little file with type definition and class that allows one's code+-- to work with different index types.+--+-- On the second thought it looks like overengineering. Let it be.++-- TODO:+-- - type-safe index? (data VertexIndex v = VertexIndex !Index, as an example)++module G500.Index+ ( Index -- index type.+ , RandomIndex(..) -- how to generate random index.+ ) where++import Data.Bits+import Data.Int+import System.Random+++-- |Big enough for Graph500 benchmark purposes.+type Index = Int64++class Bits index => RandomIndex index where+ -- |Generate random index.+ -- Accepts max value and generator.+ -- Returns random index and updated generator.+ -- Max value should be power of 2 (valid for Graph500 generator).+ randomIndex :: RandomGen g => index -> g -> (index,g)++instance RandomIndex Int64 where+ -- simple version.+ -- we assume that we get a whole 30 bits of Int in @next@ call.+ -- (this is true for StdGen)+ randomIndex maxIndex g+ | maxIndex < shiftL 1 30 = (fromIntegral low30Bits .&. maxIndex,g')+ | otherwise = (fromIntegral low30Bits .|. shiftL highBits 30, g'')+ where+ (low30Bits,g') = next g+ (highBits,g'') = randomIndex (shiftR maxIndex 30) g'
+ src/G500/Read.hs view
@@ -0,0 +1,60 @@+-- |+-- Module : G500.Read+-- Copyright : (C) 2013 Parallel Scientific Labs, LLC.+-- License : GPLv2+--+-- A module that reads files generated by Gen (graph500gen) program+-- one directory level above.+--+module G500.Read+ ( Graph500Reader+ , mkGraph500Reader+ ) where++import Control.Monad+import Data.Array.IO+import Data.Array.Unboxed+import Data.Word+import System.IO++import G500++-- |A synonym for IO action that performs a read.+type Graph500Reader = IO (Maybe (UArray Int Index))++-- |Create a graph reader structure.+-- The @handle@ should be opened to read as a binary file.+-- Also the handle gets captured in result of the function and will+-- be closed at the end of operation.+mkGraph500Reader :: Handle -- ^ File handle.+ -> Int -- ^ Size of a read.+ -> IO Graph500Reader+mkGraph500Reader handle portionSize = do+ buf <- newArray (0,bytesCount-1) 0+ return $ graph500Reader handle buf+ where+ vertexSize = 8+ edgeSize = 2*vertexSize+ bytesCount = edgeSize*portionSize+ graph500Reader h buf = do+ eof <- hIsEOF h+ if eof then do+ hClose h+ return Nothing+ else readBatchPortion h buf+ readVertexIndex :: IOUArray Int Word8 -> Int -> IO Index+ readVertexIndex a i = do+ bytes <- forM [i*vertexSize..(i+1)*vertexSize-1] $ readArray a+ return $ foldr (\x j -> j*256 + fromIntegral x) 0 $ bytes+ readBatchPortion :: Handle -> IOUArray Int Word8 -> IO (Maybe (UArray Int Index))+ readBatchPortion h buf = do+ n <- hGetArray h buf bytesCount+ let edgesCount = n `div` edgeSize+ let indicesCount = edgesCount*2+ let maxIndex = indicesCount-1+ result <- (newArray (0,maxIndex) (0 :: Index)) :: IO (IOUArray Int Index)+ forM_ [0..indicesCount-1] $ \i -> do+ idx <- readVertexIndex buf i+ writeArray result i idx+ pairs <- freeze result+ return $ Just pairs
+ src/Gen.hs view
@@ -0,0 +1,88 @@+-- |+-- Copyright : (C) 2013 Parallel Scientific Labs, LLC.+-- License : GPLv2+--+-- A program to generate various test sets.++module Main where++import Control.Monad (mplus, forM_, when)+import Control.Monad.State+import Control.Monad.Error+import Data.Bits (shiftL, shiftR)+import Data.Array.IO (IOUArray, newArray, writeArray, readArray, getBounds, hPutArray)+import Data.List (stripPrefix)+import Data.Word (Word8)+import System.Environment (getArgs)+import System.Exit (exitSuccess, exitFailure)+import System.IO (openBinaryFile, IOMode(WriteMode), hClose)++import G500+import G500.GenerateFile++-------------------------------------------------------------------------------+-- Command line arguments handling.++-- |Type of graph generated.+-- @Graph500@ is a R-MAT graph that is useful for Graph500 benchmark.+--+-- @Simple@ is a simple graph where each i-th index is connected to+-- (i+[1..edgeFactor])-th index. In the case of overflow (dest index is+-- bigger than 2^scale-1) we just connect to max index allowed.+-- This type of graph is for verification purposes.+--+type MatchM a = ErrorT String (State [String]) a++matchArgs' :: [String] -> Either String (String, GraphType, Int, Int, Maybe (Int, String))+matchArgs' args = flip evalState args $ runErrorT $ do+ filename <- getArg+ graphType <- getArg+ ty <- matchGraphType graphType+ scale <- match "scale"+ edgeFactor <- match "edgeFactor" `mplus` return 16+ actionsInfo <- liftM Just matchActionsInfo `mplus` return Nothing+ return (filename, ty, scale, edgeFactor, actionsInfo)+ where+ getArg :: MatchM String+ getArg = do+ as <- get+ case as of+ (a : as) -> do+ put as+ return a+ _ -> fail "no args."+ match :: Read x => String -> MatchM x+ match prefix = do+ opt <- getArg+ case stripPrefix (prefix ++ "=") opt of+ Just suffix -> return (read suffix)+ Nothing -> fail $ prefix++" required."+ matchActionsInfo :: MatchM (Int, String)+ matchActionsInfo = do+ n <- match "numActions"+ s <- getArg+ return (n,s)+ matchGraphType :: String -> MatchM GraphType+ matchGraphType "graph500" = return Graph500+ matchGraphType "simple" = return Simple+ matchGraphType ty = fail $ "Bad graph type "++ty++matchArgs :: [String] -> IO (String, GraphType, Int, Int, Maybe (Int, String))+matchArgs args = case matchArgs' args of+ Left err -> do+ putStrLn $ "command line error: "++err+ putStrLn "usage: graph500gen filename (simple|graph500) scale=<scale> [edgeFactor=<edge factor>] [numActions=# actionsFilename]"+ putStrLn ""+ putStrLn "The data will be stored into file filename."+ putStrLn "If edgeFactor argument is omitted edgeFactor=16 assumed."+ exitFailure+ Right r -> return r++-------------------------------------------------------------------------------+-- Main driver.++main = do+ args <- getArgs+ (fn,ty,scale,edgeFactor, actions) <- matchArgs args+ generateWriteFile fn ty scale edgeFactor+ return ()