htsn-import 0.0.8 → 0.0.9
raw patch · 30 files changed
+1591/−125 lines, 30 filesbinary-added
Files
- doc/TODO +5/−2
- doc/dbschema/MLB_earlylineXML.png binary
- doc/dbschema/earlylineXML.png binary
- doc/man1/htsn-import.1 +191/−2
- htsn-import.cabal +5/−1
- makefile +14/−2
- schema/MLB_earlylineXML.dtd +22/−0
- schema/earlylineXML.dtd +20/−0
- schemagen/MLB_earlylineXML/21160887.xml +1/−0
- schemagen/earlylineXML/21166989.xml +1/−0
- src/Main.hs +12/−0
- src/TSN/Parse.hs +15/−15
- src/TSN/Picklers.hs +358/−35
- src/TSN/Team.hs +1/−1
- src/TSN/XML/AutoRacingResults.hs +58/−39
- src/TSN/XML/AutoRacingSchedule.hs +1/−1
- src/TSN/XML/EarlyLine.hs +627/−0
- src/TSN/XML/GameInfo.hs +1/−1
- src/TSN/XML/MLBEarlyLine.hs +136/−0
- src/TSN/XML/News.hs +1/−1
- src/TSN/XML/Odds.hs +66/−21
- src/TSN/XML/SportInfo.hs +1/−1
- test/TestSuite.hs +6/−0
- test/shell/import-duplicates.test +3/−3
- test/xml/AutoRacingResultsXML-fractional-kph.xml +1/−0
- test/xml/MLB_earlylineXML.dtd +22/−0
- test/xml/MLB_earlylineXML.xml +1/−0
- test/xml/Odds_XML-missing-starters.xml +1/−0
- test/xml/earlylineXML.dtd +20/−0
- test/xml/earlylineXML.xml +1/−0
doc/TODO view
@@ -36,11 +36,9 @@ * CBATeamScheduleXML * CFLTeamScheduleXML * CFLTotalTeamScheduleXML- * earlylineXML * Minor_Baseball_TeamScheduleXML * MinorLeagueHockeyTeamScheduleXML * MLB_Boxscore_XML- * MLB_earlylineXML * MLB_IndividualStats_XML * MLB_Probable_Pitchers_XML * MLB_Roster_XML@@ -65,3 +63,8 @@ 5. Consolidate all of the make_game_time functions which take a date/time and produce a combined time.++6. Factor out test code where possible; a lot of them differ only in+ the filename.++7. Combine test XML files where possible.
+ doc/dbschema/MLB_earlylineXML.png view
binary file changed (absent → 59639 bytes)
+ doc/dbschema/earlylineXML.png view
binary file changed (absent → 59639 bytes)
doc/man1/htsn-import.1 view
@@ -106,6 +106,26 @@ should be considered a bug if they are incorrect. The diagrams are created using the pgModeler <http://www.pgmodeler.com.br/> tool. +.SH DATABASE SCHEMA COMPROMISES++There are a few places that the database schema isn't exactly how we'd+like it to be:++.IP \[bu] 2+\fIearlylineXML.dtd\fR++The database representations for earlylineXML.dtd and+MLB_earlylineXML.dtd are the same; that is, they share the same+tables. The two document types represent team names in different+ways. In order to accomodate both types with one parser, we had to+make both ways optional, and then merge the two together before+converting to the database representation.++Unfortunately, when we merge two optional things together, we get+another optional thing back. There's no way to say that \(dqat least+one is not optional.\(dq So the team names in the database schema are+optional as well, even though they should always be present.+ .SH NULL POLICY .P Normally in a database one makes a distinction between fields that@@ -204,8 +224,8 @@ Processed 1 document(s) total. .fi .P-At this point, the database schema matches the old documents, i.e. the-ones without \fIAStarter\fR and \fIHStarter\fR. If we use a new+At this point, the database schema matches the old documents, that is,+the ones without \fIAStarter\fR and \fIHStarter\fR. If we use a new version of \fBhtsn-import\fR, supporting the new fields, the migration is handled gracefully: .P@@ -285,6 +305,33 @@ is provided as test/xml/newsxml-multiple-sms.xml. .IP \[bu]+\fIMLB_earlylineXML.dtd\fR++Unlike earlylineXML.dtd, this document type has more than one <game>+associated with each <date>. Moreover, each <date> has a bunch of+<note> children that are supposed to be associated with the <game>s,+but the document structure indicates no explicit relationship. For+example,++.nf+<date>+ <note>...</note>+ <game>...</game>+ <game>...</game>+ <note>...</note>+ <game>...</game>+</date>+.fi++Here the first <note> is inferred to apply to the two <game>s that+follow it, and the second <note> applies to the single <game> that+follows it. But this is very fragile to parse. Instead, we use a hack+to facilitate (un)pickling, and then drop the notes entirely during+the database conversion.++A similar workaround is implemented for Odds_XML.dtd.++.IP \[bu] \fIOdds_XML.dtd\fR The <Notes> elements here are supposed to be associated with a set of@@ -316,6 +363,144 @@ offending document can be removed if desired. An example is provided as test/xml/weatherxml-backwards-teams.xml. +.SH DATE/TIME ISSUES++Dates and times appear in a number of places on the feed. The date+portions are usually, fine, but the times often lack important+information such as the time zone, or whether \(dq8 o'clock\(dq means+a.m. or p.m.++The most pervasive issue occurs with the timestamps that are included+in every message. A typical timestamp looks like,++.nf+<time_stamp> May 24, 2014, at 04:18 PM ET </time_stamp>+.fi++The \(dqtime zone\(dq is given as \(dqET\(dq, but unfortunately+\(dqET\(dq is not a valid time zone. It stands for \(dqEastern+Time\(dq, which can belong to either of two time zones, EST or EDT,+based on the time of the year (that is, whether or not daylight+savings time is in effect) and one's location (for example, Arizona+doesn't observe daylight savings time). It's not much more useful to+be off by one hour than it is to be off by five hours, and since we+can't determine the true offset from the timestamp, we always parse+and store these as UTC.++Here's a list of the ones that may cause surprises:++.IP \[bu] 2+\fIAutoRacingResultsXML.dtd\fR++The <RaceDate> elements contain a full date and time, but no time zone+information:++.nf+<RaceDate>5/24/2014 2:45:00 PM</RaceDate>+.fi++We parse them as UTC, which will be wrong when stored,+but \(dqcorrect\(dq if the new UTC time zone is ignored.++.IP \[bu]+\fIAuto_Racing_Schedule_XML.dtd\fR++The <Race_Date> and <Race_Time> elements are combined into on field in+the database, but no time zone information is given. For example,++.nf+<Race_Date>02/16/2013</Race_Date>+<Race_Time>08:10 PM</Race_Time>+.fi++As a result, we parse and store the times as UTC. The race times are+not always present in the database, but when they are missing, they+are presented as \(dqTBA\(dq (to be announced):++.nf+<Race_Time>TBA</Race_Time>+.fi++Since the dates do not appear to be optional, we store only the race+date in that case.++.IP \[bu]+\fIearlylineXML.dtd\fR++The <time> elements in the early lines contain neither a time zone nor+an am/pm identifier:++.nf+<time>8:30</time>+.fi++The times are parsed and stored as UTC, since we+don't have any other information upon which to base a guess. Even if+one ignores the UTC time zone, the time can possibly be off by 12+hours (due to the a.m./p.m. issue).++.IP \[bu]+\fIjfilexml.dtd\fR++The <Game_Date> and <Game_Time> elements are combined into on field in+the database, but no time zone information is given. For example,++.nf+<Game_Date>06/15/2014</Game_Date>+<Game_Time>08:00 PM</Game_Time>+.fi++As a result, we parse and store the times as UTC.++The <CurrentTimestamp> elements suffer a similar problem, sans the+date:++.nf+<CurrentTimeStamp>11:30 A.M.</CurrentTimeStamp>+.fi++They are also stored as UTC.++.IP \[bu]+\fIMLB_earlylineXML.dtd\fR++See earlylineXML.dtd.++.IP \[bu]+\fIOdds_XML.dtd\fR++The <Game_Date> and <Game_Time> elements are combined into on field in+the database, but no time zone information is given. For example,++.nf+<Game_Date>01/04/2014</Game_Date>+<Game_Time>04:35 PM</Game_Time>+.fi++As a result, we parse and store the times as UTC.++.IP \[bu]+\fISchedule_Changes_XML.dtd\fR++The <Game_Date> and <Game_Time> elements are combined into on field in+the database, but no time zone information is given. For example,++.nf+<Game_Date>06/06/2014</Game_Date>+<Game_Time>04:00 PM</Game_Time>+.fi++As a result, we parse and store the times as UTC. The game times are+not always present in the database, but when they are missing, they+are presented as \(dqTBA\(dq (to be announced):++.nf+<Game_Time>TBA</Game_Time>+.fi++Since the dates do not appear to be optional, we store only the game+date in that case.+ .SH DEPLOYMENT .P When deploying for the first time, the target database will most@@ -448,6 +633,8 @@ .IP \[bu] Auto_Racing_Schedule_XML.dtd .IP \[bu]+earlylineXML.dtd+.IP \[bu] Heartbeat.dtd .IP \[bu] Injuries_Detail_XML.dtd@@ -455,6 +642,8 @@ injuriesxml.dtd .IP \[bu] jfilexml.dtd+.IP \[bu]+MLB_earlylineXML.dtd .IP \[bu] newsxml.dtd .IP \[bu]
htsn-import.cabal view
@@ -1,5 +1,5 @@ name: htsn-import-version: 0.0.8+version: 0.0.9 cabal-version: >= 1.8 author: Michael Orlitzky maintainer: Michael Orlitzky <michael@orlitzky.com>@@ -48,6 +48,7 @@ schemagen/Cbask_Tourn_MVP_XML/*.xml schemagen/Cbask_Tourn_Records_XML/*.xml schemagen/cflpreviewxml/*.xml+ schemagen/earlylineXML/*.xml schemagen/Heartbeat/*.xml schemagen/Injuries_Detail_XML/*.xml schemagen/injuriesxml/*.xml@@ -75,6 +76,7 @@ schemagen/MLB_Matchup_XML/*.xml schemagen/mlbonbasepctxml/*.xml schemagen/MLBOPSXML/*.xml+ schemagen/MLB_earlylineXML/*.xml schemagen/MLB_Pitching_Appearances_Leaders/*.xml schemagen/MLB_Pitching_Balks_Leaders/*.xml schemagen/MLB_Pitching_CG_Leaders/*.xml@@ -273,11 +275,13 @@ TSN.XmlImport TSN.XML.AutoRacingResults TSN.XML.AutoRacingSchedule+ TSN.XML.EarlyLine TSN.XML.GameInfo TSN.XML.Heartbeat TSN.XML.Injuries TSN.XML.InjuriesDetail TSN.XML.JFile+ TSN.XML.MLBEarlyLine TSN.XML.News TSN.XML.Odds TSN.XML.ScheduleChanges
makefile view
@@ -17,13 +17,21 @@ --prefix=/ runghc Setup.hs build -doc: $(PN).cabal $(SRCS)+dist/doc: $(PN).cabal $(SRCS) runghc Setup.hs configure --user --prefix=/ runghc Setup.hs hscolour --all runghc Setup.hs haddock --all \ --hyperlink-source \ --haddock-options="--ignore-all-exports" +doc: dist/doc+++# The MLB schema is identical to the regular one.+doc/dbschema/MLB_earlylineXML.png: doc/dbschema/earlylineXML.png+ cp $< $@++ # # Testing. #@@ -41,9 +49,13 @@ # # Misc. #-dist:++# Only generate MLB_earlylineXML.png long enough to create+# the tarball.+dist: doc/dbschema/MLB_earlylineXML.png runghc Setup.hs configure --prefix=/ TAR_OPTIONS="--format=ustar" runghc Setup.hs sdist+ rm $< hlint: hlint --ignore="Use camelCase" \
+ schema/MLB_earlylineXML.dtd view
@@ -0,0 +1,22 @@+<!ELEMENT XML_File_ID (#PCDATA)>+<!ELEMENT heading (#PCDATA)>+<!ELEMENT category (#PCDATA)>+<!ELEMENT sport (#PCDATA)>+<!ELEMENT title (#PCDATA)>+<!ELEMENT note (#PCDATA)>+<!ELEMENT time (#PCDATA)>+<!ELEMENT pitcher (#PCDATA)>+<!ELEMENT line (#PCDATA)>+<!ELEMENT teamA ( ( pitcher, line ) )>+<!ELEMENT teamH ( ( pitcher, line ) )>+<!ELEMENT over_under (#PCDATA)>+<!ELEMENT game ( ( time, teamA, teamH, over_under ) )>+<!ELEMENT date ( ( note | game )+ )>+<!ELEMENT time_stamp (#PCDATA)>+<!ELEMENT message ( ( XML_File_ID, heading, category, sport, title, date, time_stamp ) )>++<!ATTLIST teamA rotation CDATA #REQUIRED>+<!ATTLIST teamA name CDATA #REQUIRED>+<!ATTLIST teamH rotation CDATA #REQUIRED>+<!ATTLIST teamH name CDATA #REQUIRED>+<!ATTLIST date value CDATA #REQUIRED>
+ schema/earlylineXML.dtd view
@@ -0,0 +1,20 @@+<!ELEMENT XML_File_ID (#PCDATA)>+<!ELEMENT heading (#PCDATA)>+<!ELEMENT category (#PCDATA)>+<!ELEMENT sport (#PCDATA)>+<!ELEMENT title (#PCDATA)>+<!ELEMENT note (#PCDATA)>+<!ELEMENT time (#PCDATA)>+<!ELEMENT teamA (#PCDATA)>+<!ELEMENT teamH (#PCDATA)>+<!ELEMENT over_under (#PCDATA)>+<!ELEMENT game ( ( time, teamA, teamH, over_under ) )>+<!ELEMENT date ( ( note, game ) )>+<!ELEMENT time_stamp (#PCDATA)>+<!ELEMENT message ( ( XML_File_ID, heading, category, sport, title, date*, time_stamp ) )>++<!ATTLIST teamA rotation CDATA #REQUIRED>+<!ATTLIST teamA line CDATA #REQUIRED>+<!ATTLIST teamH rotation CDATA #REQUIRED>+<!ATTLIST teamH line CDATA #REQUIRED>+<!ATTLIST date value CDATA #REQUIRED>
+ schemagen/MLB_earlylineXML/21160887.xml view
@@ -0,0 +1,1 @@+<?xml version="1.0" standalone="no" ?> <!DOCTYPE message PUBLIC "-//TSN//DTD Odds 1.0/EN" "MLB_earlylineXML.dtd"> <message> <XML_File_ID>21160887</XML_File_ID> <heading>AAO;MLB-EARLY-LINE</heading> <category>Odds</category> <sport>MLB</sport> <title>Major League Baseball Overnight Line</title> <date value="SATURDAY, MAY 24TH (05/24/2014)"> <note>National League</note> <game> <time>3:05</time> <teamA rotation="901" name="LOS"> <pitcher>D.Haren</pitcher> <line>-130</line> </teamA> <teamH rotation="902" name="PHI"> <pitcher>D.Buchanan</pitcher> <line></line> </teamH> <over_under>8.5o</over_under> </game> <game> <time>4:10</time> <teamA rotation="903" name="COL"> <pitcher>J.Nicasio</pitcher> <line></line> </teamA> <teamH rotation="904" name="ATL"> <pitcher>M.Minor</pitcher> <line>-130</line> </teamH> <over_under>7.5p</over_under> </game> <game> <time>4:10</time> <teamA rotation="905" name="MIL"> <pitcher>W.Peralta</pitcher> <line></line> </teamA> <teamH rotation="906" name="MIA"> <pitcher>J.Turner</pitcher> <line>-105</line> </teamH> <over_under>8p</over_under> </game> <game> <time>4:10</time> <teamA rotation="907" name="ARI"> <pitcher>J.Collmenter</pitcher> <line></line> </teamA> <teamH rotation="908" name="NYM"> <pitcher>Z.Wheeler</pitcher> <line>-120</line> </teamH> <over_under>7.5p</over_under> </game> <game> <time>7:15</time> <teamA rotation="909" name="STL"> <pitcher>J.Garcia</pitcher> <line></line> </teamA> <teamH rotation="910" name="CIN"> <pitcher>T.Cingrani</pitcher> <line>-105</line> </teamH> <over_under>7o</over_under> </game> <game> <time>7:15</time> <teamA rotation="911" name="WAS"> <pitcher>S.Strasburg</pitcher> <line></line> </teamA> <teamH rotation="912" name="PIT"> <pitcher>G.Cole</pitcher> <line>-105</line> </teamH> <over_under>6.5o</over_under> </game> <game> <time>10:10</time> <teamA rotation="913" name="CHC"> <pitcher>T.Wood</pitcher> <line></line> </teamA> <teamH rotation="914" name="SDP"> <pitcher>B.Buckner</pitcher> <line>off</line> </teamH> <over_under>off</over_under> </game> <note>American League</note> <game> <time>1:07</time> <teamA rotation="915" name="OAK"> <pitcher>J.Chavez</pitcher> <line>-120</line> </teamA> <teamH rotation="916" name="TOR"> <pitcher>R.Dickey</pitcher> <line></line> </teamH> <over_under>9p</over_under> </game> <game> <time>2:10</time> <teamA rotation="917" name="NYY"> <pitcher>V.Nuno</pitcher> <line></line> </teamA> <teamH rotation="918" name="CWS"> <pitcher>J.Danks</pitcher> <line>-105</line> </teamH> <over_under>9.5p</over_under> </game> <game> <time>4:08</time> <teamA rotation="919" name="TEX"> <pitcher>N.Martinez</pitcher> <line></line> </teamA> <teamH rotation="920" name="DET"> <pitcher>R.Porcello</pitcher> <line>-170</line> </teamH> <over_under>9u</over_under> </game> <game> <time>4:10</time> <teamA rotation="921" name="BOS"> <pitcher>J.Peavy</pitcher> <line></line> </teamA> <teamH rotation="922" name="TAM"> <pitcher>D.Price</pitcher> <line>-135</line> </teamH> <over_under>7o</over_under> </game> <note>Time change</note> <game> <time>12:35</time> <teamA rotation="923" name="CLE"> <pitcher>C.Kluber</pitcher> <line></line> </teamA> <teamH rotation="924" name="BAL"> <pitcher>U.Jimenez</pitcher> <line>-110</line> </teamH> <over_under>8.5u</over_under> </game> <game> <time>7:15</time> <teamA rotation="925" name="KAN"> <pitcher>J.Shields</pitcher> <line>-110</line> </teamA> <teamH rotation="926" name="ANA"> <pitcher>M.Shoemaker</pitcher> <line></line> </teamH> <over_under>7.5u</over_under> </game> <game> <time>10:10</time> <teamA rotation="927" name="HOU"> <pitcher>B.Oberholtzer</pitcher> <line></line> </teamA> <teamH rotation="928" name="SEA"> <pitcher>B.Maurer</pitcher> <line>off</line> </teamH> <over_under>off</over_under> </game> <note>Inter League</note> <game> <time>10:05</time> <teamA rotation="929" name="MIN"> <pitcher>S.Deduno</pitcher> <line></line> </teamA> <teamH rotation="930" name="SFG"> <pitcher>R.Vogelsong</pitcher> <line>off</line> </teamH> <over_under>off</over_under> </game> </date> <time_stamp> May 24, 2014, at 09:23 AM ET </time_stamp> </message>
+ schemagen/earlylineXML/21166989.xml view
@@ -0,0 +1,1 @@+<?xml version="1.0" standalone="no" ?> <!DOCTYPE message PUBLIC "-//TSN//DTD Odds 1.0/EN" "earlylineXML.dtd"> <message> <XML_File_ID>21166989</XML_File_ID> <heading>ACO;NBA-EARLY-LINE</heading> <category>Odds</category> <sport>NBA</sport> <title>National Basketball Association Overnight Line</title> <date value="SUNDAY, MAY 25TH (05/25/2014)"> <note>Western Conference Finals - San Antonio leads 2-0</note> <game> <time>8:30</time> <teamA rotation="511" line="">San Antonio</teamA> <teamH rotation="512" line="-2">Oklahoma City</teamH> <over_under>208</over_under> </game> </date> <date value="MONDAY, MAY 26TH (05/26/2014)"> <note>Eastern Conference Finals - Miami leads 2-1</note> <game> <time>8:30</time> <teamA rotation="513" line="">Indiana</teamA> <teamH rotation="514" line="off">Miami</teamH> <over_under>off</over_under> </game> </date> <time_stamp> May 25, 2014, at 09:06 AM ET </time_stamp> </message>
src/Main.hs view
@@ -51,12 +51,18 @@ import qualified TSN.XML.AutoRacingSchedule as AutoRacingSchedule ( dtd, pickle_message )+import qualified TSN.XML.EarlyLine as EarlyLine (+ dtd,+ pickle_message ) import qualified TSN.XML.GameInfo as GameInfo ( dtds, parse_xml ) import qualified TSN.XML.Heartbeat as Heartbeat ( dtd, verify ) import qualified TSN.XML.Injuries as Injuries ( dtd, pickle_message ) import qualified TSN.XML.InjuriesDetail as InjuriesDetail ( dtd, pickle_message )+import qualified TSN.XML.MLBEarlyLine as MLBEarlyLine (+ dtd,+ pickle_message ) import qualified TSN.XML.JFile as JFile ( dtd, pickle_message ) import qualified TSN.XML.News as News ( dtd,@@ -194,12 +200,18 @@ | dtd == AutoRacingSchedule.dtd = go AutoRacingSchedule.pickle_message + | dtd == EarlyLine.dtd =+ go EarlyLine.pickle_message+ -- GameInfo and SportInfo appear last in the guards | dtd == Injuries.dtd = go Injuries.pickle_message | dtd == InjuriesDetail.dtd = go InjuriesDetail.pickle_message | dtd == JFile.dtd = go JFile.pickle_message++ | dtd == MLBEarlyLine.dtd =+ go MLBEarlyLine.pickle_message | dtd == News.dtd = -- Some of the newsxml docs are busted in predictable ways.
src/TSN/Parse.hs view
@@ -8,7 +8,7 @@ where import Data.Either.Utils ( maybeToEither )-import Data.Time.Clock ( NominalDiffTime, UTCTime, addUTCTime )+import Data.Time.Clock ( UTCTime ) import Data.Time.Format ( parseTime ) import System.Locale ( defaultTimeLocale ) import Text.Read ( readMaybe )@@ -89,27 +89,27 @@ time_format :: String time_format = "%I:%M %p" --- | The format string for a time_stamp. This omits the leading and--- trailing space.++-- | The format string for a time_stamp. We keep the leading/trailing+-- space so that parseTime and formatTime are inverses are one+-- another, even though there is some confusion as to how these two+-- functions should behave:+--+-- <https://ghc.haskell.org/trac/ghc/ticket/9150>+-- time_stamp_format :: String-time_stamp_format = "%B %-d, %Y, at " ++ time_format ++ " ET"+time_stamp_format = " %B %-d, %Y, at " ++ time_format ++ " ET " --- | Parse a time stamp from a 'String' (maybe).------ TSN doesn't provide a proper time zone name, so we assume that--- it's always Eastern Standard Time. EST is UTC-5, so we--- add five hours to convert to UTC.++-- | Parse a time stamp from a 'String' (maybe). TSN doesn't provide a+-- proper time zone name, so we parse it as UTC, and maybe our+-- eventual consumer can figure out a way to deduce the time zone. -- parse_time_stamp :: String -> Maybe UTCTime parse_time_stamp =- fmap add_five . parseTime defaultTimeLocale time_stamp_format- where- five_hours :: NominalDiffTime- five_hours = 5 * 60 * 60+ parseTime defaultTimeLocale time_stamp_format - add_five :: UTCTime -> UTCTime- add_five = addUTCTime five_hours -- | Extract the \"time_stamp\" element from a document. If we fail
src/TSN/Picklers.hs view
@@ -2,10 +2,14 @@ -- feed. -- module TSN.Picklers (+ pickler_tests,+ xp_ambiguous_time, xp_date, xp_date_padded, xp_datetime,+ xp_early_line_date, xp_earnings,+ xp_fracpart_only_double, xp_gamedate, xp_tba_time, xp_time,@@ -14,17 +18,29 @@ where -- System imports.+import Data.Char ( toUpper ) import Data.List ( intercalate ) import Data.List.Split ( chunksOf )+import Data.Maybe ( catMaybes, listToMaybe ) import Data.String.Utils ( replace )-import Data.Time.Clock ( NominalDiffTime, UTCTime, addUTCTime )+import Data.Time.Clock ( UTCTime ) import Data.Time.Format ( formatTime, parseTime )-import System.Locale ( defaultTimeLocale )+import Data.Tree.NTree.TypeDefs ( NTree(..) )+import System.Locale ( TimeLocale( wDays, months ), defaultTimeLocale )+import Test.Tasty ( TestTree, testGroup )+import Test.Tasty.HUnit ( (@?=), testCase )+import Text.Read ( readMaybe ) import Text.XML.HXT.Arrow.Pickle ( xpText, xpWrap, xpWrapMaybe ) import Text.XML.HXT.Arrow.Pickle.Xml ( PU )+import Text.XML.HXT.Core (+ XmlTree,+ XNode( XTag, XText ),+ mkName,+ pickleDoc,+ unpickleDoc ) -- Local imports. import TSN.Parse (@@ -58,6 +74,21 @@ -- | (Un)pickle a UTCTime without the time portion. --+-- /Examples/:+--+-- This should parse:+--+-- >>> let tn = text_node "2/15/1983"+-- >>> unpickleDoc xp_date tn+-- Just 1983-02-15 00:00:00 UTC+--+-- But for some reason, it can also parse a leading zero in the+-- month. Whatever. This isn't required behavior.+--+-- >>> let tn = text_node "02/15/1983"+-- >>> unpickleDoc xp_date tn+-- Just 1983-02-15 00:00:00 UTC+-- xp_date :: PU UTCTime xp_date = (to_date, from_date) `xpWrapMaybe` xpText@@ -72,6 +103,16 @@ -- | (Un)pickle a UTCTime without the time portion. The day/month are -- padded to two characters with zeros. --+-- Examples:+--+-- >>> let tn = text_node "02/15/1983"+-- >>> unpickleDoc xp_date_padded tn+-- Just 1983-02-15 00:00:00 UTC+--+-- >>> let tn = text_node "06/07/2014"+-- >>> unpickleDoc xp_date_padded tn+-- Just 2014-06-07 00:00:00 UTC+-- xp_date_padded :: PU UTCTime xp_date_padded = (to_date, from_date) `xpWrapMaybe` xpText@@ -108,6 +149,8 @@ format_commas x = reverse (intercalate "," $ chunksOf 3 $ reverse $ show x) ++ -- | Parse \<Earnings\> from an 'AutoRaceResultsListing'. These are -- essentially 'Int's, but they look like, --@@ -117,6 +160,16 @@ -- -- * \<Earnings\>TBA\</Earnings\> --+-- Examples:+--+-- >>> let tn = text_node "1,000,191"+-- >>> unpickleDoc xp_earnings tn+-- Just (Just 1000191)+--+-- >>> let tn = text_node "TBA"+-- >>> unpickleDoc xp_earnings tn+-- Just Nothing+-- xp_earnings :: PU (Maybe Int) xp_earnings = (to_earnings, from_earnings) `xpWrap` xpText@@ -134,15 +187,66 @@ from_earnings (Just i) = format_commas i ++-- | Pickle a 'Double' that can be missing its leading zero (for+-- values less than one). For example, we've seen,+--+-- <TrackLength KPH=".805">0.5</TrackLength>+--+-- Which 'xpPrim' can't handle without the leading+-- zero. Unfortunately there's no way pickle/unpickle can be+-- inverses of each other here, since \"0.5\" and \".5\" should+-- unpickle to the same 'Double'.+--+-- Examples:+--+-- >>> let tn = text_node "0.5"+-- >>> unpickleDoc xp_fracpart_only_double tn+-- Just 0.5+--+-- >>> let tn = text_node ".5"+-- >>> unpickleDoc xp_fracpart_only_double tn+-- Just 0.5+--+-- >>> let tn = text_node "foo"+-- >>> unpickleDoc xp_fracpart_only_double tn+-- Nothing+--+xp_fracpart_only_double :: PU Double+xp_fracpart_only_double =+ (to_double, from_double) `xpWrapMaybe` xpText+ where+ -- | Convert a 'String' to a 'Double', maybe. We always prepend a+ -- zero, since it will fix the fraction-only values, and not hurt+ -- the ones that already have a leading integer.+ to_double :: String -> Maybe Double+ to_double s = readMaybe ("0" ++ s)++ from_double :: Double -> String+ from_double = show+++ -- | (Un)pickle an unpadded 'UTCTime'. Used for example on the -- \<RaceDate\> elements in an 'AutoRaceResults' message. -- -- Examples: ----- * \<RaceDate\>6/1/2014 1:00:00 PM\</RaceDate\>+-- >>> let tn = text_node "6/1/2014 1:00:00 PM"+-- >>> unpickleDoc xp_datetime tn+-- Just 2014-06-01 13:00:00 UTC ----- * \<RaceDate\>5/24/2014 2:45:00 PM\</RaceDate\>+-- >>> let tn = text_node "5/24/2014 2:45:00 PM"+-- >>> unpickleDoc xp_datetime tn+-- Just 2014-05-24 14:45:00 UTC --+-- Padded! For some reason it works with only one zero in front. I+-- dunno man. NOT required (or even desired?) behavior.+--+-- >>> let tn = text_node "05/24/2014 2:45:00 PM"+-- >>> unpickleDoc xp_datetime tn+-- Just 2014-05-24 14:45:00 UTC+-- xp_datetime :: PU UTCTime xp_datetime = (to_datetime, from_datetime) `xpWrapMaybe` xpText@@ -156,15 +260,53 @@ from_datetime = formatTime defaultTimeLocale format ++-- | Takes a 'UTCTime', and returns the English suffix that would be+-- appropriate after the day of the month. For example, if we have a+-- UTCTime representing Christmas, this would return \"th\" because+-- \"th\" is the right suffix of \"December 25th\".+--+-- Examples:+--+-- >>> import Data.Maybe ( fromJust )+-- >>> :{+-- let parse_date :: String -> Maybe UTCTime;+-- parse_date = parseTime defaultTimeLocale date_format;+-- :}+--+-- >>> let dates = [ "1/" ++ (d : "/1970") | d <- ['1'..'9'] ]+-- >>> let suffixes = map (date_suffix . fromJust . parse_date) dates+-- >>> suffixes+-- ["st","nd","rd","th","th","th","th","th","th"]+--+date_suffix :: UTCTime -> String+date_suffix t =+ case (reverse daystr) of+ [] -> []+ ('1':_) -> "st"+ ('2':_) -> "nd"+ ('3':_) -> "rd"+ _ -> "th"+ where+ daystr = formatTime defaultTimeLocale "%d" t++ -- | (Un)pickle a UTCTime from a weather forecast's gamedate. Example -- input looks like, ----- \<forecast gamedate=\"Monday, December 30th\"\>--- -- When unpickling we get rid of the suffixes \"st\", \"nd\", \"rd\", and -- \"th\". During pickling, we add them back based on the last digit -- of the date. --+-- Examples:+--+-- >>> let tn = text_node "Monday, December 30th"+-- >>> let (Just gd) = unpickleDoc xp_gamedate tn+-- >>> gd+-- 1970-12-30 00:00:00 UTC+-- >>> pickleDoc xp_gamedate gd+-- NTree (XTag "/" []) [NTree (XText "Wednesday, December 30th") []]+-- xp_gamedate :: PU UTCTime xp_gamedate = (to_gamedate, from_gamedate) `xpWrapMaybe` xpText@@ -178,37 +320,42 @@ s' = case (reverse s) of (c2:c1:cs) -> let suffix = [c1,c2] in- case suffix of- "st" -> reverse cs- "nd" -> reverse cs- "rd" -> reverse cs- "th" -> reverse cs- _ -> s -- Unknown suffix, leave it alone.+ if suffix `elem` ["st","nd","rd","th"]+ then reverse cs+ else s -- Unknown suffix, leave it alone.+ _ -> s -- The String is less than two characters long, -- leave it alone. from_gamedate :: UTCTime -> String- from_gamedate d = s ++ (suffix s)+ from_gamedate d = s ++ (date_suffix d) where s = formatTime defaultTimeLocale format d - suffix :: String -> String- suffix cs =- case (reverse cs) of- [] -> []- ('1':_) -> "st"- ('2':_) -> "nd"- ('3':_) -> "rd"- _ -> "th" --- | (Un)pickle a UTCTime without the date portion.+-- | (Un)pickle a UTCTime without the date portion. Doesn't work if+-- the fields aren't zero-padded to two characters. --+-- /Examples/:+--+-- Padded, should work:+--+-- >>> let tn = text_node "04:35 PM"+-- >>> unpickleDoc xp_time tn+-- Just 1970-01-01 16:35:00 UTC+--+-- Unpadded, should fail:+--+-- >>> let tn = text_node "4:35 PM"+-- >>> unpickleDoc xp_time tn+-- Nothing+-- xp_time :: PU UTCTime xp_time = (to_time, from_time) `xpWrapMaybe` xpText@@ -224,10 +371,23 @@ -- 'xp_time' in that it uses periods in the AM/PM part, i.e. \"A.M.\" -- and \"P.M.\" It also doesn't use padding for the \"hours\" part. ----- Examples:+-- /Examples/: ----- * \<CurrentTimeStamp\>11:30 A.M.\</CurrentTimeStamp\>+-- A standard example of the correct form: --+-- >>> let tn = text_node "11:30 A.M."+-- >>> let (Just result) = unpickleDoc xp_time_dots tn+-- >>> result+-- 1970-01-01 11:30:00 UTC+-- >>> pickleDoc xp_time_dots result+-- NTree (XTag "/" []) [NTree (XText "11:30 A.M.") []]+--+-- Another miracle, it still parses with a leading zero!+--+-- >>> let tn = text_node "01:30 A.M."+-- >>> unpickleDoc xp_time_dots tn+-- Just 1970-01-01 01:30:00 UTC+-- xp_time_dots :: PU UTCTime xp_time_dots = (to_time, from_time) `xpWrapMaybe` xpText@@ -249,6 +409,31 @@ -- | (Un)pickle a UTCTime without the date portion, allowing for a -- value of \"TBA\" (which gets translated to 'Nothing'). --+-- /Examples/:+--+-- A failed parse will return 'Nothing':+--+-- >>> let tn = text_node "YO"+-- >>> unpickleDoc xp_tba_time tn+-- Just Nothing+--+-- And so will parsing a \"TBA\":+--+-- >>> let tn = text_node "TBA"+-- >>> unpickleDoc xp_tba_time tn+-- Just Nothing+--+-- But re-pickling 'Nothing' gives only \"TBA\":+--+-- >>> pickleDoc xp_tba_time Nothing+-- NTree (XTag "/" []) [NTree (XText "TBA") []]+--+-- A normal time is also parsed successfully, of course:+--+-- >>> let tn = text_node "08:10 PM"+-- >>> unpickleDoc xp_tba_time tn+-- Just (Just 1970-01-01 20:10:00 UTC)+-- xp_tba_time :: PU (Maybe UTCTime) xp_tba_time = (to_time, from_time) `xpWrap` xpText@@ -265,23 +450,161 @@ -- | (Un)pickle the \<time_stamp\> element format to/from a 'UTCTime'.+-- The time_stamp elements look something like, ----- Example: \<time_stamp\> January 6, 2014, at 10:11 PM ET \</time_stamp\>+-- \<time_stamp\> January 6, 2014, at 10:11 PM ET \</time_stamp\> ----- TSN doesn't provide a proper time zone name, so we assume that--- it's always Eastern Standard Time. EST is UTC-5, so we--- add/subtract 5 hours to convert to/from UTC.+-- TSN doesn't provide a proper time zone name, only \"ET\" for+-- \"Eastern Time\". But \"Eastern Time\" changes throughout the+-- year, depending on one's location, for daylight-savings+-- time. It's really not any more useful to be off by one hour than+-- it is to be off by 5 hours, so rather than guess at EDT/EST, we+-- just store the timestamp as UTC. --+-- Examples:+--+-- >>> let tn = text_node " January 6, 2014, at 10:11 PM ET "+-- >>> let (Just tstamp) = unpickleDoc xp_time_stamp tn+-- >>> tstamp+-- 2014-01-06 22:11:00 UTC+-- >>> pickleDoc xp_time_stamp tstamp+-- NTree (XTag "/" []) [NTree (XText " January 6, 2014, at 10:11 PM ET ") []]+-- xp_time_stamp :: PU UTCTime xp_time_stamp = (parse_time_stamp, from_time_stamp) `xpWrapMaybe` xpText where- five_hours :: NominalDiffTime- five_hours = 5 * 60 * 60-- subtract_five :: UTCTime -> UTCTime- subtract_five = addUTCTime (-1 * five_hours)- from_time_stamp :: UTCTime -> String from_time_stamp =- formatTime defaultTimeLocale time_stamp_format . subtract_five+ formatTime defaultTimeLocale time_stamp_format++++-- | (Un)pickle an ambiguous 12-hour AM/PM time, which is ambiguous+-- because it's missing the AM/PM part.+--+-- Examples:+--+-- >>> let tn = text_node "8:00"+-- >>> unpickleDoc xp_ambiguous_time tn+-- Just 1970-01-01 08:00:00 UTC+--+xp_ambiguous_time :: PU UTCTime+xp_ambiguous_time =+ (to_time, from_time) `xpWrapMaybe` xpText+ where+ ambiguous_time_format :: String+ ambiguous_time_format = "%-I:%M"++ to_time :: String -> Maybe UTCTime+ to_time = parseTime defaultTimeLocale ambiguous_time_format++ from_time :: UTCTime -> String+ from_time =+ formatTime defaultTimeLocale ambiguous_time_format+++-- | Pickle a date value from a \<date\> element as they appear in the+-- early lines. This is a particularly wacky format, but then so is+-- the associated time (see 'xp_ambiguous_time').+--+-- Examples:+--+-- >>> let tn = text_node "SUNDAY, MAY 25TH (05/25/2014)"+-- >>> let (Just result) = unpickleDoc xp_early_line_date tn+-- >>> result+-- 2014-05-25 00:00:00 UTC+-- >>> pickleDoc xp_early_line_date result+-- NTree (XTag "/" []) [NTree (XText "SUNDAY, MAY 25TH (05/25/2014)") []]+--+-- >>> let tn = text_node "SATURDAY, JUNE 7TH (06/07/2014)"+-- >>> let (Just result) = unpickleDoc xp_early_line_date tn+-- >>> result+-- 2014-06-07 00:00:00 UTC+-- >>> pickleDoc xp_early_line_date result+-- NTree (XTag "/" []) [NTree (XText "SATURDAY, JUNE 7TH (06/07/2014)") []]+--+xp_early_line_date :: PU UTCTime+xp_early_line_date =+ (to_time, from_time) `xpWrapMaybe` xpText+ where+ -- | We need to create our own time locale that talks IN ALL CAPS.+ -- Actually, 'parseTime' doesn't seem to care about the+ -- case. But when we spit it back out again ('formatTime'),+ -- we'll want it to be in all caps.+ --+ caps_time_locale :: TimeLocale+ caps_time_locale =+ defaultTimeLocale { wDays = caps_days, months = caps_months }++ caps_days :: [(String,String)]+ caps_days = map both_to_upper (wDays defaultTimeLocale)++ caps_months :: [(String,String)]+ caps_months = map both_to_upper (months defaultTimeLocale)++ both_to_upper :: (String,String) -> (String,String)+ both_to_upper (s1,s2) = (map toUpper s1, map toUpper s2)++ wacko_date_formats :: [String]+ wacko_date_formats =+ ["%A, %B %-d" ++ suffix ++ " (" ++ date_format_padded ++ ")" |+ suffix <- ["ST", "ND", "RD","TH"] ]++ to_time :: String -> Maybe UTCTime+ to_time s =+ listToMaybe $ catMaybes possible_parses+ where+ possible_parses = [ parseTime caps_time_locale fmt s |+ fmt <- wacko_date_formats ]++ from_time :: UTCTime -> String+ from_time t =+ formatTime caps_time_locale fmt t+ where+ upper_suffix = map toUpper (date_suffix t)+ fmt = "%A, %B %-d" ++ upper_suffix ++ " (" ++ date_format_padded ++ ")"++++-- | Create an 'XmlTree' containing only the given text. This is+-- useful for testing (un)picklers, where we don't want to have to+-- bother to create a dummy XML document.+--+-- Examples:+--+-- >>> text_node "8:00"+-- NTree (XText "8:00") []+--+text_node :: String -> XmlTree+text_node s = NTree (XText s) []++++--+-- * Tasty Tests+--++-- | A list of all tests for this module. This primary exists to+-- eliminate the unused import/export warnings for 'unpickleDoc' and+-- 'text_node' which are otherwise only used in the doctests.+--+pickler_tests :: TestTree+pickler_tests =+ testGroup+ "Pickler tests"+ [ test_pickle_of_unpickle_is_identity ]+++-- | If we unpickle something and then pickle it, we should wind up+-- with the same thing we started with (plus an additional root+-- element).+--+test_pickle_of_unpickle_is_identity :: TestTree+test_pickle_of_unpickle_is_identity =+ testCase "pickle composed with unpickle is (almost) the identity" $ do+ let tn = text_node "8:00"+ let (Just utctime) = unpickleDoc xp_ambiguous_time tn+ let actual = pickleDoc xp_ambiguous_time utctime+ let expected = NTree (XTag (mkName "/") []) [tn]+ actual @?= expected
src/TSN/Team.hs view
@@ -51,7 +51,7 @@ -- | A wrapper around 'Team' that lets us distinguish between home and -- away teams. See also 'HTeam'. \"V\" (visiting) was chosen instead--- of \"A\" (away) simply because \"vteam" looks better than+-- of \"A\" (away) simply because \"vteam\" looks better than -- \"ateam\". This is purely for type-safety. -- newtype VTeam = VTeam { vteam :: Team } deriving (Eq, Show)
src/TSN/XML/AutoRacingResults.hs view
@@ -57,10 +57,13 @@ xpWrap ) -- Local imports.-import TSN.Codegen (- tsn_codegen_config )+import TSN.Codegen ( tsn_codegen_config ) import TSN.DbImport ( DbImport(..), ImportResult(..), run_dbmigrate )-import TSN.Picklers ( xp_earnings, xp_datetime, xp_time_stamp )+import TSN.Picklers (+ xp_earnings,+ xp_fracpart_only_double,+ xp_datetime,+ xp_time_stamp ) import TSN.XmlImport ( XmlImport(..), XmlImportFk(..) ) import Xml ( Child(..),@@ -352,9 +355,9 @@ -------- Database stuff.----+--+-- * Database stuff.+-- instance DbImport Message where dbmigrate _ =@@ -529,7 +532,9 @@ xp11Tuple (-- I can't think of another way to get both the -- TrackLength and its KPH attribute. So we shove them -- both in a 2-tuple. This should probably be an embedded type!- xpElem "TrackLength" $ xpPair xpText (xpAttr "KPH" xpPrim) )+ xpElem "TrackLength" $+ xpPair xpText+ (xpAttr "KPH" xp_fracpart_only_double) ) (xpElem "Laps" xpInt) (xpOption $ xpElem "AverageSpeedMPH" xpPrim) (xpOption $ xpElem "AverageSpeedKPH" xpPrim)@@ -561,7 +566,7 @@ xml_most_laps_leading m) ----- Tasty Tests+-- * Tasty Tests -- -- | A list of all tests for this module.@@ -579,24 +584,33 @@ -- test does not mean that unpickling succeeded. -- test_pickle_of_unpickle_is_identity :: TestTree-test_pickle_of_unpickle_is_identity =- testCase "pickle composed with unpickle is the identity" $ do- let path = "test/xml/AutoRacingResultsXML.xml"- (expected, actual) <- pickle_unpickle pickle_message path- actual @?= expected+test_pickle_of_unpickle_is_identity = testGroup "pickle-unpickle tests" $+ [ check "pickle composed with unpickle is the identity"+ "test/xml/AutoRacingResultsXML.xml", + check "pickle composed with unpickle is the identity (fractional KPH)"+ "test/xml/AutoRacingResultsXML-fractional-kph.xml" ]+ where+ check desc path = testCase desc $ do+ (expected, actual) <- pickle_unpickle pickle_message path+ actual @?= expected + -- | Make sure we can actually unpickle these things. -- test_unpickle_succeeds :: TestTree-test_unpickle_succeeds =- testCase "unpickling succeeds" $ do- let path = "test/xml/AutoRacingResultsXML.xml"- actual <- unpickleable path pickle_message+test_unpickle_succeeds = testGroup "unpickle tests" $+ [ check "unpickling succeeds"+ "test/xml/AutoRacingResultsXML.xml", - let expected = True- actual @?= expected+ check "unpickling succeeds (fractional KPH)"+ "test/xml/AutoRacingResultsXML-fractional-kph.xml" ]+ where+ check desc path = testCase desc $ do+ actual <- unpickleable path pickle_message+ let expected = True+ actual @?= expected @@ -604,24 +618,29 @@ -- record. -- test_on_delete_cascade :: TestTree-test_on_delete_cascade =- testCase "deleting auto_racing_results deletes its children" $ do- let path = "test/xml/AutoRacingResultsXML.xml"- results <- unsafe_unpickle path pickle_message- let a = undefined :: AutoRacingResults- let b = undefined :: AutoRacingResultsListing- let c = undefined :: AutoRacingResultsRaceInformation+test_on_delete_cascade = testGroup "cascading delete tests" $+ [ check "deleting auto_racing_results deletes its children"+ "test/xml/AutoRacingResultsXML.xml", - actual <- withSqliteConn ":memory:" $ runDbConn $ do- runMigration silentMigrationLogger $ do- migrate a- migrate b- migrate c- _ <- dbimport results- deleteAll a- count_a <- countAll a- count_b <- countAll b- count_c <- countAll c- return $ sum [count_a, count_b, count_c]- let expected = 0- actual @?= expected+ check "deleting auto_racing_results deletes its children (fractional KPH)"+ "test/xml/AutoRacingResultsXML-fractional-kph.xml" ]+ where+ check desc path = testCase desc $ do+ results <- unsafe_unpickle path pickle_message+ let a = undefined :: AutoRacingResults+ let b = undefined :: AutoRacingResultsListing+ let c = undefined :: AutoRacingResultsRaceInformation++ actual <- withSqliteConn ":memory:" $ runDbConn $ do+ runMigration silentMigrationLogger $ do+ migrate a+ migrate b+ migrate c+ _ <- dbimport results+ deleteAll a+ count_a <- countAll a+ count_b <- countAll b+ count_c <- countAll c+ return $ sum [count_a, count_b, count_c]+ let expected = 0+ actual @?= expected
src/TSN/XML/AutoRacingSchedule.hs view
@@ -363,7 +363,7 @@ constructors: - name: AutoRacingSchedule uniques:- - name: unique_auto_racing_schedule+ - name: unique_auto_racing_schedules type: constraint # Prevent multiple imports of the same message. fields: [db_xml_file_id]
+ src/TSN/XML/EarlyLine.hs view
@@ -0,0 +1,627 @@+{-# LANGUAGE FlexibleInstances #-}+{-# LANGUAGE GADTs #-}+{-# LANGUAGE QuasiQuotes #-}+{-# LANGUAGE RecordWildCards #-}+{-# LANGUAGE TemplateHaskell #-}+{-# LANGUAGE TypeFamilies #-}++-- | Parse TSN XML for the DTD \"earlylineXML.dtd\". For that DTD,+-- each \<message\> element contains a bunch of \<date\>s, and those+-- \<date\>s contain a single \<game\>. In the database, we merge+-- the date info into the games, and key the games to the messages.+--+-- Real life is not so simple, however. There is another module,+-- "TSN.XML.MLBEarlyLine" that is something of a subclass of this+-- one. It contains early lines, but only for MLB games. The data+-- types and XML schema are /almost/ the same, but TSN like to make+-- things difficult.+--+-- A full list of the differences is given in that module. In this+-- one, we mention where data types have been twerked a little to+-- support the second document type.+--+module TSN.XML.EarlyLine (+ EarlyLine, -- Used in TSN.XML.MLBEarlyLine+ EarlyLineGame, -- Used in TSN.XML.MLBEarlyLine+ dtd,+ pickle_message,+ -- * Tests+ early_line_tests,+ -- * WARNING: these are private but exported to silence warnings+ EarlyLineConstructor(..),+ EarlyLineGameConstructor(..) )+where++-- System imports.+import Control.Monad ( join )+import Data.Time ( UTCTime(..) )+import Data.Tuple.Curry ( uncurryN )+import Database.Groundhog (+ countAll,+ deleteAll,+ insert_,+ migrate,+ runMigration,+ silentMigrationLogger )+import Database.Groundhog.Core ( DefaultKey )+import Database.Groundhog.Generic ( runDbConn )+import Database.Groundhog.Sqlite ( withSqliteConn )+import Database.Groundhog.TH (+ groundhog,+ mkPersist )+import Test.Tasty ( TestTree, testGroup )+import Test.Tasty.HUnit ( (@?=), testCase )+import Text.XML.HXT.Core (+ PU,+ xp4Tuple,+ xp6Tuple,+ xp7Tuple,+ xpAttr,+ xpElem,+ xpInt,+ xpList,+ xpOption,+ xpPair,+ xpText,+ xpWrap )++-- Local imports.+import TSN.Codegen ( tsn_codegen_config )+import TSN.DbImport ( DbImport(..), ImportResult(..), run_dbmigrate )+import TSN.Picklers (+ xp_ambiguous_time,+ xp_early_line_date,+ xp_time_stamp )+import TSN.XmlImport ( XmlImport(..) )+import Xml (+ FromXml(..),+ ToDb(..),+ pickle_unpickle,+ unpickleable,+ unsafe_unpickle )+++-- | The DTD to which this module corresponds. Used to invoke dbimport.+--+dtd :: String+dtd = "earlylineXML.dtd"++--+-- * DB/XML data types+--++-- * EarlyLine/Message++-- | Database representation of a 'Message'. It lacks the \<date\>+-- elements since they're really properties of the games that they+-- contain.+--+data EarlyLine =+ EarlyLine {+ db_xml_file_id :: Int,+ db_heading :: String,+ db_category :: String,+ db_sport :: String,+ db_title :: String,+ db_time_stamp :: UTCTime }+ deriving (Eq, Show)++++-- | XML Representation of an 'EarlyLine'. It has the same+-- fields, but in addition contains the 'xml_dates'.+--+data Message =+ Message {+ xml_xml_file_id :: Int,+ xml_heading :: String,+ xml_category :: String,+ xml_sport :: String,+ xml_title :: String,+ xml_dates :: [EarlyLineDate],+ xml_time_stamp :: UTCTime }+ deriving (Eq, Show)+++instance ToDb Message where+ -- | The database analogue of a 'Message' is an 'EarlyLine'.+ --+ type Db Message = EarlyLine+++-- | The 'FromXml' instance for 'Message' is required for the+-- 'XmlImport' instance.+--+instance FromXml Message where+ -- | To convert a 'Message' to an 'EarlyLine', we just drop+ -- the 'xml_dates'.+ --+ from_xml Message{..} =+ EarlyLine {+ db_xml_file_id = xml_xml_file_id,+ db_heading = xml_heading,+ db_category = xml_category,+ db_sport = xml_sport,+ db_title = xml_title,+ db_time_stamp = xml_time_stamp }+++-- | This allows us to insert the XML representation 'Message'+-- directly.+--+instance XmlImport Message++++-- * EarlyLineDate / EarlyLineGameWithNote++-- | This is a very sad data type. It exists so that we can+-- successfully unpickle/pickle the MLB_earlylineXML.dtd documents+-- and get back what we started with. In that document type, the+-- dates all have multiple \<game\>s associated with them (as+-- children). But the dates also have multiple \<note\>s as+-- children, and we're supposed to figure out which notes go with+-- which games based on the order that they appear in the XML+-- file. Yeah, right.+--+-- In any case, instead of expecting the games and notes in some+-- nice order, we use this data type to expect \"a game and maybe a+-- note\" multiple times. This will pair the notes with only one+-- game, rather than all of the games that TSN think it should go+-- with. But it allows us to pickle and unpickle correctly at least.+--+data EarlyLineGameWithNote =+ EarlyLineGameWithNote {+ date_note :: Maybe String,+ date_game :: EarlyLineGameXml }+ deriving (Eq, Show)+++-- | XML representation of a \<date\>. It has a \"value\" attribute+-- containing the actual date string. As children it contains a+-- (non-optional) note, and a game. The note and date value are+-- properties of the game as far as I can tell.+--+data EarlyLineDate =+ EarlyLineDate {+ date_value :: UTCTime,+ date_games_with_notes :: [EarlyLineGameWithNote] }+ deriving (Eq, Show)++++-- * EarlyLineGame / EarlyLineGameXml++-- | Database representation of a \<game\> in earlylineXML.dtd and+-- MLB_earlylineXML.dtd. We've had to make a sacrifice here to+-- support both document types. Since it's not possible to pair the+-- \<note\>s with \<game\>s reliably in MLB_earlylineXML.dtd, we+-- have omitted the notes entirely. This is sad, but totally not our+-- fault.+--+-- In earlylineXML.dtd, each \<date\> and thus each \<note\> is+-- paired with exactly one \<game\>, so if we only cared about that+-- document type, we could have retained the notes.+--+-- In earlylinexml.DTD, the over/under is required, but in+-- MLB_earlylinexml.DTD it is not. So another compromise is to have+-- it optional here.+--+-- The 'db_game_time' should be the combined date/time using the+-- date value from the \<game\> element's containing+-- \<date\>. That's why EarlyLineGame isn't an instance of+-- 'FromXmlFk': the foreign key isn't enough to construct one, we+-- also need the date.+--+data EarlyLineGame =+ EarlyLineGame {+ db_early_lines_id :: DefaultKey EarlyLine,+ db_game_time :: UTCTime, -- ^ Combined date/time+ db_away_team :: EarlyLineGameTeam,+ db_home_team :: EarlyLineGameTeam,+ db_over_under :: Maybe String }+++-- | XML representation of a 'EarlyLineGame'. Comparatively, it lacks+-- only the foreign key to the parent message.+--+data EarlyLineGameXml =+ EarlyLineGameXml {+ xml_game_time :: UTCTime, -- ^ Only an ambiguous time string, e.g. \"8:30\"+ xml_away_team :: EarlyLineGameTeamXml,+ xml_home_team :: EarlyLineGameTeamXml,+ xml_over_under :: Maybe String }+ deriving (Eq, Show)++++-- * EarlyLineGameTeam / EarlyLineGameTeamXml++-- | Database representation of an EarlyLine team, used in both+-- earlylineXML.dtd and MLB_earlylineXML.dtd. It doubles as an+-- embedded type within the DB representation 'EarlyLineGame'.+--+-- The team name is /not/ optional. However, since we're overloading+-- the XML representation, we're constructing 'db_team_name' name+-- from two Maybes, 'xml_team_name_attr' and+-- 'xml_team_name_text'. To ensure type safety (and avoid a runtime+-- crash), we allow the database field to be optional as well.+--+data EarlyLineGameTeam =+ EarlyLineGameTeam {+ db_rotation_number :: Int,+ db_line :: Maybe String, -- ^ Can be blank, a Double, or \"off\".+ db_team_name :: Maybe String, -- ^ NOT optional, see the data type docs.+ db_pitcher :: Maybe String -- ^ Optional in MLB_earlylineXML.dtd,+ -- always absent in earlylineXML.dtd.+ }+++-- | This here is an abomination. What we've got is an XML+-- representation, not for either earlylineXML.dtd or+-- MLB_earlylineXML.dtd, but one that will work for /both/. Even+-- though they represent the teams totally differently! Argh!+--+-- The earlylineXML.dtd teams look like,+--+-- \<teamA rotation=\"709\" line=\"\">Miami\</teamA\>+--+-- While the MLB_earlylineXML.dtd teams look like,+--+-- <teamA rotation="901" name="LOS">+-- <pitcher>D.Haren</pitcher>+-- <line>-130</line>+-- </teamA>+--+-- So that's cool. This data type has placeholders that should allow+-- the name/line to appear either as an attribute or as a text+-- node. We'll sort it all out in the conversion to+-- EarlyLineGameTeam.+--+data EarlyLineGameTeamXml =+ EarlyLineGameTeamXml {+ xml_rotation_number :: Int,+ xml_line_attr :: Maybe String,+ xml_team_name_attr :: Maybe String,+ xml_team_name_text :: Maybe String,+ xml_pitcher :: Maybe String,+ xml_line_elem :: Maybe String }+ deriving (Eq, Show)++++instance ToDb EarlyLineGameTeamXml where+ -- | The database analogue of a 'EarlyLineGameTeamXml' is an+ -- 'EarlyLineGameTeam', although the DB type is merely embedded+ -- in another type.+ --+ type Db EarlyLineGameTeamXml = EarlyLineGameTeam+++-- | The 'FromXml' instance for 'EarlyLineGameTeamXml' lets us convert+-- it to a 'EarlyLineGameTeam' easily.+--+instance FromXml EarlyLineGameTeamXml where+ -- | To convert a 'EarlyLineGameTeamXml' to an 'EarlyLineGameTeam',+ -- we figure how its fields were represented and choose the ones+ -- that are populated. For example if the \"line\" attribute was+ -- there, we'll use it, but if now, we'll use the \<line\>+ -- element.+ --+ from_xml EarlyLineGameTeamXml{..} =+ EarlyLineGameTeam {+ db_rotation_number = xml_rotation_number,+ db_line = merge xml_line_attr xml_line_elem,+ db_team_name = merge xml_team_name_attr xml_team_name_text,+ db_pitcher = xml_pitcher }+ where+ merge :: Maybe String -> Maybe String -> Maybe String+ merge Nothing y = y+ merge x Nothing = x+ merge _ _ = Nothing+++++-- | Convert an 'EarlyLineDate' into a list of 'EarlyLineGame's. Each+-- date has one or more games, and the fields that belong to the date+-- should really be in the game anyway. So the database+-- representation of a game has the combined fields of the XML+-- date/game.+--+-- This function gets the games out of a date, and then sticks the+-- date value inside the games. It also adds the foreign key+-- reference to the games' parent message, and returns the result.+--+-- This would convert a single date to a single game if we only+-- needed to support earlylineXML.dtd and not MLB_earlylineXML.dtd.+--+date_to_games :: (DefaultKey EarlyLine) -> EarlyLineDate -> [EarlyLineGame]+date_to_games fk date =+ map convert_game games_only+ where+ -- | Get the list of games out of a date (i.e. drop the notes).+ --+ games_only :: [EarlyLineGameXml]+ games_only = (map date_game (date_games_with_notes date))++ -- | Stick the date value into the given game.+ --+ combine_date_time :: EarlyLineGameXml -> UTCTime+ combine_date_time elgx =+ UTCTime (utctDay $ date_value date) (utctDayTime $ xml_game_time elgx)++ -- | Convert an XML game to a database one.+ --+ convert_game :: EarlyLineGameXml -> EarlyLineGame+ convert_game gx =+ EarlyLineGame {+ db_early_lines_id = fk,+ db_game_time = combine_date_time gx,+ db_away_team = from_xml (xml_away_team gx),+ db_home_team = from_xml (xml_home_team gx),+ db_over_under = xml_over_under gx }+++--+-- * Database stuff+--++instance DbImport Message where+ dbmigrate _ =+ run_dbmigrate $ do+ migrate (undefined :: EarlyLine)+ migrate (undefined :: EarlyLineGame)++ dbimport m = do+ -- Insert the message and obtain its ID.+ msg_id <- insert_xml m++ -- Create a function that will turn a list of dates into a list of+ -- games by converting each date to its own list of games, and+ -- then concatenating all of the game lists together.+ let convert_dates_to_games = concatMap (date_to_games msg_id)++ -- Now use it to make dem games.+ let games = convert_dates_to_games (xml_dates m)++ -- And insert all of them+ mapM_ insert_ games++ return ImportSucceeded+++mkPersist tsn_codegen_config [groundhog|++- entity: EarlyLine+ dbName: early_lines+ constructors:+ - name: EarlyLine+ uniques:+ - name: unique_early_lines+ type: constraint+ # Prevent multiple imports of the same message.+ fields: [db_xml_file_id]+++- entity: EarlyLineGame+ dbName: early_lines_games+ constructors:+ - name: EarlyLineGame+ fields:+ - name: db_early_lines_id+ reference:+ onDelete: cascade+ - name: db_away_team+ embeddedType:+ - {name: rotation_number, dbName: away_team_rotation_number}+ - {name: line, dbName: away_team_line}+ - {name: team_name, dbName: away_team_name}+ - {name: pitcher, dbName: away_team_pitcher}+ - name: db_home_team+ embeddedType:+ - {name: rotation_number, dbName: home_team_rotation_number}+ - {name: line, dbName: home_team_line}+ - {name: team_name, dbName: home_team_name}+ - {name: pitcher, dbName: home_team_pitcher}++- embedded: EarlyLineGameTeam+ fields:+ - name: db_rotation_number+ dbName: rotation_number+ - name: db_line+ dbName: line+ - name: db_team_name+ dbName: team_name+ - name: db_pitcher+ dbName: pitcher+|]++++--+-- * Pickling+--+++-- | Pickler for the top-level 'Message'.+--+pickle_message :: PU Message+pickle_message =+ xpElem "message" $+ xpWrap (from_tuple, to_tuple) $+ xp7Tuple (xpElem "XML_File_ID" xpInt)+ (xpElem "heading" xpText)+ (xpElem "category" xpText)+ (xpElem "sport" xpText)+ (xpElem "title" xpText)+ (xpList pickle_date)+ (xpElem "time_stamp" xp_time_stamp)+ where+ from_tuple = uncurryN Message+ to_tuple m = (xml_xml_file_id m,+ xml_heading m,+ xml_category m,+ xml_sport m,+ xml_title m,+ xml_dates m,+ xml_time_stamp m)++++-- | Pickler for a '\<note\> followed by a \<game\>. We turn them into+-- a 'EarlyLineGameWithNote'.+--+pickle_game_with_note :: PU EarlyLineGameWithNote+pickle_game_with_note =+ xpWrap (from_tuple, to_tuple) $+ xpPair (xpOption $ xpElem "note" xpText)+ pickle_game+ where+ from_tuple = uncurry EarlyLineGameWithNote+ to_tuple m = (date_note m, date_game m)+++-- | Pickler for the \<date\> elements within each \<message\>.+--+pickle_date :: PU EarlyLineDate+pickle_date =+ xpElem "date" $+ xpWrap (from_tuple, to_tuple) $+ xpPair (xpAttr "value" xp_early_line_date)+ (xpList pickle_game_with_note)+ where+ from_tuple = uncurry EarlyLineDate+ to_tuple m = (date_value m, date_games_with_notes m)++++-- | Pickler for the \<game\> elements within each \<date\>.+--+pickle_game :: PU EarlyLineGameXml+pickle_game =+ xpElem "game" $+ xpWrap (from_tuple, to_tuple) $+ xp4Tuple (xpElem "time" xp_ambiguous_time)+ pickle_away_team+ pickle_home_team+ (xpElem "over_under" (xpOption xpText))+ where+ from_tuple = uncurryN EarlyLineGameXml+ to_tuple m = (xml_game_time m,+ xml_away_team m,+ xml_home_team m,+ xml_over_under m)++++-- | Pickle an away team (\<teamA\>) element within a \<game\>. Most+-- of the work (common with the home team pickler) is done by+-- 'pickle_team'.+--+pickle_away_team :: PU EarlyLineGameTeamXml+pickle_away_team = xpElem "teamA" pickle_team+++-- | Pickle a home team (\<teamH\>) element within a \<game\>. Most+-- of the work (common with theaway team pickler) is done by+-- 'pickle_team'.+--+pickle_home_team :: PU EarlyLineGameTeamXml+pickle_home_team = xpElem "teamH" pickle_team+++-- | Team pickling common to both 'pickle_away_team' and+-- 'pickle_home_team'. Handles everything inside the \<teamA\> and+-- \<teamH\> elements. We try to parse the line/name as both an+-- attribute and an element in order to accomodate+-- MLB_earlylineXML.dtd.+--+-- The \"line\" and \"pitcher\" fields wind up being double-Maybes,+-- since they can be empty even if they exist.+--+pickle_team :: PU EarlyLineGameTeamXml+pickle_team =+ xpWrap (from_tuple, to_tuple) $+ xp6Tuple (xpAttr "rotation" xpInt)+ (xpOption $ xpAttr "line" (xpOption xpText))+ (xpOption $ xpAttr "name" xpText)+ (xpOption xpText)+ (xpOption $ xpElem "pitcher" (xpOption xpText))+ (xpOption $ xpElem "line" (xpOption xpText))+ where+ from_tuple (u,v,w,x,y,z) =+ EarlyLineGameTeamXml u (join v) w x (join y) (join z)++ to_tuple (EarlyLineGameTeamXml u v w x y z) =+ (u, double_just v, w, x, double_just y, double_just z)+ where+ double_just val = case val of+ Nothing -> Nothing+ just_something -> Just just_something+++++--+-- * Tasty Tests+--++-- | A list of all tests for this module.+--+early_line_tests :: TestTree+early_line_tests =+ testGroup+ "EarlyLine tests"+ [ test_on_delete_cascade,+ test_pickle_of_unpickle_is_identity,+ test_unpickle_succeeds ]++-- | If we unpickle something and then pickle it, we should wind up+-- with the same thing we started with. WARNING: success of this+-- test does not mean that unpickling succeeded.+--+test_pickle_of_unpickle_is_identity :: TestTree+test_pickle_of_unpickle_is_identity =+ testCase "pickle composed with unpickle is the identity" $ do+ let path = "test/xml/earlylineXML.xml"+ (expected, actual) <- pickle_unpickle pickle_message path+ actual @?= expected++++-- | Make sure we can actually unpickle these things.+--+test_unpickle_succeeds :: TestTree+test_unpickle_succeeds =+ testCase "unpickling succeeds" $ do+ let path = "test/xml/earlylineXML.xml"+ actual <- unpickleable path pickle_message++ let expected = True+ actual @?= expected++++-- | Make sure everything gets deleted when we delete the top-level+-- record.+--+test_on_delete_cascade :: TestTree+test_on_delete_cascade =+ testCase "deleting early_lines deletes its children" $ do+ let path = "test/xml/earlylineXML.xml"+ results <- unsafe_unpickle path pickle_message+ let a = undefined :: EarlyLine+ let b = undefined :: EarlyLineGame++ actual <- withSqliteConn ":memory:" $ runDbConn $ do+ runMigration silentMigrationLogger $ do+ migrate a+ migrate b+ _ <- dbimport results+ deleteAll a+ count_a <- countAll a+ count_b <- countAll b+ return $ sum [count_a, count_b]+ let expected = 0+ actual @?= expected
src/TSN/XML/GameInfo.hs view
@@ -162,7 +162,7 @@ let a2 = xml_file_id t let ex2 = 21201550 let a3 = show $ time_stamp t- let ex3 = "2014-05-31 20:13:00 UTC"+ let ex3 = "2014-05-31 15:13:00 UTC" let a4 = take 9 (xml t) let ex4 = "<message>" let actual = (a1,a2,a3,a4)
+ src/TSN/XML/MLBEarlyLine.hs view
@@ -0,0 +1,136 @@+-- | Parse TSN XML for the DTD \"MLB_earlylineXML.dtd\". This module+-- is unique (so far) in that it is almost entirely a subclass of+-- another module, "TSN.XML.EarlyLine". The database representations+-- should be almost identical, and the XML schema /could/ be+-- similar, but instead, welcome to the jungle baby. Here are the+-- differences:+--+-- * In earlylineXML.dtd, each \<date\> element contains exactly one+-- game. In MLB_earlylineXML.dtd, they contain multiple games.+--+-- * As a result of the previous difference, the \<note\>s are no+-- longer in one-to-one correspondence with the games. The+-- \<note\> elements are thrown in beside the \<game\>s, and we're+-- supposed to figure out to which \<game\>s they correspond+-- ourselves. This is the same sort of nonsense going on with+-- 'TSN.XML.Odds.OddsGameWithNotes'.+--+-- * The \<over_under\> element can be empty in+-- MLB_earlylineXML.dtd (it can't in earlylineXML.dtd).+--+-- * Each home/away team in MLB_earlylineXML.dtd has a \<pitcher\>+-- that isn't present in the regular earlylineXML.dtd.+--+-- * In earlylineXML.dtd, the home/away team lines are given as+-- attributes on the \<teamH\> and \<teamA\> elements+-- respectively. In MLB_earlylineXML.dtd, the lines can be found+-- in \<line\> elements that are children of the \<teamH\> and+-- \<teamA\> elements.+--+-- * In earlylineXML.dtd, the team names are given as text within+-- the \<teamA\> and \<teamH\> elements. In MLB_earlylineXML.dtd,+-- they are instead given as attributes on those respective+-- elements.+--+-- Most of these difficulties have been worked around in+-- "TSN.XML.EarlyLine", so this module could be kept somewhat boring.+--+module TSN.XML.MLBEarlyLine (+ dtd,+ mlb_early_line_tests,+ module TSN.XML.EarlyLine -- This re-exports the EarlyLine and EarlyLineGame+ -- constructors unnecessarily. Whatever.+ )+where++-- System imports (needed only for tests)+import Database.Groundhog (+ countAll,+ deleteAll,+ migrate,+ runMigration,+ silentMigrationLogger )+import Database.Groundhog.Generic ( runDbConn )+import Database.Groundhog.Sqlite ( withSqliteConn )+import Test.Tasty ( TestTree, testGroup )+import Test.Tasty.HUnit ( (@?=), testCase )+++-- Local imports.+import TSN.DbImport ( DbImport( dbimport ) )+import TSN.XML.EarlyLine ( EarlyLine, EarlyLineGame, pickle_message )+import Xml (+ pickle_unpickle,+ unpickleable,+ unsafe_unpickle )+++-- | The DTD to which this module corresponds. Used to invoke dbimport.+--+dtd :: String+dtd = "MLB_earlylineXML.dtd"++++--+-- * Tasty Tests+--++-- | A list of all tests for this module.+--+mlb_early_line_tests :: TestTree+mlb_early_line_tests =+ testGroup+ "MLBEarlyLine tests"+ [ test_on_delete_cascade,+ test_pickle_of_unpickle_is_identity,+ test_unpickle_succeeds ]++-- | If we unpickle something and then pickle it, we should wind up+-- with the same thing we started with. WARNING: success of this+-- test does not mean that unpickling succeeded.+--+test_pickle_of_unpickle_is_identity :: TestTree+test_pickle_of_unpickle_is_identity =+ testCase "pickle composed with unpickle is the identity" $ do+ let path = "test/xml/MLB_earlylineXML.xml"+ (expected, actual) <- pickle_unpickle pickle_message path+ actual @?= expected++++-- | Make sure we can actually unpickle these things.+--+test_unpickle_succeeds :: TestTree+test_unpickle_succeeds =+ testCase "unpickling succeeds" $ do+ let path = "test/xml/MLB_earlylineXML.xml"+ actual <- unpickleable path pickle_message++ let expected = True+ actual @?= expected++++-- | Make sure everything gets deleted when we delete the top-level+-- record.+--+test_on_delete_cascade :: TestTree+test_on_delete_cascade =+ testCase "deleting (MLB) early_lines deletes its children" $ do+ let path = "test/xml/MLB_earlylineXML.xml"+ results <- unsafe_unpickle path pickle_message+ let a = undefined :: EarlyLine+ let b = undefined :: EarlyLineGame++ actual <- withSqliteConn ":memory:" $ runDbConn $ do+ runMigration silentMigrationLogger $ do+ migrate a+ migrate b+ _ <- dbimport results+ deleteAll a+ count_a <- countAll a+ count_b <- countAll b+ return $ sum [count_a, count_b]+ let expected = 0+ actual @?= expected
src/TSN/XML/News.hs view
@@ -291,7 +291,7 @@ constructors: - name: NewsTeam uniques:- - name: unique_news_team+ - name: unique_news_teams type: constraint fields: [team_name]
src/TSN/XML/Odds.hs view
@@ -147,16 +147,25 @@ instance XmlImport OddsGameCasinoXml --- * OddsGameTeamXml+-- * OddsGameTeamXml / OddsGameTeamStarterXml +-- | The XML representation of a \"starter\". It contains both an ID+-- and a name. The ID does not appear to be optional, but the name+-- can be absent. When the name is absent, the ID has always been+-- set to \"0\". This occurs even though the entire starter element+-- is optional (see 'OddsGameTeamXml' below).+--+data OddsGameTeamStarterXml =+ OddsGameTeamStarterXml {+ xml_starter_id :: Int,+ xml_starter_name :: Maybe String }+ deriving (Eq, Show)++ -- | The XML representation of a \<HomeTeam\> or \<AwayTeam\>, as -- found in \<Game\>s. We can't use the 'Team' representation -- directly because there are some other fields we need to parse. ----- The starter id/name could perhaps be combined into an embedded--- type, but can you make an entire embedded type optional with--- Maybe? I doubt it works.--- data OddsGameTeamXml = OddsGameTeamXml { xml_team_id :: String, -- ^ The home/away team IDs@@ -170,7 +179,7 @@ xml_team_rotation_number :: Maybe Int, xml_team_abbr :: String, xml_team_name :: String,- xml_team_starter :: Maybe (Int, String), -- ^ (id, name)+ xml_team_starter :: Maybe OddsGameTeamStarterXml, xml_team_casinos :: [OddsGameCasinoXml] } deriving (Eq, Show) @@ -309,16 +318,20 @@ (xml_team_rotation_number xml_home_team), db_away_team_starter_id =- (fst <$> xml_team_starter xml_away_team),+ (xml_starter_id <$> xml_team_starter xml_away_team), - db_away_team_starter_name =- (snd <$> xml_team_starter xml_away_team),+ -- Sometimes the starter element is present but the name isn't,+ -- so we combine the two maybes with join.+ db_away_team_starter_name = join+ (xml_starter_name <$> xml_team_starter xml_away_team), db_home_team_starter_id =- (fst <$> xml_team_starter xml_home_team),+ (xml_starter_id <$> xml_team_starter xml_home_team), - db_home_team_starter_name =- (snd <$> xml_team_starter xml_home_team) }+ -- Sometimes the starter element is present but the name isn't,+ -- so we combine the two maybes with join.+ db_home_team_starter_name = join+ (xml_starter_name <$> xml_team_starter xml_home_team) } -- | This lets us insert the XML representation 'OddsGameXml' directly.@@ -437,7 +450,7 @@ constructors: - name: OddsCasino uniques:- - name: unique_odds_casino+ - name: unique_odds_casinos type: constraint fields: [casino_client_id] @@ -582,9 +595,7 @@ (xpElem "HomeRotationNumber" (xpOption xpInt)) (xpElem "HomeAbbr" xpText) (xpElem "HomeTeamName" xpText)- (-- This is an ugly way to get both the HStarter ID attribute- -- and contents.- xpOption (xpElem "HStarter" $ xpPair (xpAttr "ID" xpInt) xpText))+ (xpOption pickle_home_starter) (xpList pickle_casino) where from_tuple = uncurryN OddsGameTeamXml@@ -597,6 +608,32 @@ xml_team_starter, xml_team_casinos) ++-- | Portion of the 'OddsGameTeamStarterXml' pickler that is not+-- specific to the home/away teams.+--+pickle_starter :: PU OddsGameTeamStarterXml+pickle_starter =+ xpWrap (from_tuple, to_tuple) $+ xpPair (xpAttr "ID" xpInt) (xpOption xpText)+ where+ from_tuple = uncurry OddsGameTeamStarterXml+ to_tuple OddsGameTeamStarterXml{..} = (xml_starter_id,+ xml_starter_name)++-- | Pickler for an home team 'OddsGameTeamStarterXml'+--+pickle_home_starter :: PU OddsGameTeamStarterXml+pickle_home_starter = xpElem "HStarter" pickle_starter+++-- | Pickler for an away team 'OddsGameTeamStarterXml'+--+pickle_away_starter :: PU OddsGameTeamStarterXml+pickle_away_starter = xpElem "AStarter" pickle_starter+++ -- | Pickler for an 'OddsGameTeamXml'. -- pickle_away_team :: PU OddsGameTeamXml@@ -608,9 +645,7 @@ (xpElem "AwayRotationNumber" (xpOption xpInt)) (xpElem "AwayAbbr" xpText) (xpElem "AwayTeamName" xpText)- (-- This is an ugly way to get both the AStarter ID attribute- -- and contents.- xpOption (xpElem "AStarter" $ xpPair (xpAttr "ID" xpInt) xpText))+ (xpOption pickle_away_starter) (xpList pickle_casino) where from_tuple = uncurryN OddsGameTeamXml@@ -721,7 +756,10 @@ "test/xml/Odds_XML-largefile.xml", check "pickle composed with unpickle is the identity (league name)"- "test/xml/Odds_XML-league-name.xml" ]+ "test/xml/Odds_XML-league-name.xml",++ check "pickle composed with unpickle is the identity (missing starters)"+ "test/xml/Odds_XML-missing-starters.xml" ] where check desc path = testCase desc $ do (expected, actual) <- pickle_unpickle pickle_message path@@ -745,7 +783,10 @@ "test/xml/Odds_XML-largefile.xml", check "unpickling succeeds (league name)"- "test/xml/Odds_XML-league-name.xml" ]+ "test/xml/Odds_XML-league-name.xml",++ check "unpickling succeeds (missing starters)"+ "test/xml/Odds_XML-missing-starters.xml" ] where check desc path = testCase desc $ do actual <- unpickleable path pickle_message@@ -780,6 +821,10 @@ check "deleting odds deleted its children (league name)" "test/xml/Odds_XML-league-name.xml" 35 -- 5 casinos, 30 teams+ ,+ check "deleting odds deleted its children (missing starters)"+ "test/xml/Odds_XML-missing-starters.xml"+ 7 -- 5 casinos, 2 teams ] where check desc path expected = testCase desc $ do
src/TSN/XML/SportInfo.hs view
@@ -275,7 +275,7 @@ let a2 = xml_file_id t let ex2 = 2011 let a3 = show $ time_stamp t- let ex3 = "2009-09-28 00:50:00 UTC"+ let ex3 = "2009-09-27 19:50:00 UTC" let a4 = take 9 (xml t) let ex4 = "<message>" let actual = (a1,a2,a3,a4)
test/TestSuite.hs view
@@ -1,12 +1,15 @@ import Test.Tasty ( TestTree, defaultMain, testGroup ) +import TSN.Picklers ( pickler_tests ) import TSN.XML.AutoRacingResults ( auto_racing_results_tests ) import TSN.XML.AutoRacingSchedule ( auto_racing_schedule_tests )+import TSN.XML.EarlyLine ( early_line_tests ) import TSN.XML.GameInfo ( game_info_tests ) import TSN.XML.Heartbeat ( heartbeat_tests ) import TSN.XML.Injuries ( injuries_tests ) import TSN.XML.InjuriesDetail ( injuries_detail_tests ) import TSN.XML.JFile ( jfile_tests )+import TSN.XML.MLBEarlyLine ( mlb_early_line_tests ) import TSN.XML.News ( news_tests ) import TSN.XML.Odds ( odds_tests ) import TSN.XML.ScheduleChanges ( schedule_changes_tests )@@ -19,13 +22,16 @@ "All tests" [ auto_racing_results_tests, auto_racing_schedule_tests,+ early_line_tests, game_info_tests, heartbeat_tests, injuries_tests, injuries_detail_tests, jfile_tests,+ mlb_early_line_tests, news_tests, odds_tests,+ pickler_tests, schedule_changes_tests, scores_tests, sport_info_tests,
test/shell/import-duplicates.test view
@@ -16,15 +16,15 @@ # and a newsxml that aren't really supposed to import. find ./test/xml -maxdepth 1 -name '*.xml' | wc -l >>>-26+30 >>>= 0 # Run the imports again; we should get complaints about the duplicate-# xml_file_ids. There are 2 errors for each violation, so we expect 2*22+# xml_file_ids. There are 2 errors for each violation, so we expect 2*26 # occurrences of the string 'ERROR'. ./dist/build/htsn-import/htsn-import -c 'shelltest.sqlite3' test/xml/*.xml 2>&1 | grep ERROR | wc -l >>>-44+52 >>>= 0 # Finally, clean up after ourselves.
+ test/xml/AutoRacingResultsXML-fractional-kph.xml view
@@ -0,0 +1,1 @@+<?xml version="1.0" standalone="no" ?> <!DOCTYPE message PUBLIC "-//TSN//DTD Leader 1.0/EN" "AutoRacingResultsXML.dtd"> <message> <XML_File_ID>21475733</XML_File_ID> <heading>ATX%CRAFTSMAN-FINAL-RESULTS</heading> <category>Statistics</category> <sport>NASCAR-T</sport> <RaceID>1747</RaceID> <RaceDate>7/23/2014 9:00:00 PM</RaceDate> <Title>NASCAR - Camping World - 1-800 CarCash Mudsummer Classic - Final Results</Title> <Track_Location>Eldora Speedway - New Weston, OH</Track_Location> <Laps_Remaining>0</Laps_Remaining> <Checkered_Flag>True</Checkered_Flag> <Listing> <FinishPosition>1</FinishPosition> <StartingPosition>6</StartingPosition> <CarNumber>54</CarNumber> <DriverID>1484</DriverID> <Driver>Darrell Wallace Jr.</Driver> <CarMake>Toyota</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>2</FinishPosition> <StartingPosition>3</StartingPosition> <CarNumber>30</CarNumber> <DriverID>108</DriverID> <Driver>Ron Hornaday Jr.</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>3</FinishPosition> <StartingPosition>4</StartingPosition> <CarNumber>29</CarNumber> <DriverID>1482</DriverID> <Driver>Ryan Blaney</Driver> <CarMake>Ford</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>4</FinishPosition> <StartingPosition>10</StartingPosition> <CarNumber>52</CarNumber> <DriverID>40</DriverID> <Driver>Ken Schrader</Driver> <CarMake>Toyota</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>5</FinishPosition> <StartingPosition>13</StartingPosition> <CarNumber>3</CarNumber> <DriverID>1061</DriverID> <Driver>Ty Dillon</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>6</FinishPosition> <StartingPosition>18</StartingPosition> <CarNumber>8</CarNumber> <DriverID>1106</DriverID> <Driver>John H. Nemechek</Driver> <CarMake>Toyota</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>7</FinishPosition> <StartingPosition>2</StartingPosition> <CarNumber>13</CarNumber> <DriverID>1483</DriverID> <Driver>Jeb Burton</Driver> <CarMake>Toyota</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>8</FinishPosition> <StartingPosition>5</StartingPosition> <CarNumber>98</CarNumber> <DriverID>117</DriverID> <Driver>Johnny Sauter</Driver> <CarMake>Toyota</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>9</FinishPosition> <StartingPosition>9</StartingPosition> <CarNumber>88</CarNumber> <DriverID>123</DriverID> <Driver>Matt Crafton</Driver> <CarMake>Toyota</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>10</FinishPosition> <StartingPosition>19</StartingPosition> <CarNumber>2</CarNumber> <DriverID>1321</DriverID> <Driver>Austin Dillon</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>11</FinishPosition> <StartingPosition>8</StartingPosition> <CarNumber>19</CarNumber> <DriverID>1105</DriverID> <Driver>Tyler Reddick</Driver> <CarMake>Ford</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>12</FinishPosition> <StartingPosition>24</StartingPosition> <CarNumber>77</CarNumber> <DriverID>1221</DriverID> <Driver>German Quiroga</Driver> <CarMake>Toyota</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>13</FinishPosition> <StartingPosition>21</StartingPosition> <CarNumber>31</CarNumber> <DriverID>1085</DriverID> <Driver>Ben Kennedy</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>14</FinishPosition> <StartingPosition>7</StartingPosition> <CarNumber>21</CarNumber> <DriverID>1477</DriverID> <Driver>Joey Coulter</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>15</FinishPosition> <StartingPosition>27</StartingPosition> <CarNumber>02</CarNumber> <DriverID>1486</DriverID> <Driver>Tyler Young</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>16</FinishPosition> <StartingPosition>14</StartingPosition> <CarNumber>17</CarNumber> <DriverID>550</DriverID> <Driver>Timothy Peters</Driver> <CarMake>Toyota</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>17</FinishPosition> <StartingPosition>23</StartingPosition> <CarNumber>9</CarNumber> <DriverID>1509</DriverID> <Driver>Chase Pistone</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>18</FinishPosition> <StartingPosition>16</StartingPosition> <CarNumber>63</CarNumber> <DriverID>999</DriverID> <Driver>J.R. Heffner</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>19</FinishPosition> <StartingPosition>26</StartingPosition> <CarNumber>05</CarNumber> <DriverID>1300</DriverID> <Driver>John Wes Townley</Driver> <CarMake>Toyota</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>20</FinishPosition> <StartingPosition>17</StartingPosition> <CarNumber>20</CarNumber> <DriverID>1504</DriverID> <Driver>Gray Gaulding</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>21</FinishPosition> <StartingPosition>15</StartingPosition> <CarNumber>50</CarNumber> <DriverID>241</DriverID> <Driver>T.J. Bell</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>22</FinishPosition> <StartingPosition>12</StartingPosition> <CarNumber>35</CarNumber> <DriverID>1115</DriverID> <Driver>Mason Mingus</Driver> <CarMake>Toyota</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>23</FinishPosition> <StartingPosition>20</StartingPosition> <CarNumber>99</CarNumber> <DriverID>1238</DriverID> <Driver>Bryan Silas</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>24</FinishPosition> <StartingPosition>25</StartingPosition> <CarNumber>08</CarNumber> <DriverID>1503</DriverID> <Driver>Korbin Forrister</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>150</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>25</FinishPosition> <StartingPosition>30</StartingPosition> <CarNumber>14</CarNumber> <DriverID>1307</DriverID> <Driver>Michael Annett</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>149</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>26</FinishPosition> <StartingPosition>11</StartingPosition> <CarNumber>32</CarNumber> <DriverID>965</DriverID> <Driver>Kyle Larson</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>148</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Accident</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>27</FinishPosition> <StartingPosition>29</StartingPosition> <CarNumber>6</CarNumber> <DriverID>171</DriverID> <Driver>Norm Benning</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>148</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>28</FinishPosition> <StartingPosition>28</StartingPosition> <CarNumber>80</CarNumber> <DriverID>1520</DriverID> <Driver>Jody Knowles</Driver> <CarMake>Ford</CarMake> <Points>0</Points> <Laps_Completed>148</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>29</FinishPosition> <StartingPosition>1</StartingPosition> <CarNumber>51</CarNumber> <DriverID>639</DriverID> <Driver>Erik Jones</Driver> <CarMake>Toyota</CarMake> <Points>0</Points> <Laps_Completed>144</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Running</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Listing> <FinishPosition>30</FinishPosition> <StartingPosition>22</StartingPosition> <CarNumber>03</CarNumber> <DriverID>1518</DriverID> <Driver>Michael Affarano</Driver> <CarMake>Chevrolet</CarMake> <Points>0</Points> <Laps_Completed>93</Laps_Completed> <Laps_Leading>0</Laps_Leading> <Status>Overheating</Status> <DNF>False</DNF> <Earnings>TBA</Earnings> </Listing> <Race_Information> <TrackLength KPH=".805">0.5</TrackLength> <Laps>150</Laps> <AverageSpeedMPH>50.195</AverageSpeedMPH> <AverageSpeedKPH>80.764</AverageSpeedKPH> <AverageSpeed>50.195</AverageSpeed> <TimeOfRace>1 Hr., 29 Mins., 39 Secs.</TimeOfRace> <MarginOfVictory>5.489 Seconds</MarginOfVictory> <Cautions>7 for 33 laps</Cautions> <LeadChanges>5 among 5 drivers</LeadChanges> <Most_Laps_Leading> <DriverID>1484</DriverID> <Driver>Darrell Wallace Jr.</Driver> <NumberOfLaps>0</NumberOfLaps> </Most_Laps_Leading> </Race_Information> <time_stamp> July 23, 2014, at 11:32 PM ET </time_stamp> </message>
+ test/xml/MLB_earlylineXML.dtd view
@@ -0,0 +1,22 @@+<!ELEMENT XML_File_ID (#PCDATA)>+<!ELEMENT heading (#PCDATA)>+<!ELEMENT category (#PCDATA)>+<!ELEMENT sport (#PCDATA)>+<!ELEMENT title (#PCDATA)>+<!ELEMENT note (#PCDATA)>+<!ELEMENT time (#PCDATA)>+<!ELEMENT pitcher (#PCDATA)>+<!ELEMENT line (#PCDATA)>+<!ELEMENT teamA ( ( pitcher, line ) )>+<!ELEMENT teamH ( ( pitcher, line ) )>+<!ELEMENT over_under (#PCDATA)>+<!ELEMENT game ( ( time, teamA, teamH, over_under ) )>+<!ELEMENT date ( ( note | game )+ )>+<!ELEMENT time_stamp (#PCDATA)>+<!ELEMENT message ( ( XML_File_ID, heading, category, sport, title, date, time_stamp ) )>++<!ATTLIST teamA rotation CDATA #REQUIRED>+<!ATTLIST teamA name CDATA #REQUIRED>+<!ATTLIST teamH rotation CDATA #REQUIRED>+<!ATTLIST teamH name CDATA #REQUIRED>+<!ATTLIST date value CDATA #REQUIRED>
+ test/xml/MLB_earlylineXML.xml view
@@ -0,0 +1,1 @@+<?xml version="1.0" standalone="no" ?> <!DOCTYPE message PUBLIC "-//TSN//DTD Odds 1.0/EN" "MLB_earlylineXML.dtd"> <message> <XML_File_ID>21161927</XML_File_ID> <heading>AAO;MLB-EARLY-LINE</heading> <category>Odds</category> <sport>MLB</sport> <title>Major League Baseball Overnight Line</title> <date value="SUNDAY, MAY 25TH (05/25/2014)"> <note>National League</note> <game> <time>1:10</time> <teamA rotation="951" name="MIL"> <pitcher>J.Nelson</pitcher> <line></line> </teamA> <teamH rotation="952" name="MIA"> <pitcher>R.Wolf</pitcher> <line>-105</line> </teamH> <over_under>8o</over_under> </game> <note>Game one of doubleheader</note> <game> <time>1:10</time> <teamA rotation="953" name="ARI"> <!-- Manually removed pitcher --> <pitcher>B.Arroyo</pitcher> <line></line> </teamA> <teamH rotation="954" name="NYM"> <pitcher>R.Montero</pitcher> <line>-105</line> </teamH> <over_under>7.5u</over_under> </game> <game> <time>1:35</time> <teamA rotation="955" name="LOS"> <pitcher>J.Beckett</pitcher> <line>-110</line> </teamA> <teamH rotation="956" name="PHI"> <pitcher>AJ.Burnett</pitcher> <line></line> </teamH> <!-- Manually removed over_under --> <over_under></over_under> </game> <game> <time>1:35</time> <teamA rotation="957" name="WAS"> <pitcher>D.Fister</pitcher> <line></line> </teamA> <teamH rotation="958" name="PIT"> <pitcher>F.Liriano</pitcher> <line>-115</line> </teamH> <over_under>7p</over_under> </game> <game> <time>4:10</time> <teamA rotation="959" name="CHC"> <pitcher>J.Hammel</pitcher> <line></line> </teamA> <teamH rotation="960" name="SDP"> <pitcher>I.Kennedy</pitcher> <line>-125</line> </teamH> <over_under>6.5p</over_under> </game> <game> <time>5:10</time> <teamA rotation="961" name="COL"> <pitcher>F.Morales</pitcher> <line></line> </teamA> <teamH rotation="962" name="ATL"> <pitcher>J.Teheran</pitcher> <line>-160</line> </teamH> <over_under>7.5p</over_under> </game> <game> <time>8:05</time> <teamA rotation="963" name="STL"> <pitcher>A.Wainwright</pitcher> <line>-140</line> </teamA> <teamH rotation="964" name="CIN"> <pitcher>M.Leake</pitcher> <line></line> </teamH> <over_under>6.5o</over_under> </game> <note>American League</note> <game> <time>1:05</time> <teamA rotation="965" name="TEX"> <pitcher>C.Lewis</pitcher> <line></line> </teamA> <teamH rotation="966" name="DET"> <pitcher>J.Verlander</pitcher> <line>-180</line> </teamH> <over_under>8.5u</over_under> </game> <game> <time>1:05</time> <teamA rotation="967" name="OAK"> <pitcher>D.Pomeranz</pitcher> <line>-125</line> </teamA> <teamH rotation="968" name="TOR"> <pitcher>J.Happ</pitcher> <line></line> </teamH> <over_under>9o</over_under> </game> <game> <time>1:35</time> <teamA rotation="969" name="CLE"> <pitcher>T.Bauer</pitcher> <line></line> </teamA> <teamH rotation="970" name="BAL"> <pitcher>M.Gonzalez</pitcher> <line>-120</line> </teamH> <over_under>9p</over_under> </game> <game> <time>1:40</time> <teamA rotation="971" name="BOS"> <pitcher>B.Workman</pitcher> <line></line> </teamA> <teamH rotation="972" name="TAM"> <pitcher>J.Odorizzi</pitcher> <line>-120</line> </teamH> <over_under>8p</over_under> </game> <game> <time>2:10</time> <teamA rotation="973" name="NYY"> <pitcher>M.Tanaka</pitcher> <line>-165</line> </teamA> <teamH rotation="974" name="CWS"> <pitcher>A.Rienzo</pitcher> <line></line> </teamH> <over_under>7.5o</over_under> </game> <game> <time>3:35</time> <teamA rotation="975" name="KAN"> <pitcher>J.Vargas</pitcher> <line></line> </teamA> <teamH rotation="976" name="ANA"> <pitcher>G.Richards</pitcher> <line>-155</line> </teamH> <over_under>8u</over_under> </game> <game> <time>4:10</time> <teamA rotation="977" name="HOU"> <pitcher>D.Keuchel</pitcher> <line></line> </teamA> <teamH rotation="978" name="SEA"> <pitcher>H.Iwakuma</pitcher> <line>-165</line> </teamH> <over_under>6.5o</over_under> </game> <note>Inter League</note> <game> <time>4:05</time> <teamA rotation="979" name="MIN"> <pitcher>R.Nolasco</pitcher> <line></line> </teamA> <teamH rotation="980" name="SFG"> <pitcher>M.Bumgarner</pitcher> <line>-175</line> </teamH> <over_under>7.5p</over_under> </game> <note>Write-in game - Game two of doubleheader</note> <game> <time>4:40</time> <teamA rotation="981" name="ARI"> <pitcher>Z.Spruill</pitcher> <line></line> </teamA> <teamH rotation="982" name="NYM"> <pitcher>D.Matsuzaka</pitcher> <line>off</line> </teamH> <over_under>off</over_under> </game> </date> <time_stamp> May 24, 2014, at 03:05 PM ET </time_stamp> </message>
+ test/xml/Odds_XML-missing-starters.xml view
@@ -0,0 +1,1 @@+<?xml version="1.0" standalone="no" ?> <!DOCTYPE message PUBLIC "-//TSN//DTD Statistics 1.0/EN" "Odds_XML.dtd"> <message> <XML_File_ID>21434211</XML_File_ID> <heading>AAO;BB-ODDS</heading> <category>Odds</category> <sport>MLB</sport> <Title>Las Vegas Major League Baseball Line</Title> <Line_Time>Current Line as of 9:30 A.M. ET</Line_Time> <League_Name>Inter League</League_Name> <Notes>TUESDAY, JULY 15TH</Notes> <Notes>85TH All Star Game - Target Field - Minneapolis, MN</Notes> <Game> <GameID>42056</GameID> <Game_Date>07/15/2014</Game_Date> <Game_Time>08:00 PM</Game_Time> <AwayTeam> <AwayTeamID>683</AwayTeamID> <AwayRotationNumber>945</AwayRotationNumber> <AwayAbbr>NAT</AwayAbbr> <AwayTeamName>MLB-National</AwayTeamName> <AStarter ID="0"></AStarter> <Casino ClientID="116" Name="5Dimes">-103</Casino> <Casino ClientID="111" Name="Sports Interaction">-110</Casino> <Casino ClientID="104" Name="BOVADA"></Casino> <Casino ClientID="121" Name="Mirage"></Casino> <Casino ClientID="127" Name="DonBest Consensus">-103</Casino> </AwayTeam> <HomeTeam> <HomeTeamID>684</HomeTeamID> <HomeRotationNumber>946</HomeRotationNumber> <HomeAbbr>AME</HomeAbbr> <HomeTeamName>MLB-American</HomeTeamName> <HStarter ID="0"></HStarter> <Casino ClientID="116" Name="5Dimes">-107</Casino> <Casino ClientID="111" Name="Sports Interaction">-110</Casino> <Casino ClientID="104" Name="BOVADA"></Casino> <Casino ClientID="121" Name="Mirage"></Casino> <Casino ClientID="127" Name="DonBest Consensus">-107</Casino> </HomeTeam> <Over_Under> <Casino ClientID="116" Name="5Dimes">7.5u</Casino> <Casino ClientID="111" Name="Sports Interaction">8u</Casino> <Casino ClientID="104" Name="BOVADA"></Casino> <Casino ClientID="121" Name="Mirage"></Casino> <Casino ClientID="127" Name="DonBest Consensus">8u</Casino> </Over_Under> </Game> <time_stamp> July 14, 2014, at 09:29 AM ET </time_stamp> </message>
+ test/xml/earlylineXML.dtd view
@@ -0,0 +1,20 @@+<!ELEMENT XML_File_ID (#PCDATA)>+<!ELEMENT heading (#PCDATA)>+<!ELEMENT category (#PCDATA)>+<!ELEMENT sport (#PCDATA)>+<!ELEMENT title (#PCDATA)>+<!ELEMENT note (#PCDATA)>+<!ELEMENT time (#PCDATA)>+<!ELEMENT teamA (#PCDATA)>+<!ELEMENT teamH (#PCDATA)>+<!ELEMENT over_under (#PCDATA)>+<!ELEMENT game ( ( time, teamA, teamH, over_under ) )>+<!ELEMENT date ( ( note, game ) )>+<!ELEMENT time_stamp (#PCDATA)>+<!ELEMENT message ( ( XML_File_ID, heading, category, sport, title, date*, time_stamp ) )>++<!ATTLIST teamA rotation CDATA #REQUIRED>+<!ATTLIST teamA line CDATA #REQUIRED>+<!ATTLIST teamH rotation CDATA #REQUIRED>+<!ATTLIST teamH line CDATA #REQUIRED>+<!ATTLIST date value CDATA #REQUIRED>
+ test/xml/earlylineXML.xml view
@@ -0,0 +1,1 @@+<?xml version="1.0" standalone="no" ?> <!DOCTYPE message PUBLIC "-//TSN//DTD Odds 1.0/EN" "earlylineXML.dtd"> <message> <XML_File_ID>21166989</XML_File_ID> <heading>ACO;NBA-EARLY-LINE</heading> <category>Odds</category> <sport>NBA</sport> <title>National Basketball Association Overnight Line</title> <date value="SUNDAY, MAY 25TH (05/25/2014)"> <note>Western Conference Finals - San Antonio leads 2-0</note> <game> <time>8:30</time> <teamA rotation="511" line="">San Antonio</teamA> <teamH rotation="512" line="-2">Oklahoma City</teamH> <over_under>208</over_under> </game> </date> <date value="MONDAY, MAY 26TH (05/26/2014)"> <note>Eastern Conference Finals - Miami leads 2-1</note> <game> <time>8:30</time> <teamA rotation="513" line="">Indiana</teamA> <teamH rotation="514" line="off">Miami</teamH> <over_under>off</over_under> </game> </date> <time_stamp> May 25, 2014, at 09:06 AM ET </time_stamp> </message>