diff --git a/README.md b/README.md new file mode 100644 --- /dev/null +++ b/README.md @@ -0,0 +1,15 @@ +# Kawhi : stats.NBA.com library 🏀 + +[![Build Status](https://img.shields.io/hackage/v/kawhi.svg)](https://hackage.haskell.org/package/kawhi) [![Build Status](https://travis-ci.org/hamsterdam/kawhi.svg?branch=development)](https://travis-ci.org/hamsterdam/kawhi) + +A Haskell library for working with [NBA Stats](http://stats.nba.com) data. NBA Stats contains rich data, but accessing it is difficult and tedious. Kawhi helps to automate the data retrieval, allowing users to focus on statistics. + +It's named after [Kawhi Leonard](http://cdn2.vox-cdn.com/assets/5039756/Kawhi-Leonard-portrait-oil.JPG). + +## Jump Ball + +To get started: + +1. [Get Haskell](https://haskell-lang.org/get-started). If you've never done any computer programming, this could be challenging. [Never give up!](https://www.instagram.com/p/BENA9hpN_wL/) +2. Read the [guide](https://github.com/hamsterdam/kawhi/blob/master/guide.md). +3. Check out the [example](https://github.com/hamsterdam/kawhi/tree/master/example). diff --git a/changelog.md b/changelog.md new file mode 100644 --- /dev/null +++ b/changelog.md @@ -0,0 +1,11 @@ +# Changelog + +## 0.1.0 + +### Summary + +Updates have been made to support [http-client-0.5.0](https://hackage.haskell.org/package/http-client-0.5.0). + +### API Changes + +The Utility functions have been removed as they are no longer useful. diff --git a/guide.md b/guide.md new file mode 100644 --- /dev/null +++ b/guide.md @@ -0,0 +1,77 @@ +# Guide + +## Gather requirements + +Ideally, the people responsible for [stats.nba.com](stats.nba.com) would explain exactly how to access all the data in an easy to use website. Unfortunately, that is not the current reality. For now, we have to do some detective work. Luckily, it's not too hard. + +*The following describes instructions for using Chrome on a macOS machine. The steps are similar on any modern web browser.* + +- Go to the page you want to get data from on [stats.nba.com](stats.nba.com). For example, go to the [San Antonio Spurs "advanced statistics" page](http://stats.nba.com/team/#!/1610612759/stats/advanced/). +- In the menu bar click `View -> Developer -> Developer Tools`. +- In the Developer Tools view, click the `Network` tab. +- Click `XHR` from the list of filters. +- Click the row with name that starts with `teamdashboardbygeneralsplits`. You may need to refresh. +- Click the `Preview` tab. + +All the data you'll need is presented in this tab. + +You'll need the `resource`, which is shown as `teamdashboardbygeneralsplits`. This is the unique identifier for this collection of data. In this case, it's the general splits for a specific player. **Record this value.** + +``` +teamdashboardbygeneralsplits +``` + +Additionally, you'll need data in `parameters`. Click the arrow to open `parameters` to reveal all the data. This collection of pairs, like `MeasureType` and `Advanced`, are parameters that control the data returned in the `teamdashboardbygeneralsplits` resource. The key `MeasureType` with value `Advanced` specifies that the data should contain "advanced" statistics instead of basic ones. All the pairs shown are *required* for valid data. **Record all of the keys and values.** + +``` +"MeasureType": "Advanced" +"PerMode": "PerGame" +"PlusMinus": "N" +"PaceAdjust": "N" +"Rank": "N" +"LeagueID": "00" +"Season": "2015-16" +"SeasonType": "Regular Season" +"PORound": 0 +"TeamID": 1610612759 +"Outcome": null +"Location": null +"Month": 0 +"SeasonSegment": null +"DateFrom": null +"DateTo": null +"OpponentTeamID": 0 +"VsConference": null +"VsDivision": null +"GameSegment": null +"Period": 0 +"ShotClockRange": null +"LastNGames": 0 +``` + +Finally, you'll need some data from `resultSets`. Click the arrow to reveal all the data. The rows in `resultSets` are the splits. Find the split with the `name` `MonthTeamDashboard`. **Record this value.** + +``` +MonthTeamDashboard +``` + +This contains the data for the 2015-16 regular season, split by month. It corresponds to the `Month` section on the regular webpage. Click the arrow to open the `MonthTeamDashboard` split. Look for the `headers`. Click to reveal all of its data. Look at the list of values, like `OFF_RATING`. These values correspond to the columns in the regular webpage. For example, `OFF_RATING` corresponds to the `OffRtg` column. **Record all the values corresponding to columns you are interested in collecting data for.** + +``` +SEASON_MONTH_NAME +OFF_RATING +DEF_RATING +``` + +## Make a Haskell project +Look at the [example project](example). It uses the data from this guide. The output looks like: + +``` +AdvancedStats {month = "October", offensiveRating = 102.7, defensiveRating = 93.4} +AdvancedStats {month = "November", offensiveRating = 102.5, defensiveRating = 93.4} +AdvancedStats {month = "December", offensiveRating = 111.8, defensiveRating = 91.5} +AdvancedStats {month = "January", offensiveRating = 114.0, defensiveRating = 100.7} +AdvancedStats {month = "February", offensiveRating = 110.7, defensiveRating = 99.1} +AdvancedStats {month = "March", offensiveRating = 107.8, defensiveRating = 97.2} +AdvancedStats {month = "April", offensiveRating = 102.3, defensiveRating = 103.5} +``` diff --git a/kawhi.cabal b/kawhi.cabal --- a/kawhi.cabal +++ b/kawhi.cabal @@ -1,5 +1,5 @@ name: kawhi -version: 0.0.1 +version: 0.1.0 synopsis: stats.NBA.com library description: Functions and types for interacting with stats.NBA.com homepage: https://github.com/hamsterdam/kawhi @@ -10,6 +10,11 @@ category: API build-type: Simple cabal-version: >=1.10 +extra-source-files: + README.md + guide.md + stack.yaml + changelog.md library hs-source-dirs: library @@ -22,8 +27,8 @@ aeson >= 0.11, bytestring >= 0.10, exceptions >= 0.8, - http-conduit >= 2.1, - http-client >= 0.4, + http-conduit >= 2.2, + http-client >= 0.5.1, http-types >= 0.9, mtl >= 2.2, safe >= 0.3, @@ -38,21 +43,21 @@ main-is: Tests.hs other-modules: NBA.Stats.Tests build-depends: - base, - kawhi, - aeson, - bytestring, - exceptions, - http-client, - http-types, - mtl, - scientific, - smallcheck, - tasty, - tasty-hunit, - tasty-quickcheck, - tasty-smallcheck, - text + base >= 4.8 && < 5, + kawhi >= 0.0, + aeson >= 0.11, + bytestring >= 0.10, + exceptions >= 0.8, + http-client >= 0.4.30, + http-types >= 0.9, + mtl >= 2.2, + scientific >= 0.3, + smallcheck >= 1.1, + tasty >= 0.11, + tasty-hunit >= 0.9, + tasty-quickcheck >= 0.8, + tasty-smallcheck >= 0.8, + text >= 1.2 ghc-options: -Wall -threaded -rtsopts -with-rtsopts=-N default-language: Haskell2010 diff --git a/library/Control/Monad/Http.hs b/library/Control/Monad/Http.hs --- a/library/Control/Monad/Http.hs +++ b/library/Control/Monad/Http.hs @@ -40,14 +40,19 @@ performRequest = HTTPSimple.httpLbs instance Catch.MonadThrow m => MonadHttp (HttpT m) where - performRequest _ = check + performRequest request = check where check = do response <- Reader.ask let status = HTTP.responseStatus response if status >= HTTP.ok200 && status < HTTP.multipleChoices300 then return response - else Catch.throwM $ HTTP.StatusCodeException status [] (HTTP.createCookieJar []) + else + let + badResponse = response { HTTP.responseBody = () } + body = LBS.toStrict . HTTP.responseBody $ response + in + Catch.throwM $ HTTP.HttpExceptionRequest request (HTTP.StatusCodeException badResponse body) instance Trans.MonadIO m => MonadHttp (Except.ExceptT e m) where performRequest = HTTPSimple.httpLbs diff --git a/library/NBA/Stats.hs b/library/NBA/Stats.hs --- a/library/NBA/Stats.hs +++ b/library/NBA/Stats.hs @@ -33,12 +33,9 @@ StatsPath, StatsParameters, StatsError(..), - - -- * Utility - domain, - getRequest, ) where +import qualified Control.Monad.Catch as Catch import qualified Control.Monad.Except as Except import qualified Control.Monad.Trans as Trans import qualified Control.Monad.Http as MonadHttp @@ -47,11 +44,11 @@ import Data.Aeson ((.:), (.=)) import qualified Data.ByteString.Lazy as LBS import qualified Data.ByteString as SBS +import qualified Data.ByteString.Char8 as Char8 import qualified Data.List as List import Data.Monoid ((<>)) import qualified Data.Text as Text import qualified Network.HTTP.Client as HTTP -import qualified Network.HTTP.Simple as HTTP import qualified Safe {- | @@ -88,7 +85,7 @@ The simpler version of this function, 'getSplitRows', has a concrete 'm'. -} getSplitRowsGeneric :: - (Trans.MonadIO m, MonadHttp.MonadHttp m, Except.MonadError StatsError m, Aeson.FromJSON a) + (Trans.MonadIO m, MonadHttp.MonadHttp m, Except.MonadError StatsError m, Catch.MonadThrow m, Aeson.FromJSON a) => StatsPath -- ^ The URL path for the stats web page containing the split. -> SplitName -- ^ The split name. -> StatsParameters -- ^ The parameters for customizing the stats. @@ -104,7 +101,7 @@ The simpler version of this function, 'getSplitRows', has a concrete 'm'. -} getSplitRowGeneric :: - (Trans.MonadIO m, MonadHttp.MonadHttp m, Except.MonadError StatsError m, Eq v, Show v, Aeson.FromJSON v, Aeson.FromJSON a) + (Trans.MonadIO m, MonadHttp.MonadHttp m, Except.MonadError StatsError m, Catch.MonadThrow m, Eq v, Show v, Aeson.FromJSON v, Aeson.FromJSON a) => StatsPath -- ^ The URL path for the stats web page containing the split. -> SplitName -- ^ The split name. -> SplitColumn -- ^ The column name key for a the desired row. @@ -229,23 +226,6 @@ format :: String -> String -> String format name message = name ++ " " ++ message -{- | - Generates an HTTP GET request like the ones used internally. --} -getRequest :: StatsPath -> HTTP.Request -getRequest path = HTTP.defaultRequest { - HTTP.method = "GET", - HTTP.secure = False, - HTTP.host = domain, - HTTP.path = "/stats/" <> path -} - -{- | - The NBA Stats domain name. --} -domain :: SBS.ByteString -domain = "stats.nba.com" - parseSplitRow :: (Except.MonadError StatsError m, Aeson.FromJSON a) => [SplitColumn] -> SplitRow -> m a parseSplitRow columns row = if length columns == length row @@ -267,13 +247,15 @@ -get :: (Trans.MonadIO m, MonadHttp.MonadHttp m) => StatsPath -> StatsParameters -> m (HTTP.Response LBS.ByteString) -get path params = MonadHttp.performRequest $ HTTP.setQueryString params $ getRequest path +get :: (MonadHttp.MonadHttp m, Catch.MonadThrow m) => StatsPath -> StatsParameters -> m (HTTP.Response LBS.ByteString) +get path params = do + request <- HTTP.parseRequest $ Char8.unpack $ "http://stats.nba.com/stats/" <> path + MonadHttp.performRequest $ HTTP.setQueryString params request {- $use The following is a working example of getting some "advanced statistics", split by month, for the San Antonio Spurs 2015-2016 regular season. - To learn how to find the NBA Stats values, like 'teamdashboardbygeneralsplits', for this example, read the . + To learn how to find the NBA Stats values, like 'teamdashboardbygeneralsplits', for this example, read the . @ import qualified Data.Aeson as Aeson diff --git a/stack.yaml b/stack.yaml new file mode 100644 --- /dev/null +++ b/stack.yaml @@ -0,0 +1,15 @@ +# Specifies the GHC version and set of packages available (e.g., lts-3.5, nightly-2015-09-21, ghc-7.10.2) +resolver: lts-6.7 + +# Packages to be built +packages: +- '.' + +extra-deps: + - http-client-0.5.1 + - http-conduit-2.2.0 + - http-client-tls-0.3.0 + +docker: + enable: false # Travis containerized builds don't support Docker yet + repo: "fpco/stack-build" diff --git a/tests/NBA/Stats/Tests.hs b/tests/NBA/Stats/Tests.hs --- a/tests/NBA/Stats/Tests.hs +++ b/tests/NBA/Stats/Tests.hs @@ -8,7 +8,6 @@ import qualified Data.Aeson as Aeson import qualified Data.Aeson.Types as Aeson import Data.Aeson ((.:)) -import qualified Data.ByteString.Char8 as Char8 import qualified Data.ByteString.Lazy as ByteString import Data.Monoid ((<>)) import qualified Data.Scientific as Sci @@ -23,11 +22,6 @@ tests :: Tasty.TestTree tests = Tasty.testGroup "NBA.Stats" [ - SC.testProperty "getRequest == HTTP.parseUrl" $ - \path -> SC.monadic $ do - let request = Stats.getRequest $ Char8.pack path - model <- HTTP.parseUrl $ "http://stats.nba.com/stats/" <> path - return $ show request == show model, propertyStatsErrorShow Stats.StatsResponseDecodeFailure "StatsResponseDecodeFailure", propertyStatsErrorShow Stats.SplitNameNotFound "SplitNameNotFound", propertyStatsErrorShow Stats.SplitKeyNotFound "SplitKeyNotFound",