diff --git a/CHANGELOG.org b/CHANGELOG.org --- a/CHANGELOG.org +++ b/CHANGELOG.org @@ -1,4 +1,9 @@ #+TITLE: CHANGELOG +* 0.0.2 ++ Add more documentation and examples programs ++ Export ~Paginable~ class from ~Reddit.hs~ ++ ~firstPage~ now gets the API limit of items (100) + * 0.0.1 + Initial release diff --git a/README.org b/README.org --- a/README.org +++ b/README.org @@ -3,23 +3,97 @@ ~heddit~ is a Reddit API client library for Haskell. It aims to be (almost) as comprehensive as libraries such as [[https://github.com/praw-dev/praw][praw]], from which it takes major inspiration. Although it is not quite there, it does offer access to various parts of the Reddit API, including: - comments and submissions - user accounts and private messages - - subreddits, wikis, collections, and emojis + - subreddits, multireddits, wikis, collections, and sub emojis - live threads - moderation and various mod actions - - ... and more + - ... and more! To start using ~heddit~, visit [[file:./doc/start.org][the (very) quick start guide]] or browse the haddocks in [[file:./src/Network][src/Network]] *WARNING*: ~heddit~ is /definitely/ alpha-quality at the moment. Expect bugs, instability, and incomplete documentation for now! * Table of Contents +- [[Brief example]] - [[file:./doc/start.org][Quick start]] - [[file:./doc/auth.org][Authenticating]] - [[Caveats]] - [[Examples]] +- [[Helpful libraries]] - [[Tests]] - [[License]] +* Brief example +We can briefly demonstrate what you can do with ~heddit~ by getting a list of r/haskell's moderators, printing out some information about them, and then finding out which mod has the highest link karma. You can also directly run this example with [[file:./examples/Mods.hs][the mods example program]] that contains the same code. + +#+begin_src haskell +{-# LANGUAGE OverloadedLabels #-} + +import Control.Monad.IO.Class ( MonadIO(liftIO) ) + +import Data.Foldable ( for_, maximumBy ) +import Data.Generics.Labels () +import Data.Generics.Wrapped +import Data.Ord ( comparing ) +import qualified Data.Text.IO as T + +import Lens.Micro.Platform + +import Network.Reddit +-- This module is not exported by default from Network.Reddit, as it exports +-- quite a large number of actions and types +import Network.Reddit.Moderation ( getModerators ) + +-- This is a small example of how I would use the library. Note that I'm using +-- @generic-lens@ and @microlens-platform@ for record access. This isn't entirely +-- necessary, but I decided to hide record selectors by default. This library +-- defines a lot of record types, and exporting all of the selectors would +-- steal a lot of top-level names in calling code. If you want to use them, +-- you can import the relevant modules directly. For this small example, if +-- you didn't want to use the generic-lens-based approach I'm using, you +-- would need to import Network.Reddit.Types.Account (Account(..)) +main :: IO () +main = + -- First we'll use @loadClient@ to get a @Client@ by reading the auth + -- details from an @auth.ini@ file either in the working directory or in + -- $XDG_DATA_HOME/heddit. @Nothing@ indicates that we'll use the section + -- marked @[DEFAULT]@ in the ini file. + -- + -- @runReddit@ takes a @Client@ and a @RedditT@ action to run, which + -- in this example is @modsInfo@ below + loadClient Nothing >>= (`runReddit` modsKarmaInfo) + where + modsKarmaInfo = do + mods <- + -- First, we'll get a @Seq ModAccount@, which is like an + -- @Account@ but has less information + (getModerators =<< mkSubredditName "haskell") + -- So we'll pipe each one into @getUser@ by extracting + -- the username + >>= traverse (getUser . (^. #username)) + -- Then, we can print the @linkKarma@ for each account + -- (I'm using @wrappedTo@ from Data.Generics.Wrapped, + -- which generically unwraps single-constructor types, + -- to unwrap the @Username@ and get the @Text@ value) + for_ mods $ \m -> liftIO . T.putStrLn + $ mconcat [ m ^. #username & wrappedTo + , " has " + , m ^. #linkKarma . to show . packed + ] + + -- Finally, we can see which mod account has the most + -- link karma by comparing the sequence of mods using + -- the @topKarma@ function below + liftIO . T.putStrLn + $ mconcat [ "The mod with the most karma is " + , topKarma mods ^. #username & wrappedTo + ] + + topKarma = maximumBy (comparing (^. #linkKarma)) + +#+end_src + +See the [[Examples]] section below for more. + * Caveats There are some major caveats to using ~heddit~ that you may want to consider before deciding to use it, namely: - It is currently alpha quality. It needs more tests and users to work out some of the bugs that are assuredly lurking deep within. The exposed API is a little weird and may be difficult to work with. @@ -27,11 +101,25 @@ - ~heddit~ only supports authenticating via OAuth (see [[file:./doc/auth.org][Authenticating]]) and cannot access Reddit's old JSON API. This means that it is not possible to create a truly anonymous client. Instead, you must register all applications, even those without a user context. * Examples -I would like to add some more executable examples of how to use ~heddit~, particularly to demonstrate some of the weirder points of the API. At the moment, the available example programs are: +Here are some executable examples using different features from ~heddit~ that hopefully illustrate how to use the library and accomplish different tasks. At the moment, the available example programs are: +- The [[file:./examples/Mods.hs][mods example]] shown above +- An [[file:./examples/Paginating.hs][example]] showing how to use ~Listing t a~ and ~Paginator t a~ to paginate through data returned from Reddit (or avoid it altogether). +- Infinitely [[file:./examples/Streaming.hs][streaming]] paginable things as they are created - Getting an initial [[file:./examples/RefreshTokens.hs][refresh token]] +All of the example programs live in the [[file:./examples][examples]] directory and can be run using, for example, ~cabal run exes:~ + * Tests ~heddit~ has two test suites: a simple test of JSON decoding of API responses that runs by default, and the ~io-tests~ that are disabled by default behind a flag of the same name. If you wish to run the ~io-tests~, be aware that these make real requests against Reddit's API endpoints using the authenticated account. After enabling the tests, make sure to create an ~auth.ini~ and ~profile.json~ in the working directory or in ~$XDG_CONFIG_HOME/heddit~. See the ~io-tests~ modules for more info. + +* Helpful libraries +There are some other libraries that you may wish to use with ~heddit~: +- ~containers~, for ~Seq~ +- ~conduit~, if you wish to [[file:./src/Network/Reddit.hs::stream][stream]] Reddit actions +- ~exceptions~, for ~MonadThrow~ and ~MonadCatch~ +- Possibly ~unliftio~, for the ~MonadUnliftIO~ constraint, although this is also exported by ~conduit~. This is really only needed for initializing a ~Client~, which you could just do at the top-level in IO anyway +- ~generic-lens~ if you want to use the lens labels as demonstrated above. Also provides several other helpful lenses and functions for dealing with data generically. Note that the ~generic-lens~ labels are not used internally by ~heddit~, so it is safe to use with other libraries that rely on ~-XOverloadedLabels~ +- ~microlens-platform~, ~lens~, or another lens-compatible lib to use with ~generic-lens~, should you choose to use the latter * License ~heddit~ is distributed under a three-clause BSD license. See [[file:LICENSE][the license]] for details. Parts of this library are directly inspired by and/or adapted from ~praw~, which is distributed under a simplified BSD two-clause license. See [[https://github.com/praw-dev/praw/blob/master/LICENSE.txt]] diff --git a/examples/Mods.hs b/examples/Mods.hs new file mode 100644 --- /dev/null +++ b/examples/Mods.hs @@ -0,0 +1,48 @@ +{-# LANGUAGE OverloadedLabels #-} + +module Mods where + +import Control.Monad.IO.Class ( MonadIO(liftIO) ) + +import Data.Foldable ( for_, maximumBy ) +import Data.Generics.Labels () +import Data.Generics.Wrapped +import Data.Ord ( comparing ) +import qualified Data.Text.IO as T + +import Lens.Micro.Platform + +import Network.Reddit +import Network.Reddit.Moderation ( getModerators ) + +-- | This tiny program will get the current moderators of r\/haskell, print the +-- link karma for each one, and then find the mod with the highest karma and print +-- the username. At the moment, running this example produces: +-- +-- > dons has 76424 +-- > jfredett has 748 +-- > edwardkmett has 6180 +-- > taylorfausak has 6216 +-- > Iceland_jack has 1402 +-- > BoteboTsebo has 229 +-- > AutoModerator has 1000 +-- > The mod with the most karma is dons +-- +main :: IO () +main = loadClient Nothing >>= (`runReddit` modsKarmaInfo) + where + modsKarmaInfo = do + mods <- (getModerators =<< mkSubredditName "haskell") + >>= traverse (getUser . (^. #username)) + for_ mods $ \m -> liftIO . T.putStrLn + $ mconcat [ m ^. #username & wrappedTo + , " has " + , m ^. #linkKarma . to show . packed + ] + + liftIO . T.putStrLn + $ mconcat [ "The mod with the most karma is " + , topKarma mods ^. #username & wrappedTo + ] + + topKarma = maximumBy (comparing (^. #linkKarma)) diff --git a/examples/Paginating.hs b/examples/Paginating.hs new file mode 100644 --- /dev/null +++ b/examples/Paginating.hs @@ -0,0 +1,113 @@ +{-# LANGUAGE FlexibleContexts #-} +{-# LANGUAGE OverloadedLabels #-} + +-- | This example program shows how to use 'Paginator's and 'Listing's, which +-- are used in many API endpoints. You may notice that several of the actions +-- exported by this library have a type signature similar to +-- @... Paginator t a -> Listing t a@. In many cases, when you make a request to +-- Reddit, it will respond with a \"view\" of the data along with controls that +-- allow you to paginate through entries. This is represented as the 'Listing' +-- type in @heddit@. You can use this listing to feed the next action to fetch +-- more items in the form of a 'Paginator'. In addition, many of the types that +-- are 'Paginable' (and thus can be used in a 'Paginator') have extra options +-- that can be passed to the 'Paginator' in order to filter, limit or sort items. +-- +-- There are are a couple of ways to deal with 'Listing's and 'Paginator's in +-- @heddit@, as the examples below aim to illustrate +module Paginating where + +import Data.Foldable ( for_ ) +import Data.Generics.Labels () +import qualified Data.Text.IO as T + +import Lens.Micro.Platform + +import Network.Reddit + +main :: IO () +main = do + c <- loadClient Nothing + subname <- mkSubredditName "haskell" + results <- runReddit c . sequence + $ [ ignorePagination, withOptions, secondPage ] <*> [ subname ] + for_ (zip descs results) + $ \(desc, res) -> T.putStrLn $ desc <> ": " <> getTitle res + where + descs = [ "1st result (without pagination)" + , "1st result (all-time)" + , "11th result (with pagination)" + ] + + getTitle = maybe "No results!" (^. #title) + +-- | If you do not want to deal with any of this 'Listing' or 'Paginator', +-- business, but would rather just get the first results that Reddit returns, +-- you can use the convenience function 'firstPage' with your action. This +-- ignores the pagination controls entirely, and just gets the items from the +-- endpoint. You can\'t fetch subsequent items, however, or pass any additional +-- options. You are thus limited to the first 100 items, with default options +ignorePagination :: MonadReddit m => SubredditName -> m (Maybe Submission) +ignorePagination subname = + -- 'firstPage' just gets up to the first 100 items, using the default + -- sort and other options + firstPage (getTopSubmissions subname) <&> (^? _head) + +-- | This example uses a 'Paginator' to add an option telling Reddit the timeframe +-- we are interested in. +-- +-- With actions that take a @Paginator@ and return a @Listing@, you can +-- provide an initial paginator with @emptyPaginator@. @Paginator@ has +-- a field @opts@ that holds an instance of the 'PaginateOptions' type +-- family. Check the haddocks for the options of different 'Paginable' +-- types +withOptions :: MonadReddit m => SubredditName -> m (Maybe Submission) +withOptions subname = do + -- In this case, we specify the time range that we are interested in + submissions <- getTopSubmissions subname + $ emptyPaginator & #opts . #itemTime ?~ AllTime + -- @submissions@ is a @Listing SubmissionID Submission@. The @children@ + -- field of a listing holds the actual results. The other fields will + -- be discussed below + pure $ submissions ^? #children . _head + +-- | In this example, we can use the 'Listing' resulting from an action to get +-- the next \"page\" of results +secondPage :: MonadReddit m => SubredditName -> m (Maybe Submission) +secondPage subname = do + -- All @Paginator@s have a @limit@ field to specify the number of items + -- desired, which we will set to 10 (the maximum is ostensibly 100, + -- although Reddit doesn't seem to care if you exceed this; the default + -- is 25) + firstListing <- getTopSubmissions subname firstPaginator + -- Now that we have a listing, we can turn it into a @Paginator@ to feed + -- the next API call by using the function 'nextPage'. You can optionally + -- provide the initial paginator as well to avoid having to set the same + -- options again. + -- + -- All @Listing@s have @before@ and @after@ fields which act like anchors + -- in the "slice" of data. They can be used as pagination controls, but are + -- not necessarily present. Using @nextPage@ will create a @Paginator@ with + -- corresponding @before@ and @after@ fields set to the values from the + -- @Listing@. When both fields are @Just@, @after@ takes precedence. + -- + -- The behavior of @before@ is a little counter-intuitive (at least to me). + -- It does /not/ point to data preceding the current first item in the + -- @children@ field, and will only be non-null if items have been created + -- that would precede it in the interim since obtaining the @Listing@. That + -- is, even if you are on the nth page above 1 of the data set, @before@ will + -- still be @Nothing@ unless entirely new items have been created on + -- Reddit's end + nextListing <- getTopSubmissions subname + $ nextPage (Just firstPaginator) firstListing + -- We could continue in the same vein, but we'll stop here and return the + -- first child + pure $ nextListing ^? #children . _head + where + firstPaginator = emptyPaginator & #limit .~ 10 + +nTimes :: MonadReddit m => SubredditName -> m (Maybe Submission) +nTimes = do + undefined + where + unfoldSubs paginator = do + undefined diff --git a/examples/Streaming.hs b/examples/Streaming.hs new file mode 100644 --- /dev/null +++ b/examples/Streaming.hs @@ -0,0 +1,74 @@ +{-# LANGUAGE FlexibleContexts #-} +{-# LANGUAGE OverloadedLabels #-} + +-- | This example will show how to use the 'stream' action to get an infinite +-- stream of new 'Paginable' things. In this example, we will stream new +-- submissions, filter them based on some criteria, and print out some information +-- about them. In a real program, instead of simply printing information, we could +-- take some 'MonadReddit' action instead, such as replying to them +-- +-- Running this example will produce the following type of output (truncated for +-- readability here): +-- +-- > On 2021-07-18 07:32:21 UTC u/Pututu_Life posted "What is your go to comfort .. +-- > On 2021-07-18 07:32:29 UTC u/Kenney93 posted "What do you do when you are ... +-- > On 2021-07-18 07:32:45 UTC u/CYS801 posted "What is your proudest ... +-- > On 2021-07-18 07:32:53 UTC u/Ambitious-Ad8598 posted "What is the name of ... +-- > On 2021-07-18 07:33:08 UTC u/Additional_Quantity5 posted "What is your ... +-- +module Streaming where + +import Conduit + +import Data.Generics.Labels () +import Data.Generics.Wrapped +import Data.Text ( Text ) +import qualified Data.Text as T +import qualified Data.Text.IO as T + +import Lens.Micro.Platform + +import Network.Reddit + +main :: IO () +main = loadClient Nothing >>= (`runReddit` streamed) + where + -- We can use r/askreddit, which has a fairly high and steady + -- volume of new submissions being posted + streamed = streamSubmissions =<< mkSubredditName "askreddit" + +-- | Run the stream until interrupted +streamSubmissions :: MonadReddit m => SubredditName -> m () +streamSubmissions subname = runConduit + $ stream (Just True) -- This will discard elements that were + -- already "in" the stream before starting, + -- so we only get actually new submissions + action + .| filterC isAskingWhat + .| submissionInfo + .| mapM_C (liftIO . T.putStrLn) -- This will look nicer than + -- @printC@ + where + action = getNewSubmissions subname + +-- | A silly heuristic to detect questions asking "what ... ?". Of course +-- this will miss questions using the popular format "...s of Reddit, +-- what...?", but r/askreddit is high-volume enough that it will still +-- catch a lot of submissions +isAskingWhat :: Submission -> Bool +isAskingWhat s = case s ^? #title . to (T.words . T.toLower) . _head of + Just "what" -> True + _ -> False + +-- | Print some information about the submission +submissionInfo :: Monad m => ConduitT Submission Text m () +submissionInfo = awaitForever $ \s -> yield + $ mconcat [ "On " + , s ^. #created . to show . packed + , " u/" + , s ^. #author & wrappedTo + , " posted " + , "\"" + , s ^. #title + , "\"" + ] diff --git a/heddit.cabal b/heddit.cabal --- a/heddit.cabal +++ b/heddit.cabal @@ -1,6 +1,6 @@ cabal-version: 3.0 name: heddit -version: 0.0.1 +version: 0.0.2 synopsis: Reddit API bindings description: See the README at https://gitlab.com/ngua/heddit license: BSD-3-Clause @@ -133,6 +133,63 @@ main-is: RefreshTokens.hs ghc-options: -main-is RefreshTokens + default-language: Haskell2010 + +executable mods + import: common-options, common-extensions + hs-source-dirs: examples + + if !flag(examples) + buildable: False + + build-depends: + , base >=4.13 && <5 + , generic-lens >=1.1 && <2.2 + , heddit + , microlens-platform ^>=0.4.2 + , text ^>=1.2 + + main-is: Mods.hs + ghc-options: -main-is Mods + default-language: Haskell2010 + +executable streaming + import: common-options, common-extensions + hs-source-dirs: examples + + if !flag(examples) + buildable: False + + build-depends: + , base >=4.13 && <5 + , conduit ^>=1.3 + , generic-lens >=1.1 && <2.2 + , heddit + , microlens-platform ^>=0.4.2 + , text ^>=1.2 + + main-is: Streaming.hs + ghc-options: -main-is Streaming + default-language: Haskell2010 + +executable paginating + import: common-options, common-extensions + hs-source-dirs: examples + + -- if !flag(examples) + -- buildable: False + + build-depends: + , base >=4.13 && <5 + , containers ^>=0.6 + , generic-lens >=1.1 && <2.2 + , heddit + , microlens-platform ^>=0.4.2 + , monad-loops ^>=0.4.3 + , text ^>=1.2 + + main-is: Paginating.hs + ghc-options: -main-is Paginating default-language: Haskell2010 test-suite tests diff --git a/src/Network/Reddit.hs b/src/Network/Reddit.hs --- a/src/Network/Reddit.hs +++ b/src/Network/Reddit.hs @@ -41,6 +41,7 @@ , RateLimits(RateLimits) , Listing(Listing) , Paginator(Paginator) + , Paginable , ItemOpts(ItemOpts) , defaultItemOpts , ItemSort(..) @@ -256,7 +257,7 @@ firstPage :: (MonadReddit m, Paginable a) => (Paginator t a -> m (Listing t a)) -> m (Seq a) -firstPage f = f emptyPaginator <&> (^. field @"children") +firstPage f = f emptyPaginator { limit = 100 } <&> (^. field @"children") -- | Update a 'Paginator' with a 'Listing' to make a query for the next \"page\" -- of content. If the first argument is @Nothing@, defaults will be used for