diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -1,1 +1,174 @@ -# sitepipe +# SitePipe + +### Contents: + +- [What is it?](#what-is-it) + - [What's it look like?](#whats-it-look-like) + - [Wait, another static site generator? What about + Hakyll/Jekyll?](#wait-another-static-site-generator-what-about-hakylljekyll) +- [Getting Started](#getting-started) + - [Quick Start](#quick-start) + - [Tutorial](#tutorial) +- [Concepts](#concepts) + - [How is SitePipe different from other + solutions?](#how-is-sitepipe-different-from-other-solutions) + - [Data/Metadata](#datametadata) + - [Templating](#templating) + - [Loaders](#loaders) + - [Reader](#reader) + - [Writers](#writers) + - [Loader/Writers](#loaderwriters) + - [Utilities](#utilities) +- [Issues/Troubleshooting](#issuestroubleshooting) + +## What is it? + +It's a simple to understand static site generator for making blogs, personal +websites, etc. + +## What's it look like? + +Here's a dead-simple blog generated from markdown files, you can see it in action in +[examples/starter-template](./examples/starter-template), or build on it in the [tutorial](./docs/tutorial.md) + +```haskell +{-# language OverloadedStrings #-} +module Main where + +import SitePipe + +main :: IO () +main = site $ do + -- Load all the posts from site/posts/ + posts <- resourceLoader markdownReader ["posts/*.md"] + + -- Build up a context for our index page + let indexContext :: Value + indexContext = object [ "posts" .= posts + -- The url is where the index page will be written to + , "url" .= ("/index.html" :: String) + ] + + -- write out index page and posts via templates + writeTemplate "templates/index.html" [indexContext] + writeTemplate "templates/post.html" posts +``` + +## Wait, another static site generator? What about Hakyll/Jekyll? + +Yup, yet another static site generator. The reason for it is that I tried using +Hakyll and Jekyll on different occasions and found there was too much magic +going on for me to understand how to customize things for my use-cases. They were +too opinionated without giving me escape hatches to wire in my own functionality. + +When I tried Hakyll specifically I got really bogged down; what was a +`Compiler` monad? How does an `Item` work? How do I add a custom field? Why +couldn't I just edit data directly like I'm used to doing in Haskell? + +# Getting Started + +## Quick Start + +The easiest way to get started is to clone this repo and try out the examples in the +[examples](./examples) directory. There's a starter-template which is a barebones +starting point, and also a slightly more complex blog with tags and an rss feed. +You can build either of the examples using [Stack](http://seanhess.github.io/2015/08/04/practical-haskell-getting-started.html) +by `cd`ing into the directory and running `stack build && stack exec build-site`. +This creates a 'dist' folder with the results of the build. A quick way to serve +the site is to use [Serve](https://www.npmjs.com/package/serve). + +Serving a site with [Serve](https://www.npmjs.com/package/serve): +- `npm install -g serve` +- `serve dist` +- Navigate to the port which is serving (usually http://localhost:3000) + +## Tutorial + +Read the walkthrough of the system [HERE](./docs/tutorial.md); it'll run you through the basics +of how the system works and how to make your own customizations! + +# Concepts + +How is SitePipe different from other solutions? +----------------------------------------------- + +Instead of dealing with complex contexts SitePipe works with *values*. Values +are loaded from files and can be rendered into html. What happens to the values +in-between is up to you! + +SitePipe provides a bunch of helpers for you, but at the end of the day you can +fit the pipes together however you like. + +## Data/Metadata + +Metadata for posts and content is parsed from yaml into [Aeson's `Value` +type](https://hackage.haskell.org/package/aeson); Unlike Hakyll which depends +on Pandoc's metadata blocks which can only accept Strings as values, Aeson can +easily represent nested objects or lists inside your metadata, and there's a +rich ecosystem for working with Aeson types! You can load resources in as any +object which implements `FromJSON` (or just leave them as Aeson Values) and you +have the option to edit the objects directly without worrying about monadic or +external context. + +## Templating + +SitePipe has built-in support for [Mustache +Templates](https://mustache.github.io/mustache.5.html), specifically [Justus +Adam's implementation](https://hackage.haskell.org/package/mustache) in +Haskell. This lets you use a well established templating system in your site, +complete with template functions, partials, and iteration. Since the underlying +data is based on It's clear how templates will behave since resources are based +on Aeson's JSON types. + +## Loaders + +You can load resources in to work on them using a `Loader`, A loader simply +finds and loads files into resources by employing a `Reader` on some files. A +basic `resourceLoader` loader is provided, which will load all of the files +matching a set of file-globs through the provided reader and will return an +Aeson Value containing the relevant metadata and content. You should be able to +use resourceLoader for most things by customizing the reader function which you +pass it. + +## Reader + +A reader is a function of the type `String -> IO String`; the input is the file +contents which remain after a yaml header has been stripped off (if it exists). +The most common reader is the provided `markdownReader` which runs a markdown +document through pandoc's markdown processor and outputs html. You can write +your own readers if you like, either by making a function which operates over +the content of the document and matches `String -> IO String` or by using +the provided Pandoc helpers (`mkPandocReader`, `mkPandocReaderWith`) which +allow you to use any of Pandoc's provided document formats, and optionally specify +transformations over the pandoc document before it is rendered to html or some other +output format. + +## Writers + +Writers take a list of resources (anything with a ToJSON instance, often an +Aeson Value) and will write them to the output where the static site will be. +The most common writer is `writeTemplate` which will render the given resource +through a given template, but you can also use `textWriter`, or write your own +writer; either writing to disk using `liftIO` or by using the provided +`writeWith` combinator which given a transformation from a resource to a String +`(a -> SiteM String)` will write the result of the transformation to the place +specified by the resource's url. + +## Loader/Writers + +Some things don't fit into the previous categories. For example `copyFiles` and +`copyFilesWith` are simple tools which just copy the specified files over as-is +into the output directory. You pass either of them a list of file globs and the +resulting files will be copied over. `copyFiles` sends them to the same +relative filepath from the source directory to the output directory, while +`copyFilesWith` allows you to transform the filepath to specify a new location +for each file. + +## Utilities + +Sitepipe includes a few utilities which simply make working with sites easier. +The included utilities will expand as time goes on. + +# Issues/Troubleshooting + +Feel free to file an [issue](https://github.com/chrispenner/sitepipe/issues) if you run into any trouble! diff --git a/app/Main.hs b/app/Main.hs deleted file mode 100644 --- a/app/Main.hs +++ /dev/null @@ -1,65 +0,0 @@ -{-# language OverloadedStrings #-} -{-# language DuplicateRecordFields #-} -module Main where - -import SitePipe -import qualified Data.Map as M -import Data.Text.Lens -import qualified Data.Text as T -import qualified Text.Mustache as MT -import qualified Text.Mustache.Types as MT - -main :: IO () -main = siteWithGlobals funcs $ do - posts <- fmap processPostTags <$> resourceLoader markdownReader ["posts/*.md"] - let tags = byTags posts - writeTemplate "templates/index.html" [mkIndexEnv posts tags] - writeTemplate "templates/base.html" (over (key "tags" . _Array . traverse) stripHTMLSuffix <$> posts) - writeTemplate "templates/tag.html" (stripPostsHTMLSuffix <$> tags) - staticAssets - -funcs :: MT.Value -funcs = MT.object - ["truncate" MT.~> MT.overText (T.take 30) - ] - -stripHTMLSuffix :: Value -> Value -stripHTMLSuffix obj = obj - & key "url" . _String . unpacked %~ setExt "" - -stripPostsHTMLSuffix :: Value -> Value -stripPostsHTMLSuffix tag = tag - & key "posts" . _Array . traversed . key "url" . _String . unpacked %~ setExt "" - -mkIndexEnv :: [Value] -> [Value] -> Value -mkIndexEnv posts tags = - object [ "posts" .= (stripHTMLSuffix <$> posts) - , "tags" .= (stripHTMLSuffix <$> tags) - , "url" .= ("/index.html" :: String) - ] - -staticAssets :: SiteM () -staticAssets = copyFiles - [ "css/*.css" - , "js/" - , "images/" - ] - -processPostTags :: Value -> Value -processPostTags post = post & key "tags" . _Array . traverse %~ mkSimpleTag - where - mkSimpleTag (String t) = makeTag (T.unpack t, []) - mkSimpleTag x = x - -byTags :: [Value] -> [Value] -byTags postList = makeTag <$> M.toList tagMap - where - tagMap = M.unionsWith mappend (fmap toMap postList) - toMap post = M.fromList (zip (post ^.. key "tags" . values . key "tag" . _String . unpacked) $ repeat [post]) - -makeTag :: (String, [Value]) -> Value -makeTag (tagname, posts) = object - [ "tag" .= tagname - , "url" .= ("/tag/" ++ tagname ++ ".html") - , "posts" .= posts - ] diff --git a/sitepipe.cabal b/sitepipe.cabal --- a/sitepipe.cabal +++ b/sitepipe.cabal @@ -1,7 +1,6 @@ name: sitepipe -version: 0.1.0 +version: 0.1.1 synopsis: A simple to understand static site generator -description: A simple to understand static site generator homepage: https://github.com/ChrisPenner/sitepipe#readme license: BSD3 license-file: LICENSE @@ -23,18 +22,16 @@ , SitePipe.Parse , SitePipe.Types , SitePipe.Utilities - build-depends: base >= 4.7 && < 5 + build-depends: base >= 4.9 && < 5 , optparse-applicative , unordered-containers - , containers , directory , filepath , megaparsec , mtl - , optparse-generic , pandoc , yaml - , mustache >= 2.2.2 + , mustache >= 2.2.3 , bytestring , text , parsec @@ -45,29 +42,8 @@ , aeson , shelly , MissingH - - default-language: Haskell2010 - -executable sitepipe-exe - hs-source-dirs: app - main-is: Main.hs - ghc-options: -threaded -rtsopts -with-rtsopts=-N - build-depends: base - , sitepipe - , lens , containers - , text - , unordered-containers - , mustache >= 2.2.3 - default-language: Haskell2010 -test-suite sitepipe-test - type: exitcode-stdio-1.0 - hs-source-dirs: test - main-is: Spec.hs - build-depends: base - , sitepipe - ghc-options: -threaded -rtsopts -with-rtsopts=-N default-language: Haskell2010 source-repository head diff --git a/src/SitePipe.hs b/src/SitePipe.hs --- a/src/SitePipe.hs +++ b/src/SitePipe.hs @@ -33,6 +33,7 @@ -- * Utilities , setExt , addPrefix + , getTags -- * Types , module SitePipe.Types diff --git a/src/SitePipe/Files.hs b/src/SitePipe/Files.hs --- a/src/SitePipe/Files.hs +++ b/src/SitePipe/Files.hs @@ -63,14 +63,20 @@ -- | Given a path to a mustache template file (relative to your source directory); -- this writes a list of resources to the output directory by applying each one to the template. -writeTemplate :: (ToJSON a) => FilePath -> [a] -> SiteM () +writeTemplate :: (ToJSON a) + => FilePath -- ^ Path to template (relative to site dir) + -> [a] -- ^ List of resources to write + -> SiteM () writeTemplate templatePath resources = do template <- loadTemplate templatePath writeWith (renderTemplate template) resources -- | Write a list of resources using the given processing function from a resource -- to a string. -writeWith :: (ToJSON a) => (a -> SiteM String) -> [a] -> SiteM () +writeWith :: (ToJSON a) + => (a -> SiteM String) -- ^ A function which renders a resource to a string. + -> [a] -- ^ List of resources to write + -> SiteM () writeWith resourceRenderer resources = traverse_ (writeOneWith resourceRenderer) resources @@ -85,7 +91,9 @@ liftIO $ writeFile outFile renderedContent -- | Writes the content of the given resources without using a template. -textWriter :: (ToJSON a) => [a] -> SiteM () +textWriter :: (ToJSON a) + => [a] -- ^ List of resources to write + -> SiteM () textWriter resources = writeWith (return . view (key "content" . _String . unpacked) . toJSON) resources @@ -116,7 +124,9 @@ -- this function finds all files matching any of the provided list -- of fileglobs (according to 'srcGlob') and returns a list of loaded resources -- as Aeson 'Value's. -resourceLoader :: (String -> IO String) -> [GlobPattern] -> SiteM [Value] +resourceLoader :: (String -> IO String) -- ^ A reader which processes file contents + -> [GlobPattern] -- ^ File glob; relative to the @site@ directory + -> SiteM [Value] -- ^ Returns a list of Aeson objects resourceLoader = resourceLoaderGen -- | A more generic version of 'resourceLoader' which returns any type with a diff --git a/src/SitePipe/Readers.hs b/src/SitePipe/Readers.hs --- a/src/SitePipe/Readers.hs +++ b/src/SitePipe/Readers.hs @@ -6,6 +6,9 @@ -- * Reader Generators , mkPandocReader + + -- * Pandoc Writers + , pandocToHTML ) where import Text.Pandoc @@ -15,8 +18,20 @@ -- makes a resource reader compatible with 'SitePipe.Files.resourceLoader'. -- -- > docs <- resourceLoader (mkPandocReader readDocX) ["docs/*.docx"] -mkPandocReader :: (ReaderOptions -> String -> (Either PandocError Pandoc)) -> String -> IO String -mkPandocReader pReader content = writeHtmlString def <$> runPandocReader (pReader def) content +mkPandocReader :: (ReaderOptions -> String -> Either PandocError Pandoc) -> String -> IO String +mkPandocReader pReader = mkPandocReaderWith pReader id pandocToHTML + +-- | Like `mkPandocReader`, but allows you to provide both a @'Pandoc' -> 'Pandoc'@ transformation, +-- which is great for things like relativizing links or running transforms over specific document elements. +-- See https://hackage.haskell.org/package/pandoc-lens for some useful tranformation helpers. You also specify +-- the tranformation from @Pandoc -> String@ which allows you to pick the output format of the reader. +-- If you're unsure what to use in this slot, the pandocToHTML function is a good choice. +mkPandocReaderWith :: (ReaderOptions -> String -> Either PandocError Pandoc) -> (Pandoc -> Pandoc) -> (Pandoc -> String) -> String -> IO String +mkPandocReaderWith pReader transformer writer content = writer . transformer <$> runPandocReader (pReader def) content + +-- | A simple helper which renders pandoc to HTML; good for use with 'mkPandocReaderWith' +pandocToHTML :: Pandoc -> String +pandocToHTML = writeHtmlString def -- | Runs the Pandoc reader handling errors. runPandocReader :: (MonadThrow m) => (String -> Either PandocError Pandoc) -> String -> m Pandoc diff --git a/src/SitePipe/Utilities.hs b/src/SitePipe/Utilities.hs --- a/src/SitePipe/Utilities.hs +++ b/src/SitePipe/Utilities.hs @@ -2,9 +2,15 @@ module SitePipe.Utilities ( addPrefix , setExt + , getTags ) where import System.FilePath.Posix +import Data.Aeson +import Data.Aeson.Lens +import qualified Data.Map as M +import qualified Data.Text as T +import Control.Lens hiding ((.=)) -- | Set the extension of a filepath or url to the given extension. -- Use @setExt ""@ to remove any extension. @@ -14,3 +20,26 @@ -- | Add a prefix to a filepath or url addPrefix :: String -> FilePath -> FilePath addPrefix = (++) + +-- | Given a function which creates a url from a tag name and a list of posts +-- (which have a tags property which is a list of strings) +-- this returns a list of tags which contain: +-- +-- * name: The tag name +-- * url: The tag's url +-- * posts: The list of posts matching that tag +getTags :: (String -> String) -- ^ Accept a tagname and create a url + -> [Value] -- ^ List of posts + -> [Value] +getTags makeUrl postList = uncurry (makeTag makeUrl) <$> M.toList tagMap + where + tagMap = M.unionsWith mappend (toMap <$> postList) + toMap post = M.fromList (zip (post ^.. key "tags" . values . _String . to T.unpack) $ repeat [post]) + +-- | Makes a single tag +makeTag :: (String -> String) -> String -> [Value] -> Value +makeTag makeUrl tagname posts = object + [ "tag" .= tagname + , "url" .= makeUrl tagname + , "posts" .= posts + ] diff --git a/test/Spec.hs b/test/Spec.hs deleted file mode 100644 --- a/test/Spec.hs +++ /dev/null @@ -1,2 +0,0 @@ -main :: IO () -main = putStrLn "Test suite not yet implemented"