diff --git a/LICENSE b/LICENSE new file mode 100644 --- /dev/null +++ b/LICENSE @@ -0,0 +1,20 @@ +The MIT License (MIT) +Copyright (c) 2014 Pixbi + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +of the Software, and to permit persons to whom the Software is furnished to do +so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 --- /dev/null +++ b/README.md @@ -0,0 +1,318 @@ +# duplo + +A opinionated, framework-less build tool for web applications + + +## Installation + +1. Get a [Mac](http://www.apple.com/mac/) +2. Install [Homebrew](http://brew.sh/) +3. Get [npm](http://npmjs.org/): `brew install npm` +4. Get [GMP](https://gmplib.org/): `brew install gmp` +5. Get pre-1.x.x [Component](https://github.com/componentjs): `npm install -g + component@0.19.9` +6. Get [duplo](https://github.com/pixbi/duplo): `npm install -g duplo` + + +## Usage + +* `duplo help` displays all commands. +* `duplo info` displays the version for this duplo installation. +* `duplo init ` scaffolds a new duplo repo in the current + directory. +* `duplo build` builds the project. `DUPLO_ENV` defaults to `dev`. +* `duplo dev`: starts a webserver, watches for file changes, and builds in + development environment. +* `duplo test`: build test cases and you should run it in browser by yourself. +* `duplo production`: like `duplo dev` but builds in production environment +* `duplo patch` bumps the patch version. +* `duplo minor` bumps the minor version. +* `duplo major` bumps the major version. + + +## Guiding Principle + +This is a build tool, not an application framework. It simply compiles and +builds your codebase for you and does not inject a runtime into or impose a +structure on your application. + +However, it does have opinions, specifically: + +* [Jade](http://jade-lang.com/) over HTML +* [Stylus](http://learnboost.github.io/stylus/) over CSS +* [GitHub](http://github.com/) and by extension [git](http://git-scm.com/) for + source code management +* [Heroku](https://www.heroku.com/) for application deployment +* [Selenium](http://docs.seleniumhq.org/) for automated browser testing +* [CircleCI](https://circleci.com/) for continuous integration + +The idea is to manage and deploy your code exclusively with git and have +CircleCI deals with deployment for you. However, duplo is a build tool; it +doesn't care about the exact structure of your application. This means that all +scripts are dumped into one single file, and so are the stylesheets and the +markup. + + +## File Structure + + app/ --> Application code + app/index.jade --> Entry point for markups. Only this file is compiled. + Use Jade's include system to pull in other markups. + app/index.js --> Application entry point. Only the top-level + application's `index.js` is included and run. Its + dependencies' are ignored. + app/assets/ --> Asset files are copied as-is to build's top-level + directory + app/styl/ --> It contains "special" stylesheets that get loaded + before any other stylesheets. + app/modules/ --> All other application code not listed above must be + placed here. All files at the `app/` level are not + included by default. + components/ --> Other repos imported via Component.IO + component.json --> The Component.IO manifest + dev/ --> Files here are included only when building in + development mode. + dev/assets/ --> Copied as-is just like `app/assets/`. Files here would + replace those with the same name under `app/assets/`. + dev/modules/ --> Works just like `app/modules/`. + public/ --> Built files when developing. Not committed to source + test/ --> Test files go here + + +## Development + +During development, everything in the `dev/assets/` directory is copied over +as-is *at the end* of the build process. This means that files in the directory +would replace whatever that has been built (or copied over from `app/assets/`) +at their respective locations. + +Anything under `dev/modules/` would be treated just like those under +`app/modules/`, that they would be concatenated/compiled into the respective +output files (i.e. `index.html`, `index.css`, or `index.js`). + + +## Environment + +duplo injects the `DUPLO_ENV` global variable with the value from the +environment variable of the same name when building. There is no default value. + + +## Entry Point + +Every application has a main entry point. In a duplo application, it is +`app/index.js`. Each repo may contain its own `app/index.js` but only the repo +on which duplo is run does duplo execute `app/index.js`. Note that +`app/index.js` is excluded when duplo commits via Component.IO so that the +consuming application does not see library index files when building the +project. + +Note that duplo only inspects the *top-level* `define()`. If you use +`require()`, your program may not execute as duplo is not aware of anything +other `define()` declarations. The proper way to declare an entry point in +`app/index.js` is: + +```js +define('main', [/* ... dependencies ... */], function (/* ... dependencies ... */) { + // Code here ... +}); +``` + + +## Application Parameterization + +If you need some build-time customization of the app, such as customizing each +build with a JSON object of unique IDs and metadata, you can pass any string +as the environment variable `DUPLO_IN`. The string is then turned into a +JavaScript string and stored into a global variable. + +To avoid special characters, `DUPLO_IN` must be base-64 encoded. + +For example, say you need to pass in a random ID for each build, you would +invoke: + +```sh +// Content decoded as: `{"id":"someId"}` +$ env DUPLO_IN="eyJpZCI6InNvbWVJZCJ9" duplo +``` + +Then in `app/index.js`: + +```js +var someId = DUPLO_IN.id; +``` + +Note that all newline characters are removed before the string is wrapped into +a JavaScript string. + + +## JavaScript Concatenation Order + +JavaScript files are not concatenated in any particular order. You must wrap +code inside an AMD module and declaring its dependencies. For code that needs +to be executed at initialization, utilize the environment's initialization +event such as `document.addEventListener("DOMContentLoaded")` to bootstrap the +rest of the script. + + +## CSS/Stylus Concatenation Order + +Unlike script files, where you place your CSS files within `app/` is +significant. Stylus files will be concatenated in this order: + + app/styl/variables.styl --> An optional variable file that gets injected + into every Stylus file + app/styl/keyframes.styl --> Keyframes + app/styl/fonts.styl --> Font declarations + app/styl/reset.styl --> Resetting existing CSS in the target + environment + app/styl/main.styl --> Application CSS that goes before any module + CSS + app/**/*.styl --> All other CSS files + +There is no particular concatenation order between different dependencies. + + +## HTML/Jade Concatenation Order + +Jade files are concatenated in no particular order as the Jade include system +is used for explicit ordering. + + +## Automatic rewriting for Jade + +duplo does not and cannot peek into Jade's include system. However, it does +automatically expand paths in include statements to make the inclusion process +easier. Take this example: + +```jade +include index.jade +include menu/index.jade +include pixbi-helper/index.jade +``` + +In the absence of a Component repo string (i.e. `-`), the path is +assumed to be pointing to a file under the `modules` directory in the current +repo. With a component repo string, it is assumed to also be pointing to a file +under the `modules` directory, but in the corresponding component's repo. + +The above is effectively rewritten into these paths, relative to the top-level +repo's directory. + +```jade +include app/modules/index.jade +include app/modules/menu/index.jade +include components/pixbi-helper/app/modules/index.jade +``` + + +## A note on the `modules` directory + +By now, it should be obvious that there are really two "modes" for any duplo +repo: an application mode and a library mode. In application mode, duplo acts +as the top-level program, including other duplo repos via Component as +libraries. In this scenario, `app/index.js` and `app/index.jade` are included +into the build. Contrast this to the library mode, where only those in the +"second" level (e.g. `modules/`, `assets/`, `styl/`) are included into the +build. + + +## Component Versions + +Each component's version is recorded in the `DUPLO_VERSIONS` global variable, +in the form similar to: + +```json +{ + "pixbi-main": "4.1.9", + "pixbi-launcher": "0.1.4" +} +``` + + +## Dependency Selection + +Some cases require the repo to be polymorphic in the sense that we could +generate different forms of the same codebase. For example, you may need to +build the repo in an embeddable form which would exclude certain dependencies +that are required in its standalone form. + +In this case you would include a `modes` attribute in the `component.json` +manifest file. The attribute would contain an `embeddable` and a `standalone` +attributes, each of which would then contain an array of dependencies as +specified in the `dependencies` attribute to include. + +Running duplo with the environment variable `DUPLO_MODE` set to `embeddable` +would build with the dependencies specified under `embeddable` while setting +`MODE` to `standalone` would do the same with those specified under the +`standalone` attribute. Otherwise duplo would just build with all dependencies. + +Note that dependency selection applies at the dependency level but not at the +file level within the components. + +Also note that duplo caches between builds. When you switch dependency +selection, remember to `duplo clean` your repo first. + +Putting it all together, an example of a `component.json`: + +```json +{ + "dependencies": { + "pixbi/sdk": "1.1.1", + "pixbi/embeddable": "2.2.2", + "pixbi/standalone": "3.3.3" + }, + "modes": { + "embeddable": [ + "pixbi/standalone" + ], + "standalone": [ + "pixbi/embeddable" + ] + } +} +``` + + +## Duplo Log + +Note that tasks are run in parallel so the display log may look scrambled from +line to line. This is normal. + + +## Testing + +Follow these steps: + +1. Create the directory `test` in your project root directory, alongside with + `dev` and `app`. +2. Create your unit test cases in the directory `test/modules`. +3. Put the following in the file `test/modules/index.jade`: + +```jade +html + head + title Testing Result + link(rel="stylesheet", href="vender/mocha.css") + script(src="vender/mocha.js"). + script. + mocha.setup('bdd'); + + body + div#mocha + p + a(href=".") Index + div#messages + div#fixtures + + script(src="./index.js"). + script. + mocha.run() +``` + +Then run `duplo test`. Open `public/index.html` in your browser for the test +results. + +## Copyright and License + +Code and documentation copyright 2014 Pixbi. Code released under the MIT +license. Docs released under Creative Commons. diff --git a/Setup.hs b/Setup.hs new file mode 100644 --- /dev/null +++ b/Setup.hs @@ -0,0 +1,2 @@ +import Distribution.Simple +main = defaultMain diff --git a/duplo.cabal b/duplo.cabal new file mode 100644 --- /dev/null +++ b/duplo.cabal @@ -0,0 +1,52 @@ +name: duplo +version: 1.6.0 +synopsis: Frontend development build tool +description: Intuitive, simple building blocks for building composable, completely self-managed web applications +license: MIT +license-file: LICENSE +author: Kenneth Kan +maintainer: ken@pixbi.com +category: Web +build-type: Simple +extra-source-files: README.md +cabal-version: >=1.10 + +source-repository head + type: git + location: https://github.com/pixbi/duplo.git + +executable duplo + main-is : Duplo.hs + build-depends : base == 4.7.* + , MissingH == 1.3.* + , aeson == 0.8.* + , aeson-pretty == 0.7.* + , base64-bytestring == 1.0.* + , bytestring == 0.10.* + , containers == 0.5.* + , directory == 1.2.* + , executable-path == 0.0.* + , filepath == 1.3.* + , filepather == 0.3.* + , fsnotify == 0.1.* + , http-types == 0.8.* + , language-javascript == 0.5.* + , lens == 4.5.* + , mtl == 2.2.* + , process == 1.2.* + , regex-compat == 0.95.* + , scotty == 0.9.* + , shake == 0.14.* + , system-fileio == 0.3.* + , system-filepath == 0.4.* + , text == 1.1.* + , transformers == 0.4.* + , unordered-containers == 0.2.* + , utf8-string == 0.3.* + , wai == 3.0.* + , warp == 3.0.* + , ansi-terminal == 0.5.* + , text-format == 0.3.* + hs-source-dirs : src/ + default-language : Haskell2010 + default-extensions : OverloadedStrings diff --git a/src/Duplo.hs b/src/Duplo.hs new file mode 100644 --- /dev/null +++ b/src/Duplo.hs @@ -0,0 +1,215 @@ +{-# LANGUAGE ScopedTypeVariables #-} + +import Control.Applicative ((<$>)) +import Control.Exception (catch, handle, throw, Exception) +import Control.Lens.Operators +import Control.Monad (void, when, unless) +import Control.Monad.Trans.Maybe (MaybeT(..), runMaybeT) +import Data.ByteString.Base64 (decode) +import Data.ByteString.Char8 (pack, unpack) +import Data.Maybe (fromMaybe) +import Data.String.Utils (replace) +import Development.Duplo.Server (serve) +import Development.Duplo.Shake (shakeMain) +import Development.Duplo.Utilities (logStatus, errorPrintSetter) +import Development.Duplo.Watcher (watch) +import Development.Shake (cmd, ShakeException(..)) +import Development.Shake.FilePath (()) +import GHC.Conc (forkIO) +import System.Console.GetOpt (getOpt, OptDescr(..), ArgDescr(..), ArgOrder(..)) +import System.Directory (getCurrentDirectory, createDirectoryIfMissing) +import System.Environment (lookupEnv, getArgs, getExecutablePath) +import System.FilePath.Posix (takeDirectory) +import System.Process (proc, createProcess, waitForProcess) +import qualified Control.Lens +import qualified Development.Duplo.Component as CM +import qualified Development.Duplo.Types.AppInfo as AI +import qualified Development.Duplo.Types.Builder as TB +import qualified Development.Duplo.Types.Config as TC +import qualified Development.Duplo.Types.Options as OP +import qualified Filesystem.Path +import qualified GHC.IO + +main = do + -- Command-line arguments + args <- getArgs + + let (cmdName:cmdArgs) = + if (length args > 0) + -- Normal case (with command name) + then args + -- Edge cases (with nothing provided) + else ("":[]) + + -- Deal with options + let (actions, nonOptions, errors) = getOpt Permute OP.options args + options <- foldl (>>=) (return OP.defaultOptions) actions + + -- Port for running dev server + port <- fmap read $ fromMaybe "8888" <$> lookupEnv "PORT" + -- Environment - e.g. dev, staging, production + duploEnvMB <- lookupEnv "DUPLO_ENV" + -- Build mode, for dependency selection + duploMode <- fromMaybe "" <$> lookupEnv "DUPLO_MODE" + -- Application parameter + duploIn <- fromMaybe "" <$> lookupEnv "DUPLO_IN" + -- Current working directory + cwd <- getCurrentDirectory + -- Duplo directory, assuming this is a build cabal executable (i.e. + -- `./dist/build/duplo/duplo`) + duploPath <- fmap (( "../../../") . takeDirectory) getExecutablePath + + -- Base64 decode + let duploInDecoded = case (decode $ pack $ duploIn) of + Left _ -> "" + Right input -> unpack input + + -- Paths to various relevant directories + let nodeModulesPath = duploPath "node_modules/.bin/" + let utilPath = duploPath "util/" + let distPath = duploPath "dist/build/" + let miscPath = duploPath "etc/" + let defaultsPath = miscPath "static/" + let appPath = cwd "app/" + let devPath = cwd "dev/" + let testPath = cwd "test/" + let assetsPath = appPath "assets/" + let depsPath = cwd "components/" + let targetPath = cwd "public/" + + -- Extract environment + let duploEnv' = case cmdName of + -- `build` is a special case. It takes `production` as the + -- default. + "build" -> maybe "production" id duploEnvMB + -- By default, `dev` is the default. + _ -> maybe "dev" id duploEnvMB + + -- Internal command translation + let (cmdNameTranslated, bumpLevel, duploEnv, toWatch) = + case cmdName of + "info" -> ("version", "", duploEnv', False) + "ver" -> ("version", "", duploEnv', False) + "new" -> ("init", "", duploEnv', False) + "bump" -> ("bump", "patch", duploEnv', False) + "release" -> ("bump", "patch", duploEnv', False) + "patch" -> ("bump", "patch", duploEnv', False) + "minor" -> ("bump", "minor", duploEnv', False) + "major" -> ("bump", "major", duploEnv', False) + "dev" -> ("build", "", "dev", True) + "live" -> ("build", "", "production", True) + "production" -> ("build", "", "production", True) + "build" -> ("build", "", duploEnv', False) + "test" -> ("build", "", "test", False) + _ -> (cmdName, "", duploEnv', False) + + -- Certain flags turn into commands. + let cmdNameWithFlags = if (OP.optVersion options) + then "version" + else cmdNameTranslated + + -- Display version either via command or option. We need to do this + -- before any `readManifest` as it throws an error when there isn't one, + -- as it should. + when (cmdNameWithFlags == "version") $ do + let command = utilPath "display-version.sh" + let process = createProcess $ proc command [duploPath] + + -- Run the command. + (_, _, _, handle) <- process + -- Remember to do it synchronously. + void $ waitForProcess handle + + -- We only care about exceptions thrown by the builder. + let ignoreManifestError' (e :: TB.BuilderException) = case e of + -- Only when missing manifest + TB.MissingManifestException _ -> return "" + -- Re-throw other builder exceptions. + _ -> throw e + -- Helper function to ignore exceptions, only for this stage, before + -- Shake is run. + let ignoreManifestError io = catch io ignoreManifestError' + + -- Gather information about this project + appName <- ignoreManifestError $ fmap AI.name CM.readManifest + appVersion <- ignoreManifestError $ fmap AI.version CM.readManifest + appId <- ignoreManifestError $ fmap CM.appId CM.readManifest + + -- We may need custom builds with mode + let depManifestPath = cwd "component.json" + dependencies <- CM.getDependencies $ case duploMode of + "" -> Nothing + a -> Just a + let depIds = fmap (replace "/" "-") dependencies + + -- Display additional information when verbose. + when (OP.optVerbose options) $ + -- More info about where we are + putStr $ "\n" + ++ ">> Current Directory\n" + ++ "Application name : " + ++ appName ++ "\n" + ++ "Application version : " + ++ appVersion ++ "\n" + ++ "Component.IO repo ID : " + ++ appId ++ "\n" + ++ "Current working directory : " + ++ cwd ++ "\n" + ++ "duplo is installed at : " + ++ duploPath ++ "\n" + -- Report back what's given for confirmation. + ++ "\n" + ++ ">> Environment Variables\n" + ++ "DUPLO_ENV (runtime environment) : " + ++ duploEnv ++ "\n" + ++ "DUPLO_MODE (build mode) : " + ++ duploMode ++ "\n" + ++ "DUPLO_IN (app parameters) : " + ++ duploInDecoded ++ "\n" + + -- Construct environment + let buildConfig = TC.BuildConfig { TC._appName = appName + , TC._appVersion = appVersion + , TC._appId = appId + , TC._cwd = cwd + , TC._duploPath = duploPath + , TC._env = duploEnv + , TC._mode = duploMode + , TC._nodejsPath = nodeModulesPath + , TC._dist = distPath + , TC._input = duploIn + , TC._utilPath = utilPath + , TC._miscPath = miscPath + , TC._defaultsPath = defaultsPath + , TC._appPath = appPath + , TC._devPath = devPath + , TC._testPath = testPath + , TC._assetsPath = assetsPath + , TC._depsPath = depsPath + , TC._targetPath = targetPath + , TC._bumpLevel = bumpLevel + , TC._port = port + , TC._dependencies = depIds + } + + -- Construct the Shake command + let shake' = shakeMain cmdNameWithFlags cmdArgs buildConfig options + let shake = shake' `catch` handleExc + + -- Watch or just build + unless toWatch $ shake + when toWatch $ do + -- Start a local server + _ <- forkIO $ serve port + + let targetDirs = [devPath, appPath, depsPath] + -- Make sure we have these directories to watch + mapM_ (createDirectoryIfMissing True) targetDirs + -- Watch for file changes + watch shake targetDirs + +-- | Handle all errors +handleExc (e :: ShakeException) = do + putStr $ show e + logStatus errorPrintSetter "Build failed" + putStrLn ""