espial-0.0.39: README.md
# Espial
Espial is an open-source, web-based bookmarking server.
It supports multiple user accounts and is primarily intended for self-hosted deployments.
Bookmarks are stored in a SQLite database to keep setup and maintenance straightforward.
Espial also includes internationalization support.
### Adding Bookmarks
The easist way for logged-in users to add bookmarks, is with the "bookmarklet", found on the Settings page.
Espial also supports file import options.
## Demo Server
Log in with:
- username: `demo`
- password: `demo`
https://espdemo.ae8.org/u:demo
## Related Projects
Also, see the android app for adding bookmarks via an Android Share intent:
https://github.com/jonschoning/espial-share-android
## Installation
### Docker Setup (Recommended Method)
Docker installation is the recommended approach for most deployments.
See:
https://github.com/jonschoning/espial-docker
### Setup From Source
1. Install Haskell tooling (choose one):
- **Stack**: https://docs.haskellstack.org/en/stable/
- **GHCup** (installs GHC, Stack, and more): https://www.haskell.org/ghcup/install/
2. Build executables:
```bash
stack build
```
3. Create the database:
```bash
stack exec migration -- createdb
```
4. Create a user:
```bash
stack exec migration -- createuser --userName myusername --userPassword myuserpassword
```
5. Import a pinboard bookmark file for a user (optional):
```bash
stack exec migration -- importbookmarks --userName myusername --bookmarkFile sample-bookmarks.json
```
6. Import a firefox bookmark file for a user (optional):
```bash
stack exec migration -- importfirefoxbookmarks --userName myusername --bookmarkFile firefox-bookmarks.json
```
7. Start a production server:
```bash
stack exec espial
```
## Configuration
See `config/settings.yml` for changing default run-time parameters & environment variables.
- `config/settings.yml` is embedded into the app executable when compiled and also read once when the app starts. Current settings in `config/settings.yml` will override the embedded compile-time settings.
- `config/settings.yml` values formatted like `_env:ENV_VAR_NAME:default_value` can be overridden by the specified environment variable.
- Example:
- `_env:PORT:3000`
- environment variable `PORT`
- default app http port: `3000`
## Internationalization
Espial's frontend supports a selectable UI language per account, on the Account Settings (`settings`) page.
The Server Language default is controlled by `language-default` in `config/settings.yml`, which can also be set with environment variable `LANGUAGE_DEFAULT` to the language code; the default value is `en`.
#### Supported Languages:
| Code | English name | Native name |
| --------- | --------------------- | ------------------ |
| `en` | English | English |
| `de` | German | Deutsch |
| `es` | Spanish | Español |
| `fr` | French | Français |
| `it` | Italian | Italiano |
| `ja` | Japanese | 日本語 |
| `ko` | Korean | 한국어 |
| `pl` | Polish | Polski |
| `pt-BR` | Portuguese (Brazil) | Português (Brasil) |
| `ru` | Russian | Русский |
| `tr` | Turkish | Türkçe |
| `uk` | Ukrainian | Українська |
| `zh-Hans` | Chinese (Simplified) | 简体中文 |
| `zh-Hant` | Chinese (Traditional) | 繁體中文 |
## Request IP Logging
Espial supports the `IP_FROM_HEADER` environment variable for request logging.
- `IP_FROM_HEADER=true`: log the client IP from the `X-Real-IP` or `X-Forwarded-For` header when present, and fall back to the peer address if neither header is available.
- `IP_FROM_HEADER=false`: log the peer address from the HTTP connection.
Only set `IP_FROM_HEADER=true` if your application is safely positioned **behind a trusted reverse proxy**.
## TLS / Reverse Proxy
A reverse proxy is the recommended approach for production and most self-hosted deployments. For simple local or LAN setups where that is impractical, Espial can also terminate TLS directly — see [Optional: In-Process TLS](#optional-in-process-tls) below.
Set `SSL_ONLY=true` whenever Espial is served over HTTPS (via reverse proxy or in-process TLS) to enable the `Secure` cookie flag and HTTP→HTTPS redirects.
### Recommended: Reverse Proxy (Caddy, nginx, Cloudflare Tunnel, …)
Running Espial behind a reverse proxy is the recommended approach for production and most self-hosted deployments. The proxy terminates TLS and forwards plain HTTP to Espial. This gives you automatic certificate management, HTTP/2 and HTTP/3 at the edge, and cleaner separation of concerns.
For container-based deployment examples, including production-oriented layouts, see the `espial-docker` repository:
- https://github.com/jonschoning/espial-docker
Minimal [Caddy](https://github.com/caddyserver/caddy) example:
Localhost without a real domain:
```caddyfile
https://localhost:3050 {
reverse_proxy localhost:3000
}
```
or with a domain:
```caddyfile
espial.example.com {
reverse_proxy 127.0.0.1:3000
}
```
With the domain setup:
- Caddy terminates TLS for `espial.example.com`.
- Espial continues listening on HTTP, locally on `127.0.0.1:3000`
- If using Docker Compose, it would look like `espial:3000`
- Set `IP_FROM_HEADER=true` only when Espial is reachable solely through that trusted proxy.
If you are using Cloudflare:
- Prefer Cloudflare SSL mode `Full (strict)`.
- use `header_up X-Forwarded-For {http.request.header.CF-Connecting-IP}`
- If traffic can reach Espial directly without passing through your trusted proxy, do not enable `IP_FROM_HEADER=true`, because client IP headers can be spoofed.
#### Running on a Subpath
Espial can also be served under a path prefix on a shared domain, e.g. `https://www.domain.com/espial` alongside other apps on the same host. This needs two pieces working together:
1. **`APPROOT`** — tell Espial the full external URL (including the subpath) it's being served at, so generated links, redirects, and static asset URLs come out correct:
```yaml
approot: "_env:APPROOT:https://www.domain.com/espial"
```
or via environment variable:
```bash
APPROOT=https://www.domain.com/espial stack exec espial
```
2. **Reverse proxy** — strip the `/espial` prefix before forwarding to Espial, since Espial itself always routes as if mounted at `/`. Also redirect the bare `/espial` (no trailing slash) to `/espial/` so relative URLs in the page resolve against the right base.
Minimal Caddy example (see `caddy/Caddyfile-Standalone-Subpath`, which runs Caddy in Docker alongside an Espial instance on the host):
```caddyfile
:80 {
redir /espial /espial/
handle_path /espial/* {
# Caddy pools and reuses idle connections to the backend
# by default, but Espial's Warp server closes idle connections after 30s
# set keepalive to off or 15s
reverse_proxy host.docker.internal:3000 {
transport http {
keepalive off
}
}
}
}
```
`handle_path` strips the `/espial` prefix before proxying, matching what `APPROOT` told Espial to expect. If Caddy and Espial run on the same host (not in Docker), replace `host.docker.internal:3000` with `localhost:3000`.
### Optional: In-Process TLS
For simple local or LAN deployments where adding a reverse proxy is impractical, Espial can terminate TLS directly using your own certificate and key files (PEM format, unencrypted key).
Set the `TLS_CERT_FILE` and `TLS_KEY_FILE` environment variables (or the corresponding `tls-cert-file` / `tls-key-file` keys in `config/settings.yml`) to the paths of your certificate and private key:
```bash
TLS_CERT_FILE=/path/to/cert.pem TLS_KEY_FILE=/path/to/key.pem stack exec espial
```
Or in `config/settings.yml`:
```yaml
tls-cert-file: "/path/to/cert.pem"
tls-key-file: "/path/to/key.pem"
```
When both values are set Espial listens on HTTPS with HTTP/2 enabled. When either is absent Espial falls back to plain HTTP.
**Certificate rotation** — Espial reloads the certificate from disk automatically every 12 hours without restarting or dropping existing connections. To trigger an immediate reload send `SIGHUP` to the process:
```bash
kill -HUP <espial-pid>
```
**Notes:**
- Let's Encrypt certificates (`fullchain.pem` + `privkey.pem`) work directly.
- A self-signed certificate for local testing can be generated with:
```bash
openssl req -x509 -newkey rsa:2048 -nodes \
-keyout key.pem -out cert.pem -days 3650 \
-subj "/CN=localhost" \
-addext "subjectAltName=DNS:localhost,IP:127.0.0.1"
```
- A reverse proxy is still preferred for production: it provides HTTP/3, edge caching, and hides the Espial process from the public internet.
## Archive Backends
Espial supports configurable archive backends for saving bookmark snapshots.
Set the backend with `archive-backend` in `config/settings.yml`:
- `disabled`: archiving is turned off (default).
- `wayback-machine`: enables submission to the Internet Archive Wayback Machine.
Wayback Machine support requires the following settings:
- `wayback-machine-access-key`
- `wayback-machine-secret-key`
Create these by signing in to your Internet Archive account and generating S3-style API credentials at `https://archive.org/account/s3.php`. \
If `wayback-machine` is selected but the access key or secret key is missing, archiving is disabled at runtime.
- `archivebox07`: queues the URL in a local ArchiveBox 0.7 instance and stores an ArchiveBox link on the bookmark.
**IMPORTANT - ArchiveBox stores all archive data in a single global index space, so this arcive-backend is best suited to single-user Espial instances.**
**Recommended setup is to use Docker Compose to run the ArchiveBox instance**
- Simple example, running on localhost: [docker-compose.archivebox07.yml](docker-compose.archivebox07.yml)
- See https://github.com/jonschoning/espial-docker for more examples intended for deployment
- In all examples, you must change the `ARCHIVEBOX_PASSWORD` from it's default value.
ArchiveBox support requires the following settings:
- `archivebox-url`
`archivebox-url` is the URL espial uses to sign in to ArchiveBox and submit URLs through the web UI. In Docker Compose this is typically `http://archivebox:8000`.
- `archivebox-public-url` (optional)
Public ArchiveBox URL stored on bookmarks.
- `archivebox-username` plus `archivebox-password`
Espial signs in to the ArchiveBox web UI with these credentials before submitting URLs.
- `archivebox-tag` (optional)
A tag Espial adds to submissions (example: `espial`).
- `archivebox-plugins` (optional)
Comma-separated list of ArchiveBox methods (plugins) to request when submitting URLs, e.g. `title,favicon,singlefile,screenshot`.
Set the ArchiveBox admin credentials in the override path by supplying:
- `ARCHIVEBOX_USERNAME=...`
- `ARCHIVEBOX_PASSWORD=...`
The `Makefile` includes the following helpers:
- `docker-compose-up-archivebox07`
- `docker-compose-up-d-archivebox07`
- `docker-compose-exec-archivebox07`
Or start the instance manually via docker compose, example:
```bash
docker compose -f docker-compose.archivebox07.yml up
```
Configure the enrivonment variable `ARCHIVE_METHODS` to control which archive methods ArchiveBox uses:
```yaml
environment:
- ARCHIVE_METHODS=title,favicon,singlefile,screenshot
```
Available ARCHIVE_METHODS plugins:
- `archive_org`, `dom`, `favicon`, `git`, `headers`, `htmltotext`, `media`, `mercury`, `pdf`, `readability`, `screenshot`, `singlefile`, `title`, `wget`
If `ARCHIVE_METHODS` is unset/not-present, ArchiveBox will uses all plugins.
For additional information and configuration, refer to the [ArchiveBox repository](https://github.com/ArchiveBox/ArchiveBox)
Optional proxy settings for archive requests:
- `archive-socks-proxy-host`
- `archive-socks-proxy-port`
## CLI
Migration commands are run via:
```bash
stack exec migration -- <command> [options]
```
All commands take an optional `--conn` parameter for the database location; if omitted, the database location is loaded from `config/settings.yml` or environment variable `SQLITE_DATABASE`
### Commands
| Command | Example |
| ---------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `createdb` | `stack exec migration -- createdb` |
| `createuser` | `stack exec migration -- createuser --userName myusername --userPassword myuserpassword` |
| `createuser` (password file) | `stack exec migration -- createuser --userName myusername --userPasswordFile mypassword.txt` |
| `deleteuser` | `stack exec migration -- deleteuser --userName myusername` |
| `createapikey` | `stack exec migration -- createapikey --userName myusername` |
| `deleteapikey` | `stack exec migration -- deleteapikey --userName myusername` |
| `importbookmarks` | `stack exec migration -- importbookmarks --userName myusername --bookmarkFile sample-bookmarks.json` |
| `importfirefoxbookmarks` | `stack exec migration -- importfirefoxbookmarks --userName myusername --bookmarkFile firefox-bookmarks.json` |
| `importnetscapebookmarks` | `stack exec migration -- importnetscapebookmarks --userName myusername --bookmarkFile bookmarks.html` |
| `importnotes` | `stack exec migration -- importnotes --userName myusername --noteDirectory ./notes` |
| `exportbookmarks` | `stack exec migration -- exportbookmarks --userName myusername --bookmarkFile exported-bookmarks.json` |
| `exportnetscapebookmarks` | `stack exec migration -- exportnetscapebookmarks --userName myusername --bookmarkFile exported-bookmarks.html` |
| `printmigratedb` | `stack exec migration -- printmigratedb` |
| `showuser` | `stack exec migration -- showuser --userName myusername` |
### `importbookmarks` Command Notes:
See `sample-bookmarks.json`, which contains a JSON array, each line containing a `FileBookmark` object.
Example:
```json
[
{
"href": "http://raganwald.com/2018/02/23/forde.html",
"description": "Forde's Tenth Rule, or, \"How I Learned to Stop Worrying and \u2764\ufe0f the State Machine\"",
"extended": "",
"time": "2018-02-26T22:57:20Z",
"shared": "yes",
"toread": "yes",
"tags": "raganwald"
},
,
{
"href": "http://downloads.haskell.org/~ghc/latest/docs/html/users_guide/flags.html",
"description": "7.6. Flag reference \u2014 Glasgow Haskell Compiler 8.2.2 User's Guide",
"extended": "-fprint-expanded-synonyms",
"time": "2018-02-26T21:52:02Z",
"shared": "yes",
"toread": "no",
"tags": "ghc haskell"
}
]
```