lock_files.md

# Lock Files

Stack attempts to provide reproducible build plans. This involves reproducibly
getting the exact same contents of source packages and configuration options
(like Cabal flags and GHC options) for a given set of input files. There are a
few problems with making this work:

* Entering all of the information to fully provide reproducibility is tedious.
This would include things like Hackage revisions, hashes of remote tarballs,
etc. Users do not want to enter this information.

* Many operations in Stack rely upon a "snapshot hash," which transitively
includes the completed information for all of these dependencies. If any of
that information is missing when parsing the `stack.yaml` file or snapshot
files, it could be expensive for Stack to calculate it.

To address this, we follow the (fairly standard) approach of having a
_lock file_. The goal of the lock file is to cache completed locations of
project, snapshot packages and snapshots themselves so that:

* These files can be stored in source control
* Users on other machines can reuse these lock files and get identical build
plans given that the used project packages and local snapshots are the same on
those machines
* Rerunning `stack build` in the future is deterministic in the build plan, not
depending on mutable state in the world like Hackage revisions

!!! note

If, for example, a tarball available remotely is deleted or the hash
changes, it will not be possible for Stack to perform the build.
However, by deterministic, we mean it either performs the same build or
fails, never accidentally doing something different.

This document explains the contents of a lock file, how they are used, and how
they are created and updated.

## stack.yaml and snapshot files

Relevant to this discussion, Stack's project-level configuration file
(`stack.yaml`, by default) specifies:

* the parent snapshot (the [`snapshot`](../configure/yaml/project.md#snapshot)
key)
* extra-deps

Some of this information can be incomplete. Consider this `stack.yaml` file:

~~~yaml
snapshot: lts-24.43
extra-deps:
- acme-missiles-0.3
~~~

This information is _incomplete_. For example, the extra-deps may change in the
future. Instead, you could specify enough information in the `stack.yaml` file
to fully resolve that package. That looks like:

~~~yaml
extra-deps:
- hackage: acme-missiles-0.3@sha256:2ba66a092a32593880a87fb00f3213762d7bca65a687d45965778deb8694c5d1,613
pantry-tree:
size: 226
sha256: 614bc0cca76937507ea0a5ccc17a504c997ce458d7f2f9e43b15a10c8eaeb033
~~~

The `lts-24.43` information is also incomplete. While we assume in general that
Haskell LTS snapshots never change, there is nothing that prohibits that from
happening. Instead, the complete version of that key is:

~~~yaml
snapshots:
- url: https://raw.githubusercontent.com/commercialhaskell/stackage-snapshots/master/lts/24/43.yaml
size: 729011
sha256: 3c412a7c13dba6d3d808455a458e0776c58b6cf99b8a7961a2f5e55589d6f1d6
~~~

Users do not particularly feel like writing all of that. Therefore, it is common
to see _incomplete_ information in a `stack.yaml` file.

## Recursive snapshot layers

Snapshot files can be _recursive_, where `stack.yaml` refers to
`mySnapshotA.yaml`, which refers to `mySnapshotB.yaml`, which refers to a remote
snapshot file (available via an HTTP(S) URL).

We need to encode information from _all_ of these snapshot layers and the
`stack.yaml` file in the lock file, to ensure that we can detect if anything
changes.

## Performance

In addition to acting as a pure correctness mechanism, the design of a lock file
given here also works as a performance improvement. Instead of requiring that
all snapshot files be fully parsed on each Stack invocation, we can store
information in the lock file and bypass parsing of the additional files in the
common case of no changes.

## Lock file contents

The lock file contains the following information:

* Completed package locations for extra-deps and packages in snapshot files

!!! note

This only applies to _immutable_ packages. Mutable packages are not
included in the lock file.

* Completed information for the snapshot locations

It looks like the following:

~~~yaml
# This file was autogenerated by Stack.
# You should not edit this file by hand.
# For more information, please see the documentation at:
# https://docs.haskellstack.org/en/stable/topics/lock_files

packages:
# Listed in the order that they are encountered in the layers of snapshots,
# starting at the lowest layer:
- completed:
hackage: acme-missiles-0.3@sha256:2ba66a092a32593880a87fb00f3213762d7bca65a687d45965778deb8694c5d1,613
pantry-tree:
sha256: 614bc0cca76937507ea0a5ccc17a504c997ce458d7f2f9e43b15a10c8eaeb033
size: 226
original:
hackage: acme-missiles-0.3
snapshots:
- completed:
sha256: 3c412a7c13dba6d3d808455a458e0776c58b6cf99b8a7961a2f5e55589d6f1d6
size: 729011
url: https://raw.githubusercontent.com/commercialhaskell/stackage-snapshots/master/lts/24/43.yaml
original: lts-24.43
~~~

## Creation procedure

Whenever a project-level configuration file (`stack.yaml`, by default) is
loaded, Stack checks for a lock file in the same file path, with a `.lock`
extension added. For example, if you command:

~~~text
stack --stack-yaml my-stack.yaml build
~~~

~~~text
stack --stack-yaml my-stack.yaml build --dry-run
~~~

then Stack will use a lock file in the location `my-stack.yaml.lock`. For the
rest of this document, we will assume that the files are simply `stack.yaml` and
`stack.yaml.lock`.

If the lock file does not exist, subject to Stack's
[`--lock-file`](../configure/global_flags.md#-lock-file-option) option, it will
be created by:

* Loading the `stack.yaml`
* Loading all snapshot files
* Completing all missing information
* Writing out the new `stack.yaml.lock` file to the disk

## Update procedure

Whenever a project-level configuration file (`stack.yaml`, by default) is
loaded, all completed package or snapshot locations (even those completed using
information from a lock file) get collected to form a new lock file in memory.
Subject to Stack's
[`--lock-file`](../configure/global_flags.md#-lock-file-option) option, that new
lock file is compared against the one on disk and, if there are any differences,
written out to the disk.

## `stack config build-files` command

The
[`stack config build-files`](../commands/config_command.md#the-stack-config-build-files-command)
loads a project-level configuration file (see above) without taking any other
build steps (other than generating, when applicable, a Cabal file from a package
description in the Hpack format).