stacks User Guide

stacks is a Terraform code pre-processor written in Python and distributed as a Click-based command-line tool. It assembles a "layer" of Terraform code on the fly from environment-specific variables, shared modules, Jinja templates, and RSA-encrypted secrets, then drives terraform against the rendered output. A small set of state-surgery commands round out the toolset for safely moving, importing, and editing resources after a layer has been rendered.

This guide walks end users — platform engineers and developers who deploy infrastructure with stacks — from a first install to day-to-day usage and recipe-level workflows. Architecture, code internals, and the full configuration reference live in the companion Administrator & Developer Guide.

1. Quick Start

Get a working stacks CLI on a workstation in under a minute, then verify it can drive Terraform against a layer.

1.1 What stacks is in 30 seconds

• A pre-processor for Terraform that renders a single "layer directory" into a self-contained Terraform working directory called stacks.out/.
• A CLI built on top of Click with the entry point stacks (Python module stacks.main:cli).
• Layer state lives in regular Terraform state — stacks shells out to your installed terraform binary, it does not reimplement Terraform.
• Inline secrets are encrypted with RSA + AES-GCM and embedded as ENC[...] strings inside *.tfvars so the same files are safe to commit.
• State-surgery commands (stacks surgery list/import/remove/move/rename/edit) let operators repair or reorganise resources without leaving the tool.

1.2 Prerequisites checklist

• [ ] Python 3 with pip available (the package is installable from the standard Python tooling).
• [ ] A terraform binary on PATH (or set STACKS_TERRAFORM_PATH to the absolute path of the binary you want).
• [ ] git available on PATH — required by stacks diff.
• [ ] bash available — stacks surgery edit invokes bash -c to redirect Terraform output.
• [ ] An editor for stacks surgery edit — defaults to vi if neither STACKS_EDITOR nor EDITOR is set.

1.3 Install (lowest-friction path)

The repository ships a pyproject.toml and is structured for pip-based installation. Install the CLI directly from a clone:

git clone https://github.com/cisco-open/stacks.git
cd stacks
pip install .

Verify the binary is on PATH:

stacks version

Stacks 1.x.x
Python 3.x.x

Tip

The repository's lockfile is uv.lock, which means contributors typically use uv for reproducible builds. End users who only want the CLI on their machine can install with plain pip — the pyproject.toml is the canonical metadata source.

1.4 First render

stacks render is the workhorse. It must be run from inside a layer directory (a directory of the form stacks/<stack>/layer/<env>[@<subenv>][_<instance>]):

cd stacks/my-stack/layer/dev
stacks render

stacks will:

1. Clear the output directory stacks.out/ while preserving any cached .terraform/ state.
2. Copy *.tf and *.tfvars.jinja files from the global, stack, base, and layer scopes into stacks.out/ with prefix-namespacing (common_*, stack_*, base_*, layer_*).
3. Render every *.tfvars.jinja and *.tf file with Jinja2 using merged variables from environments/<env>/env.tfvars, sub-environment directories, and any .tfvars found upstream in the stack tree.
4. Emit stacks.tf.json (the variable universe) and zzz.auto.tfvars.json (resolved variables) at the top of stacks.out/.
5. Run terraform -chdir=stacks.out init automatically when no .terraform/terraform.tfstate exists yet — controlled by --init (auto, always, never).

Important

stacks render, stacks diff, stacks terraform, and every stacks surgery subcommand assert at startup that you are in a valid layer directory — they walk four parents up to find the repository root and require environments/<env>/ to exist. Running from elsewhere raises an AssertionError.

1.5 What to try next

• Run stacks terraform plan to drive the wrapped Terraform binary against the rendered output (see Section 4.4).
• Encrypt your first secret with stacks encrypt (Section 5.3).
• Walk through the layer directory layout in Section 3.

2. Installation

stacks is distributed as a standard Python package. Three install paths are supported.

2.1 System requirements

2.2 Install methods

From source (canonical)

git clone https://github.com/cisco-open/stacks.git
cd stacks
pip install .

Editable / development install

git clone https://github.com/cisco-open/stacks.git
cd stacks
pip install -e .

uv (matches lockfile)

git clone https://github.com/cisco-open/stacks.git
cd stacks
uv sync
uv run stacks version

Warning

The published Python distribution registers the entry point as stacks. If your shell already has another stacks binary on PATH (some unrelated Python tools also use the name), prepend the venv's bin/ directory to PATH or invoke the tool as python -m stacks.main.

2.3 Runtime dependencies

The CLI imports the following third-party packages at runtime — they are declared in pyproject.toml and installed automatically by pip install .:

2.4 Verify the install

stacks --help

Usage: stacks [OPTIONS] COMMAND [ARGS]...

  Stacks, the Terraform code pre-processor.

  All commands MUST run within a layer directory, unless noted otherwise.

Commands:
  decrypt    Decrypt an encrypted string using a private key.
  diff       Render and compare Git HEAD vs current uncommitted changes.
  encrypt    Encrypt a secret string using a public key.
  render     Render a layer into working Terraform code.
  surgery    Terraform state surgery utilities.
  terraform  Terraform command wrapper.
  version    Show version information.

Note

Two commands are intentionally hidden from --help: stacks preinit (an internal stage of render, exposed only for advanced debugging) and stacks genkey (the key pair encrypts a fixed format, so generating a key pair is a one-time operation). Both still resolve when invoked explicitly.

2.5 Install troubleshooting

stacks: command not found after pip install .

The Python interpreter's bin/ (or Scripts/ on Windows) is not on PATH. Either activate the virtualenv that owns the install or invoke python -m stacks.main.

ModuleNotFoundError: No module named 'hcl2' at startup

A previous install was performed against a different Python interpreter. Re-run pip install . in the interpreter you actually use, or use a clean virtualenv.

terraform: command not found when running stacks render

stacks invokes the binary named in STACKS_TERRAFORM_PATH (default terraform). Either install Terraform globally or set the environment variable to an absolute path:

export STACKS_TERRAFORM_PATH=/usr/local/bin/terraform

3. Configuration

stacks itself is configured by a small set of environment variables — the heavy configuration lives in the layer / stack / environment directory tree that the tool reads during render.

3.1 Environment variables (user-facing)

All defaults are visible in src/stacks/cmd/config.py:

Important

STACKS_PRIVATE_KEY_PATH accepts a comma-separated list. stacks tries each key in order and keeps the first that successfully decrypts — this is how you rotate keys without re-encrypting every secret in one change.

3.2 Repository layout stacks expects

stacks walks four directory levels up from the current working directory to locate the repository root:

<repo-root>/
  environments/                      # STACKS_ENVIRONMENTS_DIR
    <env>/
      env.tfvars                     # Required if `<env>/` is used
      <subenv>/
        *.tfvars                     # Optional sub-environment overrides
  stacks/                            # STACKS_STACKS_DIR
    <stack>/
      base/                          # STACKS_BASE_DIR (modules + .tf)
        *.tf
        modules/...
      layer/                         # STACKS_LAYERS_DIR per the layer name
        <env>[@<subenv>][_<instance>]/
          *.tfvars.jinja             # The layer body
          stacks.out/                # Created by `stacks render`

3.3 Layer directory naming convention

Layer directory names encode the environment, optional sub-environment, and optional instance:

The Context object splits on _ once, then splits the head on @ once, so sub-environment and instance can co-exist freely.

3.4 Predefined Jinja variables

When stacks render evaluates *.tfvars.jinja and *.tf templates, it makes a fixed set of predefined variables available under var:

Note

All other variables are merged from the on-disk .tfvars files in ascending priority order: env → sub-env → stacks-wide → stack-wide → layer. The Admin Guide documents the full precedence chain in Section 3.

4. Core Usage

stacks exposes nine public commands plus a surgery subgroup. Every command is annotated below with its purpose, required directory context, and the Click-defined arguments and options.

4.1 stacks version

Prints the installed stacks version and the underlying Python version. Can be run from any directory.

stacks version

Stacks 1.x.x
Python 3.x.x

4.2 stacks render

Renders the current layer into stacks.out/ and (optionally) runs terraform init.

stacks render [--init auto|always|never]

Behaviour:

1. Calls the internal preinit stage — clears stacks.out/ (preserving .terraform, .terraformrc, .terraform.lock.hcl), copies in *.tf and *.tfvars.jinja from common/stack/base/layer with prefixed names, renders all Jinja templates with the merged variable set, and writes stacks.tf.json and zzz.auto.tfvars.json.
2. Runs terraform -chdir=stacks.out init according to --init.

Tip

zzz.auto.tfvars.json is named with three leading zs on purpose — it ensures the file has the highest precedence when Terraform applies auto-loaded variables. Don't rename it.

4.3 stacks diff

Renders the layer twice — once against git stash-ed working tree (the HEAD copy) and once against the live working tree — and runs diff -ur --color between the two output directories.

stacks diff

The temporary directories stacks.out.old/ and stacks.out.new/ are removed at the end.

Warning

stacks diff calls git stash and git stash pop against the repository the layer lives in. If git stash fails (e.g., the working tree contains staged changes the stash refuses to take), Terraform will be rendered against an inconsistent file set. Commit or stash known-good changes before running.

4.4 stacks terraform

Wraps the configured terraform binary, but first re-renders the layer:

stacks terraform [--init auto|always|never] -- <terraform-args>...

Examples:

stacks terraform plan
stacks terraform apply
stacks terraform output
stacks terraform fmt

The wrapper sets -chdir=<stacks.out> for you, so you do not need to navigate into the output directory manually.

4.5 stacks encrypt

Encrypts a string with a public key and prints the resulting ENC[...] ciphertext. Can be run from any directory.

stacks encrypt --public-key-path <PATH> <STRING>
stacks encrypt --public-key-path <PATH> --from-stdin < secret.txt

Output format:

ENC[<sym_key_b64>;<gcm_tag_b64>;<iv_b64>;<ciphertext_b64>]

The ciphertext can be embedded directly in *.tfvars files — stacks recognises the ENC[...] envelope on read and decrypts it before passing the value to Jinja or Terraform.

Caution

The CLI prints the ciphertext to stdout. Be deliberate about where you redirect it (a file, a clipboard, a CI variable) — copy/paste into shell history is a real disclosure path for the metadata even though the plaintext stays sealed.

4.6 stacks decrypt

Decrypts an ENC[...] string using a private key.

stacks decrypt --private-key-path <PATH> "ENC[...]"

Warning

Issue #14 (closed) reported stacks decrypt --private-key-path <...> raising AttributeError. If you hit that on an older build, upgrade to commit a1fadd8 or later.

4.7 stacks surgery list

Lists every Terraform-managed resource in the current layer's state by address. Wraps terraform state list after re-rendering the layer.

stacks surgery list

4.8 stacks surgery import

Imports an existing cloud resource into the current layer's state by id.

stacks surgery import <ADDRESS> <RESOURCE-ID>

Example:

stacks surgery import aws_instance.web i-0123456789abcdef0

4.9 stacks surgery remove

Removes a resource from the current layer's state without destroying the underlying infrastructure (delegates to terraform state rm).

stacks surgery remove <ADDRESS>

4.10 stacks surgery move

Moves a resource from the current layer's state into another layer's state, preserving its real-world identity. Internally re-renders the current layer, captures the resource id from terraform state show, imports it into the destination layer (a Context is created at <repo-root>/<TO-PATH>), then removes it from the source.

stacks surgery move <FROM-ADDRESS> <TO-ADDRESS> <TO-PATH>

4.11 stacks surgery rename

Renames a resource within the current layer's state. Implemented as a stacks surgery move whose destination path is the same layer.

stacks surgery rename <FROM-ADDRESS> <TO-ADDRESS>

4.12 stacks surgery edit

Pulls the current Terraform state, copies it to a temporary file, opens it with STACKS_EDITOR / EDITOR / vi, shows a colour diff against the original, and pushes the edited state back if you confirm.

stacks surgery edit

Caution

The state serial is not incremented automatically, on purpose — it protects non-experts from accidentally corrupting state. If you need a serial bump (e.g., to force remote backends to accept your push), edit the serial field manually inside vi.

4.13 Hidden commands

stacks genkey \
  --public-key-path keys/stacks.pub \
  --private-key-path keys/stacks.key

5. Common Tasks

Realistic workflows assembled from the commands above.

5.1 Stand up a new layer for a new environment

Goal: add a staging environment to an existing stack.

1. Create the env directory with at least one .tfvars file:

   mkdir -p environments/staging
   echo 'region = "us-west-2"' > environments/staging/env.tfvars
   

2. Create the layer directory and a minimal *.tfvars.jinja:

   mkdir -p stacks/<stack>/layer/staging
   cat > stacks/<stack>/layer/staging/main.tfvars.jinja <<'EOF'
   region = "{{ var.region }}"
   EOF
   

3. Render and inspect:

   cd stacks/<stack>/layer/staging
   stacks render
   stacks terraform plan
   

5.2 Promote a change from working tree to a real apply

Goal: preview every Terraform-visible side effect of an in-flight edit before running apply.

cd stacks/<stack>/layer/prod
stacks diff           # rendered HEAD vs rendered working tree
stacks terraform plan # Terraform-level plan against the rendered layer
stacks terraform apply

Tip

stacks diff shows changes to rendered Terraform — i.e. what the Jinja templates evaluated to — which catches variable-resolution bugs that terraform plan cannot.

5.3 Bootstrap encryption keys and store a secret

Goal: start using encrypted ENC[...] strings in .tfvars.

1. Generate a key pair (one-time, hidden command):

   stacks genkey \
     --public-key-path keys/stacks.pub \
     --private-key-path keys/stacks.key
   

2. Encrypt your first secret:

   stacks encrypt \
     --public-key-path keys/stacks.pub \
     'super-secret-value'
   

   ENC[NWE5...;ZGYz...;NjBh...;OWUz...]
   

3. Drop the result into a .tfvars:

   db_password = "ENC[NWE5...;ZGYz...;NjBh...;OWUz...]"
   

4. Make STACKS_PRIVATE_KEY_PATH point at the private key wherever stacks render runs (CI runners, operator workstations):

   export STACKS_PRIVATE_KEY_PATH=keys/stacks.key
   

Caution

Commit only the public key. Distribute the private key out-of-band and store it on disk only where renders happen.

5.4 Rotate the RSA key without re-encrypting every secret

STACKS_PRIVATE_KEY_PATH accepts a comma-separated list, and the decrypt helper tries each key in order. Use this to roll keys gradually:

export STACKS_PRIVATE_KEY_PATH=keys/stacks.key.new,keys/stacks.key.old

Re-encrypt secrets at your leisure with stacks encrypt --public-key-path keys/stacks.pub.new. Once every ENC[...] payload decrypts under the new key, drop the old one from the list.

5.5 Reorganise resources across layers

Goal: a resource lives in layer/dev but logically belongs in layer/dev_alice.

cd stacks/<stack>/layer/dev
stacks surgery move \
  aws_s3_bucket.user_data \
  aws_s3_bucket.user_data \
  stacks/<stack>/layer/dev_alice

The third argument is the destination layer path relative to the repository root, not relative to the current layer.

5.6 Adopt an existing cloud resource

Goal: import an EC2 instance that was created by hand.

cd stacks/<stack>/layer/prod
stacks surgery import aws_instance.bastion i-0123456789abcdef0
stacks terraform plan   # confirm there is no diff

Note

Always follow import with terraform plan. If the imported resource's Terraform configuration drifts from what cloud reports, the next apply will rewrite or destroy it.

5.7 Run an in-place state edit

Goal: fix a corrupted serial or remove an orphaned resource block by hand.

cd stacks/<stack>/layer/prod
stacks surgery edit

stacks pulls the current state, copies it to .terraform/new.tfstate, opens that file in your editor, prints a diff against the original on save, and asks Proceed with changes? [y/N] before pushing.

5.8 Clean rebuild after upstream changes

Goal: force a fresh terraform init after pinning a new provider version or pulling new modules.

cd stacks/<stack>/layer/prod
stacks render --init always

--init always re-runs terraform init even when local state already exists.

6. Troubleshooting

The patterns below are derived from the exact exception strings in the shipped source and from real GitHub issues filed against the project.

Error: please set the STACKS_PRIVATE_KEY_PATH environment variable

Cause: stacks encountered an ENC[...] payload while rendering or running decrypt, but no private key path is configured.

Solution:

1. Export the variable, pointing at a private key that owns the encrypted payloads:

   export STACKS_PRIVATE_KEY_PATH=keys/stacks.key
   

2. If you want decryption to be optional (e.g. in a CI lint job), call the underlying helpers.decrypt(..., must_decrypt=False) in code — the CLI itself does not expose a soft-decrypt flag.

Prevention: Set the variable in the same shell profile / CI secret store that holds the rest of your stack credentials.

Error: Failure to render <PATH>: <jinja2-error>

Cause: a Jinja2 template referenced a variable that was not defined in the merged variable set. stacks uses jinja2.StrictUndefined, so any unknown variable raises UndefinedError.

Solution:

1. Re-read the wrapped Jinja error after the colon — it names the missing variable.
2. Confirm the variable is declared in one of the precedence layers (environments/<env>/env.tfvars, <subenv>/*.tfvars, stacks/*.tfvars, stack/*.tfvars, layer-level *.tfvars).
3. Run stacks preinit to re-run only the rendering stage and inspect stacks.out/ without invoking Terraform.

Prevention: Validate every new template by running stacks render in the lowest environment first.

Error: Failure to render <PATH> at line <N>, in statement <STMT>: <jinja2-error>

Cause: Jinja2 raised a TemplateSyntaxError — the template body has malformed {% %} / {{ }} syntax.

Solution:

1. Open the file at the named line; fix the syntax.
2. Test in isolation with python -c "import jinja2; jinja2.Template(open('PATH').read())".

Prevention: Keep templates terse. The shipped helpers do not provide debug-time template linting — a CI step that calls stacks render against every layer is the practical guard.

Error: No input received from stdin.

Cause: stacks encrypt --from-stdin was invoked with no piped data.

Solution: Either pipe data in (echo 'secret' | stacks encrypt --from-stdin --public-key-path ...) or pass the secret as a positional argument.

Error: You must provide either a string argument or use --from-stdin.

Cause: stacks encrypt was invoked without any plaintext source.

Solution: Provide the string as a positional argument or set --from-stdin.

Error: AttributeError from stacks decrypt --private-key-path

Cause: Reported in #14 (closed) — affected older revisions of helpers/crypto.py.

Solution: Upgrade to commit a1fadd8 or later. The current code splits private_key_path correctly and iterates over each key.

Symptom: stacks render errors with AssertionError and no message

Cause: The current working directory is not a valid layer directory. The Context constructor walks four levels up (path.parent.parent.parent.parent) to find the repository root and asserts that environments/<env>/ exists, where <env> is parsed from the layer directory name.

Solution:

1. cd into a directory of the form <repo>/stacks/<stack>/layer/<env>[@<subenv>][_<instance>].
2. Confirm environments/<env>/ exists from the repo root.
3. Confirm any sub-environment directory (environments/<env>/<subenv>/) exists when the layer name contains @<subenv>.

Prevention: Wrap operator workflows in shell aliases that cd to the right layer before invoking stacks.

Symptom: stacks diff shows no differences but terraform plan does

Cause: stacks diff only sees changes that affect rendered Terraform. A pure variable-file change that flows through unchanged Jinja produces identical rendered output but Terraform will still pick up new variable values from zzz.auto.tfvars.json.

Solution: Trust terraform plan for Terraform-level diffs. Use stacks diff to confirm template changes evaluate as expected.

Symptom: stacks surgery edit opens the wrong editor

Cause: Editor resolution order is STACKS_EDITOR → EDITOR → vi.

Solution: Set STACKS_EDITOR explicitly:

export STACKS_EDITOR=nano
stacks surgery edit

Symptom: stacks render succeeds but terraform init fails to fetch modules

Cause: stacks render does not pre-fetch modules — it copies the base/ modules directly under stacks.out/ and relies on terraform init for everything else. Network or registry credentials missing from the shell will surface as init errors.

Solution: Configure your ~/.terraformrc and credential helpers exactly as a non-stacks Terraform project would.

Tip

When in doubt, cd stacks.out && terraform init -upgrade reproduces the exact init the wrapper would run, but lets you append flags interactively.

7. FAQ

Q: What problem is stacks solving over plain Terraform plus tfvars?

stacks lets you keep variable definitions, environment-specific values, shared modules, and templated .tf files in one Git repository, then materialise a per-environment Terraform working directory on demand. The key value-adds are Jinja templating across both *.tfvars.jinja and *.tf files, inline ENC[...] secret encryption, and a surgery group of state-management commands.

Q: Where do my Terraform plan/apply cycles run?

Inside stacks.out/ (controlled by STACKS_OUTPUT_DIR), which stacks render rebuilds before each invocation. You generally never need to cd into it — stacks terraform and stacks surgery set -chdir=stacks.out for you.

Q: Does stacks ship its own Terraform binary?

No. It shells out to whatever STACKS_TERRAFORM_PATH points to (default: terraform on PATH). Use this to pin Terraform versions per-stack.

Q: How do I share encrypted secrets between teams?

Distribute the public key (stacks.pub) widely; restrict the private key (stacks.key) to operators who need to render. Anyone can encrypt new payloads against the public key without ever holding the private key.

Q: Can I rotate the RSA key without re-encrypting every secret in one shot?

Yes — set STACKS_PRIVATE_KEY_PATH to a comma-separated list of paths. stacks tries each key in order and uses the first one that successfully decrypts. Re-encrypt secrets piecewise, then drop the old key from the list.

Q: Why is stacks preinit hidden?

Because it executes only the rendering and variable-resolution step of render, with no terraform init. Useful for debugging template logic, but not for normal operation.

Q: Why is stacks genkey hidden?

Because the key pair format is fixed (RSA-2048 with public exponent 2**16 + 1, PEM-encoded, PKCS#8 unencrypted private) and the operation is done once per project.

Q: What state backend does stacks use?

Whatever you configure inside the layer. stacks does not impose a state backend — it only renders the Terraform code and runs terraform init. Configure remote state via the usual terraform { backend "..." } block.

Q: Does stacks surgery edit increment the state serial automatically?

No. By design — to protect non-experts from accidentally corrupting remote state. Bump serial manually inside the editor when you need to.

Q: What is the project's status?

Issue #8 (closed) tracked an inquiry about project status. The codebase is concise (24 source files, ~30k characters of code at commit a1fadd8) and currently lives under the cisco-open organisation. See the Admin Guide for governance notes.

Q: How does stacks handle nested encrypted values inside lists or maps?

The helpers.decrypt function recurses into lists and dicts, so any ENC[...] string at any depth in a parsed .tfvars value is decrypted. Plain strings without the ENC[ prefix are returned untouched.