Secrets on servers without files on disk

The simplest way to give a server its secrets is a file. Copy a .env to /opt/app/.env, point the service at it, done. It is also one of the hardest setups to keep clean. The file survives every deploy, rides along into backups and disk snapshots, gets baked into images by accident, and is readable by anything that can read that path. Nobody remembers to update it everywhere when a value changes, and nobody knows which servers still have the old one.

This post walks through the usual options, what each one leaks, and how an agent that holds secrets in memory with a time-limited lease changes the picture. It also says plainly what an agent cannot protect, because the last mile of any secret is a running process that needs the value.

The usual options and what they leak

A file on disk. Easy to reason about and easy to forget. Its main risks are persistence and spread: snapshots, backups, a careless scp, a container image built from the wrong directory. File permissions limit who can read it on the live machine, but they do not follow the copies.

Environment variables set by the service manager. systemd's Environment= and EnvironmentFile= are popular. The systemd manual itself says environment variables are not suitable for passing secrets: variables set for a unit are exposed to unprivileged clients over D-Bus, and they are passed down the process tree, including across security boundaries. It recommends its credentials mechanism instead (systemd.exec(5)).

Environment variables in containers. Docker's documentation on Compose secrets makes a similar point: environment variables are often available to all processes, access is hard to track, and they can end up in logs when debugging (Docker Docs).

Any environment variable, anywhere. On Linux, a process's starting environment is visible in /proc/<pid>/environ to anyone who passes the kernel's ptrace read check, which in practice means the same user or root (proc_pid_environ(5)).

None of this means environment variables are wrong. Most applications read configuration that way, and rewriting them is rarely worth it. It means the question is where the values come from, how long they live, and how many copies exist.

The agent model

Crunu's answer for servers is the crunu agent running as a systemd service on each machine. The idea is to replace a long-lived file with a short-lived, revocable grant.

1. The machine is enrolled once. An admin creates the machine in the web app and gets a single-use enrollment code, valid for 15 minutes by default and at most 24 hours, optionally tied to the machine's addresses. On the server, crunu enroll reads the code from standard input, never from a command-line argument, generates the machine's own keys and prints a fingerprint. The admin approves the machine only after checking that the fingerprint matches.
2. The agent fetches and verifies. The service polls the server once a minute. It checks signatures on everything it receives, decrypts the environments granted to this machine and keeps the result in memory.
3. Programs ask the agent. The agent serves environments over a local socket that only root and members of the crunu group can use. Your application starts through crunu run, which asks the agent for the variables.
4. A lease keeps it honest. The agent renews its lease every five minutes. If the server becomes unreachable, the agent keeps serving the last verified state until the lease ends, 24 hours by default, and then stops.

The enrollment step looks like this on the server, with the code pasted from the web app:

printf '%s\n' "$CODE" | sudo crunu enroll --server https://vault.company.example -
sudo systemctl enable --now crunu-agent
sudo crunu status

The web app's New machine dialog shows the full install-and-enroll command, including checksums for the installer and binary. If the same code is ever presented a second time, the machine is flagged as a possible interception and an alert is raised.

Running the application

With the agent in place, the application's unit file contains no values at all:

# /etc/systemd/system/payments-api.service
[Service]
User=payments
ExecStart=/usr/local/bin/crunu run --env payments-api/production --restart on-change -- /opt/payments/bin/server

Add the service user to the agent's group once with usermod -aG crunu payments.

crunu run starts the program with a clean environment: a short allowlist (PATH, HOME, USER, LANG, TZ and a few others) plus the variables of the named environment. It forwards signals and returns the program's exit code. Its own credential variables are never passed to the child. With --restart on-change, when a value changes in Crunu the agent notices, waits two seconds for further changes, stops the program gracefully and starts it again with the new variables. The journal records only the names of the changed variables.

Compare that with the file approach. There is no .env on disk, no value in the unit file, nothing for a snapshot to capture, and rotating a database password is a save in the web app rather than an SSH session to every server.

Two ways to deliver to a machine

When you grant a machine an environment, you choose how it is delivered:

• Environment key. The machine receives the environment key and always sees the latest version of all its files. Revoking it means rotating the environment key.
• Selected files. The machine receives a sealed copy of only the files you pick, refreshed whenever they are saved. It can be revoked without rotating the key.

Selected files suit machines that need one configuration file out of a large environment, for example a worker that only needs the queue credentials.

When a program really needs a file

Some software insists on reading a file: a TLS key, a cloud credentials file, a tool with no environment variable support. For those, crunu pull writes the environment's files to their paths:

sudo crunu pull --env payments-api/production --env-out /etc/payments/env

It writes with permissions 0600, in two phases so a half-written file is never visible, keeps the previous version as .prev, does not follow symbolic links, and refuses to write into dangerous places such as .bashrc, .ssh/ or .config/systemd/. The --env-out option writes the variables as a single file suitable for systemd's EnvironmentFile=.

This brings back a file on disk, with the trade-offs described above. Use it where you must, not by default.

Revocation and what "within minutes" means

When an admin revokes a machine, the agent learns about it on its next check. While the machine is online, that happens within minutes: the agent polls every minute, and the product's own bound for revocation reaching an online machine is five minutes. On revocation the agent wipes the environments from memory and refuses to serve them, and new crunu run calls fail with exit code 4. Crunu also rotates the keys of environments where the machine held a key, so a copy of the machine's key opens nothing new.

If the machine is offline, revocation takes effect at the latest when its lease ends. Choose the lease length with that in mind: shorter leases mean faster cut-off for offline machines and less tolerance for network outages.

The agent's exit codes make failures explicit for scripts and supervisors. The full list is in environment variables and exit codes.

If an environment is restricted to certain networks and a machine's address changes, a machine pass lets it keep receiving content for a while, 24 hours by default, so a cloud host that gets a new IP does not go dark immediately.

Hardening the agent itself

A process that holds decrypted secrets is an attractive target. On Linux the agent marks itself non-dumpable (PR_SET_DUMPABLE=0), so other processes running as the same user cannot attach to it, and core dumps are disabled. The systemd unit runs it in a sandbox with a read-only system (ProtectSystem=strict), a system call filter and no capabilities. Updates go through crunu self-update, which checks the signature of the release manifest against a key built into the binary and the SHA-256 of the file.

What the agent cannot hide

It would be misleading to stop there. An agent improves where secrets live and how long they last. It does not make a compromised machine safe.

• Root sees everything granted to the machine. A machine is a legitimate recipient. Anyone with root on it can ask the agent, read the application's memory or attach a debugger to the application. Grant each machine the minimum.
• The application still holds environment variables. crunu run hands values to your program the usual way, as its starting environment. That avoids the unit file and the D-Bus exposure systemd warns about, but the program's own /proc/<pid>/environ is still readable by its user and root, and the program keeps the values until it restarts.
• Revocation does not reach running processes. A program started before revocation keeps its variables until it exits. Restart services after revoking access, and reissue values at providers if the machine may have been compromised.
• Platform scope. The agent service runs on Linux with systemd, on amd64 and arm64. On macOS the command-line tool works for people and service tokens, but there is no background service. There is no Windows build.

If a machine's key or a service token may have leaked, revoke it, rotate the environments it received and reissue the values at the providers. Crunu's documentation has a runbook for that case, and the machines feature page shows the full lifecycle.

A short migration plan

1. Pick one service and load its production .env into a Crunu environment.
2. Enroll the production machines and grant them that environment, with Environment key or Selected files delivery.
3. Change the unit file to start through crunu run --restart on-change.
4. Delete the old .env from the servers, and check images, snapshots and backup jobs for copies.
5. Reissue the values that lived in those files at their providers, since the old copies may still exist somewhere.
6. Change the deploy pipeline so it only ships code and restarts the service. The post on giving CI only the secrets it needs covers that side.

More in machines and leases, install the agent, and the command references for crunu enroll and the daemon, crunu run and crunu pull.

Sources

• systemd.exec(5), freedesktop.org, checked October 27, 2026.
• Manage secrets securely in Docker Compose, Docker Docs, checked October 27, 2026.
• proc_pid_environ(5), Linux manual page, checked October 27, 2026.