OpenRC reads conf.d but your daemon never sees it
10 min read
A daemon that takes all of its configuration from environment variables was moved onto an OpenRC host. The init script was a dozen lines, the settings went into /etc/conf.d/exampled where settings go, and the service came up clean:
* Starting exampled ... [ ok ]
* status: started
It was also listening on loopback instead of the address in the config, pointing at a database that did not exist, and had no API token. Every value it was running with was a compiled-in default. Nothing was logged, because from the daemon's point of view nothing was wrong: an unset variable means "use the default", and every variable was unset.
The file was fine. OpenRC had read it. The daemon never saw a line of it.
Prove it from the process, not the file
The config file tells you what you intended. The kernel will tell you what the process actually received, in /proc/<pid>/environ. Entries are NUL-separated, so translate them to newlines:
tr '\0' '\n' < /proc/$(cat /run/exampled.pid)/environ | sort
That pidfile is the daemon's own under the default start-stop-daemon. Under supervisor=supervise-daemon it is the supervisor's, and the check needs the adjustment shown under fix three.
Here is the conf.d file:
# /etc/conf.d/exampled
EXAMPLED_LISTEN="0.0.0.0:8080"
EXAMPLED_DATABASE_URL="postgres://exampled@db.example.net/exampled"
EXAMPLED_API_TOKEN="s3cr3t-token"
And here is what the running daemon has, trimmed of the long PATH-style entries:
HOME=/var/lib/exampled
LANG=C.UTF-8
PWD=/
SHLVL=1
TERM=xterm
USER=exampled
_=/bin/start-stop-daemon
Narrow it to the variables you care about and the result is blunter:
tr '\0' '\n' < /proc/$(cat /run/exampled.pid)/environ | grep EXAMPLED; echo "rc=$?"
rc=1
No match. Not one of the three made it. That single command separates "the daemon is ignoring its config" from "the daemon was never given its config", which are very different bugs with very different owners.
Why: sourced is not exported
An OpenRC service script is a shell script run by openrc-run. Before your script is loaded, the wrapper sources the matching conf.d file. On OpenRC 0.63 the relevant lines of /usr/libexec/rc/sh/openrc-run.sh are:
if ! sourcex -e "$_conf_d/$RC_SVCNAME.$RC_RUNLEVEL"; then
sourcex -e "$_conf_d/$RC_SVCNAME"
fi
sourcex is a wrapper around the shell's . builtin. Sourcing a file full of NAME=value lines creates shell variables in that one shell process. A shell variable is only copied into a child's environment if it has been marked for export, and nothing here marks them.
That design is deliberate, and it suits the way most init scripts work. The conf.d file exists to parameterise the script: the script reads ${EXAMPLED_OPTS} and turns it into command_args, and the daemon receives command-line flags. The variables never needed to leave the shell.
It falls apart for a twelve-factor style daemon that has no flags and reads only its environment. There the conf.d file looks exactly like an environment file, and anyone coming from systemd's EnvironmentFile= will reasonably assume it is one. It is not. It is a shell fragment.
There is a second filter in front of this one. OpenRC scrubs the caller's environment before running a service, so this does not work either:
EXAMPLED_LISTEN=0.0.0.0:8080 rc-service exampled start
The variable is dropped before the script runs. /etc/rc.conf documents an rc_env_allow setting to let named variables through, but an allowlist of things that happen to be set in the shell of whoever restarted the service is not configuration. Leave it alone.
Fix one: re-source with auto-export
The shell option set -a (allexport) marks every variable assigned from that point on for export. Turn it on, source the conf.d file a second time, turn it off:
#!/sbin/openrc-run
description="exampled"
command="/usr/local/bin/exampled"
command_user="exampled"
command_background=true
pidfile="/run/${RC_SVCNAME}.pid"
start_pre() {
set -a
. "/etc/conf.d/${RC_SVCNAME}"
set +a
}
Restart and repeat the narrowed check:
EXAMPLED_LISTEN=0.0.0.0:8080
EXAMPLED_DATABASE_URL=postgres://exampled@db.example.net/exampled
EXAMPLED_API_TOKEN=s3cr3t-token
rc=0
This works because start_pre and the default start run in the same shell, so the exports are still in place when start-stop-daemon is launched. The same block at the top level of the script, outside any function, also works; putting it in start_pre keeps it out of status, stop and dependency scans, which have no use for it.
The property that matters is that there is no list. Add a fourth variable to conf.d next month and it reaches the daemon without anyone touching the init script. If the conf.d file is optional, wrap the block in if [ -f "/etc/conf.d/${RC_SVCNAME}" ]; then ... fi so a missing file does not fail the start. Do not shorten it to [ -f ... ] && { ... }: as the last command in start_pre that returns 1 when the file is absent, and OpenRC aborts the start without saying why.
Fix two: export in the conf.d file
Since the file is shell, it can export for itself:
export EXAMPLED_LISTEN="0.0.0.0:8080"
export EXAMPLED_DATABASE_URL="postgres://exampled@db.example.net/exampled"
EXAMPLED_API_TOKEN="s3cr3t-token"
No init script change is needed, which is the appeal when the script belongs to a package and you would rather not fork it. The cost is in that third line. It is a perfectly valid conf.d entry, it looks like its neighbours, and the daemon will not receive it:
EXAMPLED_LISTEN=0.0.0.0:8080
EXAMPLED_DATABASE_URL=postgres://exampled@db.example.net/exampled
rc=0
Every future edit has to remember the keyword, and the failure when someone forgets is the same silent fall-back to defaults you started with. It is tempting to avoid that with a bare set -a as the first line of the conf.d file. Do not. Nothing turns it back off, so it stays in force while the init script itself is loaded, and the daemon inherits command, command_user, pidfile, description and, where /bin/sh is bash, the bodies of OpenRC's internal shell functions as BASH_FUNC_ entries. It works, and it turns the environ check into several screens of noise.
Fix three: pass them with --env
Both of OpenRC's process launchers accept -e, --env VAR=VALUE, and the init script can feed that through start_stop_daemon_args:
start_stop_daemon_args="--env 'EXAMPLED_LISTEN=${EXAMPLED_LISTEN}' --env 'EXAMPLED_DATABASE_URL=${EXAMPLED_DATABASE_URL}'"
For supervisor=supervise-daemon the variable is supervise_daemon_args; openrc-run(8) says it falls back to start_stop_daemon_args when undefined. Under supervise-daemon the pidfile also changes meaning: supervise-daemon(8) says of --pidfile that "this is not the pid file of the process that is being supervised". It is the supervisor's, and the supervisor does not carry --env values in its own environment, so the earlier check reports rc=1 for a daemon that has every variable. Check the supervisor's child instead:
tr '\0' '\n' < /proc/$(pgrep -P "$(cat /run/exampled.pid)")/environ | grep EXAMPLED; echo "rc=$?"
Passing variables this way is explicit, and it is the only one of the three where the init script itself chooses which variables the daemon gets. It also has two sharp edges, and both showed up in testing.
Values are re-parsed by the shell. OpenRC passes the argument string through eval, so a space splits the value and &, ; or $(...) in it is executed as root. The inner single quotes in the line above are what prevent that. Without them, --env EXAMPLED_MOTD=${EXAMPLED_MOTD} with EXAMPLED_MOTD="hello there world" reported [ ok ] on start and then:
* status: crashed
The value was cut at the first space, and there and world were handed to the daemon as two extra arguments it rejected on startup. That is the start-stop-daemon symptom. Under supervise-daemon the same mistake fails at start instead, with supervise-daemon: (null) does not exist and [ !! ].
A database URL ending in ?sslmode=require&connect_timeout=5 did more damage. Unquoted, the & ended the command early: the start reported [ !! ], the status was stopped, and a daemon was left running anyway, outside OpenRC's control, with its URL cut at the &. A value containing $(...) had the command inside it run as root.
Written out in full, the safe form of the first example is:
start_stop_daemon_args="--env 'EXAMPLED_MOTD=${EXAMPLED_MOTD}'"
It holds until a value contains a single quote.
Secrets end up on a command line. supervise-daemon stays running for the life of the service, and its arguments stay with it:
ps -eo pid,args | grep '[s]upervise-daemon'
4126 supervise-daemon exampled --start --pidfile /run/exampled.pid --user exampled --env EXAMPLED_API_TOKEN=s3cr3t-token /usr/local/bin/exampled --
/proc/<pid>/cmdline is world-readable; /proc/<pid>/environ is mode 0400 and owned by the process's user. Moving a token from the environment to an argument takes it from "readable by the service account and root" to "readable by every local user". Do not use --env for credentials.
Which one
For a daemon that is configured entirely through its environment, use the set -a re-source. It is three lines, it cannot drift out of step with the config file, and it handles spaces and secrets without any special care.
Use export in conf.d when you cannot or should not edit the init script. Use --env for one or two non-secret values where being explicit is worth more than being generic.
One side effect of set -a to be aware of: it exports everything in the file, including any OpenRC settings you keep there such as rc_need or command_args. The daemon gets a few variables it has no interest in. That is normally harmless, but if the daemon treats unknown variables in its own prefix as an error, keep the conf.d file to the daemon's variables only.
Lock the file down
Once the conf.d file carries a database password or an API token, its default permissions are wrong. A file created by an editor or dropped in by an install script is usually 0644, readable by every account on the box:
chown root:root /etc/conf.d/exampled
chmod 0600 /etc/conf.d/exampled
This does not break a service that runs as an unprivileged user. The init script runs as root, sources the file as root, and only then drops to command_user. Verified with the daemon running as nobody:
-rw------- 1 root root 156 Nov 10 09:41 /etc/conf.d/exampled
USER PID COMMAND
nobody 4127 /usr/local/bin/exampled
The daemon had all of its variables, and cat /etc/conf.d/exampled as nobody returned Permission denied. The process can still read its own environment, which is the point, but it can no longer read the file, and neither can anything else running under a different account.
One thing does change for non-root users. openrc-run sources conf.d on every invocation, status included, so an unprivileged rc-service exampled status now fails with Permission denied and exit code 1 instead of reporting the state. Anything that polls service status without root needs another route.
Checklist
- Read
/proc/<pid>/environbefore reading the daemon's source. If the variable is absent there, the daemon is not at fault. - Under
supervise-daemon, read theenvironof the supervisor's child; the pidfile points at the supervisor. - Treat
/etc/conf.d/<service>as a shell fragment sourced byopenrc-run, not as an environment file. - For an env-configured daemon, re-source conf.d between
set -aandset +ainstart_pre. - If you rely on
exportin conf.d instead, check every line has it, including the one added last week. - Keep credentials out of
--env;psshows them to everyone. - Single-quote every
--envvalue inside the args string; it goes througheval, so spaces,&,;and$(...)are live. Checkrc-service <service> statusa second after start rather than trusting[ ok ]. chmod 0600any conf.d file that holds a secret, and expect non-rootrc-service <service> statusto stop working for it.- After every config change, restart and re-run the
environcheck. A green start proves the process exists, not that it is configured.