SSH management¶

SSH access is managed entirely through Ansible. This page covers how operator keys are stored, how to add new users, and how to configure extra system users for branch or environment-specific deployments.

Operator SSH keys¶

SSH public keys for the operators are stored in keys/ at the repo root:

keys/
└── pt.pub

These keys are installed during initial server provisioning by the bootstrap_root.yml playbook, which creates the operator accounts and authorizes the keys.

Adding new users¶

When adding a new user to a server:

• Always create a dedicated SSH key pair for the user — do not reuse existing operator keys
• Add the user in host_vars for that specific server — not in group_vars
• Avoid global user configuration — access should remain environment- and server-specific

User accounts are managed by the access_users role via system.users in each host's host_vars:

# inventories/prod/host_vars/mysite-web-01/main.yml
system:
  users:
    - name: mysite
      sudo: true
      ssh:
        generation: true      # generate ed25519 key pair on the server
        authorized_key: true  # self-authorize the generated key
        connect: true         # add to SSH AllowUsers

See Inventory & variables for how host_vars files are structured, and New server for a full provisioning walkthrough.

Revoked access¶

access_users_denied in group_vars/all/common.yml lists names that must never be granted access again:

access_users_denied:
  - sd

Adding a name here does two things: it blocks the name from ever being granted access again, and it actively strips the access it already has.

Blocked¶

The playbook run fails rather than quietly re-granting access:

• access_users fails if the name appears in access_users, system.users, system.access_users_groups, or the deployment role's system_users, system_users_base and system_users_extra — so it can be re-created neither as a user nor as a group. The deployment lists matter because that role runs after this one and would otherwise recreate the account minutes after it was stripped
• ssh_hardening fails if the name reaches AllowUsers through any of base_ssh_allow_users, access_users, system.users or sites

The first check also means you cannot lock out an operator who is still declared: put pt on the list while pt is still in access_users, and the run fails before anything is touched.

Stripped¶

On every host where the account still exists, access_users converges it to no access:

The account, its home directory and its files are not touched. Deleting those destroys data and cannot be undone by re-running the playbook, so it stays a manual step:

sudo pkill -u <name>; sudo userdel -r <name>; sudo groupdel <name>

Terminating the processes matters as much as the rest. Unix credentials are per process: a session that was already open keeps the supplementary sudo and docker group ids it started with, so until it is gone the account can still act as root and re-authorize itself. The one exception is the account this run connects as - killing that would abort the play, so it is reported instead and left for another operator to end by hand.

An offboarded operator's key is routinely authorized for root and for service accounts as well as their own, so every authorized_keys and authorized_keys2 under /root and /home is scanned for it and the matching line removed. Both filenames, because sshd's default AuthorizedKeysFile is .ssh/authorized_keys .ssh/authorized_keys2 and the managed sshd_config does not narrow it. Two limits are worth knowing:

• Only keys the repo holds can be revoked. The scan uses keys/<name>.pub; if there is no such file, the run says so and moves on. Keys the repo has never seen are invisible here and must be removed by hand.
• It refuses to remove an account's last usable key. On some hosts the offboarded key is the only one authorized for root — removing it would leave no way in at all. The run fails, names the account, and asks you to authorize a replacement first. root_keys.yml does that for the operator key. The count is per account across both key files and ignores repeats of the key being revoked, so an account is only flagged when nothing would actually survive.

Do not reach for exclusive: true to make the repo authoritative over authorized_keys. Deploy keys that live only on the servers — the *-bedrock-github keys the site users authenticate with — are not in the repo, and exclusive would delete them and break deployments.

Sudoers rules are found by scanning for the name, not by trusting a filename — a <name> ... or %<name> ... rule is removed wherever it lives, and each file is validated with visudo -cf before the change is kept. Other users' rules in the same file are left alone, and Defaults lines never match. This matters because the repo is not the only thing that writes sudoers rules: cloud-init grants the image's first user passwordless root in /etc/sudoers.d/90-cloud-init-users, and hand-rolled rules end up directly in /etc/sudoers. If cloud-init ever re-runs on a host, it can recreate its file — re-running the playbook removes the rule again.

Enforcement is idempotent and only ever touches accounts that already exist, so it is safe on every run. It refuses to act on a uid 0 account. Dry-run it first with --check --diff. To disable it for a host — leaving only the blocking checks — set access_users_denied_enforce: false.

Note that removing a name from the inventory without adding it here only stops Ansible from managing it. The roles are otherwise additive and never delete an account, its keys or its group memberships. Dropping the name from base_ssh_allow_users does block SSH login on the next run, because sshd_config is rewritten from the template every time — but the key, the account and its sudo rights stay behind. That is the gap access_users_denied closes.

Managing extra deployment users¶

For environments that need dynamically-named users (e.g. branch deployments), the deployment role supports a system_users_extra list that is merged with system_users_base at run time.

system_users_base is defined in group_vars/all/deployment.yml and always includes the deploy CI/CD user. system_users_extra extends that list with additional per-environment users.

Basic usage¶

# inventories/prod/group_vars/web/main.yml
system_users_extra:
  - name: "{{ system_user_branch_name | default('') }}"
    pwd: "{{ system_user_branch_password | default('') }}"
    groups: [sudo]
    ssh:
      create_keys: true       # generate an SSH key pair for this user
      authorized_keys: true   # self-authorize the generated key

Appending to the list¶

If you need to accumulate entries across multiple variable files or plays, use Jinja2 list concatenation to append rather than replace:

system_users_extra: "{{ system_users_extra | default([]) + [
  {
    'name': system_user_branch_name,
    'pwd': system_user_branch_password,
    'groups': ['sudo'],
    'ssh': {
      'create_keys': true,
      'authorized_keys': true
    }
  }
] }}"

SSH key file¶

The key file path for generated keys is controlled by the role-level variable deployment_ssh_key_file (default: .ssh/id_ed25519). Override it per-host in host_vars if needed:

deployment_ssh_key_file: ".ssh/id_app_deploy"

SSH tunnels (database clients)¶

Tunnels are blocked on all hosts – the ssh_hardening role writes AllowTcpForwarding no. A client such as Sequel Ace or TablePlus fails with:

channel 2: open failed: administratively prohibited: open failed

The key options in authorized_keys do not help here; sshd_config wins.

Grant it per host, for named users, towards named destinations only:

# inventories/stage/host_vars/web-01/main.yml
ssh_tcp_forwarding_users:
  - pt
ssh_tcp_forwarding_permitopen:
  - localhost:3320

Then reapply the host so ssh_hardening rewrites sshd_config:

ansible-playbook -i inventories/stage/hosts.yml baseline.yml -l web-01 --ask-vault-pass

Both variables are required together – see ssh_hardening.

Only local forwarding (ssh -L) is granted. ssh -R remains blocked, which is all a database client needs.

PermitOpen is matched literally – sshd does no name lookup – so the client must ask for the destination exactly as it is written here. In Sequel Ace the MySQL host field is the forwarding destination, so it has to say localhost with port 3320; asking for 127.0.0.1:3320 is refused by the allowlist.

Once the tunnel is up, a local mysql client is the opposite case: point it at 127.0.0.1 and the forwarded port, because with localhost it ignores the port and looks for a unix socket instead:

Can't connect to local MySQL server through socket '/tmp/mysql.sock'

See Database access from a GUI client for the full Sequel Ace walkthrough.