Skip to content

VDI console tokens (Kerbside integration)

Since v0.8, Shaken Fist can hand a user a proxied graphical console instead of requiring direct network access to the hypervisor. Rather than exposing the raw SPICE port, Shaken Fist mints a short lived, cryptographically signed token and points the user's viewer at a Kerbside proxy, which validates the token and relays the SPICE session on the user's behalf.

This page is the operator runbook for that integration: what it is, how to enable it, how the signing key is stored and rotated, and how the per-node certificate subject flows through to the proxy. The user-facing side (how a person actually opens a console) lives in the consoles user guide.

What the integration does

When a user requests a proxied console, sf-api mints an Ed25519 JSON Web Token (JWT, algorithm EdDSA) describing the instance, the requesting namespace, and an expiry. It returns a URL of the form <KERBSIDE_URL>/sf-console.vv?token=<jwt>. The user's viewer fetches that URL from Kerbside, which:

  • verifies the token's signature offline against the public key Shaken Fist publishes — there is no callback to sf-api on the connection hot path;
  • checks the audience, expiry, and single-use identifier (jti); and
  • proxies the SPICE session to the correct hypervisor, pinning the backend certificate subject to the value Shaken Fist published for that node.

The trust model is therefore offline signature verification: Shaken Fist is the sole signer, Kerbside is a verifier that never holds private key material, and a compromised proxy cannot mint valid console tokens.

Note

Two unrelated KERBSIDE_* configuration namespaces exist and must not be conflated. KERBSIDE_URL and KERBSIDE_TOKEN_DURATION described on this page are Shaken Fist cluster configuration (set via SHAKENFIST_-prefixed environment, cluster_config, or sf-ctl). The Kerbside proxy daemon has its own, separate KERBSIDE_-prefixed environment variables (documented in the Kerbside configuration page), such as KERBSIDE_SF_CONSOLE_TOKEN_AUDIENCE. When you see a KERBSIDE_* name, check which side owns it.

Enabling the integration

The integration is disabled by default: KERBSIDE_URL is empty, and every mint request returns HTTP 404 while it stays empty. Instances still offer the direct-to-hypervisor console described in the consoles user guide; enabling this integration only adds the proxied path.

Note

Kerbside deployment itself is not yet automated by the Shaken Fist ansible collection, so enabling the integration is a manual step today. You deploy and configure a Kerbside proxy out of band (see the Kerbside component documentation), then point Shaken Fist at it as below.

To enable it, set KERBSIDE_URL to your Kerbside deployment's public base URL. You can do this at deploy time by setting the collection's kerbside_url variable in your group_vars/all.yml (see Installation), which renders it into every node's /etc/sf/config:

kerbside_url: https://kerbside.example.com

or on a running cluster with sf-ctl (a cluster_config value overrides the rendered file at process start):

sf-ctl set-config KERBSIDE_URL https://kerbside.example.com

KERBSIDE_URL is both the base for the returned console URL and the token audience (aud) claim. Kerbside's expected audience — its own SF_CONSOLE_TOKEN_AUDIENCE, or a value derived from its PUBLIC_FQDN if that is unset — must equal this string exactly, or verification fails.

Optionally set KERBSIDE_TOKEN_DURATION, the lifetime in seconds of a minted token. It defaults to 300 seconds (five minutes), which is ample for a viewer to redeem the URL:

sf-ctl set-config KERBSIDE_TOKEN_DURATION 300

Both values are read from cluster configuration at sf-api start, so change them before (or restart sf-api after) enabling the integration.

Signing key custody

The private key that signs console tokens lives in a single cluster_config row named KERBSIDE_JWT_SIGNING_KEY. Treat its custody the same way you treat AUTH_SECRET_SEED: it is cluster-wide secret material, and anyone who can read it can mint console tokens for any instance.

The stored value is a small JSON document holding the active key id and a newest-first list of Ed25519 keypairs (private and public PEM). Because the configuration name ends in _KEY, sf-ctl show-config redacts it by default; never pass --show-secrets where the output could be logged, and never print or event private key material.

The row is also excluded from the cluster configuration Shaken Fist exports into each daemon's environment at startup. A row is withheld when its name looks like a secret and it is not a declared configuration option; ordinary rows, including options this release does not know about, are still exported. So the signing key no longer sits in every daemon's /proc/<pid>/environ on every node, and is no longer inherited by the processes privexec spawns. Nothing an operator does turns that off.

It is worth being precise about what that does and does not buy, because the distinction matters if you are reasoning about blast radius. Shaken Fist reads cluster configuration a whole table at a time, so the key is still sent to every daemon at startup and to the cluster daemon on each maintenance pass — it is simply no longer retained in their environment. An attacker who can read a daemon's memory is therefore not shut out by this change; one who can read its environment, or the environment of a command privexec ran, now is.

The key is not created automatically: it must be provisioned explicitly before the first console is opened. Until it exists, the vdiconsoleproxy endpoint returns HTTP 500 naming the command to run below — minting never self-bootstraps a key. Create it — for example so you can publish the public key to Kerbside before anyone opens a console — with the idempotent:

sf-ctl ensure-kerbside-signing-key

This prints the active key id and how many keys are published; it never prints private material and does nothing if a key already exists.

Provision the key before upgrading a cluster Kerbside already scrapes

Do this before you roll the daemons, if Kerbside is already configured with this cluster as a console source.

Until the key exists, GET /admin/vditokenpubkey returns HTTP 404. Kerbside v0.5.0 and earlier treat that failure as making the whole source unusable: the scrape is abandoned before it enumerates any consoles, and the cleanup pass that follows then removes every console Kerbside had previously recorded for this cluster. That deletes the direct and proxy console routes too, not only the token-based ones, so users lose consoles which have nothing to do with this feature.

Provisioning the key first avoids it entirely. If you have already hit it, provision the key and let Kerbside scrape again — the inventory is rebuilt from Shaken Fist, so the loss is an outage rather than permanent damage.

Kerbside v0.6.0 fixes this, in two parts. A missing key is downgraded to "this source has no token consoles yet" rather than failing the source (kerbside#412), and the cleanup pass no longer removes consoles belonging to a source it did not successfully enumerate (kerbside#413). The second half matters on its own: before it, any early exit from a scrape — an unreachable cluster, an error fetching the cluster CA — deleted that source's consoles, whether or not tokens were involved. On v0.5.0 and earlier, provision the key before you upgrade.

Publishing the public keys

Kerbside verifies tokens with the public half of the signing key, which Shaken Fist serves to admin callers at GET /admin/vditokenpubkey. The response is the active key id and every currently published public key:

{
    "active_kid": "3f2a9c1e",
    "keys": [
        {
            "kid": "3f2a9c1e",
            "alg": "EdDSA",
            "public_pem": "-----BEGIN PUBLIC KEY-----\n...\n-----END PUBLIC KEY-----\n",
            "created": 1789000000
        }
    ]
}

It returns HTTP 404 until a signing key exists (run sf-ctl ensure-kerbside-signing-key first). Kerbside fetches this endpoint to learn the verification keys; a token is accepted if it is signed by any published key, which is what makes rotation safe.

Rotation runbook

Rotation publishes a fresh signing key while keeping the previous one verifiable, so a rotation never invalidates tokens that are still in flight. The stored key list is a two-key window: rotation prepends a new key, marks it active, and trims the list to the two newest entries.

To rotate:

sf-ctl rotate-kerbside-signing-key

This prints the new active key id and the published key count (which is 2 after the first rotation). New tokens are signed only with the active key; tokens signed by the previous key still verify because Kerbside accepts any published key.

Warning

Do not rotate more often than KERBSIDE_TOKEN_DURATION. Each rotation drops the third-oldest key from the two-key window, and any token signed by a dropped key immediately becomes unverifiable. Waiting at least one token lifetime between rotations guarantees every in-flight token was signed by a key that is still published. Rotating twice in quick succession can strand live sessions.

Note

Run the signing-key commands (ensure-kerbside-signing-key, rotate-kerbside-signing-key) one at a time. Both perform an unlocked read-modify-write of the KERBSIDE_JWT_SIGNING_KEY cluster config, so two running concurrently could lose one write and strand a key that should still be published. These are manual, infrequent operator actions, so serialising them is trivial — just don't fire two at once.

A safe cadence is therefore: rotate, wait for Kerbside to refresh its published keys and for at least KERBSIDE_TOKEN_DURATION to elapse, then rotate again if needed.

Host subject enforcement

Kerbside pins the backend it connects to by certificate subject, so a misdirected or spoofed hypervisor address cannot silently receive a session. The subject it enforces for a given node is the value Shaken Fist publishes as that node's spice_server_cert_subject attribute.

Each hypervisor reads its own SPICE server certificate at node-observation time and publishes the parsed subject (for example C=US,O=Shaken Fist,CN=hv1). Nodes without a SPICE server certificate — a non-hypervisor node, or a hypervisor whose certificate is missing or unreadable — publish nothing, which leaves host-subject enforcement disabled for that backend rather than breaking the node. Kerbside consumes this value during its cluster-wide scrape and uses it as the connection host_subject. See the Kerbside console sources page for the proxy-side view.

There is a second way to publish nothing, and it is easier to hit on your own PKI than on the one the deployer builds: Shaken Fist can only render a subject whose every attribute has a SPICE host-subject short name (C, ST, L, O, OU, CN, DC, emailAddress). A subject qualified with anything else — a serialNumber, a custom OID — cannot be rendered exactly, and a partial rendering would wrongly reject the backend, so the node publishes nothing and enforcement is disabled for it. If your CA issues such subjects this happens on every node at once. It is not silent: each node logs a warning naming the offending attribute and its OID once per episode rather than once per read — a certificate which is repaired and later regresses warns a second time, because that recurrence is the event you need to see. Grep your logs for no SPICE host-subject short name if spice_server_cert_subject is unexpectedly absent from a node in GET /nodes.

A certificate with an empty subject — a SAN-only certificate, which some modern CAs issue by default — has nothing to pin and is treated the same way: nothing is published, enforcement is disabled for that node, and the node logs empty subject.

📝 Report an issue with this page