Console Sources

Kerbside can connect to the following platforms:

• Shaken Fist
• oVirt, an Open Source Red Hat supported virtualization system
• OpenStack, an Open Source cloud compute platform
• Proxmox VE, an Open Source virtualization platform
• A fixed list of QEMU targets, for CI and debugging (the static source)

The connection to each platform (a source of consoles, so "console sources") is defined in the sources.yaml configuration file in YAML format. The path to this file is configured via the SOURCES_PATH setting.

How Console Sources Work

Kerbside handles different source types in different ways:

• oVirt: These sources are queried regularly (once a minute) for a list of available consoles. The consoles are stored in the database and presented in the Kerbside administrative interface.

• Proxmox VE: Proxmox sources are scraped the same way, once a minute. The scrape never touches a VM's console password; a ticket is minted when a session's first channel connects. See the Proxmox VE section below.

• Shaken Fist: Shaken Fist sources are also scraped regularly, but with a system credential, so the scrape covers the whole cluster rather than a single namespace. Access then goes through a token exchange rather than opening a console straight from the scraped table: Shaken Fist mints a short-lived, Ed25519-signed token that Kerbside verifies entirely offline at /sf-console.vv. See the Shaken Fist section below.

• OpenStack: OpenStack sources are handled differently. Rather than periodic scraping, OpenStack uses on-demand authentication. When a user requests a console via Nova's SPICE console API, Nova provides an authentication token that Kerbside validates against the configured OpenStack source. This means OpenStack consoles appear dynamically when requested rather than being pre-discovered.

It is possible to have more than one console source for a given type, so for example the VDI proxy could be used to combine virtual machines from two OpenStack clusters together seamlessly.

What happens when a source fails

Each scraping pass ends by cleaning up consoles it did not re-see, on the reasoning that they have been deleted at the source. That reasoning only holds for a source which was actually enumerated, so the cleanup is scoped to those: a source which errored, raised, or was skipped keeps its consoles. A cluster which is briefly unreachable, presents a mismatched CA during a rotation, or returns an API error therefore does not lose its console inventory; the consoles stay listed and are reconciled on the next successful pass.

The one exception is a source removed from sources.yaml altogether. It is deleted along with its consoles, because nothing will ever scrape it again and its consoles would otherwise be orphaned.

That exception is why a sources.yaml which cannot be used at all changes nothing. If the file is missing, unreadable or not valid YAML, if it is not a list, or if any entry is not a mapping or lacks a source or type key, the pass logs why and stops before touching the database: every source and console stays as the last pass which loaded the file left them, and the next pass, a minute later, tries again. Reconciling against such a file would read it as "no sources" and delete the lot. An empty file is refused the same way, since a truncated file is the likelier cause; a deployment which really has no sources says so with []. The YAML error is logged by problem, line and column only -- never the offending line itself, which in this file may well hold a password.

Erroring still marks the source as errored in the administrative interface, which is where a persistent failure should be diagnosed from. Retaining consoles is not the same as reporting the source as healthy.

Retention is not time bounded. A cluster which has been decommissioned but left in sources.yaml keeps its consoles listed indefinitely, and users can still be issued tokens for virtual machines which no longer exist. That is the deliberate trade: a transient failure is common and used to be silently destructive, whereas a permanently dead source is visible as an errored source in the administrative interface, and removing it from sources.yaml deletes it and its consoles on the next pass.

Shaken Fist

Shaken Fist sources are periodically scraped for their available consoles, but both the scrape and the console-access flow differ from the other scraped sources (oVirt and Proxmox VE), so they are described here in full.

Cluster-wide scrape. The source's username must be system, and Kerbside scrapes the entire cluster in one pass: every namespace's instances are enumerated, so all of the cluster's SPICE consoles become reachable through a single source. A source configured with any other namespace does not narrow the scrape to that namespace: only the instance listing uses the configured namespace, while the CA fetch, the signing-key fetch and the node listing are always made as system with the same key, so a credential that cannot authenticate as system fails at startup (see issue #444). Each pass also refreshes the cluster's node list so a console can be associated with the hypervisor currently hosting it. An instance is only kept when it is in the created state and its VDI video model is a SPICE variant.

Token exchange. Unlike oVirt, a Shaken Fist console is not opened directly from the scraped table. Shaken Fist mints a short-lived, Ed25519-signed JWT and hands the viewer an exchange URL of the form <KERBSIDE_URL>/sf-console.vv?token=<jwt>. Kerbside verifies that JWT entirely offline against the cluster's signing public keys, which it caches when the source is initialized (and refetches exactly once if it sees an unknown key id, so a signing-key rotation is tolerated). Verification checks the signature, the audience (aud), and the expiry (exp), and enforces single use by recording the token's jti so a replayed token is rejected — no callback to Shaken Fist happens on this path. Only once the token verifies does Kerbside look the scraped console up, confirm that the console's source matches the source whose key verified the token, and issue the usual Kerbside console token and virt-viewer (.vv) file. The audience Kerbside accepts is set by SF_CONSOLE_TOKEN_AUDIENCE (or, when unset, derived from PUBLIC_FQDN) and must equal Shaken Fist's KERBSIDE_URL exactly; see Configuration.

The exchange is offline so that opening a console does not depend on the cluster's API being up. The signing keys therefore live in the database (sf_token_keys): the daemon's source writes them, and the API process, which holds no source objects, reads them there. The only call to Shaken Fist from the exchange path is the unknown-key-id refetch, debounced to one per minute per worker so a stream of tokens with random key ids cannot turn it into load on the cluster. The exchange never creates a console, unlike the OpenStack one: a token for a console not yet scraped is answered 404 without spending its jti, since a token can outlive the gap until the next scrape and a retry should then succeed. The issued .vv points at the proxy, from Kerbside's own configuration; the proxy then finds the backend from the scraped console row, never from the token's claims.

What the exchange audits. A rejection that happens after the token verifies -- an unknown console, or a replayed jti -- names a verified source and console uuid, and is recorded as an audit event against that console. A rejection that happens before verification -- a malformed token, an unknown key id, a bad signature, an expired token, a wrong audience -- is logged only. /sf-console.vv is unauthenticated by design, since the Shaken Fist JWT is itself the credential, so those rejections can be provoked by anyone with no credential at all; auditing them would let an unauthenticated caller grow the audit_events table without limit, and nothing reaps it.

Backend certificate pinning. At scrape time each console's host_subject is pinned from the hypervisor node's published SPICE server certificate subject (spice_server_cert_subject), so the proxy's backend TLS leg can verify it is talking to the expected hypervisor. A node that publishes no subject (an older cluster, or a node without a cert) leaves host_subject unset, and the proxy skips host-subject enforcement for that backend rather than refusing it. The optional synthesize_host_subject knob (below) lets an operator on the stock cn=hostname PKI turn enforcement on before the node-side change is deployed.

The following options are used to configure a Shaken Fist console source (type: shakenfist).

Option	Description
source	The name of the source (used as an identifier)
type	The type of the source: shakenfist
url	The API URL for the Shaken Fist cluster
username	The Shaken Fist namespace to authenticate to; must be system, which scrapes the whole cluster
password	The API key/password to authenticate with
ca_cert	Required: the SSL CA public key certificate to validate API and VDI connections against
synthesize_host_subject	Optional (default false): when a node publishes no spice_server_cert_subject, synthesize CN=<node> so the proxy still enforces the backend host subject. Only correct on a stock cn=hostname PKI — a mismatch would wrongly reject the backend — so it is off by default.

Note: The CA certificate is verified against the cluster's advertised certificate during initialization, and the cluster's VDI token signing keys are fetched at the same time. If the CA certificates do not match, the source is marked as errored and is not scraped.

A failed signing key fetch is treated differently, because token exchange is optional and console scraping is not. The source is not errored: it is scraped as usual, its consoles stay in the inventory, and only /sf-console.vv token exchange is unavailable for that cluster until a later fetch succeeds. The most common cause is a cluster on which no signing key exists yet, because sf-ctl ensure-kerbside-signing-key has not been run -- /admin/vditokenpubkey returns 404 until it has, and Shaken Fist never creates the key lazily. That case is logged at info level; a genuine failure such as an unreachable or unauthenticated cluster is logged as a warning. Kerbside refetches the keys on every maintenance pass, because the daemon constructs each source afresh on every pass and the fetch is part of construction; there is no separate refresh timer, so a change that keeps sources alive between passes must add one. Nothing needs to be restarted once the key is created.

Each SPICE instance is matched to its hypervisor through the cluster's node list, by node UUID. An instance whose node is missing from that list is skipped with a warning; one or two during node churn are expected. If none of the SPICE instances match a node, the scrape fails instead: the source is marked errored and keeps the consoles it had, rather than reporting zero consoles and deleting them all. That pattern means Kerbside and the cluster disagree about how nodes are identified, usually after a Shaken Fist API change, and the error names the node references it could not resolve.

oVirt

The following options are used to configure an oVirt console source (type: ovirt).

Option	Description
source	The name of the source (used as an identifier)
type	The type of the source: ovirt
url	The oVirt Engine URL (e.g., https://ovirt.example.org/ovirt-engine). Must not end in /api -- Kerbside appends that itself
username	The username to authenticate to the source as. oVirt 4.4.7 and later name the built-in administrator admin@ovirt@internalsso; older deployments use admin@internal
password	The password to authenticate with
ca_cert	Required: the SSL CA public key certificate to validate API and VDI connections against. Inline PEM text, not a path

Note: The CA certificate is verified against the engine's PKI certificate during initialization. If they don't match, the source will be marked as errored. A source that errors as soon as it is configured usually means the pasted CA is stale or truncated, rather than that the engine is unreachable.

The engine URL must use the name the engine's certificate was issued for, and that name must resolve from the Kerbside host: an IP address URL fails certificate verification.

The account needs to list VMs, list hosts (Kerbside pins each VM's host certificate subject), read graphics consoles, and acquire console tickets -- the last requires the RECONNECT_TO_VM action group on the VMs. Listing hosts is an administrative operation, so a plain VM-portal role is not enough; only SuperUser has been tested.

oVirt console tickets

Kerbside asks the engine for a graphics-console ticket when the first channel of a session connects, not when the .vv file is downloaded. The ticket is requested with an explicit lifetime (the engine would otherwise apply two hours): the token's remaining life plus 30 seconds. It is stored on the session's consoletokens row, which every proxy node shares, so the session's other channels reuse it and only one channel mints, under a row lock. A stored ticket with less than 10 seconds left is replaced. The maintenance loop clears the ticket once it or the token has expired. Static sources are different: their ticket is the configured console password, stored on the console.

QEMU's set_password, which oVirt uses to set a SPICE ticket, keeps clients that are already connected, so replacing a session's ticket leaves its established channels up. This is exercised against real QEMU by tools/direct-qemu/verify-ticket-rotation.sh.

The mint is bounded as a whole at 20 seconds, because ovirtsdk4 applies no timeout to its login; each engine request is separately bounded at 15 seconds. A mint that fails or times out denies the channel (console ticket unavailable) and is audited with the exception's class name. A successful mint is audited with its lifetime. An unreachable engine therefore breaks connection setup as well as discovery. Direct consoles (/console/direct, administrators only; oVirt, not Proxmox) mint through the same deadline with a lifetime of CONSOLE_TOKEN_DURATION; an errored source answers 404 and any other failure 502.

Only one client is served per VM at a time: when a second session connects to a VM, the SPICE server drops the first session's client, regardless of tickets. That is unchanged by how tickets are minted.

Kerbside connects to the hypervisor's SPICE ports directly, so it needs L3 reachability to every hypervisor, and oVirt's own SPICE proxy (SpiceProxyDefault) is not in the path. See Kerbside for oVirt for the deployment architecture, the network prerequisites, and what is and is not proven.

Proxmox VE

The following options are used to configure a Proxmox VE console source (type: proxmox).

Option	Description
source	The name of the source (used as an identifier)
type	The type of the source: proxmox
url	The URL of the API of one node of the cluster, on port 8006 (e.g., https://pve1.example.org:8006). It must be https: any other scheme marks the source errored, since the token would otherwise cross the network in the clear. Its host is also the CONNECT proxy for every console in the source, so it must be a node, or something which forwards ports 8006 and 3128 to one
username	The ID of an API token, in the form user@realm!tokenid (e.g., kerbside@pve!console). A password login is not supported
password	The token's secret
ca_cert	Required: the cluster's CA certificate, inline PEM text, not a path. It is /etc/pve/pve-root-ca.pem on any node. Kerbside uses it to validate the API, and as the CA for the TLS connection to each VM's SPICE server, which the same CA signed

The URL must use a name the node's certificate was issued for, and that name must resolve from the Kerbside host.

Proxmox VE account

The token needs only the VM.Audit and VM.Console privileges on /vms. Create a role holding exactly those, a user to own the token, and a privilege-separated token, and give both the role on /vms:

pveum role add KerbsideConsole --privs "VM.Audit VM.Console"
pveum user add kerbside@pve
pveum acl modify /vms --roles KerbsideConsole --users kerbside@pve
pveum user token add kerbside@pve console --privsep 1
pveum acl modify /vms --roles KerbsideConsole --tokens 'kerbside@pve!console'

pveum user token add prints the token's secret once. A privilege-separated token holds only the privileges that both it and its user have, so the user needs the role as well as the token. Giving the user the role does not let the token do more than the token's own ACL allows. The token ID, kerbside@pve!console, is the source's username.

Proxmox VE network

The Kerbside broker must reach the host in the source's url on both 8006 (the API) and 3128 (the node's SPICE proxy). Kerbside asks Proxmox to name that same host as the proxy for each console, so the nodes' own names never need to resolve from the broker, and refuses a ticket whose proxy is any other host. Kerbside reaches a VM's SPICE server only through that proxy, with an HTTP CONNECT. The CONNECT itself is plaintext, so the ticket's signed connect host crosses the network between the broker and that host in the clear; only the SPICE session inside the tunnel is TLS. Keep that network one you trust.

Which VMs are listed

Each scrape lists the cluster's VMs in one request, and then reads the configuration of those that are candidates. A VM becomes a console only when it is:

• a running QEMU VM, not a container and not a template;
• configured with a qxl, qxl2, qxl3 or qxl4 display, or any virtio display. The default std display has no SPICE server, so such a VM is skipped;
• configured with an SMBIOS UUID. A VM created with qm create or the web interface has one; a VM without one is skipped, and the scrape logs its VM ID at info level.

Kerbside reads each VM's running configuration, not changes still pending until its next restart.

Each node's SPICE certificate subject is read once per scrape and pinned for that node's consoles. A node which answers but has no pve-ssl.pem subject yields no consoles, and the scrape logs a warning. If any request fails, for the VM list, a node's certificates or one VM's configuration, the source is marked errored for that scrape and keeps every console it had. A scrape that finished without those consoles would remove them, and with them the tokens of any session using them. While a source is errored, no new consoles are discovered in it.

A VM which migrates to another node still mints on its new node, but its console keeps the old node's pinned subject until the next scrape, so for up to a minute after a migration the backend connection fails its subject check and the console cannot be reached.

Proxmox VE console uuid

A Proxmox console's uuid has the form <vmid>-<smbios uuid>, with the SMBIOS UUID lower-cased (e.g., 101-0b3c9d6a-1f4e-4c8a-9d52-7a1e5f0c2b44). Proxmox hands out the lowest free VM ID, so a deleted VM's ID is reused by the next VM created. A VM ID alone would give two different machines one console, one audit trail and one set of tokens. The SMBIOS UUID tells them apart, and the VM ID lets Kerbside find the VM without reading every VM's configuration. A mint checks that the VM still has the SMBIOS UUID the console was discovered with, and refuses if it does not.

Proxmox VE console tickets

Kerbside asks Proxmox for a SPICE ticket when the first channel of a session connects, not when the .vv file is downloaded, and shares it between the session's channels as described for oVirt. A Proxmox ticket lasts 30 seconds from the mint, a length Proxmox does not let a caller choose, so CONSOLE_TOKEN_DURATION does not change it. A session's channels reuse the ticket until 10 seconds before it expires, and a channel which connects after that mints a fresh one. Minting a ticket replaces the VM's password, which does not disconnect clients already connected. A second session to the same console does disconnect the first, because qemu's SPICE server serves one client at a time, as it does for any SPICE client.

A mint reads the VM list and the VM's configuration before it asks for the ticket, and each request is bounded at 5 seconds. A failure denies the channel (console ticket unavailable) and is audited with the exception's class name. An unreachable API therefore breaks connection setup as well as discovery.

The API's response also names a CA. Kerbside ignores it and verifies the connection against the source's ca_cert, so the API cannot choose what the connection is checked against.

Proxmox VE has no direct consoles

A Proxmox console is proxied only. There is no direct .vv file for it: a 30 second ticket is too short for a file to be downloaded and opened, and only Kerbside can make the CONNECT the ticket is for. A request to /console/direct/<source>/<uuid>/console.vv for a Proxmox console is answered with 409, this source offers proxied consoles only. The refusal is made before the source's secrets are read or a ticket is minted, and is audited. The consoles page does not offer Direct for these consoles.

OpenStack

OpenStack sources work differently from Shaken Fist and oVirt. Instead of periodically scraping for available consoles, Kerbside validates authentication tokens issued by Nova when users request SPICE direct console access.

Nova 2025.1 (Epoxy) and later includes native support for SPICE direct consoles via the "spice-direct" console type, which the Kerbside developers proposed to Nova and landed with the Nova team (see Kerbside for OpenStack). When a user requests a console, Nova returns a URL pointing to Kerbside with an authentication token. Kerbside validates this token via Nova's /os-console-auth-tokens/ API and establishes the proxied connection to the hypervisor.

The following options are used to configure an OpenStack console source (type: openstack).

Option	Description
source	The name of the source (used as an identifier)
type	The type of the source: openstack
url	The Keystone authentication URL (e.g., http://keystone.example.org:5000)
username	The username for the service account
password	The password for the service account
verify	Optional: how to verify TLS certificates on the session Kerbside uses to reach this cloud -- both the Keystone authentication and the Nova console token validation call, since they share one session. It does not affect ca_cert, which is the hypervisor VDI leg. Accepts a boolean, the strings true/false case-insensitively, or a path to a CA bundle on the Kerbside container's filesystem, which is what a deployment with a private Keystone CA usually wants. Defaults to true. Any other string is passed through as a bundle path and fails at connection time, so a typo disables nothing and breaks the exchange
interface	Optional: the Keystone catalogue interface Kerbside resolves Nova on for console token validation -- internal, public or admin. Defaults to internal, because Kerbside is a control plane service that already sits on the management network to reach the compute nodes. Use public only for a Kerbside outside the cloud it brokers, such as a multi-cloud deployment. Keystone authentication itself goes to url and is not affected. Any other value is a configuration error and fails the exchange
project_name	The OpenStack project name for the service account
user_domain_id	The OpenStack user domain ID (typically "default")
project_domain_id	The OpenStack project domain ID (typically "default")
ca_cert	Optional: inline PEM text for the CA which validates the hypervisor's VDI TLS port, used by the proxy on the backend leg. This is not the Keystone CA -- see verify above

Note: OpenStack integration requires Nova 2025.1+ with SPICE direct console support enabled. See the Kerbside Patches repository for Kolla-Ansible deployment support, and the Nova specification for configuration details.

Static source

The static source driver (type: static) reads its VM-to-console mapping entirely from an inline consoles: list in the sources.yaml entry. No external API calls are made and no control plane is needed.

The list is re-read at startup and once per 60-second maintenance cycle, so adding, removing or editing an entry takes effect without a restart. An edited entry is applied in place, and logged and audit logged as Console configuration changed with the names of the fields that changed -- ticket among them when the password was rotated, but never its value.

It is intended for CI pipelines that boot a QEMU guest directly and need kerbside to front it — the direct-qemu CI workflow uses this driver — and for ad-hoc debugging against a hand-rolled QEMU. The case for the deployment, and what it can and cannot do, is in Kerbside standalone.

Not intended for production use. There is no liveness check on the QEMU process behind the ticket, so kerbside will mint a .vv file for a target which is not listening and the user discovers it by failing to connect.

The following options are used to configure a static console source (type: static).

Option	Description
source	The name of the source (used as an identifier)
type	The type of the source: static
ca_cert	PEM CA certificate used to verify the target's TLS certificate when the proxy escalates to the secure port. Required for any console in this source which declares a secure_port; without it the target is verified against the public web trust store, which an internal certificate will not satisfy
consoles	A list of console entry dicts (see fields below). Required: a source with no consoles key, or a misspelled one, is marked errored and keeps the consoles it had published. Write consoles: [] for a source with no consoles, which deletes any it had

Each entry in the consoles list requires the following fields:

Field	Description
uuid	Identifier for this console, unique within this source; another source may use the same one without the two colliding
name	Human-readable display name
hypervisor	Hostname of the hypervisor (used if hypervisor_ip is empty)
hypervisor_ip	IP address of the hypervisor
insecure_port	SPICE port (plaintext)
ticket	SPICE password / authentication ticket

Optional fields (default to null):

Field	Description
secure_port	SPICE TLS port (if QEMU exposes one)
host_subject	TLS host subject for certificate verification. Enforced: when set, the proxy refuses hypervisors whose certificate subject does not match (exact attribute count/order/type).

Kerbside reads every unquoted number in sources.yaml exactly as it is written. A text field such as ticket: 012345 is the text 012345, not the octal number 5349 that YAML 1.1 would make of it, and the same holds for spellings like 0x1F, 1_000 and 12:34. A port may be quoted or not, as long as it is a whole number from 1 to 65535. Any other type -- a list, a mapping, a boolean, a port that is not one -- marks the source errored, and it keeps the consoles it had published. The same rule applies to the fields every source shares (source, type, url, username, password and the like), except that a bad one refuses the whole file and keeps the last configuration that loaded.

Example sources.yaml entry for a static source:

- source: ci-direct-qemu
  type: static
  consoles:
    - uuid: "6f4e2c1a-0000-0000-0000-000000000001"
      name: "sextant-ci-vm"
      hypervisor: "localhost"
      hypervisor_ip: "127.0.0.1"
      insecure_port: 5910
      ticket: "my-spice-password"

A two-console example with inline comments is available at etc/example-static-sources.yaml.

The static source pairs cleanly with Ryll's control socket for end-to-end direct-QEMU testing. A test harness boots a QEMU guest, declares it via the static source, points kerbside at that source, and then drives the SPICE session through Ryll's --control-socket interface — sending keystrokes, pasting text, and capturing screenshots without a GUI. The control-socket protocol is the bridge between the test driver and the SPICE session it asserts against.

The direct-qemu lane's final step runs the Sextant scenario tempest test (tests/scenario/test_sextant_scenario.py), which drives the full Awaiting → Booting → bootloader-ignore → paste → Parked → shutdown sequence and asserts both the live digest_updated QR event stream and the post-mortem serial drain. The test skips when CONF.kerbside.control_socket_path is unset, so it is safe to load the plugin on the OpenStack lane where these direct-qemu options are not configured. tools/direct-qemu/run-scenario.sh handles tempest venv setup, config file generation, and test execution.

Example sources.yaml

An example configuration follows:

- source: sfmel
  type: shakenfist
  url: https://sfmel.example.org/api
  username: sfvdi
  password: ...omitted...
  ca_cert: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----

- source: ovirt
  type: ovirt
  url: https://ovirt.example.org/ovirt-engine
  username: kerbside@internal
  password: ...omitted...
  ca_cert: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----

- source: pve
  type: proxmox
  url: https://pve1.example.org:8006
  username: kerbside@pve!console
  password: ...omitted...
  ca_cert: |
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----

- source: kolla
  type: openstack
  url: http://keystone.example.org:5000
  username: kerbside
  password: ...omitted...
  project_name: service
  user_domain_id: default
  project_domain_id: default

Related Documentation

• Configuration - General configuration reference
• Proxy Architecture - Internal proxy design
• Kerbside for Shaken Fist - The Shaken Fist use-case guide: value proposition, setup, and the sf-e2e worked example
• Kerbside for OpenStack - The OpenStack use-case guide: the spice-direct token exchange, setup, and the openstack_matrix worked example
• Kerbside standalone - The standalone use-case guide: the static source's no-control-plane model, setup, and the direct-qemu worked example

📝 Report an issue with this page