Installation¶
This page takes you from nothing to a working, proxied SPICE console, and then points at the pages on deploying Kerbside for real. If you would rather see Kerbside working before reading about it, skip to Try it: the demo stack.
Before you install: OS packages¶
pip install kerbside builds mysqlclient from source — that package
ships no wheel — so a clean machine needs a compiler, the MariaDB
client headers and pkg-config before pip will get anywhere. Without
them the install fails at Can not find valid pkg-config name, or
later in the compile, which reads as a Python problem and is not one.
On Debian:
sudo apt-get install build-essential pkg-config python3-dev \
python3-venv libmariadb-dev-compat libxml2-dev libxslt1-dev
On Ubuntu the same list applies with libmysqlclient-dev in place of
libmariadb-dev-compat. On RHEL, CentOS and Fedora:
If you will use an ovirt source, the ovirt-engine-sdk-python
client library (see
Client libraries for console sources)
is also a C extension and needs libcurl4-openssl-dev and
libssl-dev on Debian and Ubuntu, or libcurl-devel and
openssl-devel on RHEL, CentOS and Fedora.
You will also want a MariaDB or MySQL server, either locally or reachable over the network — see What a running Kerbside needs.
From a checkout, tox -e bindep reports the same thing against
bindep.txt, per platform. It needs the source tree and tox, so it
is a developer convenience rather than a step for someone installing
from PyPI.
Installing with pip¶
Kerbside is split into two packages: the pure-Python kerbside (the
REST API, the console-source drivers, the SQLAlchemy data model, and
the daemon that supervises the proxy) and the Rust kerbside-proxy
(the SPICE proxy itself, rust/kerbside-proxy/ in the source tree).
kerbside-proxy is published to PyPI as a separate maturin
bindings = "bin" wheel that carries the compiled binary and lands it
on PATH (the daemon finds it via shutil.which('kerbside-proxy'));
you do not build or install it separately. A release install —
pip install kerbside==X.Y.Z — gets the exact-pinned kerbside-proxy
release it was built and tested against: the two packages release in
lockstep from a single v* tag. A git or dev install of kerbside
instead resolves whatever kerbside-proxy is newest on PyPI, which may
be a rolling dev wheel rather than a tagged release. That wheel is
published unattended from develop, so no per-commit human review
stands behind it and nothing on the install path verifies its build
provenance. Production deployments should install a tagged
kerbside==X.Y.Z, which carries the exact == pin of the proxy it was
built and tested against. Prebuilt manylinux wheels are published for
x86_64 and aarch64 (no source distribution — an unsupported platform
gets a clean pip error).
For development you can instead point KERBSIDE_PROXY_BIN at a
locally built binary, or let the daemon pick up the in-repo
cargo build output.
At startup, the daemon runs kerbside-proxy --contract-hash and
refuses to launch a binary whose gRPC contract does not match this
kerbside version's — the error names both hashes and lists the ways
to fix it. KERBSIDE_SKIP_CONTRACT_CHECK=1 is the explicit, unsupported
escape hatch that downgrades the refusal to a logged warning. See
proxy-architecture.md for how the check works.
Client libraries for console sources¶
Some features import a client library that is not a Kerbside
dependency, so pip install kerbside does not bring it in. Install
what you use into the same virtual environment as Kerbside:
| You use | Install | Without it |
|---|---|---|
A shakenfist source |
shakenfist-client 0.8.3 or later |
The source is marked errored and ignored, and the daemon logs why. Older versions cannot fetch signing keys, which leaves token exchange unavailable for that source. |
An ovirt source |
ovirt-engine-sdk-python |
The source is marked errored and ignored, and the daemon logs why. |
An openstack source (token exchange) |
openstacksdk, keystoneauth1 and python-keystoneclient |
The API answers 500 to the token exchange request and logs the import failure. |
| Interactive login, whatever your sources | keystoneauth1 and python-keystoneclient |
The API answers 500 to login and logs the import failure. |
A proxmox source |
Nothing: it uses requests, which Kerbside already needs |
|
Only static sources |
Nothing |
Login is Keystone-only today (issue #300), so interactive login needs the Keystone libraries even if none of your sources are OpenStack. There is no tested version floor for the three OpenStack libraries.
What a running Kerbside needs¶
Installing the package gives you commands, not a running system. This section is the mental model: what has to exist before a console works. configuration.md is the reference for every setting; this is the short list of things you cannot skip.
Two processes. The REST API is a WSGI application served by gunicorn, and the SPICE proxy is a daemon that supervises the Rust binary:
Neither is optional and they are not interchangeable: the API is where
brokers and users ask for consoles, and daemon run is what discovers
consoles and moves SPICE traffic. daemon run launches the Rust
proxy as its own child process, and the two talk over a unix socket
(API_SOCKET_PATH, default /run/kerbside/api.sock), so the daemon
and its proxy always run together on one host or in one container.
The API never dials that socket. It reaches the daemon only through
the database, so it may run on another host. That host still needs
the same sources.yaml (SOURCES_PATH) and CA certificate
(CACERT_PATH) as the daemon's, because the API reads both from disk:
the sources for token exchange, and the CA for the console files it
issues. Note that the API's port is a gunicorn argument rather than a
Kerbside setting; PUBLIC_FQDN is how Kerbside tells clients where to
find it. Every port the two processes listen on or dial is listed in
network-ports.md.
That gunicorn line is the demo's shape, not a recommendation: it
serves plain HTTP on every interface, and the API carries bearer
tokens and Keystone credentials. A real deployment binds it to an
interface you have chosen and puts TLS in front of it — a reverse
proxy, or gunicorn's own --certfile and --keyfile. The proxy leg's
TLS settings below do not cover the API leg; they are separate.
A database. MySQL or MariaDB, reached via SQL_URL. Create the
database and its user yourself, then let Kerbside create its own
schema:
Run that before starting either process, and again after every
upgrade. It is idempotent, so a service unit or container entrypoint
can call it unconditionally. It reads SQL_URL from the configuration
described below, so write that first: with no configuration at all,
SQL_URL still has a Kolla-shaped placeholder default and the command
fails with Unknown server host 'kolla', which is a missing config
file rather than a database problem.
TLS material for the proxy leg. Clients reach the proxy over TLS
and verify it against a CA you provide. Four settings carry this:
CACERT_PATH, PROXY_HOST_CERT_PATH, PROXY_HOST_CERT_KEY_PATH and
PROXY_HOST_SUBJECT. The last is the trap — it is the subject string
clients are told to expect, so it must match the subject of the
certificate the proxy actually presents. When those disagree, the
viewer refuses the connection and the error does not explain itself.
A console source. Kerbside proxies consoles it has discovered, so
it needs at least one source in the file named by SOURCES_PATH,
conventionally /etc/kerbside/sources.yaml. The drivers are Shaken
Fist, oVirt, OpenStack, Proxmox VE, and static for fixed targets — see
console-sources.md.
The minimum configuration set. Configuration is an INI file at
/etc/kerbside/kerbside.ini; that path is hardcoded, and environment
variables prefixed with KERBSIDE_ override it. These eight settings
are the ones with no useful default, and they are exactly the ones
left live in
etc/kerbside.conf.example,
which documents every setting Kerbside has:
| Setting | What it is |
|---|---|
sql_url |
The database, as a SQLAlchemy URL |
auth_secret_seed |
The seed session tokens are signed with; generate a fresh one per deployment with openssl rand -hex 32. The API refuses to start until it is set |
sources_path |
Where sources.yaml lives |
public_fqdn |
The name clients should reach this Kerbside by |
cacert_path |
The CA clients verify the proxy against |
proxy_host_cert_path |
The proxy's certificate |
proxy_host_cert_key_path |
Its key |
proxy_host_subject |
The subject clients are told to expect |
Try it: the demo stack¶
demo/ in this repository brings all of that up in three containers,
so you can watch a proxied console work before deciding how to deploy
one. It needs nothing but Docker Engine 23.0 or newer with the Compose
v2 plugin — no Python, no database, and no certificates of your own.
On Debian 12, install Docker from
Docker's own repository
rather than from the distribution: bookworm's engine is too old to build
the image, and its only compose is the end-of-life Python v1. Debian 13
packages both well enough (Engine 26.1.5, Compose 2.26.1 as a
docker compose plugin), but Docker's repository is what Docker's own
instructions assume and what kerbside's CI installs.
pip install does not give you demo/, so start from a checkout:
One thing to know about that pairing: the clone lands on develop,
but the image installs the most recent release from PyPI, so the
demo's glue comes from the checkout while Kerbside itself does not.
That is deliberate — what you evaluate is what you can install — and
CI does not exercise it, because the lane builds from the checkout
instead. If you have changed anything under demo/, or want a
stack that matches the tree you are sitting in, build it that way:
KERBSIDE_SOURCE=/src docker compose build kerbside, which needs an
ordinary clone rather than a git worktree.
demo/README.md
covers both forms.
The first run builds the image and takes a few minutes; afterwards the stack comes up in seconds. When it returns, three containers are running — MariaDB, a disk-less qemu with a SPICE server, and Kerbside with both of its processes:
$ docker compose ps --format 'table {{.Service}}\t{{.Status}}'
SERVICE STATUS
db Up 7 minutes (healthy)
kerbside Up 7 minutes (healthy)
spice-target Up 7 minutes
Now mint a token and fetch a console file:
The script mints a bearer token, finds the demo console, writes
demo-console.vv, and then proves the TLS leg rather than assuming
it: it connects to the port the .vv advertises and verifies the
presented certificate against the CA embedded in that same file.
[demo] Waiting for the stack to finish starting...
[demo] Minting a bearer token...
WARNING: this is a demonstration token, minted directly from
AUTH_SECRET_SEED because kerbside has no non-Keystone login (issue
#300). Do not use this pattern in production.
Token written to /tmp/demo-token.169928
[demo] Waiting for the console list...
[demo] Looking up the console...
[demo] Console: demo-console (demo/0d3f6a52-0000-0000-0000-0000000dec01)
[demo] Fetching the .vv file...
[demo] Verifying the TLS leg...
TLSv1.3, certificate verified against the CA in the .vv
subject matches host-subject=C=US,O=Kerbside CI,CN=kerbside-ci
[demo] Wrote ./demo-console.vv
(The script then repeats the remote-viewer command and the
expectation below, which is the next step here.)
Then open it. remote-viewer ships in the virt-viewer package
(sudo apt-get install virt-viewer, or sudo dnf install
virt-viewer), and it is the only thing you need on the client side:
The viewer deletes demo-console.vv once it has read it. The file
sets delete-this-file=1 because the ticket inside it is single-use,
and remote-viewer and ryll both honour that. To open another
session, run ./get-console.sh again.
Expect a black screen with boot firmware text, and expect the boot to fail. The demo VM has no disk, so it tries the network, then a floppy, then gives up:
iPXE (PCI 00:02.0) starting execution...ok
net0: 52:54:00:12:34:56 using 82574l on 0000:00:02.0 (Ethernet) [open]
Nothing to boot: No such file or directory
Booting from Floppy...
Boot failed: could not read the boot disk
No bootable device.
That screen is the SPICE session: it is being drawn by a real qemu and relayed through Kerbside over TLS. The viewer's title bar names the proxy session it is attached to. If you want to confirm the traffic took the TLS port rather than the plaintext one, count established sockets while the viewer is open — they should all be on 5900, and none on 5901:
ss -H -tn state established '( dport = :5900 or sport = :5900 )' | wc -l
ss -H -tn state established '( dport = :5901 or sport = :5901 )' | wc -l
On a live session that prints 12 and 0. The -H matters: without
it ss emits a header line and both counts come back one too high,
which makes the plaintext port look used when it is not.
When you are finished:
That removes the containers, the database, and the generated CA and signing seed — both of which are created on first start rather than baked into the image.
Where the token comes from¶
get-console.sh runs kerbside demo token inside the container, and
that command prints a warning every time it succeeds. The warning is
worth reading: the command exists because interactive login is
Keystone-only today
(#300), so a
static deployment gives a human no way to log in at all. It stands in
for authentication in a demonstration; it is not a preview of how your
users will get consoles.
It also refuses to mint unless every configured source is static,
and refuses while the signing seed is unconfigured. Those refusals are
deliberate rather than configuration problems to work around — in a
real deployment a broker mints session tokens through the API. If
kerbside demo token refuses on the demo stack, something else is
genuinely wrong.
What the demo is not¶
| Limitation | Detail |
|---|---|
| No real authentication | Login is Keystone-only (#300), and the session JWT scheme has no revocation or issuance audit (#301). The web UI on http://127.0.0.1:13002 cannot be logged into |
| A self-signed CA | Generated into a volume on first start, and destroyed by down -v |
| Loopback only | Every published port binds to 127.0.0.1 on purpose. Widening it puts an unauthenticated console proxy on your network |
| Two processes in one container | The shortest way to a working demo, not a shape to copy. Real deployments run the daemon and its proxy on one host, and the API may run elsewhere |
| A plaintext backend leg | Kerbside talks plaintext SPICE to the qemu container. Real deployments should pin the backend too; ovirt.md shows what that looks like |
| A static source, not a cloud | The static driver points at a fixed target, so nothing is being discovered from a real cloud |
demo/README.md is the reference for the stack itself: what each container is, how to build it against a checkout rather than the released package, and why each of these choices was made.
Deploying for real¶
The default worth reaching for is a Kerbside deployed in front of the clouds rather than by one of them, so that one Kerbside can broker consoles from several sources and sit wherever its users are.
There is a page for each way of deploying it: by hand, with pip, a venv and systemd, with the Ansible role, with Kolla-Ansible from kerbside-patches, which deploys it inside one OpenStack cloud rather than in front of it, or in containers. Deployment Options compares them: which sources each configures, and what Kerbside's CI tests.
What each cloud needs from Kerbside, and what Kerbside needs from it, is in that cloud's page under Use Cases.
However you deploy it, the pieces described in What a running Kerbside needs are the same, and two references apply throughout: configuration.md for every setting, and console-sources.md for configuring sources.
Upgrading¶
kerbside db upgrade is the whole of a normal upgrade — see
What a running Kerbside needs. This
section records the upgrades which need something from you as well.
Upgrading past the unconfigured signing seed guard¶
In Kerbside v0.6.0 and earlier, auth_secret_seed defaulted to the
sentinel ~~unconfigured~~ and nothing rejected it. A deployment that
never set it signed session tokens with a constant published in the
Kerbside source tree, so anyone could forge a token for any user
(issue #131). The API now refuses to start while the seed is that
sentinel or blank.
If your deployment never set it, the upgrade will look like a failed
boot: gunicorn reports Worker failed to boot and exits, and the log
carries Refusing to start: AUTH_SECRET_SEED is unset. Generate a
seed with openssl rand -hex 32, set it as auth_secret_seed in
/etc/kerbside/kerbside.ini or as KERBSIDE_AUTH_SECRET_SEED, and
start the API again. Kolla-Ansible deployments built from
kerbside-patches are not affected: they render the seed from
passwords.yml, which kolla-genpwd fills.
Setting the seed invalidates every token signed with the old one, forged or not, so users will need to log in again.
Upgrading past the source credential disclosure¶
In Kerbside v0.6.0 and earlier, GET /source/<name> returned the
source record verbatim, including the cleartext password Kerbside uses
to authenticate to the backend cloud, to any holder of a valid API
token. The .vv handler for direct connections separately wrote the
same credential into the daemon log on every request, along with the
console's SPICE ticket.
A static source's ticket was exposed more widely still: console discovery logged it on every maintenance pass, so it reached the daemon log roughly once a minute whether or not anybody ever requested a console. Do not assume a log is clean because nothing connected to it.
Those are fixed in code, but code cannot reach backwards.
The first question is usually "was I affected?", and the honest answer
is that you cannot tell. Kerbside writes no audit event when a source
is read, so there is no record of who called GET /source/<name> or
when. The audit trail can neither confirm nor exclude access, which is
why the guidance below is to rotate rather than to investigate:
rotation is the only action available that actually closes the
exposure.
If you ran v0.6.0 or earlier:
- Rotate the credentials in
sources.yaml— the oVirt, OpenStack or Shaken Fist service account password for every configured source — and restart the daemon so it picks them up. Anyone who held an API token, and anyone who could read the daemon's log, could have read the old ones. - Rotate the SPICE passwords of any static sources, in
sources.yamland on the SPICE servers themselves. A static source's ticket is a console password you configured rather than one minted per request, so unlike an oVirt ticket it does not expire on its own. oVirt, OpenStack and Shaken Fist tickets need no action: oVirt tickets are minted for each session and last no longer than its console token plus 30 seconds, and the others are short lived. - Scrub or expire the daemon logs from before the upgrade, including anywhere they were shipped: a log aggregator, a backup, a support bundle, or CI artifacts from a deployment lane.
The credentials being rotated are management plane accounts on the cloud Kerbside proxies for, so they are worth treating as a real exposure rather than a formality.
Upgrading past admin-only direct consoles¶
Earlier releases let any logged in user download a direct .vv file
for any console from the consoles page's Direct button. That file
points the client straight at the hypervisor with the hypervisor's own
SPICE ticket, so it bypasses the proxy's firewall, session tracking and
termination entirely.
Direct console access is now restricted to members of the Keystone
group named by KEYSTONE_ADMIN_GROUP, which defaults to empty. After
upgrading, nobody can use direct access until you set it. If you rely on
direct .vv files, create or choose a Keystone group for the users who
should have them and set KEYSTONE_ADMIN_GROUP to its name; everyone
else keeps using the proxied download. Membership is read at login, so
a user gains or loses direct access when their current session expires
(API_TOKEN_DURATION) and they log in again. See
the configuration reference.
Upgrading past connect-time oVirt tickets¶
Earlier releases minted an oVirt graphics-console ticket when a .vv
file was downloaded and stored it on the console, for the engine's
two-hour default. Kerbside now mints it when a session's first channel
connects, with a lifetime of the console token's remaining life plus
30 seconds, and holds it on the session's token row. kerbside db
upgrade adds the two columns for that and clears the oVirt tickets
stored on consoles, since nothing reads them any more.
The proxy's authorization call is what mints, so a daemon from before
this change cannot authorize an oVirt channel against a database
upgraded to it: it would present the cleared, empty ticket, and the
proxy would log hypervisor connection failed with PermissionDenied
for each oVirt channel. Upgrade every node's daemon together, not one
at a time. Kerbside also now
needs the oVirt engine to be reachable when a session connects, not
only for discovery; see Console Sources.