Instances (/instances/)
Instances sit at the core of Shaken Fist's functionality, and are the component which ties most of the other concepts in the API together. Therefore, they are also the most complicated part of Shaken Fist to explain. This description is broken into basic functionality -- showing information about instances -- and then moves onto more advanced topics like creation, deletion, and other lifecycle events.
Fetching information about an instance
There are two main ways to fetch information about instances -- you can list all instances visible to your authenticated namespace, or you can collect information about a specific instance by providing its UUID or name.
Info
Note that the amount of information visible in an instance response will change over the lifecycle of the instance -- for example when you first request the instance be created versus when the instance has had its disk specification calculated.
REST API calls
- GET /instances: List all instances visible to your authenticated namespace.
- GET /instances/{instance_ref}: Get information about a specific instance.
Python API client: get all visible instances
import json
from shakenfist_client import apiclient
sf_client = apiclient.Client()
instances = sf_client.get_instances()
print(json.dumps(instances, indent=4, sort_keys=True))
Python API client: get information about a specific instance
import json
from shakenfist_client import apiclient
sf_client = apiclient.Client()
i = sf_client.get_instance('317e9b70-8e26-46af-a1c4-76931c0da5a9')
print(json.dumps(i, indent=4, sort_keys=True))
Instance creation
Instance creation is by far the most complicated call in Shaken Fist in terms of the arguments that it takes. The code in the Python command line client is helpful if you need a fully worked example of every possible permutation. The OpenAPI documentation at https://sfcbr.shakenfist.com/api/apidocs/#/instances/post_instances provides comprehensive and up to date documentation on all the arguments to the creation call.
The instance creation API call also takes three data structures: the diskspec
;
the networkspec
; and the videospec
. These structures are not well documented
in the OpenAPI interface, so are documented here instead.
diskspec
A diskspec
consists of the following fields as a JSON dictionary:
- size (integer): the size of the disk in gigabytes.
- base (string): the base image for the disk. This can be a variety of URL-like strings, as documented on the artifacts page in the user guide. For a blank disk, omit this value.
- bus (enum): the hardware bus the disk device should be attached to on the instance. In general you shouldn't care about this and can omit this value. However, in some cases, such as unmodified Microsoft Windows images it is required. The options available here are: ide; sata; scsi; usb; virtio (the default); and nvme.
- type (enum): the type of device. The default is "disk", but in some cases you might want "cdrom".
A full example of a diskspec
is therefore:
{
'size': 20,
'base': 'debian:11',
'bus': None,
'type': None
}
networkspec
Similarly, a networkspec
consists of the following fields in a JSON dictionary:
- network_uuid (uuid): the UUID of the network the interface should exist on.
- macaddress (string): the MAC address of the interface. Omit this value to be allocated a MAC address automatically.
- address (string): the IPv4 address to assign to the interface. Omit this value to be allocated a random address.
- model (enum): the model of the network interface card. In general you should not have to set this, although it can matter in some cases, such as unmodified Microsoft Windows images. The options include: i82551; i82557b; i82559er; ne2k_pci; pcnet; rtl8139; e1000; and virtio (the default).
- float (boolean): whether to associate a floating IP with this interface to enable external accessibility to the instance. Note that you can float and unfloat an interface after instance creation if desired.
videospec
A videospec
differs from a diskspec
and a networkspec
in that it is not
passed as a list. You only have one videospec
per instance. Once again, a
videospec
is a JSON dictionary with the following fields:
- model (enum): the model of the video card to attach to the instance. Possible options include: vga; cirrus (the default); and qxl.
- memory (integer): the amount of video RAM the video card should have, in kibibytes (blocks of 1024 bytes).
REST API calls
- POST /instances/: Create an instance.
Python API client: create and then delete a simple instance
from shakenfist_client import apiclient
import time
sf_client = apiclient.Client()
i = sf_client.create_instance(
'example', 1, 1024, None,
[{
'size': 20,
'base': 'debian:11',
'bus': None,
'type': 'disk'
}],
None, None)
time.sleep(30)
i = sf_client.delete_instance(i['uuid'])
Other instance lifecycle operations
A variety of other lifecycle operations are available on instances, including deletion, and power management.
The power management actions available are:
- soft reboot: gracefully request a reboot from the instance operating system via ACPI. This is not guaranteed to actually work, but if it does is much less likely to cause disk corruption on the instance.
- hard reboot: the equivalent of holding the reset switch down on a physical machine until it reboots without operating system involvement.
- power on: turn the instance on, as if the power switch was pressed.
- power off: turn the instance immediately off, as if the power switch was held down on a physical machine.
- pause: suspend execution of the instance, but leave it hot in RAM ready to restart.
- unpause: unsuspend execution of the instance.
REST API calls
- DELETE /instances/{instance_ref}: Delete an instance.
- DELETE /instances/: Delete all instances in a namespace.
- POST /instances/{instance_ref}/rebootsoft: Soft (ACPI) reboot the instance.
- POST /instances/{instance_ref}/reboothard: Hard (reset switch) reboot the instance.
- POST /instances/{instance_ref}/poweron: Power the instance on.
- POST /instances/{instance_ref}/poweroff: Power the instance off, as if holding the power switch down.
- POST /instances/{instance_ref}/pause: Pause an instance.
- POST /instances/{instance_ref}/unpause: Unpause an instance.
Python API client: create and then delete a simple instance
from shakenfist_client import apiclient
import time
sf_client = apiclient.Client()
i = sf_client.create_instance(
'example', 1, 1024, None,
[{
'size': 20,
'base': 'debian:11',
'bus': None,
'type': 'disk'
}],
None, None)
time.sleep(30)
i = sf_client.delete_instance(i['uuid'])
Python API client: attempt a soft reboot, and hard reboot if required
Note that this example assumes the instance is running an image with the Shaken Fist in guest agent installed.
import time
from shakenfist_client import apiclient
import sys
sf_client = apiclient.Client()
i = sf_client.create_instance(
'example', 1, 1024, None,
[{
'size': 20,
'base': 'debian:11',
'bus': None,
'type': 'disk'
}],
None, None, side_channels=['sf-agent'])
# Wait for the instance to be created, or error out. Use instance events to
# provide status updates during boot.
while i['state'] not in ['created', 'error']:
events = sf_client.get_instance_events(i['uuid'])
print('Waiting for the instance to start: %s' % events[0]['message'])
time.sleep(5)
i = sf_client.get_instance(i['uuid'])
# Check the instance is created correctly
if i['state'] != 'created':
print('Instance is not in a created state!')
sys.exit(1)
print('Instance is created')
# Wait for the agent to report the reboot time
while not i['agent_system_boot_time']:
print('Waiting for agent to start: %s' % i['agent_state'])
time.sleep(20)
i = sf_client.get_instance(i['uuid'])
initial_boot = i['agent_system_boot_time']
print('Instance booted at %d' % initial_boot)
# Now try to soft reboot the instance, wait up to 60 seconds for a reboot to
# be detected
sf_client.reboot_instance(i['uuid'], hard=False)
print('Soft rebooting instance')
time.sleep(60)
i = sf_client.get_instance(i['uuid'])
# Wait for the agent to report the reboot time again
while not i['agent_system_boot_time']:
print('Waiting for agent to start: %s' % i['agent_state'])
time.sleep(20)
i = sf_client.get_instance(i['uuid'])
if i['agent_system_boot_time'] != initial_boot:
print('Boot time changed from %d to %s'
% (initial_boot, i['agent_system_boot_time']))
else:
# We failed to soft reboot, let's hard reboot instead
sf_client.reboot_instance(i['uuid'], hard=True)
print('Instance did not reboot, hard rebooting')
Sample output:
$ python3 example.py
Waiting for the instance to start: schedule complete
Instance is created
Waiting for agent to start: not ready (no contact)
Waiting for agent to start: not ready (no contact)
Waiting for agent to start: not ready (no contact)
Instance booted at 1684404969
Soft rebooting instance
Boot time changed from 1684404969 to 1684405036.0
Python API client: power off and then on an instance
import time
from shakenfist_client import apiclient
import sys
sf_client = apiclient.Client()
i = sf_client.create_instance(
'example', 1, 1024, None,
[{
'size': 20,
'base': 'debian:11',
'bus': None,
'type': 'disk'
}],
None, None)
# Wait for the instance to be created, or error out. Use instance events to
# provide status updates during boot.
while i['state'] not in ['created', 'error']:
events = sf_client.get_instance_events(i['uuid'])
print('Waiting for the instance to start: %s' % events[0]['message'])
time.sleep(5)
i = sf_client.get_instance(i['uuid'])
# Check the instance is created correctly
if i['state'] != 'created':
print('Instance is not in a created state!')
sys.exit(1)
print('Instance is created')
# Check the instance is created correctly
if i['power_state'] != 'on':
print('Instance is not in powered on state!')
sys.exit(1)
# Power the instance off
sf_client.power_off_instance(i['uuid'])
while i['power_state'] != 'off':
print('Waiting for the instance to power off')
time.sleep(5)
i = sf_client.get_instance(i['uuid'])
time.sleep(30)
# Power the instance on
sf_client.power_on_instance(i['uuid'])
while i['power_state'] != 'on':
print('Waiting for the instance to power on')
time.sleep(5)
i = sf_client.get_instance(i['uuid'])
print('Done')
Waiting for the instance to start: set attribute
Instance is created
Waiting for the instance to power off
Waiting for the instance to power on
Done
Python API client: pause and unpause an instance
import time
from shakenfist_client import apiclient
sf_client = apiclient.Client()
sf_client.pause_instance('foo')
time.sleep(30)
sf_client.unpause_instance('foo')
Other instance information
We can also request other information for an instance. For example, we can list the instance's network interfaces, or the events for the instance. See the user guide for a general introduction to the Shaken Fist event system.
REST API calls
- GET /instances/{instance_ref}/interfaces: Request information on the instance's network interfaces, if any.
- GET /instances/{instance_ref}/events: Fetch events for a specific instance.
Python API client: list network interfaces for an instance
Note that the interface details for an instance wont be populated until the instance has started being created on the hypervisor node. Specifically, this can be some time later if an image needs to be fetched from the Internet and transcoded. Therefore in this example we wait for the instance to be created before displaying interface details.
import json
from shakenfist_client import apiclient
import time
sf_client = apiclient.Client()
i = sf_client.create_instance(
'example', 1, 1024, None,
[{
'size': 20,
'base': 'debian:11',
'bus': None,
'type': 'disk'
}],
None, None)
# Wait for the instance to be created, or error out. Use instance events to
# provide status updates during boot.
while i['state'] not in ['created', 'error']:
events = sf_client.get_instance_events(i['uuid'])
print('Waiting for the instance to start: %s' % events[0]['message'])
time.sleep(5)
i = sf_client.get_instance(i['uuid'])
# Check the instance is created correctly
if i['state'] != 'created':
print('Instance is not in a created state!')
sys.exit(1)
print('Instance is created')
# Fetch and display interface details
ifaces = sf_client.get_instance_interfaces(i['uuid'])[0]
print(json.dumps(ifaces, indent=4, sort_keys=True))
$ python3 example.py
Waiting for the instance to start: Fetching required blob ffdfce7f-728e-4b76-83c2-304e252f98b1, 30% complete
Instance is created
[
{
"floating": null,
"instance_uuid": "d512e9f5-98d6-4c36-8520-33b6fc6de15f",
"ipv4": "10.0.0.6",
"macaddr": "02:00:00:73:18:66",
"metadata": {},
"model": "virtio",
"network_uuid": "6aaaf243-0406-41a1-aa13-5d79a0b8672d",
"order": 0,
"state": "created",
"uuid": "b1981e81-b37a-4176-ba37-b61bc7208012",
"version": 3
}
]
Python API client: list events for an instance
import json
from shakenfist_client import apiclient
sf_client = apiclient.Client()
interfaces = sf_client.get_instance_events('c0d52a77-0f8a-4f19-bec7-0c05efb03cb4')
print(json.dumps(interfaces, indent=4, sort_keys=True))
Note that events are returned in reverse chronological order and are limited to the 100 most recent events.
$python3 example.py
[
...
{
"duration": null,
"extra": {
"cpu usage": {
"cpu time ns": 357485828000,
"system time ns": 66297716000,
"user time ns": 291188112000
},
"disk usage": {
"vda": {
"actual bytes on disk": 956301312,
"errors": -1,
"read bytes": 406776320,
"read requests": 12225,
"write bytes": 2105954304,
"write requests": 3657
},
"vdb": {
"actual bytes on disk": 102400,
"errors": -1,
"read bytes": 279552,
"read requests": 74,
"write bytes": 0,
"write requests": 0
}
},
"network usage": {
"02:00:00:1d:24:ae": {
"read bytes": 147084732,
"read drops": 0,
"read errors": 0,
"read packets": 16484,
"write bytes": 2166754,
"write drops": 0,
"write errors": 0,
"write packets": 13144
}
}
},
"fqdn": "sf-2",
"message": "usage",
"timestamp": 1685229509.9592097,
"type": "usage"
},
...
]
Out-of-band interactions with instances
Shaken Fist supports three types of instance consoles, which provide out-of-band management of instances -- that is, the instance does not need to have functioning networking for these consoles to work. You can read a general introduction of Shaken Fist's console functionality in the user guide. This page focuses on the API calls which are used to implement the console functionality in the Shaken Fist client.
- Read only console: to download the most recent portion of the read only text
serial console, or clear the console, use the
/instances/{instance_ref}/consoledata
API calls below. - Interactive serial console: lookup the console port from the instance details fetch (as described above), and then connect to that port on the hypervisor node with a TCP client such as telnet.
- Interactive VDI console: lookup the VDI console port from the instance details
fetch (as described above), and then connect to that port on the hypervisor
with the correct client (currently one of VNC or SPICE). Alternatively, use
the
/instances/{instance_ref}/vdiconsolehelper
API call described below to download avirt-viewer
configuration file and then connect withvirt-viewer
. See the example below for more details.
REST API calls
- GET /instances/{instance_ref}/consoledata: Fetch read only serial console data for an instance
- DELETE /instances/{instance_ref}/consoledata: Clear the read only serial console for an instance.
- GET /instances/{instance_ref}/vdiconsolehelper: Generate and return a
virt-viewer
configuration file for connecting to the interactive VDI console for the instance (if configured).
Python API client: connect seamlessly to a VDI console using virt-viewer
import os
from shakenfist_client import apiclient
import subprocess
import tempfile
import time
sf_client = apiclient.Client()
i = sf_client.create_instance(
'example', 1, 1024, None,
[{
'size': 20,
'base': 'debian:11',
'bus': None,
'type': 'disk'
}],
None, None)
# Wait for the instance to be created, or error out. Use instance events to
# provide status updates during boot.
while i['state'] not in ['created', 'error']:
events = sf_client.get_instance_events(i['uuid'])
print('Waiting for the instance to start: %s' % events[0]['message'])
time.sleep(5)
i = sf_client.get_instance(i['uuid'])
# Check the instance is created correctly
if i['state'] != 'created':
print('Instance is not in a created state!')
sys.exit(1)
print('Instance is created')
# We don't use NamedTemporaryFile as a context manager as the .vv file
# will also attempt to clean up the file.
(temp_handle, temp_name) = tempfile.mkstemp()
os.close(temp_handle)
try:
with open(temp_name, 'w') as f:
f.write(sf_client.get_vdi_console_helper(i['uuid']))
p = subprocess.run('remote-viewer %s' % temp_name, shell=True)
print('Remote viewer process exited with %d return code' % p.returncode)
finally:
if os.path.exists(temp_name):
os.unlink(temp_name)
Metadata
All objects exposed by the REST API may have metadata associated with them. This metadata is for storing values that are of interest to the owner of the resources, not Shaken Fist. Shaken Fist does not attempt to interpret these values at all, with the exception of the instance affinity metadata values. The metadata store is in the form of a key value store, and a general introduction is available in the user guide.
Info
Note that for affinity metadata to be processed by the scheduler, it must be present in the instance create API call, which is why that call takes a metadata argument. Adding affinity metadata after instance creation will not affect the placement of that instance, but would affect the placement of future instances.
REST API calls
- GET /instances/{instance_ref}/metadata: Get metadata for an instance.
- POST /instances/{instance_ref}/metadata: Create a new metadata key for an instance.
- DELETE /instances/{instance_ref}/metadata/{key}: Delete a specific metadata key for an instance.
- PUT /instances/{instance_ref}/metadata/{key}: Update an existing metadata key for an instance.
Python API client: set metadata on an instance
from shakenfist_client import apiclient
sf_client = apiclient.Client()
sf_client.set_instance_metadata_item(instance_uuid, 'foo', 'bar')
Python API client: get metadata for an instance
import json
from shakenfist_client import apiclient
sf_client = apiclient.Client()
md = sf_client.get_instance_metadata(instance_uuid)
print(json.dumps(md, indent=4, sort_keys=True))
Python API client: delete metadata for an instance
import json
from shakenfist_client import apiclient
sf_client = apiclient.Client()
sf_client.delete_instance_metadata_item(instance_uuid, 'foo')