Skip to content

Instances (/instances/)

Instances sit at the core of Shaken Fist's functionality, and are the component which ties most of the other concepts in the API together. Therefore, they are also the most complicated part of Shaken Fist to explain. This description is broken into basic functionality -- showing information about instances -- and then moves onto more advanced topics like creation, deletion, and other lifecycle events.

Fetching information about an instance

There are two main ways to fetch information about instances -- you can list all instances visible to your authenticated namespace, or you can collect information about a specific instance by providing its UUID or name.

Info

Note that the amount of information visible in an instance response will change over the lifecycle of the instance -- for example when you first request the instance be created versus when the instance has had its disk specification calculated.

REST API calls
Python API client: get all visible instances 
import json
from shakenfist_client import apiclient

sf_client = apiclient.Client()
instances = sf_client.get_instances()
print(json.dumps(instances, indent=4, sort_keys=True))
Python API client: get information about a specific instance 
import json
from shakenfist_client import apiclient

sf_client = apiclient.Client()
i = sf_client.get_instance('317e9b70-8e26-46af-a1c4-76931c0da5a9')
print(json.dumps(i, indent=4, sort_keys=True))

Instance creation

Instance creation is by far the most complicated call in Shaken Fist in terms of the arguments that it takes. The code in the Python command line client is helpful if you need a fully worked example of every possible permutation. The OpenAPI documentation at https://sfcbr.shakenfist.com/api/apidocs/#/instances/post_instances provides comprehensive and up to date documentation on all the arguments to the creation call.

The instance creation API call also takes three data structures: the diskspec; the networkspec; and the videospec. These structures are not well documented in the OpenAPI interface, so are documented here instead.

diskspec

A diskspec consists of the following fields as a JSON dictionary:

  • size (integer): the size of the disk in gigabytes.
  • base (string): the base image for the disk. This can be a variety of URL-like strings, as documented on the artifacts page in the user guide. For a blank disk, omit this value.
  • bus (enum): the hardware bus the disk device should be attached to on the instance. In general you shouldn't care about this and can omit this value. However, in some cases, such as unmodified Microsoft Windows images it is required. The options available here are: ide; sata; scsi; usb; virtio (the default); and nvme.
  • type (enum): the type of device. The default is "disk", but in some cases you might want "cdrom".

A full example of a diskspec is therefore:

{
    'size': 20,
    'base': 'debian:11',
    'bus': None,
    'type': None
}

networkspec

Similarly, a networkspec consists of the following fields in a JSON dictionary:

  • network_uuid (uuid): the UUID of the network the interface should exist on.
  • macaddress (string): the MAC address of the interface. Omit this value to be allocated a MAC address automatically.
  • address (string): the IPv4 address to assign to the interface. Omit this value to be allocated a random address.
  • model (enum): the model of the network interface card. In general you should not have to set this, although it can matter in some cases, such as unmodified Microsoft Windows images. The options include: i82551; i82557b; i82559er; ne2k_pci; pcnet; rtl8139; e1000; and virtio (the default).
  • float (boolean): whether to associate a floating IP with this interface to enable external accessibility to the instance. Note that you can float and unfloat an interface after instance creation if desired.

videospec

A videospec differs from a diskspec and a networkspec in that it is not passed as a list. You only have one videospec per instance. Once again, a videospec is a JSON dictionary with the following fields:

  • model (enum): the model of the video card to attach to the instance. Possible options include: vga; cirrus (the default); and qxl.
  • memory (integer): the amount of video RAM the video card should have, in kibibytes (blocks of 1024 bytes).
REST API calls
Python API client: create and then delete a simple instance 
from shakenfist_client import apiclient
import time

sf_client = apiclient.Client()
i = sf_client.create_instance(
    'example', 1, 1024, None,
    [{
        'size': 20,
        'base': 'debian:11',
        'bus': None,
        'type': 'disk'
    }],
    None, None)

time.sleep(30)

i = sf_client.delete_instance(i['uuid'])

Other instance lifecycle operations

A variety of other lifecycle operations are available on instances, including deletion, and power management.

The power management actions available are:

  • soft reboot: gracefully request a reboot from the instance operating system via ACPI. This is not guaranteed to actually work, but if it does is much less likely to cause disk corruption on the instance.
  • hard reboot: the equivalent of holding the reset switch down on a physical machine until it reboots without operating system involvement.
  • power on: turn the instance on, as if the power switch was pressed.
  • power off: turn the instance immediately off, as if the power switch was held down on a physical machine.
  • pause: suspend execution of the instance, but leave it hot in RAM ready to restart.
  • unpause: unsuspend execution of the instance.
REST API calls
Python API client: create and then delete a simple instance 
from shakenfist_client import apiclient
import time

sf_client = apiclient.Client()
i = sf_client.create_instance(
    'example', 1, 1024, None,
    [{
        'size': 20,
        'base': 'debian:11',
        'bus': None,
        'type': 'disk'
    }],
    None, None)

time.sleep(30)

i = sf_client.delete_instance(i['uuid'])
Python API client: attempt a soft reboot, and hard reboot if required

Note that this example assumes the instance is running an image with the Shaken Fist in guest agent installed.

import time
from shakenfist_client import apiclient
import sys

sf_client = apiclient.Client()
i = sf_client.create_instance(
    'example', 1, 1024, None,
    [{
        'size': 20,
        'base': 'debian:11',
        'bus': None,
        'type': 'disk'
    }],
    None, None, side_channels=['sf-agent'])

# Wait for the instance to be created, or error out. Use instance events to
# provide status updates during boot.
while i['state'] not in ['created', 'error']:
    events = sf_client.get_instance_events(i['uuid'])
    print('Waiting for the instance to start: %s' % events[0]['message'])
    time.sleep(5)
    i = sf_client.get_instance(i['uuid'])

# Check the instance is created correctly
if i['state'] != 'created':
    print('Instance is not in a created state!')
    sys.exit(1)
print('Instance is created')

# Wait for the agent to report the reboot time
while not i['agent_system_boot_time']:
    print('Waiting for agent to start: %s' % i['agent_state'])
    time.sleep(20)
    i = sf_client.get_instance(i['uuid'])

initial_boot = i['agent_system_boot_time']
print('Instance booted at %d' % initial_boot)

# Now try to soft reboot the instance, wait up to 60 seconds for a reboot to
# be detected
sf_client.reboot_instance(i['uuid'], hard=False)
print('Soft rebooting instance')
time.sleep(60)
i = sf_client.get_instance(i['uuid'])

# Wait for the agent to report the reboot time again
while not i['agent_system_boot_time']:
    print('Waiting for agent to start: %s' % i['agent_state'])
    time.sleep(20)
    i = sf_client.get_instance(i['uuid'])

if i['agent_system_boot_time'] != initial_boot:
    print('Boot time changed from %d to %s'
        % (initial_boot, i['agent_system_boot_time']))

else:
    # We failed to soft reboot, let's hard reboot instead
    sf_client.reboot_instance(i['uuid'], hard=True)
    print('Instance did not reboot, hard rebooting')

Sample output:

$ python3 example.py
Waiting for the instance to start: schedule complete
Instance is created
Waiting for agent to start: not ready (no contact)
Waiting for agent to start: not ready (no contact)
Waiting for agent to start: not ready (no contact)
Instance booted at 1684404969
Soft rebooting instance
Boot time changed from 1684404969 to 1684405036.0
Python API client: power off and then on an instance 
import time
from shakenfist_client import apiclient
import sys

sf_client = apiclient.Client()
i = sf_client.create_instance(
        'example', 1, 1024, None,
        [{
            'size': 20,
            'base': 'debian:11',
            'bus': None,
            'type': 'disk'
        }],
        None, None)

# Wait for the instance to be created, or error out. Use instance events to
# provide status updates during boot.
while i['state'] not in ['created', 'error']:
    events = sf_client.get_instance_events(i['uuid'])
    print('Waiting for the instance to start: %s' % events[0]['message'])
    time.sleep(5)
    i = sf_client.get_instance(i['uuid'])

# Check the instance is created correctly
if i['state'] != 'created':
    print('Instance is not in a created state!')
    sys.exit(1)
print('Instance is created')

# Check the instance is created correctly
if i['power_state'] != 'on':
    print('Instance is not in powered on state!')
    sys.exit(1)

# Power the instance off
sf_client.power_off_instance(i['uuid'])
while i['power_state'] != 'off':
    print('Waiting for the instance to power off')
    time.sleep(5)
    i = sf_client.get_instance(i['uuid'])

time.sleep(30)

# Power the instance on
sf_client.power_on_instance(i['uuid'])
while i['power_state'] != 'on':
    print('Waiting for the instance to power on')
    time.sleep(5)
    i = sf_client.get_instance(i['uuid'])

print('Done')

Waiting for the instance to start: set attribute
Instance is created
Waiting for the instance to power off
Waiting for the instance to power on
Done
Python API client: pause and unpause an instance 
import time
from shakenfist_client import apiclient

sf_client = apiclient.Client()
sf_client.pause_instance('foo')

time.sleep(30)

sf_client.unpause_instance('foo')

Other instance information

We can also request other information for an instance. For example, we can list the instance's network interfaces, or the events for the instance. See the user guide for a general introduction to the Shaken Fist event system.

REST API calls
Python API client: list network interfaces for an instance

Note that the interface details for an instance wont be populated until the instance has started being created on the hypervisor node. Specifically, this can be some time later if an image needs to be fetched from the Internet and transcoded. Therefore in this example we wait for the instance to be created before displaying interface details.

import json
from shakenfist_client import apiclient
import time

sf_client = apiclient.Client()
i = sf_client.create_instance(
        'example', 1, 1024, None,
        [{
            'size': 20,
            'base': 'debian:11',
            'bus': None,
            'type': 'disk'
        }],
        None, None)

# Wait for the instance to be created, or error out. Use instance events to
# provide status updates during boot.
while i['state'] not in ['created', 'error']:
    events = sf_client.get_instance_events(i['uuid'])
    print('Waiting for the instance to start: %s' % events[0]['message'])
    time.sleep(5)
    i = sf_client.get_instance(i['uuid'])

# Check the instance is created correctly
if i['state'] != 'created':
    print('Instance is not in a created state!')
    sys.exit(1)
print('Instance is created')

# Fetch and display interface details
ifaces = sf_client.get_instance_interfaces(i['uuid'])[0]
print(json.dumps(ifaces, indent=4, sort_keys=True))

$ python3 example.py
Waiting for the instance to start: Fetching required blob ffdfce7f-728e-4b76-83c2-304e252f98b1, 30% complete
Instance is created
[
    {
        "floating": null,
        "instance_uuid": "d512e9f5-98d6-4c36-8520-33b6fc6de15f",
        "ipv4": "10.0.0.6",
        "macaddr": "02:00:00:73:18:66",
        "metadata": {},
        "model": "virtio",
        "network_uuid": "6aaaf243-0406-41a1-aa13-5d79a0b8672d",
        "order": 0,
        "state": "created",
        "uuid": "b1981e81-b37a-4176-ba37-b61bc7208012",
        "version": 3
    }
]
Python API client: list events for an instance 
import json
from shakenfist_client import apiclient

sf_client = apiclient.Client()
interfaces = sf_client.get_instance_events('c0d52a77-0f8a-4f19-bec7-0c05efb03cb4')
print(json.dumps(interfaces, indent=4, sort_keys=True))

Note that events are returned in reverse chronological order and are limited to the 100 most recent events.

$python3 example.py
[
    ...
    {
        "duration": null,
        "extra": {
            "cpu usage": {
                "cpu time ns": 357485828000,
                "system time ns": 66297716000,
                "user time ns": 291188112000
            },
            "disk usage": {
                "vda": {
                    "actual bytes on disk": 956301312,
                    "errors": -1,
                    "read bytes": 406776320,
                    "read requests": 12225,
                    "write bytes": 2105954304,
                    "write requests": 3657
                },
                "vdb": {
                    "actual bytes on disk": 102400,
                    "errors": -1,
                    "read bytes": 279552,
                    "read requests": 74,
                    "write bytes": 0,
                    "write requests": 0
                }
            },
            "network usage": {
                "02:00:00:1d:24:ae": {
                    "read bytes": 147084732,
                    "read drops": 0,
                    "read errors": 0,
                    "read packets": 16484,
                    "write bytes": 2166754,
                    "write drops": 0,
                    "write errors": 0,
                    "write packets": 13144
                }
            }
        },
        "fqdn": "sf-2",
        "message": "usage",
        "timestamp": 1685229509.9592097,
        "type": "usage"
    },
    ...
]

Out-of-band interactions with instances

Shaken Fist supports three types of instance consoles, which provide out-of-band management of instances -- that is, the instance does not need to have functioning networking for these consoles to work. You can read a general introduction of Shaken Fist's console functionality in the user guide. This page focuses on the API calls which are used to implement the console functionality in the Shaken Fist client.

  • Read only console: to download the most recent portion of the read only text serial console, or clear the console, use the /instances/{instance_ref}/consoledata API calls below.
  • Interactive serial console: lookup the console port from the instance details fetch (as described above), and then connect to that port on the hypervisor node with a TCP client such as telnet.
  • Interactive VDI console: lookup the VDI console port from the instance details fetch (as described above), and then connect to that port on the hypervisor with the correct client (currently one of VNC or SPICE). Alternatively, use the /instances/{instance_ref}/vdiconsolehelper API call described below to download a virt-viewer configuration file and then connect with virt-viewer. See the example below for more details.
REST API calls
Python API client: connect seamlessly to a VDI console using virt-viewer 
import os
from shakenfist_client import apiclient
import subprocess
import tempfile
import time

sf_client = apiclient.Client()
i = sf_client.create_instance(
        'example', 1, 1024, None,
        [{
            'size': 20,
            'base': 'debian:11',
            'bus': None,
            'type': 'disk'
        }],
        None, None)

# Wait for the instance to be created, or error out. Use instance events to
# provide status updates during boot.
while i['state'] not in ['created', 'error']:
    events = sf_client.get_instance_events(i['uuid'])
    print('Waiting for the instance to start: %s' % events[0]['message'])
    time.sleep(5)
    i = sf_client.get_instance(i['uuid'])

# Check the instance is created correctly
if i['state'] != 'created':
    print('Instance is not in a created state!')
    sys.exit(1)
print('Instance is created')

# We don't use NamedTemporaryFile as a context manager as the .vv file
# will also attempt to clean up the file.
(temp_handle, temp_name) = tempfile.mkstemp()
os.close(temp_handle)
try:
    with open(temp_name, 'w') as f:
        f.write(sf_client.get_vdi_console_helper(i['uuid']))

    p = subprocess.run('remote-viewer %s' % temp_name, shell=True)
    print('Remote viewer process exited with %d return code' % p.returncode)
finally:
    if os.path.exists(temp_name):
        os.unlink(temp_name)

Metadata

All objects exposed by the REST API may have metadata associated with them. This metadata is for storing values that are of interest to the owner of the resources, not Shaken Fist. Shaken Fist does not attempt to interpret these values at all, with the exception of the instance affinity metadata values. The metadata store is in the form of a key value store, and a general introduction is available in the user guide.

Info

Note that for affinity metadata to be processed by the scheduler, it must be present in the instance create API call, which is why that call takes a metadata argument. Adding affinity metadata after instance creation will not affect the placement of that instance, but would affect the placement of future instances.

REST API calls
Python API client: set metadata on an instance 
from shakenfist_client import apiclient

sf_client = apiclient.Client()
sf_client.set_instance_metadata_item(instance_uuid, 'foo', 'bar')
Python API client: get metadata for an instance 
import json
from shakenfist_client import apiclient

sf_client = apiclient.Client()
md = sf_client.get_instance_metadata(instance_uuid)
print(json.dumps(md, indent=4, sort_keys=True))
Python API client: delete metadata for an instance 
import json
from shakenfist_client import apiclient

sf_client = apiclient.Client()
sf_client.delete_instance_metadata_item(instance_uuid, 'foo')