Drivers

Overview

Provisioning drivers

Provisioning drivers are what makes PB tick. There are several for different backends. Currently it’s possible to:

  • provision OpenStack virtual machines (OpenStackDriver)
  • provision Docker containers from a host pool that consists of OpenStack virtual machines (DockerDriver)
  • provision containers from OpenShift (OpenShiftDriver)

The ones for DockerDriver are prepended DD_* . All drivers require the variable M2M_CREDENTIAL_STORE for credentials to OpenStack/OpenShift. The variable should point to a JSON file that contains

For OpenStackDriver and DockerDriver

  • OS_USERNAME
  • OS_PASSWORD
  • OS_TENANT_NAME
  • OS_AUTH_URL

To a v2 endpoint (v3 support will come eventually).

For OpenShiftDriver:

  • OSD_*_BASE_URL
  • OSD_*_SUBDOMAIN
  • OSD_*_USER
  • OSD_*_PASSWORD

Where * is the name of the OpenShift installation (the system can handle multiple openshifts).

Drivers are implemented as plugins that are loaded dynamically by stevedore .

Base Driver

Drivers abstract resource provisioning strategies to the system and user.

A driver object can be instantiated to connect to some end point to CRUD resources like Docker containers or OpenStack virtual machines.

class pebbles.drivers.provisioning.base_driver.ProvisioningDriverBase(logger, config)[source]

This class functions as the base for other classes.

OpenStack Driver

ToDo: document OpenStack driver on user level here.

class pebbles.drivers.provisioning.openstack_driver.OpenStackDriver(logger, config)[source]

ToDo: document Openstack driver on developer/sysadmin level here.

Docker Driver

DockerDriver maintains a pool of hosts running Docker and starts containers on one host.

For safety reasons and to avoid stuck situations the hosts expire after 4 hours and are respawned after the last container on them has been removed.

An important concept to DD are slots. A slot is roughly 512MB of memory and it is used to assign containers to pool hosts. A pool host has a number of slots and the driver won’t assign any more containers to a host that doesn’t have the slots for it.

The system maintains a number of hosts to reach DD_FREE_SLOT_TARGET unless DD_SHUTDOWN_MODE is True, in which case it waits for all containers on a host to finish and then shuts down the host.

DockerDriver configurations are available via the UI admin dashboard under “Driver Configs”.

Config Description
DD_FREE_SLOT_TARGET Number of “slots” the system should have available
DD_HOST_DATA_VOLUME_DEVICE ToDo
DD_HOST_DATA_VOLUME_FACTOR ToDo
DD_HOST_DATA_VOLUME_TYPE  
DD_HOST_EXTRA_SGS Extra security groups (name) to add to the hosts. Note that security group must exist in tenant!
DD_HOST_FLAVOR_NAME_LARGE OpenStack flavor of a large host.
DD_HOST_FLAVOR_NAME_SMALL OpenStack flavor of a small host.
DD_HOST_FLAVOR_SLOTS_LARGE How many slots a large instance provides
DD_HOST_FLAVOR_SLOTS_SMALL How many slots a small instance provides
DD_HOST_IMAGE The image a host should have
DD_HOST_MASTER_SG The security group that the pebbles instance is on. A security group rule is created to allow traffic from this security group to the pool host.
DD_HOST_NETWORK UUID of the network to which the pool host should be added. Can be “auto” if there is only one network in a tenant.
DD_MAX_HOSTS Do not spawn more than this many hosts.
DD_HOST_ROOT_VOLUME_SIZE How large a volume to create to the hosts
DD_SHUTDOWN_MODE Stop all hosts when they become free.

OpenShift Driver

To set up the OpenShift driver you need

  • a workin OpenShift instance
  • a user in the OpenShift instance (a separate machine to machine account is recommended)

1. Start by adding the url, username, password and subdomain in the creds file. names are “OSO_XXX_URL”, where XXX is the name of your installation (there can be multiple installations) 2. Restart Pebbles 3. Check out https://github.com/cscfi/notebook-images/ 4. Log in as the M2M user using the oc command line utility 5. run build_openshift.sh to build and publish images to the OpenShift Docker registry 6. Enable OpenShiftDriver in the Admin UI

class pebbles.drivers.provisioning.openshift_driver.OpenShiftDriver(logger, config)[source]

OpenShift Driver allows provisioning instances in an existing OpenShift cluster. It creates a project per user, identified by user eppn, and optionally a persistent volume claim (PVC) for user data.

The driver needs credentials for the cluster. The credentials are placed in the same m2m creds file that OpenStack and Docker driver use. The keys are as follows:

“OSD_[cluster_id]_BASE_URL”: “https://oso-cluster-api.example.org:8443”, “OSD_[cluster_id]_SUBDOMAIN”: “oso-cluster.example.org”, “OSD_[cluster_id]_USER”: “pebbles-m2m-user”, “OSD_[cluster_id]_PASSWORD”: “sickritt”

Replace [cluster_id] with a unique string to a cluster. When creating a blueprint template, refer to the cluster id in the configuration, key ‘openshift_cluster_id’ .You can have multiple credentials configured in the creds file.

Dummy Driver

class pebbles.drivers.provisioning.dummy_driver.DummyDriver(logger, config)[source]

Dummy driver mostly pretends to be a real driver for system testing and development purposes.

It runs a time-consuming process (ping) using run_logged_process, writes the public SSH key to the user for the user to a file and logs from the ping to the right places. It reports a random IP address.