Administering LAVA using Docker¶
Prerequisites¶
For buster or later, use docker.io from Debian https://tracker.debian.org/pkg/docker.io
add yourself to the docker group to avoid the need for
sudo
.Ensure that your docker installation has created a suitable network bridge (often called
docker0
).See also
Not all docker images are available for all architectures. LAVA has support for amd64 and arm64 docker images.
Using LAVA with Docker is also a regular topic of discussion on the lava-devel mailing list.
https://docs.docker.com/engine/reference/commandline/run/
Docker Networking¶
https://docs.docker.com/engine/reference/commandline/network_create/
LAVA typically uses network connections not just for the communication between master and worker but also to control the DUT. It is important to have reliable networking for the docker components.
If you have not done so already, create a new network for docker to
use, based on the docker0
bridge which should already have been
created by docker. The subnet you choose is up to you.
$ docker network create --subnet=172.18.0.0/16 dockernet
f9ca89ac46ab97e5d905a0858a03f8983941e8f4583b71e1996bbaa0d57d3138
When you start containers, you can specify that network name and then
provide a static IPv4 address for the container. This is particularly
important if you want to start a lava-server
container and then
connect a worker. The worker can be another docker container or it can
use lava-dispatcher
installed directly on this or another system.
You will need to make sure that the docker network is accessible to any
worker you want to attach to the new container.
$ docker run --net dockernet --ip 172.18.0.5 -it registry.gitlab.com/lava/lava/amd64/lava-dispatcher:2023.06
This IP address (or a hostname if you configure local DNS
appropriately) can then be used in commands to docker run to start a
lava-dispatcher
container by updating the MASTER_URL and LOGGER_URL
variables.
...
-e "LOGGER_URL=tcp://172.18.0.5:5555" \
-e "MASTER_URL=tcp://172.18.0.5:5556" \
...
Official LAVA Software Docker images¶
Official LAVA Software Docker images are available via
registry.gitlab.com
and hub.docker.com
. In each case, two
images are built at the same version: lava-dispatcher
and
lava-server
, each image is built for two architectures, amd64
and aarch64
.
Daily builds¶
registry.gitlab.com
hosts CI images regularly built from the
master
branch of lava
. Images are listed in GitLab:
Official LAVA Releases using Docker¶
The lavasoftware
organization on hub.docker.com
hosts releases
of LAVA (https://hub.docker.com/u/lavasoftware/) by retagging images
from registry.gitlab.com and pushing to hub.docker.com . Users are
free to use either hub.
Note
Due to naming conventions on hub.docker.com, the architecture
is included in the image name amd64-lava-server
when tagged for
hub.docker.com.
lava-dispatcher¶
https://hub.docker.com/r/lavasoftware/lava-dispatcher
docker pull lavasoftware/lava-dispatcher:2019.01
or
docker pull registry.gitlab.com/lava/lava/amd64/lava-dispatcher:2023.06
lava-server¶
https://hub.docker.com/r/lavasoftware/lava-server/
docker pull lavasoftware/lava-server:2019.01
or
docker pull registry.gitlab.com/lava/lava/amd64/lava-server:2023.06
Command lines¶
The use of docker with LAVA is an active area of development, including how to configure containers for a variety of situations and how to manage a LAVA lab where docker is in use. If you are doing work in this area, please subscribe to the lava-devel mailing list and ask for advice on how to use LAVA and docker for your use case.
POSIX shell¶
Command lines get long, so use wrapper scripts, e.g.:
#!/bin/sh
set -e
set -x
docker run \
-e "DISPATCHER_HOSTNAME=--hostname=calvin-2018.7-88" \
-e "LOGGER_URL=tcp://calvin:5555" \
-e "MASTER_URL=tcp://calvin:5556" \
--name calvin-docker-88-3 \
registry.gitlab.com/lava/lava/lava-dispatcher/master:2023.06.dev0001
Python¶
If you are using docker for more than a few test containers, you will probably find the Python docker SDK library very useful.
Debian
apt install python3-docker
https://packages.debian.org/unstable/python3-docker - If you install the fulllava
set on Debian Buster or newer,python3-docker
anddocker.io
will be installed by thelava
metapackage.Documentation: https://docker-py.readthedocs.io/en/stable/
The Python Docker SDK lets you do anything the docker command does, but from within Python apps – run containers, manage containers, manage Swarms, etc.
import docker
client = docker.from_env()
container_id = client.containers.run("debian", detach=True)
lava-dispatcher in docker¶
The official LAVA Software docker images for lava-dispatcher
do not
include details like ser2net
configuration or pdudaemon
or
other remote power control scripts. These will need to be added
according to your local lab configuration. Depending on the size of
your lab, you may choose to use a docker volume or docker build
to
create one or more customized docker images based on the official
images.
See also
Docker documentation on volumes and Docker documentation on building images.
Mocking up LXC inside docker¶
LXC cannot be installed/used inside a Docker container and the Docker container can replace the need for the LXC. This has the useful advantages that specialized tools which need to be isolated inside an LXC can be pre-installed in a docker container instead of needing to be installed or compiled within the LXC.
However, there are also disadvantages:
The Docker is persistent - currently,
lava-worker
andlava-run
need to be inside the container, so the next test job for that worker picks up the changes to the docker from this test job.The test job would need modification to not call LXC.
Work is underway to solve the persistence problem. In the meantime, it is possible to run test jobs using Docker if the persistence is handled correctly but this is usually only practical for single-user developer instances.
lava-lxc-mocker
exists to solve the second problem. By mocking up
the calls to lxc-*
utilities, lava-lxc-mocker
allows the same
test job to be run on a device managed by a lava-worker
in Docker
as on a device managed by a lava-worker
running on bare metal.
lava-lxc-mocker
is pre-installed in all Official LAVA Software Docker images.
lava-server in docker¶
The official LAVA Software docker images for lava-server
currently
include PostgreSQL. Work is planned to use an external PostgreSQL.
See also
LAVA Coordinator¶
lava-coordinator
is neither installed nor configured in any
official LAVA Software docker image. Therefore, a worker running from
one of these images will not have the configuration file
/etc/lava-coordinator/lava-coordinator.conf
to use
lava-coordinator
, so cannot run MultiNode LAVA test jobs. The
configuration file would need to be provided (configured for an
external coordinator installed using packages), either using a docker
volume used as a fileshare or by a modification to the docker image for
lava-dispatcher.
Work is planned to refactor lava-coordinator
to not require
external configuration or packaging.
CI images¶
lava/lava/lava-dispatcher/master
on registry.gitlab.com
contains images like 2018.7-101-g5987db8b5
See also
lava-dispatcher¶
This example runs a new worker for an existing master which can be:
on the same machine but installed from packages, not docker
on a different machine and accessible through DNS
In either case, the machine running lava-server
is accessible on
the network as calvin
. (Replace this hostname with your local
machine hostname.)
To run both master and worker on a single machine, both using docker, see Running lava-server & lava-dispatcher together.
DISPATCHER_HOSTNAME=--name=calvin-2018.7-88
/usr/bin/lava-worker --level $LOGLEVEL --log-file $LOGFILE --url $SERVER_URL $DISPATCHER_HOSTNAME
$ docker run -e "DISPATCHER_HOSTNAME=--name=calvin-2018.7-88" -e "URL=http://calvin/" --name calvin-docker-88-4 registry.gitlab.com/lava/lava/amd64/lava-dispatcher:2023.06.dev0008
2018-10-03 15:08:32,852 INFO [INIT] LAVA worker has started.
2018-10-03 15:08:32,852 INFO [INIT] Using protocol version 3
If you make mistakes, set the worker to Retired in the Django admin
interface and use docker rm <name>
to allow you to re-use the same
container with different arguments next time.
lava-server¶
$ docker run --net dockernet --ip 172.18.0.5 -it registry.gitlab.com/lava/lava/amd64/lava-server:2023.06.dev0008
Note
the dockernet
docker network needs to already exist and
is just an example name - choose your own name according to your own
preferences. See https://docs.docker.com/network/bridge/#differences-between-user-defined-bridges-and-the-default-bridge
See also
Superusers¶
There is no superuser in the lava-server docker container, admins need to login to the container and create an initial superuser:
$ docker exec -it a936cc14b913 lava-server manage users add --staff --superuser --email <EMAIL> --passwd <PASSWORD> <USERNAME>
Then this user can login through the normal UI and create Authentication Tokens.
See also
Running lava-server & lava-dispatcher together¶
The worker must be on the same docker network as the master because docker only exposes the master ports to that network.
Containers connected to the same user-defined bridge network automatically expose all ports to each other, and no ports to the outside world. This allows containerized applications to communicate with each other easily, without accidentally opening access to the outside world.
So to run a worker in docker to work with a master in docker on the
same machine, the worker must be given the --net dockernet
option.
Depending on the tasks, you should also assign an IP address to the worker, on the same docker network.
$ docker run --net dockernet --ip 172.18.0.6 ....
(This is why docker start up scripts are going to be so useful.)
See also
Separate services in separate Docker containers¶
Work is beginning to extend the Docker support to have different parts of LAVA in different containers. Some parts of this are easier to implement than others, so the support will arrive in stages.
See also