The “Checkbox Stack”

The Checkbox Stack is a collection of projects that together constitute a complete testing and certification solution. It is composed of the following parts (see table below for extra details). All of the projects are linked to from the Launchpad project group.

Architecture Diagram

This diagram contains a high-level approximation of the current Checkbox architecture. There are three main “pillars”. On the left we have end products. Those are actual tools that certification and engineers are using. On the right we have the test market. This is a open market of tests vendors and suppliers. The tests are wrapped in containers known as providers. In the center we have three shared components. Those implement the bulk of the framework and user interfaces for test execution. Finally in the bottom-left corner there is a part of checkbox (a library) that is shared with HEXR for certain tasks. HEXR is a out-of-scope web application used by part of the certification process. Arrows imply communication with the shape of the arrow shows who calls who.

As mentioned before, in the center column there are three main components of shared code (shared by everyone using the end products that are discussed below). The shared code is composed of plainbox, checkbox and checkbox-converged. Component responsibilities are discussed in more detail in the table below. Here we can see that checkbox and checkbox-converged use plainbox API. checkbox-converged does so using pyotherside, and checkbox uses this api directly through python 3.

In the right hand side column there are various test providers. The checkbox project is producing and maintaining a number of providers (see the table below) but it is expected that our downstream users will also produce their own providers (specific to a customer or project). Eventually some providers may come from third parties that will adopt the format.

Lastly in the bottom-left corner, the shared library, this library contains many parsers of various file formats and output formats. Technically this library is a dependency of HEXR, checkbox and of providers. As an added complexity the library needs to be called from python3 code and python2 code.

Note

The communication between checkbox and plainbox is bi-directional. Plainbox offers some base interfaces and extension points. Those are all exposed through plainbox (using common APIs) but some of those are actually implemented in checkbox-ng.

Warning

All internal APIs is considered unstable. Stable APIs include:

• unit definitions
• SessionAssistant API
• launcher syntax

Component Descriptions

Project	Responsible for	Type
Checkbox-Converged	The QML user interface The graphical launcher for providers, e.g. checkbox-certification-client	Application
Checkbox (CLI)	The python command-line interface the text user interface the SRU testing command Additional certification APIs sending data to Launchpad sending data to HEXR	Application
Client Certification Provider	canonical-certification-client executable client certification test plans	Provider
Server Certification Provider	server certification test plans additional server test plans	Provider
System-on-Chip Server Certification Provider	SoC server certification test plans	Provider
Checkbox Provider	Almost all job definitions Most of custom “scripts” Default and SRU test plans	Provider
Resource Provider	Almost all resource jobs Almost all resource “scripts”	Provider
Checkbox Support	Support code for various providers Parsers for many text formats	Library
PlainBox	Almost all core logic RFC822 (job definition) parser Configuration handling Testing session (suspend/resume) Job runner Trusted launcher Dependency resolver Command line handling The XML, HTML and XSLX exporters and more... Provider development toolkit ‘plainbox startprovider’ ‘manage.py’ implementation	Library and Development Toolkit