File format and location¶

Jobs are expressed as sections in text files that conform somewhat to the rfc822 specification format. Our variant of the format is described in rfc822. Each record defines a single job.

Job Fields¶

Following fields may be used by the job unit:

id:

(mandatory) - A name for the job. Should be unique, an error will be generated if there are duplicates. Should contain characters in [a-z0-9/-]. This field used to be called name. That name is now deprecated. For backwards compatibility it is still recognized and used if id is missing.

summary:

(mandatory) - A human readable name for the job. This value is available for translation into other languages. It is used when listing jobs. It must be one line long, ideally it should be short (50-70 characters max).

plugin:

(mandatory) - For historical reasons it’s called “plugin” but it’s better thought of as describing the “type” of job. The allowed types are:

manual:	jobs that require the user to perform an action and then decide on the test’s outcome.
shell:	jobs that run without user intervention and automatically set the test’s outcome.
user-interact:	jobs that require the user to perform an interaction, after which the outcome is automatically set.
user-interact-verify:
	jobs that require the user to perform an interaction, run a command after which the user is asked to decide on the test’s outcome. This is essentially a manual job with a command.
attachment:	jobs whose command output will be attached to the test report or submission.
resource:	A job whose command output results in a set of rfc822 records, containing key/value pairs, and that can be used in other jobs’ requires expressions.
qml:	A test with GUI defined in a QML file.

requires:

(optional). If specified, the job will only run if the conditions expressed in this field are met.

Conditions are of the form <resource>.<key> <comparison-operator> 'value' (and|or) ... . Comparison operators can be ==, != and in. Values to compare to can be scalars or (in the case of the in operator) arrays or tuples. The not in operator is explicitly unsupported.

Requirements can be logically chained with or and and operators. They can also be placed in multiple lines, respecting the rfc822 multi-line syntax, in which case all requirements must be met for the job to run ( and ed).

depends:

(optional). If specified, the job will only run if all the listed jobs have run and passed. Multiple job names, separated by spaces, can be specified.

after:

(optional). If specified, the job will only run if all the listed jobs have run (regardless of the outcome). Multiple job names, separated by spaces, can be specified.

This feature is available since plainbox 0.24.

command:

(optional). A command can be provided, to be executed under specific circumstances. For manual, user-interact and user-verify jobs, the command will be executed when the user presses a “test” button present in the user interface. For shell jobs, the command will be executed unconditionally as soon as the job is started. In both cases the exit code from the command (0 for success, !0 for failure) will be used to set the test’s outcome. For manual, user-interact and user-verify jobs, the user can override the command’s outcome. The command will be run using the default system shell. If a specific shell is needed it should be instantiated in the command. A multi-line command or shell script can be used with the usual multi-line syntax.

Note that a shell job without a command will do nothing.

purpose:

(mandatory). Purpose field is used in tests requiring human interaction as an information about what a given test is supposed to do. User interfaces should display content of this field prior to test execution. This field may be omitted if the summary field is supplied. Note that this field is applicable only for human interaction jobs.

steps:

(optional). Steps field depicts actions that user should perform as a part of job execution. User interfaces should display the content of this field upon starting the test. Note that this field is applicable only for jobs requiring the user to perform some actions.

verification:

(optional). Verification field is used to inform the user how they can resolve a given job outcome. Note that this field is applicable only for jobs the result of which is determined by the user.

user:

(optional). If specified, the job will be run as the user specified here. This is most commonly used to run jobs as the superuser (root).

environ:

(optional). If specified, the listed environment variables (separated by spaces) will be taken from the invoking environment (i.e. the one Checkbox is run under) and set to that value on the job execution environment (i.e. the one the job will run under). Note that only the variable names should be listed, not the values, which will be taken from the existing environment. This only makes sense for jobs that also have the user attribute. This key provides a mechanism to account for security policies in sudo and pkexec, which provide a sanitized execution environment, with the downside that useful configuration specified in environment variables may be lost in the process.

estimated_duration:

(optional) This field contains metadata about how long the job is expected to run for, as a positive float value indicating the estimated job duration in seconds.

Since plainbox version 0.24 this field can be expressed in two formats. The old format, a floating point number of seconds is somewhat difficult to read for larger values. To avoid mistakes test designers can use the second format with separate sections for number of hours, minutes and seconds. The format, as regular expression, is (\d+h)?[: ]*(\d+m?)[: ]*(\d+s)?. The regular expression expresses an optional number of hours, followed by the h character, followed by any number of spaces or : characters, followed by an optional number of minutes, followed by the m character, again followed by any number of spaces or : characters, followed by the number of seconds, ultimately followed by the s character.

The values can no longer be fractional (you cannot say 2.5m you need to say 2m 30s). We feel that sub-second granularity does is too unpredictable to be useful so that will not be supported in the future.

flags:

(optional) This fields contains list of flags separated by spaces or commas that might induce plainbox to run the job in particular way. Currently, following flags are inspected by plainbox:

preserve-locale:

This flag makes plainbox carry locale settings to the job’s command. If this flag is not set, plainbox will neuter locale settings. Attach this flag to all job definitions with commands that use translations .

win32:

This flag makes plainbox run jobs’ commands in windows-specific manner. Attach this flag to jobs that are run on Windows OS.

noreturn:

This flag makes plainbox suspend execution after job’s command is run. This prevents scenario where plainbox continued to operate (writing session data to disk and so on), while other process kills it (leaving plainbox session in unwanted/undefined state). Attach this flag to jobs that cause killing of plainbox process during their operation. E.g. run shutdown, reboot, etc.

explicit-fail:

Use this flag to make entering comment mandatory, when the user manually fails the job.

has-leftovers:

This flag makes plainbox silently ignore (and not log) any files left over by the execution of the command associated with a job. This flag is useful for jobs that don’t bother with maintenance of temporary directories and just want to rely on the one already created by plainbox.

simple:

This flag makes plainbox disable certain validation advice and have some sesible defaults for automated test cases. This simiplification is meant to cut the boiler plate on jobs that are closer to unit tests than elaborate manual interactions.

In practice the following changes are in effect when this flag is set:

• the plugin field defaults to shell
• the description field is entirely optional
• the estimated_duration field is entirely optional
• the preserve-locale flag is entirely optional

A minimal job using the simple flag looks as follows:

id: foo
command: echo "Jobs are simple!"
flags: simple

preserve-cwd:

This flag makes plainbox run the job command in the current working directory without creating a temp folder (and running the command from this temp folder). Sometimes needed on snappy (See http://pad.lv/1618197)

Additional flags may be present in job definition; they are ignored.

imports:

(optional) This field lists all the resource jobs that will have to be imported from other namespaces. This enables jobs to use resources from other namespaces. You can use the “as ...” syntax to import jobs that have dashes, slashes or other characters that would make them invalid as identifiers and give them a correct identifier name. E.g.:

imports: from 2013.com.canonical.certification import cpuinfo
requires: 'armhf' in cpuinfo.platform

imports: from 2013.com.canonical.certification import cpu-01-info as cpu01
requires: 'avx2' in cpu01.other

The syntax of each imports line is:

IMPORT_STMT :: "from" <NAMESPACE> "import" <PARTIAL_ID>
               | "from" <NAMESPACE> "import" <PARTIAL_ID> AS <IDENTIFIER>

Test Plan Fields¶

The following fields can be used in a test plan. Note that not all fields need to be used or even should be used. Please remember that Checkbox needs to maintain backwards compatibility so some of the test plans it defines may have non-typical constructs required to ensure proper behavior. You don’t have to copy such constructs when working on a new test plan from scratch

id:

Each test plan needs to have a unique identifier. This is exactly the same as with other units that have an identifier (like job definitions and categories).

This field is not used for display purposes but you may need to refer to it on command line so keeping it descriptive is useful

name:

A human-readable name of the test plan. The name should be relatively short as it may be used to display a list of test plans to the test operator.

Remember that the user or the test operator may not always be familiar with the scope of testing that you are focusing on. Also consider that multiple test providers may be always installed at the same time. The translated version of the name (and icon, see below) is the only thing that needs to allow the test operator to pick the right test plan.

Please use short and concrete names like:

• “Storage Device Certification Tests”
• “Ubuntu Core Application’s Clock Acceptance Tests”
• “Default Ubuntu Hardware Certification Tests”.

The field has a soft limit of eighty characters. It cannot have multiple lines. This field should be marked as translatable by prepending the underscore character (_) in front. This field is mandatory.

description:

A human-readable description of this test plan. Here you can include as many or few details as you’d like. Some applications may offer a way of viewing this data. In general it is recommended to include a description of what is being tested so that users can make an informed decision but please in mind that the name field alone must be sufficient to discriminate between distinct test plans so you don’t have to duplicate that information in the description.

If your tests will require any special set-up (procuring external hardware, setting some devices or software in special test mode) it is recommended to include this information here.

The field has no size limit. It can contain newline characters. This field should be marked as translatable by prepending the underscore character (_) in front. This field is optional.

include:

A multi-line list of job identifiers or patterns matching such identifiers that should be included for execution.

This is the most important field in any test plan. It basically decides on which job definitions are selected by (included by) the test plan. Separate entries need to be placed on separate lines. White space does not separate entries as the id field may (sic!) actually include spaces.

You have two options for selecting tests:

• You can simply list the identifier (either partial or fully qualified) of the job you want to include in the test plan directly. This is very common and most test plans used by Checkbox actually look like that.
• You can use regular expressions to select many tests at the same time. This is the only way to select generated jobs (created either by template units. Please remember that the dot character has a special meaning so unless you actually want to match any character escape the dot with the backslash character (\).

Regardless of if you use patterns or literal job identifiers you can use their fully qualified name (the one that includes the namespace they reside in) or an abbreviated form. The abbreviated form is applicable for job definitions that reside in the same namespace (but not necessarily the same provider) as the provider that is defining the test plan.

Plainbox will catch incorrect references to unknown jobs so you should be relatively safe. Have a look at the examples section below for examples on how you can refer to jobs from other providers (you simply use their fully qualified name for that)

mandatory_include:

A multi-line list of job identifiers or patterns matching such identifiers that should always be executed.

This optional field can be used to specify the jobs that should always run. This is particularly useful for specifying jobs that gather vital info about the tested system, as it renders imposible to generate a report with no information about system under test.

For example, session results meant to be sent to the Ubuntu certification website must include the special job: miscellanea/submission-resources

Example:

mandatory_include:

miscellanea/submission-resources

Note that mandatory jobs will always be run first (along with their dependant jobs)

bootstrap_include:

A multi-line list of job identifiers that should be run first, before the main body of testing begins. The job that should be included in the bootstrapping sections are the ones generating or helping to generate other jobs.

Example:

bootstrap_include:

graphics/generator_driver_version

Note that each entry in the bootstrap_include section must be a valid job identifier and cannot be a regular expression pattern. Also note that only resource jobs are allowed in this section.

exclude:

A multi-line list of job identifiers or patterns matching such identifiers that should be excluded from execution.

This optional field can be used to prevent some jobs from being selected for execution. It follows the similarly named -x command line option to the plainbox run command.

This field may be used when a general (broad) selection is somehow made by the include field and it must be trimmed down (for example, to prevent a specific dangerous job from running). It has the same syntax as the include.

When a job is both included and excluded, exclusion always takes priority.

category-overrides:

A multi-line list of category override statements.

This optional field can be used to alter the natural job definition category association. Currently Plainbox allows each job definition to associate itself with at most one category (see plainbox-category-units(7) and plainbox-job-units(7) for details). This is sub-optimal as some tests can be easily assigned equally well to two categories at the same time.

For that reason, it may be necessary, in a particular test plan, to override the natural category association with one that more correctly reflects the purpose of a specific job definition in the context of a specific test plan.

For example let’s consider a job definition that tests if a specific piece of hardware works correctly after a suspend-resume cycle. Let’s assume that the job definition has a natural association with the category describing such hardware devices. In one test plan, this test will be associated with the hardware-specific category (using the natural association). In a special suspend-resume test plan the same job definition can be associated with a special suspend-resume category.

The actual rules as to when to use category overrides and how to assign a natural category to a specific test is not documented here. We believe that each project should come up with a workflow and semantics that best match its users.

The syntax of this field is a list of statements defined on separate lines. Each override statement has the following form:

apply CATEGORY-IDENTIFIER to JOB-DEFINITION-PATTERN

Both ‘apply’ and ‘to’ are literal strings. CATEGORY-IDENTIFIER is the identifier of a category unit. The JOB-DEFINITION-PATTERN has the same syntax as the include field does. That is, it can be either a simple string or a regular expression that is being compared to identifiers of all the known job definitions. The pattern can be either partially or fully qualified. That is, it may or may not include the namespace component of the job definition identifier.

Overrides are applied in order and the last applied override is the effective override in a given test plan. For example, given the following two overrides:

apply cat-1 to .*
apply cat-2 to foo

The job definition with the partial identifier foo will be associated with the cat-2 category.

estimated_duration:

An approximate time to execute this test plan, in seconds.

Since plainbox version 0.24 this field can be expressed in two formats. The old format, a floating point number of seconds is somewhat difficult to read for larger values. To avoid mistakes test designers can use the second format with separate sections for number of hours, minutes and seconds. The format, as regular expression, is (\d+h)?[: ]*(\d+m?)[: ]*(\d+s)?. The regular expression expresses an optional number of hours, followed by the h character, followed by any number of spaces or : characters, followed by an optional number of minutes, followed by the m character, again followed by any number of spaces or : characters, followed by the number of seconds, ultimately followed by the s character.

The values can no longer be fractional (you cannot say 2.5m you need to say 2m 30s). We feel that sub-second granularity does is too unpredictable to be useful so that will not be supported in the future.

This field is optional. If it is missing it is automatically computed by the identical field that may be specified on particular job definitions.

Since sometimes it is easier to think in terms of test plans (they are typically executed more often than a specific job definition) this estimate may be more accurate as it doesn’t include the accumulated sum of mis-estimates from all of the job definitions selected by a particular test plan.

Examples¶

A simple test plan that selects several jobs:

id: foo-bar-and-froz
_name: Tests Foo, Bar and Froz
_description:
    This example test plan selects the following three jobs:
        - Foo
        - Bar
        - Froz
include:
    foo
    bar
    froz

A test plan that uses jobs from another provider’s namespace in addition to some of its own definitions:

id: extended-tests
_name: Extended Storage Tests (By Corp Inc.)
_description:
    This test plan runs an extended set of storage tests, customized
    by the Corp Inc. corporation. In addition to the standard Ubuntu
    set of storage tests, this test plan includes the following tests::

    - Multipath I/O Tests
    - Degraded Array Recovery Tests
include:
    2013.com.canonical.certification:disk/.*
    multipath-io
    degrade-array-recovery

A test plan that generates jobs using bootstrap_include section:

unit: test plan
id: test-plan-with-bootstrapping
_name: Tests with a bootstrapping stage
_description:
    This test plan uses bootstrapping_include field to generate additional
    jobs depending on the output of the generator job.
include: .*
bootstrap_include:
    generator

unit: job
id: generator
plugin: resource
_description: Job that generates Foo and Bar resources
command:
 echo "my_resource: Foo"
 echo
 echo "my_resource: Bar"

unit: template
template-unit: job
template-resource: generator
plugin: shell
estimated_duration: 1
id: generated_job_{my_resource}
command: echo {my_resource}
_description: Job instantiated from template that echoes {my_resource}

A test plan that marks some jobs as mandatory:

unit: test plan
id: test-plan-with-mandatory-jobs
_name: Test plan with mandatory jobs
_description:
    This test plan runs some jobs regardless of user selection.
include:
    Foo
mandatory_include:
    Bar

unit: job
id: Foo
_name: Foo job
_description: Job that might be deselected by the user
plugin: shell
command: echo Foo job

unit: job
id: Bar
_name: Bar job (mandatory)
_description: Job that should *always* run
plugin: shell
command: echo Bar job

Category Fields¶

There are two fields that are used by the category unit:

id:

This field defines the partial identifier of the category. It is similar to the id field on the job definition units.

This field is mandatory.

name:

This field defines a human readable name of the category. It may be used in application user interfaces for displaying a group of tests.

This field is translatable. This field is mandatory.

unit: category
id: audio
_name: Audio tests

id: audio/speaker-headphone-plug-detection
category_id: audio
plugin: manual
_description: Plug in your headphones and ensure the system detected them

Resources¶

Resources are a mechanism that allows to constrain certain job to execute only on devices with appropriate hardware or software dependencies. This mechanism allows some types of jobs to publish resource objects to an abstract namespace and to a way to evaluate a resource program to determine if a job can be started.

Resources also serve as a ‘generator’ for template units. See Template Unit

plugin: resource

package.name == "fwts"

device.category == 'CDROM'
optical_drive.cd == 'writable'

id: rtc
estimated_duration: 0.02
plugin: resource
command:
  if [ -e /sys/class/rtc ]
  then
      echo "state: supported"
  else
      echo "state: unsupported"
  fi
_description: Creates resource info for RTC

plugin: shell
category_id: 2013.com.canonical.plainbox::power-management
id: power-management/rtc
requires:
  rtc.state == 'supported'
  package.name == 'util-linux'
  cpuinfo.other != 'emulated by qemu'
user: root
command: hwclock -r
estimated_duration: 0.02
_summary: Test that RTC functions properly (if present)
_description:
 Verify that the Real-time clock (RTC) device functions properly, if present.

package.name == desired_package.name

package.name == 'xorg' and package.name == 'procps'

A == True and A == False

package.name == 'xorg'
package.name == 'procps'

xinput.device_class == 'XITouchClass' and xinput.touch_mode != 'dependent'

device_class: XITouchClass
touch_mode: dependant

device_class: XITouchClass
touch_mode: something else

Template-Specific Fields¶

There are four fields that are specific to the template unit:

template-unit:

Name of the unit type this template will generate. By default job definition units are generated (as if the field was specified with the value of job) eventually but other values may be used as well.

This field is optional.

template-resource:

Name of the resource job (if it is a compatible resource identifier) to use to parametrize the template. This must either be a name of a resource job available in the namespace the template unit belongs to or a valid resource identifier matching the definition in the template-imports field.

This field is mandatory.

template-imports:

A resource import statement. It can be used to refer to arbitrary resource job by its full identifier and (optionally) give it a short variable name.

The syntax of each imports line is:

IMPORT_STMT ::  "from" <NAMESPACE> "import" <PARTIAL_ID>
              | "from" <NAMESPACE> "import" <PARTIAL_ID>
                 AS <IDENTIFIER>

The short syntax exposes PARTIAL_ID as the variable name available within all the fields defined within the template unit. If it is not a valid variable name then the second form must be used.

This field is sometimes optional. It becomes mandatory when the resource job definition is from another provider namespace or when it is not a valid resource identifier and needs to be aliased.

template-filter:

A resource program that limits the set of records from which template instances will be made. The syntax of this field is the same as the syntax of typical job definition unit’s requires field, that is, it is a python expression.

When defined, the expression is evaluated once for each resource object and if it evaluates successfully to a True value then that particular resource object is used to instantiate a new unit.

This field is optional.

Instantiation¶

When a template is instantiated, a single record object is used to fill in the parametric values to all the applicable fields. Each field is formatted using the python formatting language. Within each field the record is exposed as the variable named by the template_resource field. Record data is exposed as attributes of that object.

The special parameter __index__ can be used to iterate over the devices matching the template-filter field.

Basic example¶

The following example contains a simplified template that instantiates to a simple storage test. The test is only instantiated for devices that are considered physical. In this example we don’t want to spam the user with a long list of loopback devices. This is implemented by exposing that data in the resource job itself:

id: device
plugin: resource
command:
    echo 'path: /dev/sda'
    echo 'has_media: yes'
    echo 'physical: yes'
    echo
    echo 'path: /dev/cdrom'
    echo 'has_media: no'
    echo 'physical: yes'
    echo
    echo 'path: /dev/loop0'
    echo 'has_media: yes'
    echo 'physical: no'

The template defines a test-storage-XXX test where XXX is replaced by the path of the device. Only devices which are physical according to some definition are considered for testing. This means that the record related to /dev/loop0 will be ignored and will not instantiate a test job for that device. This feature can be coupled with the existing resource requirement to let the user know that we did see their CD-ROM device but it was not tested as there was no inserted media at the time:

unit: template
template-resource: device
template-filter: device.physical == 'yes'
requires: device.has_media == 'yes'
id: test-storage-{path}
plugin: shell
command: perform-testing-on --device {path}

Real life example¶

Here is a real life example of a template unit that generates a job for each hard drive available on the system:

unit: template
template-resource: device
template-filter: device.category == 'DISK'
plugin: shell
category_id: 2013.com.canonical.plainbox::disk
id: disk/stats_{name}
requires:
 device.path == "{path}"
 block_device.{name}_state != 'removable'
user: root
command: disk_stats_test {name}
_description: This test checks {name} disk stats, generates some activity and rechecks stats to verify they've changed. It also verifies that disks appear in the various files they're supposed to.

The template-resource used here (device) refers to a resource job using the udev_resource script to get information about the system. The udev_resource script returns a list of items with attributes such as path and name, so we can use these directly in our template.

File format and location¶

Exporter entry units are regular plainbox units and are contained and shipped with plainbox providers. In other words, they are just the same as job and test plan units, for example.

Fields¶

Following fields may be used by an exporter unit.

id:

(mandatory) - Unique identifier of the exporter. This field is used to look up and store data so please keep it stable across the lifetime of your provider.

summary:

(optional) - A human readable name for the exporter. This value is available for translation into other languages. It is used when listing exporters. It must be one line long, ideally it should be short (50-70 characters max).

entry_point:

(mandatory) - This is a key for a pkg_resources entry point from the plainbox.exporters namespace. Allowed values are: jinja2, text, xlsx, json and rfc822.

file_extension:

(mandatory) - Filename extension to use when the exporter stream is saved to a file.

options:

(optional) - comma/space/semicolon separated list of options for this exporter entry point. Only the following options are currently supported.

text and rfc822:

• with-io-log
• squash-io-log
• flatten-io-log
• with-run-list
• with-job-list
• with-resource-map
• with-job-defs
• with-attachments
• with-comments
• with-job-via
• with-job-hash
• with-category-map
• with-certification-status

json:

Same as for text and additionally:

• machine-json

xlsx:

• with-sys-info
• with-summary
• with-job-description
• with-text-attachments
• with-unit-categories

jinja2:

No options available

data:

(optional) - Extra data sent to the exporter code, to allow all kind of data types, the data field only accept valid JSON. For exporters using the jinja2 entry point, the template name and any additional paths to load files from must be defined in this field. See examples below.

Example¶

This is an example exporter definition:

unit: exporter
id: my_html
_summary: Generate my own version of the HTML report
entry_point: jinja2
file_extension: html
options:
 with-foo
 with-bar
data: {
 "template": "my_template.html",
 "extra_paths": [
     "/usr/share/javascript/lib1/",
     "/usr/share/javascript/lib2/",
     "/usr/share/javascript/lib3/"]
 }

The provider shipping such unit can be as follow:

├── data
│   ├── my_template.css
│   └── my_template.html
├── units
    ├── my_test_plans.pxu
    └── exporters.pxu

Note that exporters.pxu is not strictly needed to store the exporter units, but keeping them in a dedidated file is a good practice.

How to use exporter units?¶

In order to call an exporter unit from provider foo, you just need to use in in the launcher.

Example of a launcher using custom exporter unit:

#!/usr/bin/env checkbox-cli

[launcher]
launcher_version = 1

[transport:local_file]
type = file
path = /tmp/submission.html

[exporter:my_html]
unit = 2013.com.foo.bar::my_html

[report:local_html]
transport = local_file
exporter = my_html

For more information about generating reports see Generating reports

File format and location¶

Manifest entry units are regular plainbox units and are contained and shipped with plainbox providers. In other words, they are just the same as job and test plan units, for example.

Fields¶

Following fields may be used by a manifest entry unit.

id:

(mandatory) - Unique identifier of the entry. This field is used to look up and store data so please keep it stable across the lifetime of your provider.

name:

(mandatory) - A human readable name of the entry. This should read as in a feature matrix of a device in a store (e.g., “802.11ac wireless capability”, or “Thunderbolt support”, “Number of hard drive bays”). This is not a sentence, don’t end it with a dot. Please capitalize the first letter. The name is used in various listings so it should be kept reasonably short.

The name is a translatable field so please prefix it with _ as in _name: Example.

value-type:

(mandatory) - Type of value for this entry. Currently two values are allowed: bool for a yes/no value and natural for any natural number (negative numbers are rejected).

value-units:

(optional) - Units in which value is measured in. This is only used when value-type is equal to natural. For example a “Screen size” manifest entry could be measured in “inch” units.

resource-key:

(optional) - Name of the resource key used to store the manifest value when representing the manifest as a resource record. This field defaults to the so-called partial id which is just the id: field as spelled in the unit definition file (so without the name space of the provider)

Example¶

This is an example manifest entry definition:

unit: manifest entry
id: has_thunderbolt
_name: Thunderbolt Support
value-type: bool

Naming Manifest Entries¶

To keep the code consistent there’s one naming scheme that should be followed. Entries for boolean values must use the has_XXX naming scheme. This will allow us to avoid issues later on where multiple people develop manifest entries and it’s all a bit weird what them mean has_thunderbolt or thunderbolt_supported or tb or whatever we come up with. It’s a convention, please stick to it.

Using Manifest Entries in Jobs¶

Manifest data can be used to decide if a given test is applicable for a given device under test or not. When used as a resource they behave in a standard way, like all other resources. The only special thing is the unique name-space of the resource job as it is provided by plainbox itself. The name of the resource job is: 2013.com.canonical.plainbox. In practice a simple job that depends on data from the manifest can look like this:

unit: job
id: ...
plugin: ...
requires:
 manifest.has_thunderbolt == 'True' and manifest.ns == '2013.com.canonical.checkbox'
imports: from 2013.com.canonical.plainbox import manifest

Note that the job uses the manifest job from the 2013.com.canonical.plainbox name-space. It has to be imported using the imports: field as it is in a different name-space than the one the example unit is defined in (which is arbitrary). Having that resource it can then check for the has_thunderbolt field manifest entry in the 2013.com.canonical.checkbox name-space. Note that the name-space of the manifest job is not related to the manifest.ns value. Since any provider can ship additional manifest entries and then all share the flat name-space of resource attributes looking at the .ns attribute is a way to uniquely identify a given manifest entry.

Collecting Manifest Data¶

To interactively collect manifest data from a user please include this job somewhere early in your test plan: 2013.com.canonical.plainbox::collect-manifest.

Supplying External Manifest¶

The manifest file is stored in $HOME/.local/share/plainbox/machine-manifest.json. If the provisioning method ships a valid manifest file there it can be used for fully automatic but manifest-based deployments.

File format and location¶

Packaging meta-data units are regular plainbox units and are contained and shipped with plainbox providers. In other words, they are just the same as job and test plan units, for example.

Fields¶

Following fields may be used by a manifest entry unit.

os-id:

(mandatory) - the identifier of the operating system this rule applies to. This is the same value as the ID field in the file /etc/os-release. Typical values include debian, ubuntu or fedora.

os-version-id:

(optional) - the identifier of the specific version of the operating system this rule applies to. This is the same as the VERSION_ID field in the file /etc/os-release. If this field is not present then the rule applies to all versions of a given operating system.

The remaining fields are custom and depend on the packaging driver. The values for Debian are:

Depends:

(optional) - a comma separated list of dependencies for the binary package. The syntax is the same as in normal Debian control files (including package version dependencies). This field can be split into multiple lines, for readability, as newlines are discarded.

Suggests:

(optional) - same as Depends.

Recommends:

(optional) - same as Depends.

Matching Packaging Meta-Data Units¶

The base Linux distribution driver parses the /etc/os-release file, looks at the ID, ID_VERSION and optionally the ID_LIKE fields. They are used as a standard way to determine the distribution for which packaging meta-data is being collected for.

The id and version match strategy requires that both the os-id and os-dependencies fields are present and that they match the ID and ID_VERSION values. This strategy allows the test maintainer to express each dependency accurately for each operating system they wish to support.

The id match strategy is only used when the os-version is not defined. It is useful when a single definition is applicable to many subsequent releases. This is especially useful when job works well with sufficiently old version of a third party dependency and there is no need to repeatedly re-state the same dependency for each later release of the operating system.

The id_like match strategy is only used as a last resort and can be seen as a weaker id match strategy. This time the os-id field is compared to the ID_LIKE field (if present). It is useful for working with Debian derivatives, like Ubuntu.

Each matching packaging meta-data unit is then passed to the driver to generate packaging meta-data.

Example¶

This is an example packaging meta-data unit, as taken from the resource provider:

unit: packaging meta-data
os-id: debian
Depends:
 python3-checkbox-support (>= 0.2),
 python3 (>= 3.2),
Recommends:
 dmidecode,
 dpkg (>= 1.13),
 lsb-release,
 wodim

This will cause the binary provider package to depend on the appropriate version of python3-checkbox-support and python3 in both Debian, Ubuntu and, for example, Elementary OS. In addition the package will recommend some utilities that are used by some of the jobs contained in this provider.

Using Packaging Meta-Data in Debian¶

To make use of the packaging meta-data, follow those steps:

• Ensure that /etc/os-release exists in your build chroot. On Debian it is a part of the base-files package which is not something you have to worry about but other distributions may use different strategies.

• Mark the binary package that contains the provider with the X-Plainbox-Provider: yes header.

• Add the ${plainbox:Depends}, ${plainbox:Recommends} and ${plainbox:Suggests} variables to the binary package that contains the provider.

• Override the gen_control debhelper rule and run the python3 manage.py packaging command in addition to running dh_gencontrol:

  override_dh_gencontrol:
      python3 manage.py packaging
      dh_gencontrol
  

Backus–Naur Form¶

An approximated syntax can be summarized as the following BNF:

record-list: record-list '\n' record
             | record
record: entry-list '\n\n' entry
        | entry
entry: KEY ':' VALUE
KEY: ^[^:]+
VALUE: .+\n([ ].+)*

There are two quirks which not handled by this syntax (see below). Otherwise the syntax is very simple. It defines a list of records. Each record is a list of entries. Each entry is a key-value pair. Values can be multi-line, which allows for convenient expression of longer text fragments.

Quirk 1 – the magic dot¶

Due to the way the multi-line VALUE syntax is defined, it would be impossible (or possible but dependant only on whitespace, which is not friendly) to include two consecutive newlines. For that reason a line consisting of a single space, followed by a single dot is translated to an empty line.

The example below:

key:
 .
 more value

Is parsed as an ENTRY (in python syntax):

("key", "\nvalue")

Quirk 2 – the # comments¶

Since it’s a line-oriented format and people are used to being able to insert comments anywhere with the # comment notation, any line that _starts_ with a hash or pound character is discarded. This happens earlier than other parts of parsing so comments are invisible to the rest of the parser. They can be included anywhere, including in the middle of a multi-line value.

Example:

# this is a comment
key: value
 multi-line
# comment!
 and more