plainbox.impl.resource – job resources

Warning

THIS MODULE DOES NOT HAVE STABLE PUBLIC API

exception plainbox.impl.resource.CodeNotAllowed(node)

Bases: plainbox.impl.resource.ResourceProgramError

Exception raised when unsupported computing is detected inside requirement expression.

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception plainbox.impl.resource.ExpressionCannotEvaluateError(expression, resource_id)

Bases: plainbox.impl.resource.ExpressionFailedError

Exception raised when a resource could not be evaluated because it requires an unavailable resource.

Unlike the base class, this exception is raised before even running the expression. As in the base class the exception object is meant to have enough data to provide rich and meaningful error messages to the operator.

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception plainbox.impl.resource.ExpressionFailedError(expression)

Bases: Exception

Exception raise when a resource expression failed to produce a true value.

This class is meant to be consumed by the UI layers to provide meaningful error messages to the operator. The expression attribute can be used to obtain the text of the expression that failed as well as the resource id that is used by that expression. The resource id can be used to lookup the (resource) job that produces such values.

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class plainbox.impl.resource.FakeResource(accessed_attributes=None)

Bases: object

A resource that seemingly has any accessed attribute.

All attributes resolve back to the their name. All accessed attributes are recorded and can be referenced from a set that needs to be passed to the initializer. Knowledge about accessed attributes can be helpful in various forms of static analysis.

exception plainbox.impl.resource.NoResourcesReferenced

Bases: plainbox.impl.resource.ResourceProgramError

Exception raised when an expression does not reference any resources.

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

class plainbox.impl.resource.RequirementNodeVisitor

Bases: ast.NodeVisitor

A NodeVisitor subclass used to analyze package requirement expressions.

generic_visit(node)

Called if no explicit visitor function exists for a node.

packages_seen

set() of ast.Str().id values seen joined with the “|” operator for use in debian/control files

visit(node)

Visit a node.

visit_Str(node)

Internal method of NodeVisitor.

This method is called whenever generic_visit() looks at an instance of ast.Str().

class plainbox.impl.resource.Resource(data=None)

Bases: object

A simple container for key-value data

Resource objects are used when evaluating expressions as containers for data read from resource scripts. Each RFC822 record produced by a resource script is converted to a new Resource object

class plainbox.impl.resource.ResourceExpression(text, implicit_namespace=None, imports=None)

Bases: object

Class representing a single line of an requirement program.

Each valid expression references exactly one resource. In practical terms each resource expression is a valid python expression that has no side effects (calls almost no methods, does not assign anything) that can be evaluated against a single variable which references a Resource object.

evaluate(*resource_list_list)

Evaluate the expression against a list of resources

Each subsequent resource from the list will be bound to the resource id in the expression. The return value is True if any of the attempts return a true value, otherwise the result is False.

implicit_namespace

implicit namespace for partial identifiers, may be None

resource_alias_list

The alias of the resource object this expression operates on

This is different from resource_id() in that it is always a valid python identifier. The alias is either the partial identifier of the resource job or an arbitrary identifier, as used by the job definition.

resource_id_list

The id of the resource this expression depends on

This is different from resource_alias() in that it may not be a valid python identifier and it is always (ideally) a fully-qualified job identifier.

text

The text of the original expression

class plainbox.impl.resource.ResourceNodeVisitor

Bases: ast.NodeVisitor

A NodeVisitor subclass used to analyze requirement expressions.

Warning

Implementation of this class requires understanding of some of the lower levels of python. The general idea is to use the ast (abstract syntax tree) module to allow the ResourceExpression class to execute safely (by not permitting various unsafe operations) and quickly (by knowing which resources are required so no O(n) operations over all resources are ever needed.

Resource expressions are written one per line, each line is like a separate min-program. This visitor will be applied to the root (module) node resulting from parsing each of those lines.

Each actual expression can only use a small subset of python syntax, most stuff is actually disallowed. Only basic expressions are permitted. Function calls are also disallowed, with the notable exception of ‘bool’, ‘int’, ‘float’ and ‘len’.

One very important aspect of each expression is the id of the resource it is computing against. This is visible as the ‘object’ the expressions are operating on, such as:

package.name == ‘fwts’

As a rule of a thumb exactly one such id is allowed per expression. This allows the code that evaluates this to know which resource to use. As resources are actually lists of records (where record values are available as object attribute) only one object/record is exposed to each expression. Using more than one object (by intent or simple typo) would lead to expression that will never match. This visitor class facilitates detecting that by computing the ids_seen set.

One notable fact is that storing is not allowed so it is (presumably) safe to evaluate the code in the context of the current python interpreter.

How this works:

Using the ast.NodeVisitor we can visit any node type by defining the visit_<class name> method. We care about Name and Call nodes and they have custom validation implemented. For all other nodes the generic_visit() method is called instead.

On each visit to ast.Name node we record the referenced ‘id’ (the id of the object being referenced, in simple terms)

On each visit to ast.Call node we check if the called function is in the allowed list of ids. This also takes care of stuff like foo()() which would call the return value of foo.

On each visit to any other ast.Node we check if the class is in the white-list.

All violation cause a CodeNotAllowed exception to be raised with the node that was rejected as argument.

generic_visit(node)

Internal method of NodeVisitor.

Called for all ast.Node() subclasses that don’t have a dedicated visit_xxx() method here. Only needed to all the _check_node() method.

ids_seen_list

list() of ast.Name().id values seen

ids_seen_set

set() of ast.Name().id values seen

visit(node)

Visit a node.

visit_Call(node)

Internal method of NodeVisitor.

This method is called whenever generic_visit() looks at an instance of ast.Call(). Since white-listing Call in general would be unsafe only a small subset of calls are allowed.

visit_Name(node)

Internal method of NodeVisitor.

This method is called whenever generic_visit() looks at an instance of ast.Name(). It records the node identifier and calls _check_node()

class plainbox.impl.resource.ResourceProgram(program_text, implicit_namespace=None, imports=None)

Bases: object

Class for storing and executing resource programs.

This is used by job requirement expressions

evaluate_or_raise(resource_map)

Evaluate the program with the given map of resources.

Raises a ExpressionFailedError exception if the any of the expressions that make up this program cannot be executed or executes but produces a non-true value.

Returns True

Resources must be a dictionary of mapping resource id to a list of Resource objects.

expression_list

A list of ResourceExpression instances

required_resources

A set() of resource ids that are needed to evaluate this program

exception plainbox.impl.resource.ResourceProgramError

Bases: Exception

Base class for errors in requirement programs.

This class of errors are based on static analysis, not runtime execution. Typically they encode unsupported or disallowed python code being used by an expression somewhere.

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

exception plainbox.impl.resource.ResourceSyntaxError

Bases: plainbox.impl.resource.ResourceProgramError

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

plainbox.impl.resource.parse_imports_stmt(imports)

Parse the ‘imports’ line and compute the imported symbols.

Return generator for a sequence of pairs (job_id, identifier) that describe the imported job identifiers from arbitrary namespace.

The syntax of each imports line is:

IMPORT_STMT :: “from” <NAMESPACE> “import” <PARTIAL_ID>

“from” <NAMESPACE> “import” <PARTIAL_ID> AS <IDENTIFIER>

comments powered by Disqus