Orcraft Configuration Fields

Config fields are how a component declares what can be configured about it. They are plain class attributes, and they are read by the controller to build its run configuration panel, to validate a run before it starts, and to render the config table in the log.

Declaration

Config fields come from the OrcraftConfigField namespace and are declared on a config class:

class MyTestConfig(OrcraftTestConfig):
	name = OrcraftConfigField.Text(default="My Test")

	app_folder = OrcraftConfigField.PackageFolder(
		default="",
		label="App Folder",
		required=True,
		layout=OrcraftConfigField.Layout.NewRow,
	)

	attempts = OrcraftConfigField.Numeric(default=3, label="Attempts")

Inside the component the field reads as an ordinary attribute - self.config.app_folder - holding either the configured value or the declared default. There is no lookup by string key and no dictionary to keep in sync.

Output fields work the same way, from the OrcraftResultField namespace - see Orcraft Result Fields.

Inherited fields

Every config class starts with fields it did not declare. They fall into three groups by who may change them. All the overridable ones are optional - each has a working default, and redeclaring one in your config replaces that default without changing what the field is for.

Overridable in code

Set in your config class, fixed for the run.

Overridable in code or GUI

Set in your config class, and changeable again from the run configuration panel before a run.

Reserved

Written by the framework while a run is in progress. A config that redeclares one of these, or code that assigns one, is refused with an error naming the field.

Common config field options

Every field type is constructed with keyword arguments, and these are the ones every type accepts. They describe the field to the controller - its default, how it is labelled and laid out, whether a run may start without it - rather than what the value means to your test:

input_folder = OrcraftConfigField.Folder(
	default="",
	label="Input Folder",
	required=True,
	layout=OrcraftConfigField.Layout.NewRow,
	input_placeholder="Folder containing the input files",
)

Type-specific arguments - options for a selection, port for an endpoint, endpoints for a package field - are covered with their types below.

Config field types

Every field type in the OrcraftConfigField namespace, grouped by what it selects.

Basic types

name = OrcraftConfigField.Text(default="Boolean Parameter")
verbose = OrcraftConfigField.Boolean(default=False, label="Verbose")
attempts = OrcraftConfigField.Numeric(default=3, label="Attempts")
timeout = OrcraftConfigField.Numeric(default=None, label="Timeout", required=False)

Selection types

Use the lambda variants when the choices are not knowable when the test is written - a list of firmware versions on a server, branches in a repository, devices currently attached. The callable runs when the run configuration is opened, so the list is current rather than committed to source.

mode = OrcraftConfigField.Select(
	default="fast",
	label="Mode",
	options=("fast", "thorough"),
)

version = OrcraftConfigField.SelectLambda(
	default="",
	label="Version",
	options_lambda=list_available_versions,
)

Filesystem types

These give the GUI a browse button and store the chosen path. The path is used as-is: the file stays where it is, and the test reads it from there.

Package upload types

A package field is a filesystem field that travels. The controller uploads what it points at to the endpoints that need it before the run, and inside the test the value resolves to the path on the machine the code is executing on. That is the difference that matters: a Folder is a path on the controller, a PackageFolder is a path that exists wherever the step runs.

The endpoints option binds a package field to specific endpoint fields, so files upload only where they are actually needed:

test_endpoint = OrcraftConfigField.Endpoint(default="localhost", label="Test Endpoint")
aux_endpoints = OrcraftConfigField.MultiSelectEndpoints(default=("localhost",), label="Aux Endpoints")

input_file = OrcraftConfigField.PackageFile(
	default="",
	label="Input File",
	required=True,
	endpoints=[test_endpoint],
)

input_folders = OrcraftConfigField.PackageFolderList(
	default=(),
	label="Input Folders",
	required=True,
	endpoints=[test_endpoint, aux_endpoints],
)

Without endpoints, a package field is uploaded to every endpoint the test uses.

Endpoint types

All three accept port, secure_enabled and shared in addition to the shared options, and all three take an endpoint pool as their default in place of a machine name. Orcraft Endpoints covers what they do and when to reach for each field type.

Composition types

Groups and Tests always use default_factory, because they construct component instances:

tests = OrcraftConfigField.Tests(
	default_factory=lambda: [
		MyFirstTest(),
		MySecondTest(),
	]
)

Matrix and Dimension have options of their own - generated values, exclusion rules, randomization and seeding. They are covered in Orcraft Matrix, alongside what they produce.

Layout and presentation

log_orientation = OrcraftConfigField.Format(
	default=OrcraftConfigField.LogTableOrientation.VERTICAL
)

Every component logs two tables: a configuration table before it runs and a result table after. Each has its own log_orientation - the one on the config class controls the configuration table, the one on the result class controls the result table - so you can turn off one and keep the other. Turning off the configuration table also hides the description banner printed above it.

Test cycles, matrix cases and case cycles are built by the framework and have no config class of their own, so their Configuration tables follow the test's config: horizontal or vertical as the test's is, and OFF on a test's config hides the framework's cycle, case and matrix tables beneath it. The test's own Result table follows the result class the test assigns to self.result, and a case's Result table follows the class the body assigns to case.result, or reads as the test's does when the body assigns none. The demo suite's test_log_tables_off runs a matrix with two test cycles and two case cycles and prints nothing but plain log lines.

For example, a step whose inputs are obvious from its name can drop its configuration table and still log its result:

class MyStepConfig(OrcraftTestStepConfig):
	name = OrcraftConfigField.Text(default="My Step")
	log_orientation = OrcraftConfigField.Format(
		default=OrcraftConfigField.LogTableOrientation.OFF
	)

Plain attributes

Not everything a config carries has to be an Orcraft field. A plain class attribute is a legitimate way to hand a value to a component when nobody needs to see or edit it - the framework leaves it alone entirely:

class MyConsumeStepConfig(OrcraftTestStepConfig):
	name = OrcraftConfigField.Text(default="My Consume Step")
	sample_rate = OrcraftConfigField.Numeric(default=48000, label="Sample Rate")

	# Plain attributes: filled in by the calling flow, invisible everywhere else.
	calibration = None
	channel_map = ()

A flow sets them the same way it sets declared fields, through run_step keywords or the config constructor:

second_result = self.run_step(
	MyConsumeStep,
	sample_rate=self.config.sample_rate,
	calibration=first_result.calibration,
	channel_map=first_result.channel_map,
)

The one rule is that the attribute must be declared on the class, even if only as None. run_step checks that the config has the name before setting it, so a keyword that matches nothing raises at the call site instead of silently creating an attribute nobody reads.

Use a plain attribute for data that is an implementation detail between a flow and its steps - a calibration table, a handle, a structure too large or too structured to be a GUI input. Use an Orcraft field for anything a person should be able to see or change before a run.

Not every inherited field is user-facing. name and description, for instance, are configured in code but not editable from the GUI, and fields whose value is at its default - a single cycle, an empty matrix - are hidden from the run panel rather than shown as clutter.

The two policy fields take a fixed set of values. upload_test_package:

update_endpoint_app: