Matrix Lambda Dimensions

Some values only exist when the test is configured: the releases currently published, the devices currently attached, the branches currently open. A lambda dimension takes its values from a function the test supplies, and the matrix is built from whatever that function returns. This page follows Matrix Dimensions Randomization and uses the Matrix Lambda Dimensions test of the Advanced Configuration Parameters group.

Select the Matrix Lambda Dimensions test:

The panel reports Total Cases: 18. The Maze App Version column header carries a lambda icon, and its values, maze-v3.0.0, maze-v2.0.0 and maze-v1.0.0, are the release tags the test's function fetched from GitHub while the panel was being built. There is no Maze Executable Folder field this time: every case obtains the binary of its own version, so the test needs nothing configured before it runs.

Defining a lambda dimension

Source directory:

The dimension is declared with Dimension.lambda_choice, whose factory is a function of no arguments that returns the values:

matrix = OrcraftConfigField.Matrix(
	OrcraftConfigField.Dimension.lambda_choice(label="Maze App Version", factory=lambda: fetch_maze_app_release_versions()),
	OrcraftConfigField.Dimension(label="command", values=("generate", "solve", "save")),
	OrcraftConfigField.Dimension(label="algorithm", values=(None, "BFS", "DFS", "ASTAR")),

	exclude=[
		lambda case: case.params["command"] in ("generate", "save") and case.params["algorithm"] is not None,
	],
)

The factory runs when the test is configured, which is when the controller builds its run panel, not when the test file is imported. Its result becomes the dimension's value list, in the order returned, and the matrix is the product of that list with the two listed dimensions. Three releases, three commands and four algorithms give 36 combinations; the exclusion rule removes the generate and save cases that name an algorithm, leaving 18.

The label can be any text, and it is the key under which the value reaches the test: case.params["Maze App Version"]. The function here is the same GitHub helper that Lambda Configuration Parameters uses for its selector fields. The difference is what the values are for. A SelectLambda field offers them for the user to pick one; a lambda dimension turns every one of them into cases.

Using the fetched values

Each case receives one release tag and resolves it to a folder holding that release's binary:

def _run_case(self, case: OrcraftTestCase):

	bin_folder = get_maze_app_bin_folder(case.params["Maze App Version"])

	maze_app = MazeApp(bin_folder, case.params["command"], algorithm=case.params["algorithm"])

	maze_app.run()

	return case.result

get_maze_app_bin_folder lives in the suite's shared github_repo_access module. It downloads the release asset for the current platform into a folder under the system's temporary directory and returns that folder, or returns it at once when the binary is already there. The first run of a version therefore downloads it, and every later case of that version reuses the copy.

The fetched value list

Click the Maze App Version column header. The Dimension Filter dialog lists the values the factory returned:

The values are checkable, and Select All and Clear All work as they do for a listed dimension, so the matrix can be narrowed to one release for a configuration in the way Filtering Matrix Dimensions describes. There is no Random Configuration group and no Add value field: the list is what the function returned, and a value that is not a published release could not be run.

Running across releases

Click Run and confirm with Yes. Nothing had to be configured, so Update Plan stayed disabled and the run executes the plan as loaded. The test reports 18 planned, done and passed cases, and the log's result table shows which release each case ran:

The cases run version by version, newest first, because that is the order the factory returned. Each case's log states whether its binary was downloaded or taken from the cache before the Maze application's own output.

Inspecting the run

Click Details for the completed run. The List Dimensions table records the three release tags as the dimension's default, planned and run values, next to the listed commands and algorithms:

This is the record that matters for a dimension whose values change over time. When a fourth release is published, the next configuration of the test fetches four tags and plans 24 cases, and the details of this run still show the three it covered.

This completes the matrix pages of the tutorial. The next group, Statuses and Results, begins with Orcraft Statuses: how a test tells the controller that a case passed, passed with a warning, failed or could not run at all.