Running a group

We've run individual tests and reviewed every parameter type in the Basic Configuration Parameters group. Now we'll configure the group as one run and execute all nine tests together.

Reloading the session

For this demonstration, we'll first reload the session so every test starts from its source defaults. Select the session node, then click Reload:

The Reload action reloads every reloadable suite and top-level group in the current session, not only the item currently selected. We can use it here because our session contains only one added item - the Tutorial suite. In a normal session containing several items, we usually wouldn't reload the entire session just to prepare one group for a run.

Selecting the group

In the Test Explorer, select the Basic Configuration Parameters group rather than one of its child tests. The Configuration tab switches to a multi-test view: the tests appear in a tree on the left, and selecting a test opens its configuration on the right.

This is still one group run, but every test keeps its own fields, cases and planned values. Move through the test list to configure each one before starting the run. The table provides a quick overview of all test values, while the editor below it lets us configure the selected test.

All action buttons at the bottom now apply to the entire group - this is the purpose of a group. For example, Update Plan saves the configuration of every child test, and Reset To Defaults resets every child test - not only the test currently selected in the editor. The same group-wide scope applies to Export for CI and Run. Selecting a child in the table only chooses which test's fields are shown for editing.

Configuring the tests

Every child test must have valid values. Fields marked as required can't be left empty:

Set Maze Executable Folder for Folder, Boolean, Selector, File, File List, Numeric, Multi-Selector and String Parameter.
Add at least one entry to Maze Executable Folders for Folder List Parameter.
Select a saved maze file for Maze Save/Load File in File Parameter.
Select one or more saved maze files for Maze Save/Load Files in File List Parameter.

The other fields already have usable defaults. You can leave them unchanged for the first group run, or vary them to see how each test turns its selected values into a Maze application command.

Setting repeated values

Several tests need the same Maze Executable Folder. Start with Folder Parameter, click Browse and select the Maze application's v3.0.0 folder. The selected value appears in both the test editor and its row in the overview table:

Instead of entering the same folder separately for every test, use the value from that table cell. Hover over the configured Maze Executable Folder value and select Apply this value to all tests:

The folder is copied to every test with the same Maze Executable Folder field:

Folder List Parameter remains empty because it declares the separate Maze Executable Folders list field. Configure that test independently by adding one or more Maze application folders.

Updating the plan

Changed values are shown as unplanned until they are added to the run plan. If we click Update Plan now, Orcraft validates the whole group and stops because three required fields are still empty:

The dialog lists Maze Executable Folders for Folder List Parameter, Maze Save/Load File for File Parameter and Maze Save/Load Files for File List Parameter. No values are committed to the plan, and the existing plan remains unchanged. After we close the dialog, Orcraft takes us to the first missing field so we can continue configuring the group.

Clicking OK selects Folder List Parameter, scrolls the overview table to the missing Maze Executable Folders column and reveals the field in the editor. Add one or more Maze application folders here, then repeat the update attempt to find and configure any remaining missing arguments.

After configuring the folder list and the required file fields, every test is valid. The orange diamonds and purple values show that these changes are complete but still different from the current plan:

Select Update Plan again. Validation now passes, so Orcraft asks us to confirm that the current configuration should overwrite the existing planned values:

Select Yes to update the plan for every test in the group. The values turn blue, the orange diamonds disappear and Update Plan becomes disabled because the current configuration now matches the plan:

We could also run the group with unplanned values, just as we did with an individual test, but every required field would still need to be complete.

Test configuration is preserved

The group configuration isn't a separate copy. Every value configured here is also available when we select that child test for an individual run. While we're editing the group, those values are preserved as the test's draft configuration; after Update Plan, they become that test's planned values as well. There is no need to configure the test again.

For example, selecting Folder List Parameter on its own still shows the three Maze application folders we set while configuring the group:

Running the group

Let's select the Basic Configuration Parameters group again and click Run. Even though we selected Folder List Parameter on its own in the previous step, every test's configuration is still present when we return to the group - changing the selection doesn't discard any values. Orcraft then confirms the complete scope before starting: in this configuration, 9 test cases across 9 tests.

Select Yes to continue. Orcraft validates every selected test before the run begins. If a required value is missing, the Missing Arguments dialog identifies the test and field that still need attention; return to that test in the configuration tree, set the value and run again.

Once validation passes, Orcraft creates a run for every child test and starts all nine in parallel. Their test run rows appear beneath the group, with live progress in the Status column and a Running 9 summary above the tree:

Each child keeps its own log and result, while the group node provides one place to follow the complete run.

Reading the results

When execution finishes, the summary changes to Pass 9 and every child run shows its own Pass status:

Expand the completed group in the execution tree to inspect each test. Select a child result to read its command, captured Maze application output and final status.

Group details

The GUI doesn't assign a status to the group itself - running a group creates and runs its child tests, and each test run has its own status. To inspect the combined execution data, right-click the group and select Details:

The Details dialog brings the group configuration and child results together. On the left, each test compares its default, planned and run values. On the right, the results tree lists every test and cycle with its endpoint, status, timing and controller version, while the summary reports aggregate counts and progress:

This group contains one case and one cycle per test, so the hierarchy is still simple. The Details view becomes more useful later in the tutorial, when runs include multiple cases, flows and steps that can be inspected at their corresponding levels.

Details aren't limited to groups. Double-click any node in the execution tree - from the Session node through suites, groups and tests down to an individual test-run instance - to open the Details view for that scope. These details can also be exported to an XLSX workbook. We'll cover the deeper result hierarchy and XLSX export later in the tutorial.

Rerunning and clearing

Rerunning with a change

We can rerun the entire group with a changed value without updating the plan first. Here we change only Folder Parameter's Maze Executable Folder from v3.0.0 to v2.0.0, leave the orange diamond in place and click Run:

Running the group creates a new run for every child test, even though only one value changed. The latest Folder Parameter run receives a Plan Mismatch badge because it used v2.0.0 instead of its planned v3.0.0; the other eight new runs still match their plans:

The Details dialog makes the difference explicit by placing the planned and run values side by side. Folder Parameter's v3.0.0 Plan Value and v2.0.0 Run Value are highlighted, while the other tests retain matching values:

Cleaning up superseded runs

After repeated group runs, the execution tree can accumulate several run instances under every test. Select the group and choose Clean Up to remove all but the latest running or finished instance under each child:

The older, dimmed superseded runs disappear, leaving the latest run for each test. The Folder Parameter run keeps its Plan Mismatch badge because cleanup changes only which run instances are retained:

Clearing all run instances

To remove every run instance beneath the group, including running, queued and completed runs, select Clear. This keeps the group and all of its child tests:

After clearing, no test-run rows remain, but the configuration is untouched. The unplanned v2.0.0 Folder Parameter value is still present, along with the other test values, so clearing run history doesn't reset drafts or planned configuration: