Endpoints Config Types

Up to this point every step of the tutorial ran on the controller's own machine. Orcraft can send a step to any endpoint registered in the controller instead, and the test decides where its steps go through endpoint fields on its config class. An endpoint field is a configuration field like the folder and selector fields seen earlier, except that its value is a machine rather than a path or a number. The test body never names a machine: the flows hand each step the endpoint field to run on, and the value of that field is chosen in the run panel. The same suite therefore runs on the local machine while it is being written and across a lab once it is done, with no change to its code.

The framework offers a few field types, and the difference between them is what happens when more than one machine is selected:

• Endpoint holds one machine. A test declares one such field per role it needs: a generator and a solver, a server and a client, a host and a device.
• MultiSelectEndpoints holds a set of machines that take part together in one run. The test loops over them and decides what each one does.
• EndpointFanout also holds a set of machines, but each machine gets a run of its own. The same test is repeated once per selected endpoint, and the runs execute in parallel.

In the run panel every endpoint field shows its chosen machines and an edit button that opens a dialog listing the endpoints registered on the previous page. Two rows accompany the endpoint fields. Upload Test Package decides whether the suite's own code is sent to the endpoint before the run, and Update Endpoint App whether the endpoint application on the machine is brought to the controller's version first; both default to doing so only when needed, and both are explained on the next page. A package field such as Maze Executable Folder additionally shows an Endpoints line and an Upload choice, because its files travel to the machines where they are used.

The Endpoints group of the tutorial suite declares one test per field type. All of them reuse the flows and steps of Structure and Lifecycle and all need their Maze Executable Folder set, and this page follows them with the Windows, Linux and macOS machines registered earlier.

A single endpoint

An Endpoint field holds one machine. Its value is one of the endpoints registered in the controller, and every step of the test that does not name another endpoint runs on that machine. It is the simplest way to move a test off the controller: nothing in the test changes, only the machine chosen in the panel. A test that declares a single endpoint field runs entirely on one machine, whichever it is.

Source directory:

Every test config inherits a test_endpoint field that defaults to localhost. It stays hidden in the run panel until a test declares it, which Default Endpoint and Package Upload does with one line:

test_endpoint = OrcraftConfigField.Endpoint(default="localhost", label="Test Endpoint")

A step that names no endpoint of its own runs on this one, so declaring the field is enough to move the whole test to another machine. Select the test and click the edit button at the right of Test Endpoint. Choose the Linux machine:

Click Update Plan, then Run and confirm. Open the run's Details when it finishes:

Every step row carries GMG-TEST-3 in its Endpoints column. The steps ran on the Linux machine, with the Linux build of the Maze application that the package folder also holds, and their results came back to the controller exactly as they did from localhost.

Endpoints with roles

Source directory:

Some scenarios need more than one machine at a time, each with its own job: something that produces and something that consumes, a server and a client, a host and the device attached to it. For that a test declares several Endpoint fields, one per role, and its flows send each step to the field of the role the step belongs to. The roles are fixed by the test; which machine plays which role is chosen in the panel, and the same machine may be chosen for more than one role.

Multiple Endpoints and Files Transfer declares two such roles. It generates mazes on the first and solves them on the second, moving the files between them in the meantime:

endpoint_1 = OrcraftConfigField.Endpoint(default="localhost", label="Endpoint 1")
endpoint_2 = OrcraftConfigField.Endpoint(default="localhost", label="Endpoint 2")

The flows pass an endpoint argument to run_step so that each step goes to the machine of its role, and the file transfer between the two is a framework call the flow makes between the steps. Set Endpoint 1 to the Windows machine and Endpoint 2 to the macOS machine, update the plan and run. The Details show the split:

Within each flow the Delete Existing Maze File and Generate / Save steps ran on GMG-TEST-1 and the Load / Solve steps on GMG-TEST-5. The configuration table on the left records the plan and run values of both fields with their ports, so a run can always be traced back to the machines it used.

A set of endpoints

Source directory:

A MultiSelectEndpoints field holds a set of machines rather than one. The set is still one role in one run: the test receives all the selected endpoints together, loops over them, and decides what each one does. This is the type for a role whose size is not fixed, such as any number of devices that one host drives, or any number of solvers fed by one generator. Selecting three machines does not repeat the test; it gives the single run three machines to work with.

Multi-Select Endpoints keeps one generator and takes any number of solvers:

test_endpoint = OrcraftConfigField.Endpoint(default="localhost", label="Test Endpoint")
aux_endpoints = OrcraftConfigField.MultiSelectEndpoints(default=("localhost",), label="Aux Endpoints")

A MultiSelectEndpoints field holds a tuple of endpoints, and the test body loops over it, running the three transfer flows against each solver in turn. Choose the Windows machine as the Test Endpoint and the Linux and macOS machines as Aux Endpoints:

The field lists its selected endpoints one per line. It is still one run: the four cases each visit both solvers, and the endpoints play parts in one scenario rather than repeating it.

Fan-out

Source directory:

An EndpointFanout field also holds a set of machines, but with the opposite meaning: the test is repeated once per selected endpoint, and each repetition is a run of its own. The runs start together and execute in parallel, each appears as its own Test Run row in the execution tree with the machine it ran on, and the test's plan is the case count multiplied by the number of machines. This is the type for running the same verification on every machine of a lab, for example the same application check on Windows, Linux and macOS.

Endpoint Fanout runs the flows of the first test, but redeclares the inherited field:

test_endpoint = OrcraftConfigField.EndpointFanout(default=("localhost",), label="Test Endpoint")

An EndpointFanout field also holds a tuple, but each selected endpoint gets a run of its own. Select all three machines:

Total Cases reads 12 for a matrix of four cases: the four cases three times over, once per machine. Update the plan and run:

The test row plans 12 and holds three Test Run rows that ran at the same time. Each row's Endpoints column names its machine with a fan-out mark, and each has its own Pass in the Status column. Fan-out is the way to run one verification everywhere; the multi-select field is the way to let several machines take part in one run.

Queued runs

Source directory:

A run holds the endpoints it uses for as long as it runs, because a test usually owns the machine it is working on: two runs writing to the same files or driving the same device at once would corrupt each other's results. When a second run needs an endpoint that a first run is holding, the second run is queued and starts only when the endpoint is released. Queueing is automatic; the tree shows it as a status.

This matters when a fan-out and a fixed endpoint meet in one test. In Endpoint Fanout - Queued the generator, Endpoint 1, is a plain endpoint, and the solver, Endpoint 2, is a fan-out, so every fanned-out run needs the same generator. Choose the Windows machine as the generator and the Linux and macOS machines as solvers, then run:

Two runs are created, one per solver, but they do not start together. Both need the generator, so the second row shows Queued in its Status column until the first run releases GMG-TEST-1, then runs in turn. Machines that can serve several runs at once are marked as shared on their field, which lifts the queueing; the framework reference describes the option under field options.

Permutations

Source directory:

A test can declare more than one fan-out field. The runs are then the combinations, or permutations, of the selections: every machine chosen for the first role paired with every machine chosen for the second. Two machines in each of two roles give four runs, three in each give nine. Because that grows quickly, the panel lists the combinations and lets you keep only the ones worth running.

Endpoint Fanout - Permutations declares both its roles as fan-outs. Select the Windows and the Linux machine for both Endpoint 1 and Endpoint 2:

A Fanout Endpoints Permutations table appears with one row per combination, four here, each with a check mark in Select. Clearing a row drops that combination from the plan, which is how a machine can be excluded from solving its own mazes without touching either field. With all four selected, the four cases run 16 times. Update the plan and run:

Four Test Run rows, one per combination, each naming its generator and solver in the Endpoints column, and the test row sums them to 16 planned and 16 passed.

Every endpoint on this page was chosen by name. The next page, Endpoint Pools, lets a test ask for any available endpoint that fits instead, and the run pick its endpoints when it starts.