Orcraft 1.1.3 public beta released 08-Nov-2026. The Free Plan and the Team trial are open; card payments and invoices open shortly.

Orcraft 1.1.3 public beta released 08-Nov-2026. The Free Plan and the Team trial are open; card payments and invoices open shortly.

Orcraft

The platform for test development and orchestration
that lets you and your AI Agents run the entire testing operation

Watch the demo

Running Folder Parameter test

Unlike the Hello World test, this test can't just be run as-is: its maze_executable_folder parameter is required and has no meaningful default, so a value has to be supplied before the test can do anything.

Running from CLI

Required field unset error

Running the test file as-is fails, because the required folder was never set:

python test_folder_parameter.py

With no folder to run the executable from, the case ends in an ERROR status, and the reason is carried into both the case's and the test's result tables:

Orcraft Controller (v0.1.0.dev139)
----------------------------------

                            Test: Folder Parameter | Description                            

Demonstrates a folder configuration parameter.
Runs the maze app executable with no arguments to display its help page and logs the output.


                                                  Test: Folder Parameter | Configuration                                                  

Name                   | Version | Update Endpoint App | Cycles | Case Cycles | Start Time               | Matrix | Maze Executable Folder
------------------------------------------------------------------------------------------------------------------------------------------
Test: Folder Parameter | -       | on_mismatch         | 1      | 1           | 2026-08-16  11:37:48.219 | -      | -                     


                               Test Cycle 1 | Configuration                          

    Name         | Start Time               | Number Of Cycles | Cycle | Total Cycles
    ---------------------------------------------------------------------------------
    Test Cycle 1 | 2026-08-16  11:37:48.220 | 1                | 1     | 1           

    Running from :
    	maze-win.exe

Traceback (most recent call last):
    ...
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect

                                                                  Test Cycle 1 | Result                                                               

    Name         | Status | Error Description                                                                | Finish Time              | Elapsed Time
    --------------------------------------------------------------------------------------------------------------------------------------------------
    Test Cycle 1 | ERROR  | [WinError 123] The filename, directory name, or volume label syntax is incorrect | 2026-08-16  11:37:48.227 | 0.007 s     


                                                              Test: Folder Parameter | Result                                                               

Name                   | Status | Error Description                                                                | Finish Time              | Elapsed Time
------------------------------------------------------------------------------------------------------------------------------------------------------------
Test: Folder Parameter | ERROR  | [WinError 123] The filename, directory name, or volume label syntax is incorrect | 2026-08-16  11:37:48.229 | 0.010 s     

To run it from the CLI, set the parameter's default in the config class to the folder you downloaded the Maze application into:

	maze_executable_folder = OrcraftConfigField.Folder(
		default=r"C:\Orcraft\Maze\bin\v3.0.0",
		label="Maze Executable Folder",
		required=True,
	)

Now the test has a value to work with when run directly, without changing anything else.

Pass run output

With a valid folder, the test runs the Maze executable and logs its output as part of the test's own log:

Orcraft Controller (v0.1.0.dev139)
----------------------------------

                            Test: Folder Parameter | Description                            

Demonstrates a folder configuration parameter.
Runs the maze app executable with no arguments to display its help page and logs the output.


                                                    Test: Folder Parameter | Configuration                                                    

Name                   | Version | Update Endpoint App | Cycles | Case Cycles | Start Time               | Matrix | Maze Executable Folder    
----------------------------------------------------------------------------------------------------------------------------------------------
Test: Folder Parameter | -       | on_mismatch         | 1      | 1           | 2026-08-16  11:22:24.002 | -      | C:\Orcraft\Maze\bin\v3.0.0


                               Test Cycle 1 | Configuration                          

    Name         | Start Time               | Number Of Cycles | Cycle | Total Cycles
    ---------------------------------------------------------------------------------
    Test Cycle 1 | 2026-08-16  11:22:24.004 | 1                | 1     | 1           

    Running from C:\Orcraft\Maze\bin\v3.0.0:
    	maze-v3.0.0-win.exe

    
    Maze App (v3.0.0)
    =================
    
    USAGE
      maze <command> [options]
      maze -h | --help
    
    COMMANDS
      generate    Generate and print a maze (no solving)
      solve       Generate or load a maze, then solve and print the path
      save        Generate a maze and save it to a JSON file
      load        Load a maze from a JSON file and print it
    
    GENERATION OPTIONS  (generate / solve / save)
      --width   INT       Maze width in columns            (default: 20)
      --height  INT       Maze height in rows              (default: 10)
      --seed    INT       Random seed for reproducibility  (default: random)
      --start   ROW,COL   Start coordinate                 (default: 0,0)
      --end     ROW,COL   End coordinate                   (default: last cell)
    
    SOLVE OPTIONS
      --algorithm NAME    Solver: bfs | dfs | astar        (default: bfs)
      --file    FILE      Load maze from FILE instead of generating
    
    SAVE OPTIONS
      --output  FILE      Output file path                 (default: maze.json)
    
    LOAD OPTIONS
      --file    FILE      Path to a saved maze JSON file   (required)
    
    EXAMPLES
      # Generate and print a 30×15 maze with a fixed seed
      maze generate --width 30 --height 15 --seed 42
    
      # Solve the same maze using A* (custom start/end)
      maze solve --width 30 --height 15 --seed 42 \
          --start 0,0 --end 14,29 --algorithm astar
    
      # Save a maze to file, then load and solve it with DFS
      maze save --width 20 --height 10 --seed 7 --output my_maze.json
      maze load --file my_maze.json
      maze solve --file my_maze.json --algorithm dfs
    

                         Test Cycle 1 | Result                     

    Name         | Status | Finish Time              | Elapsed Time
    ---------------------------------------------------------------
    Test Cycle 1 | PASS   | 2026-08-16  11:22:24.664 | 0.660 s     


    Test Cycles | Results

    Test Cycle   | Status
    ---------------------
    Test Cycle 1 | PASS  



                     Test: Folder Parameter | Result                     

Name                   | Status | Finish Time              | Elapsed Time
-------------------------------------------------------------------------
Test: Folder Parameter | PASS   | 2026-08-16  11:22:24.665 | 0.663 s     

Notice the Maze Executable Folder column in the configuration table - every config parameter a test declares shows up there, so the log always records exactly what the test was run with. The indented block in the middle is the Maze application's own stdout, captured and logged by the test.

We could keep this default as is, if as test developers we want whoever executes the test to always start from this folder. For demonstration purposes, though, let's reset it back to an empty default:

	maze_executable_folder = OrcraftConfigField.Folder(
		default="",
		label="Maze Executable Folder",
		required=True,
	)

Running from GUI

Let's click on the Folder Parameter test. Its configuration tab now shows the Maze Executable Folder field alongside the built-in Cycles field we met earlier. You can see the field is empty and marked in orange - meaning it's a required field that isn't set:

Missing argument

If we click Run now, we'll see a warning regarding the missing arguments:

Setting the folder

Let's click Browse and select version 2.0.0 of the Maze app to test it:

As you can see now, the field changed from its default, but is unplanned yet. Remember the details from the Editing run configuration chapter's summary:

We can run it as unplanned, but we'll click Update Plan:

Running the test

Let's click Run and see the same log we saw earlier from the CLI - note that here we ran with the v2.0.0 folder, while in the CLI we ran with the default v3.0.0:

Type to search the docs.

    ↑↓ navigate ↵ open Esc close