Orcraft 1.3.0 released 19-Nov-2026.

Orcraft 1.3.0 released 19-Nov-2026.

Orcraft

Run the entire lab testing operation with your AI Agents.

Watch the demo

Orcraft vs Robot Framework

Both are test automation frameworks with Python underneath, and both produce logs and reports of every run. They differ in the language a test is written in, and in what happens around the test: the GUI of a run, the machines it runs on and the combinations it covers.

See the note on other products

Side by side

yes  ·  no  ·  words where a tool does it in its own way

Orcraft Robot Framework
What it is Test framework, controller and endpoint application Keyword-driven test framework
Test is code in Python Keywords
Writing a structured test suite with AI agents The whole suite, from instructions written for AI agents Keyword files, as text
Running tests with AI agents across the lab On the machines the request names On its own machine
Code first yes no
Code once yes Variables on the command line
Code only yes no
GUI Built from the test code Separate editors, such as RIDE
Remote machines in a run yes Through a remote library
Machine chosen per step yes Per keyword library
Test package upload yes no
Endpoint software update yes no
Live logs From every machine, in the GUI Console progress; the log after the run
Results tree At every level, from the session down to the step, with the configuration beside the results and a filter by status Log file, down to the keyword
Results export XLSX, session files, text for AI agents HTML, XML, xUnit
Machines with roles yes no
Fan-out: one run per machine yes no
Machines shared by the team yes no
Machine pools yes no
Case matrix From dimensions: randomized, fetched at run time, with exclusion rules, filtered in the GUI Test templates
Reboot inside a test yes no
Run in a CI pipeline A snapshot of the run, exported from the GUI The same command
Past runs loaded into the GUI Runs from the command line, CI or another machine, with their configuration, to change and rerun Log and report files, to read in a browser
OS Windows, Linux, macOS Windows, Linux, macOS

Keyword files and code

Robot Framework describes a test in a keyword language, and Python libraries carry out the keywords. A team maintains both: the keyword files and the code behind them.

In Orcraft the test is the Python code, and the controller derives the configuration GUI, the run plan and the reports from it. There is no second layer to keep aligned, and no resource files to distribute to the machines that run the tests: the controller uploads the test package to each endpoint when the run starts.

This is a flow and its step from the tutorial's Hello Endpoint test. The flow runs one step, and the step runs a command on the machine chosen for the run:

class FlowHelloEndpoint(OrcraftTestFlow):
	result_type = FlowHelloEndpointResult

	def __init__(self, config: FlowHelloEndpointConfig) -> None:
		super().__init__(config)

	def _run_flow(self) -> None:
		# No endpoint is named here: the step goes to the test's Test Endpoint.
		step_result = self.run_step(StepHostname)

		if self.handle_errors(step_result, self.result):
			return

		self.result.hostname = step_result.hostname
class StepHostname(OrcraftTestStep):
	def __init__(self, config: StepHostnameConfig | None = None) -> None:
		super().__init__(config or StepHostnameConfig())

	def _run_step(self) -> OrcraftTestStepResult:
		self.result = StepHostnameResult()

		# The step's body runs on the endpoint it was sent to, so the command
		# answers with that machine's name.
		completed = subprocess.run(["hostname"], capture_output=True, text=True)
		hostname = completed.stdout.strip()

		self.checkpoint(
			completed.returncode == 0 and bool(hostname),
			f"hostname answered: {hostname}",
			f"hostname failed with exit code {completed.returncode}: {completed.stderr.strip()}",
		)

		Logger.log(f"Hello from {hostname}", color=Logger.Color.BLUE)

		self.result.hostname = hostname

		return self.result

Machines

A Robot Framework run takes place on the machine where robot is started. Keywords can also come from a remote library, a server that is installed and started on the other machine.

In Orcraft the other machine is an endpoint: a Windows, Linux or macOS machine with the endpoint application installed. The controller uploads the test package to it when the run starts, and every step names the machine it runs on, so one test can act on several machines in order. The same test can also fan out to several endpoints, one independent run each.

Case matrices

Robot Framework repeats a test over data with test templates, and the data is written out in the files. In Orcraft, dimensions declared on a test expand it into a planned matrix of cases. The cases are filtered in the GUI and run in a defined or a randomized order.

AI agents

An AI agent can write keyword files as text, and the Python libraries behind the keywords: two layers to keep aligned. An Orcraft suite is one layer of plain Python in a defined structure, so an AI agent writes the test, configures a run, runs it on the lab's machines and reads the results. Every part of the work can be done by a person or by an AI agent. The AI Agents section shows how.

Existing Python libraries

A step is ordinary Python code. A Python library written for the keywords of a Robot Framework project can be called from an Orcraft step as it is.

All other product names are trademarks of their respective owners and are used for identification and comparison purposes only. Orcraft is not affiliated with, sponsored by or endorsed by any of them. The descriptions of other products on this page compare features and functionality to help you decide whether Orcraft fits your needs; they are not intended to disparage any product or its owner. They are summaries written by us, reflect our understanding at the time of writing, and may be inaccurate or out of date. For an accurate description of any product, visit its own site. If you believe a description is wrong, tell us and we will correct it.

Type to search the docs.

    ↑↓ navigate ↵ open Esc close