Orcraft vs labgrid
Both are Python tools for a lab where tests reach hardware on other machines. labgrid is a library that gives a test access to embedded boards; the test itself is written for pytest. Orcraft is the test framework and the system that runs it: the steps of a test run on the lab's machines, and a run has a GUI, a plan and results.
See the note on other products
Side by side
yes · no · words where a tool does it in its own way
| Orcraft | labgrid | |
|---|---|---|
| What it is | Test framework, controller and endpoint application | Python library for embedded boards in a lab |
| Test is code in | Python | Python |
| Writing a structured test suite with AI agents | The whole suite, from instructions written for AI agents | pytest tests, as Python |
| Running tests with AI agents across the lab | On the machines the request names | On boards, from the command line |
| Code first | yes | yes |
| Code once | yes | Environment file |
| Code only | yes | no |
| GUI | Built from the test code | no |
| Remote machines in a run | yes | Boards, driven from the test host |
| Machine chosen per step | yes | Per command, a board |
| Test package upload | yes | Files synced to the lab host |
| Endpoint software update | yes | no |
| Live logs | From every machine, in the GUI | Terminal output |
| Results tree | At every level, from the session down to the step, with the configuration beside the results and a filter by status | Per test |
| Results export | XLSX, session files, text for AI agents | As pytest |
| Machines with roles | yes | Named targets in the environment file |
| Fan-out: one run per machine | yes | no |
| Machines shared by the team | yes | Places, through the coordinator |
| Machine pools | yes | Reservations by tags |
| Case matrix | From dimensions: randomized, fetched at run time, with exclusion rules, filtered in the GUI | As pytest |
| Reboot inside a test | yes | Power cycles the board |
| Run in a CI pipeline | A snapshot of the run, exported from the GUI | As pytest |
| Past runs loaded into the GUI | Runs from the command line, CI or another machine, with their configuration, to change and rerun | no |
| OS | Windows, Linux, macOS | Linux |
Boards and machines
labgrid is built around the embedded board: a test takes a board, powers it, talks to its bootloader and its console. Exporters expose the boards connected to each lab host, a coordinator manages access to them, and a client on the test host takes a board over the network. labgrid is not a test framework: the tests are written for pytest with the labgrid plugin and run on the test host, so the GUI, the run plan and the roll-up of results are whatever pytest and its plugins provide.
Orcraft is built around the machine. An endpoint is any machine that can reach the hardware: a lab PC wired to a device under test, a server on the same network as the instruments, a machine with the vendor's tools installed. The steps of a test run there, as ordinary Python, and talk to the hardware over whatever interface it offers.
A run
The suite's structure gives every run a configuration GUI, a plan, live logs and results rolled up from step to suite. A run configured in the GUI is exported as a snapshot, and a CI pipeline runs that snapshot from the command line. A test can reboot its machine and continue when it comes back.
AI agents
An AI agent can write a pytest test that uses labgrid, and the environment file that describes the boards. With Orcraft it writes one Python suite in a defined structure, takes the machines from a file the controller saves, runs the test on the machines the request names, or on any free one that fits, and reads the results. Every part of the work can be done by a person or by an AI agent. The AI Agents section shows how.
All other product names are trademarks of their respective owners and are used for identification and comparison purposes only. Orcraft is not affiliated with, sponsored by or endorsed by any of them. The descriptions of other products on this page compare features and functionality to help you decide whether Orcraft fits your needs; they are not intended to disparage any product or its owner. They are summaries written by us, reflect our understanding at the time of writing, and may be inaccurate or out of date. For an accurate description of any product, visit its own site. If you believe a description is wrong, tell us and we will correct it.