Vision-Assisted Cartesian Robotic System for 3D Block Construction

Abstract

Small-scale automated assembly is one of those problems that looks solved from a distance and stops looking solved the moment a real machine has to do it. A gantry that moves to a coordinate is straightforward. A machine that takes a structure a person designed, feeds itself one part at a time, picks each part up, turns it the right way round, stacks it on top of the parts already placed, and knows at every moment whether that actually happened, is a different problem, and most of the difference is feedback rather than motion. This is where our project comes in: a vision-assisted Cartesian robotic cell that builds three-dimensional structures out of 2.2 x 6.0 x 1.5 cm wooden blocks with no human placing a single block by hand. The system is built around three controllers and one brain. A Raspberry Pi 5 is the sole master: it runs the camera, the web server, the orchestration and every safety rule. An Arduino MEGA 2560 runs the gantry, a CoreXY X/Y stage with an added Z axis, a servo claw and a 28BYJ-48 rotation stepper, and executes a fourteen-phase pick-place-park cycle for every block. An Arduino Uno runs a separate feeder module, a two-stage hopper gate, a belt and an alignment servo, and confirms with two HC-SR04 ultrasonic sensors that exactly one block left the hopper and arrived at the fixed pickup point. The two Arduinos never talk to each other; the Pi is the only thing that couples them, and it does so over two independent USB serial links. The block can be laid either way round, which is why the machine carries two separately calibrated grids and not one: a vertical grid of 7 x 6 addressable cells and a horizontal grid of 3 x 10, sharing one 22.8 x 38.0 cm holder-travel envelope and one feeder cell at [0,0]. An overhead 160-degree fisheye camera watches the build surface through a colour-corrected, lens-corrected pipeline, detects the wooden blocks by warm-colour segmentation, and maps camera pixels to physical centimetres to grid cells through a saved workspace homography. The operator side is a browser application served from the Pi: a click-to-build console over the live camera image, and a 3D Build Studio in which a person designs a structure, gets immediate physics feedback on support, collisions and toppling, compiles the design into an ordered program of B col row level commands separated by grid-mode latches, and runs it while a live digital twin mirrors the real build from the firmware's own phase telemetry. The completed system placed blocks on hardware with a measured cycle time of 19.8 to 32.2 seconds per block, with the fixed phases of the cycle repeating to within 0.01 s across sixteen logged placements, and stacked a five-level tower on a single cell. The software is covered by 492 browser tests and 68 Python protocol tests. The work demonstrates that the useful engineering in a machine like this is not the kinematics: it is the discipline of never letting one part of the system assume what another part has actually done