Robot Rabbit

#195 · Alongside deeproute we tried to teach a neural network to…

2026-10-12T00:00:00+11:00

Alongside deeproute we tried to teach a neural network to route boards the way people do. All in one day, 11 October, on rented H100 and H200 GPUs at Runpod.

Data: 52,233 KiCad boards from GitHub, GitLab, Codeberg and open benchmarks, 6.56 million routing steps. Each human trace is cut into steps "direction, length, layer or via", and the model learns to guess the next step. Boards similar to the test ones were removed, and Rabbit was never in the training set.

What we tried:

a CNN over a 64×64 picture around the trace end: guesses the next step 72% of the time
a ViT over the picture at three scales: 72.7%
RouteTok, a transformer over board objects (pads, tracks, vias) instead of pixels, 14.6M parameters: 73.8% on 4 thousand boards, 79.5% on 26 thousand, 80.2% on 52 thousand
the same model at 40M parameters: 78.4%, worse than the small one
Qwen3-0.6B as the backbone: 66.9% with pretrained weights, 69.5% from random ones. Knowing text only hurts on geometry
DAgger: the model drives the trace and the classical A* says at every step what it would have done. The best thing that worked: the share of connections the network routes itself went from 77% to 93%
GRPO reinforcement learning: the model routed more connections itself but took up space, and the board as a whole routed worse. Classic reward hacking
an LLM writing coordinates as text: 45 times more tokens, about 2% of traces finished, dropped at once

The main lessons: data matters, not model size. Next-step accuracy says almost nothing about whether the model will route a board. And honestly: the best network leaves 35 unconnected on 46 test boards, the classical algorithm 2. So all of deeproute's records come from the classical core, and the networks remain a research branch for now.

The price of all this from the Runpod bill: $115.76, of which $103.69 for GPUs, about 23-24 GPU-hours (H100 about 17 hours, H200 about 6). One 14.6M model trains in about an hour on an H100 and costs a couple of dollars. Datasets on disk: 64 GB of raw boards and about 50 GB of prepared tokens.

#194 · The board is fully routed: 0 unconnected, 0 DRC errors

2026-10-12T00:00:00+11:00

The board is fully routed: 0 unconnected, 0 DRC errors. Last time fastroute stopped at 24 connections and 5 errors, and I thought the rest would be by hand.

While it was grinding, Claude Code wrote its own router in the next window in one day, deeproute: 14 thousand lines of Rust, 115 commits. The idea differs from Freerouting: all nets are routed at once and may overlap, and every overlap gets more expensive until they move apart. Then the stuck nets are ripped up together with the nets in their way and routed again. On my board it reached 15 unconnected and 0 errors in 4 minutes.

Then the two agents started working with each other. My agent ran DRC and found problems, deeproute changed the router after each one:

power tracks may not neck down to 0.15 mm mid-path, only right at the pad: immediately 69 unconnected
controller pins on power nets are sense pins, not the power path, 0.25 mm is enough: 53
reshaped the power layer from its diagnostics (separate copper for the Raspberry Pi, the arm and the spare 5 V): 22
rip up stuck nets together with their neighbours: 10

It also found that KiCad's export cuts the corners of the motor drivers' L-shaped pads, and I found that it put vias 0.04 mm apart, which no fab can make. The funniest one: nobody could route two connections because the keepout around the camera cable cutout covered two resistors. I finished the last 10 by hand and with a small A*.

Then two independent reviews. Power: the router honestly put one via where 15 A goes from a pour into the power plane. Added 3-8 each. Manufacturing: copper reached the unplated cutout, and the inner signal layers had 4% copper against 90% on the planes, so the 2 mm board could bow. Poured ground on them.

deeproute on an open set of 1113 boards: 86.6% routed clean, fastroute 69%, Freerouting 66%, and 5 times faster than Freerouting. They also tried machine learning for the router, on 52 thousand boards from GitHub, but the records above come from the classical core.

Files for JLCPCB are ready: black mask, plated edge, 8 layers, 2 mm.

#193 · The board for the factory turned out harder than I thought

2026-10-11T00:00:00+11:00

The board for the factory turned out harder than I thought. It replaces the RoboClaw, the buck converters, the servo driver and a pile of wires: an STM32, two motor drivers, the Raspberry Pi right on the board, an electronic fuse with current readback on every load, the xArm, 6 ToF sensors, 3 lidars.

The fun part was the routing. We started with Freerouting: on this board it spent 37 minutes just fanning the pads out to vias. Then we found fastroute, a Rust port of Freerouting: the same stage in 25 seconds. After that we ran experiments and watched how many links stayed unrouted:

6 layers, power pours handed to the router as obstacles: 350 unrouted after 4 hours
8 layers and pours after routing: 91
a split power plane, and the 10-15 A nets only on the 2 oz outer copper: 76
threw out 92 of 763 parts, updated fastroute from 0.1.7 to 0.1.13: same plateau, 1.5-2x faster
widened the board up to the wheels and lengthened the tail, +30% area, tuned the router rules: 56
Fable redid the part placement from scratch: the microcontroller block moved to the tail, a 2 mm ring around the fine-pitch QFNs, an empty band between each motor driver and its connector for copper: 27

Funny that the new placement gave more than the extra layers and area combined. And Fable separately caught that a 1.2 mm track at 10 A would burn, even though DRC happily passes it. The last 27 links we finish by hand.

We'll make it at JLCPCB in one order, with almost nothing done by hand: a script exports the factory files and checks each of the 166 BOM lines against their stock through the API, and the factory solders everything on both sides. In the same box come 10 sensor boards that an agent designed in an evening (it turned out the VL53L8CX can't run on 3.3 V alone, it needs its own 1.8 V regulator and a level shifter) and nylon brackets printed there too. Roughly ~$1.5-1.8k.

#192 · How the process works

2026-10-11T00:00:00+11:00

How the process works. Everything that can be code is code, and agents run the loop "edit, build, check":

mechanics: build123d on OpenCascade, a live three.js viewer, collision checks with manifold3d in 3 seconds instead of 8 minutes of exact booleans, renders for me
the board: schematic and placement in design.py, KiCad builds the board from a script, fastroute (a Rust port of Freerouting: fanout in 25 s instead of 37 minutes) routes it, kicad-cli does DRC and the factory files, the JLCPCB API gives stock and prices
models: Claude does the work, independent agents review against datasheets, Fable gives a second opinion, Jev from TypeSafe cross-checked the chassis part names
all datasheets now live in the repo, and every number in the schematic points to a page

The main rule: one agent does, another checks. Every review writes a report with numbered findings, and the fixes are recorded separately, so you can see what was found and what was done about it.

And it's a great feeling: a program for any task gets written in minutes, right along the way. Needed a 3D renderer, a ray tracer for the sensors, a collision check, a script for the JLCPCB stock, and there it is, made exactly for this project. We don't care what that code looks like, we're not publishing it: these are ad-hoc tools to get to the goal. Basically, nothing holds you back.

I counted from the logs what this costs. From 2 to 11 October: 8 sessions over 6 working days, 3.3 billion tokens, almost all of it the context re-read from cache, and 3.1 million output tokens. Without a subscription, at Opus 5.5 API prices, it would have been ~$1,200-1,500.

#191 · Now agents change the robot's design

2026-10-11T00:00:00+11:00

Now agents change the robot's design. The chassis STEP was split into 134 named parts, and everything of mine is Python code, so any change can be built, measured and checked for collisions in a couple of seconds.

The coolest part is the proximity sensors. I said "I want 8 around the robot", and Claude wrote a ray tracer: from every point of the board edge it casts the sensor's 8x8 zones through the whole model, with the wheels steered ±30°. The sides turned out 20-100% blocked by wheels, gearboxes and steering links, so 6 sensors remained: front and rear centre and on the chamfers. Then I said "I want the brackets to sit square on the edge", and the board changed shape: the chamfers turned to face the sensors, the nose grew 21 mm for the ZED 2i camera and the tail 23 mm.

#190 · I wanted to move to BLDC motors so the robot would drive…

2026-10-11T00:00:00+11:00

I wanted to move to BLDC motors so the robot would drive faster and quieter. One condition: drop-in, fitting the stock 37D brackets without rework.

with Claude I went through everyone who makes 37 mm BLDC motors. Nothing fits "like the original" with an A/B encoder: the Chinese JGB37 BLDC ones come with Hall sensors only, and Hiwonder doesn't sell them at all
from maxon we downloaded the whole 73 MB catalogue and parsed every motor, gearhead and encoder into JSON to find a combination for 1.25 m/s. The winner was ECX FLAT 22 + GPX 22 + ENX 22, but that's €430 each plus an adapter plate that has to be milled
we put the maxon into the robot's 3D model and checked the clearance to the deck: 4.3 mm, it fits

Then I worked out that for a flat the stock motors already do 1.25 m/s. I went back to the stock JGB37 and simply threw out the RoboClaw: the motors are now driven by my own board.

#189 · The robot's middle deck won't be acrylic with a pile of…

2026-10-03T00:00:00+10:00

The robot's middle deck won't be acrylic with a pile of modules and wires anymore, but a PCB of the same shape. The outline and standoff holes were taken straight from the chassis 3D model, so it drops in where the acrylic was.

On the board: the Raspberry Pi, the RoboClaw, step-downs, a fuse per branch, an INA4235 with shunts, a PCA9685 for steering, a clock and all the connectors along the edges, each facing its device. The bottom is empty, that's where the Jetson and the motors are. 4 layers, the battery path carries ~16 A.

The coolest part is that nobody drew it with a mouse. The board is described in Python, KiCad builds it from the script, Freerouting routes the signals, and kicad-cli does the checks and the factory files. Change something in the code, rerun, get a new board. 5 boards with assembly at JLCPCB come to ~$200-260.

The motor section is deliberately kept in one corner: I want to move to BLDC motors and later replace the RoboClaw with a controller right on the board without moving everything else.

#188 · Drew all the wiring for the new architecture with Claude…

2026-10-03T00:00:00+10:00

Drew all the wiring for the new architecture with Claude: every wire, pin, protocol and gauge. A few fun things came up along the way:

the RoboClaw E-stop latches by default and its input is pulled up, so a broken wire means "go ahead". We flipped it: the Pi holds the line only while the safety process is alive, and a broken wire or a hang means stop
the bumpers don't go to the stop line, otherwise a robot that hit a wall couldn't back away from it
the RoboClaw datasheet says outright that USB drops in a noisy environment and doesn't recover on its own. The main link will be UART
each of the four ToF sensors sits on its own I2C bus, so no address changes and one hung bus doesn't kill the others
the LD19 is discontinued, the lidar will be an RPLIDAR C1

The shopping list came to ~A$825, more than half of everything I already have.

#187 · I want to move the robot to a new architecture

2026-10-03T00:00:00+10:00

I want to move the robot to a new architecture. Right now one Jetson runs everything: the camera, maps, navigation, the database, the web and the tunnel, with 8 GB of memory shared by all of them and the GPU. Plus the robot is blind behind, hence the 70 seconds of pushing a wall with its rear.

The idea is to split it into a brain and a body. The Jetson keeps only what needs the GPU: the camera, depth, localization, the map and the planner. My old Raspberry Pi 4 with 8 GB becomes the body: motors, steering, power, a 360° lidar, ToF sensors and bumpers, a rear camera, Forge, the HUD and video. And most importantly, collision protection on the Pi won't depend on whether the camera process is alive.

Along the way I went through the power telemetry. The 12 V step-down for the motors hits its limit on starts and sags to 11.3 V, and the battery current sensor showed 7.6 A against a ceiling of 8.2 A, so it most likely just clipped the peaks. In the new layout every branch has its own fuse, and the Pi can power off and restart a hung Jetson. And there will be a proper power button: 93 of 104 Jetson boots came after the power was simply pulled.

#186 · After a few days with Claude trying to make navigation and…

2026-10-03T00:00:00+10:00

After a few days with Claude trying to make navigation and positioning precise and accurate, I now understand why my robot vacuum is so dumb.

I carried the robot to another room and for 10 minutes it couldn't figure out where it was. A vacuum in that situation honestly asks you to build the map again
the camera sometimes confidently reports that the robot flew off 133 metres. Had to write a filter that doesn't believe jumps over half a metre
there's a 30 cm blind zone in front and nothing at all behind. The robot pushed its rear into a wall for 70 seconds thinking it was driving. Now it's clear why the vacuum has a bumper it keeps kicking things with: it's the cheapest and most reliable sensor
while exploring the flat the robot leaves a room without finishing the corners and then comes back. Exactly like the vacuum

And that's one flat, good light and a robot that knows everything about itself: geometry, turning radius, where the camera is. A $2000 vacuum has a lidar, a bumper and cliff sensors and it still gets stuck under the sofa. I respect it now.

#185 · A separate note on how we split resources

2026-10-03T00:00:00+10:00

A separate note on how we split resources. Orin Nano has 6 cores and 8 GB of memory for everything, GPU included, and at the start the camera alone was eating almost two cores.

all interrupts sat on CPU0 and it was 100% busy. Spread them out: the camera's USB on CPU2, the RoboClaw UART on CPU4, I2C on CPU5. The Wi-Fi interrupt can't be moved, on this platform it's nailed to core zero
py-spy showed that 90% of the camera process CPU isn't my Python but the tracking optimizer inside the SDK. No point rewriting it in C++, it just needs to be called less
the camera has three modes: 30 fps when something is moving, 5 fps when the UI is open, 1 fps when nobody is watching. Depth on every second frame: GPU from 34% to 19%, power from 13.6 to 12.4 W. The 15 fps cap I actually removed: same CPU, but the pose arrives 55 ms later
the map is built by nvblox on the GPU: 184 MB and ~2 ms per frame, the built-in ZED map was +2.1 GB
ClickHouse did an insert every second into every table and ate 75% of a core. Once every 10 seconds and the load halved

The camera dropped from 170-250% to 30-100%, memory from 6+ GB to 4. The real bottleneck turned out to be memory, the GPU is only 20-25% busy. While recording an SVO memory ran low, nvblox couldn't allocate a chunk on the GPU and took the camera process down. On a Jetson video memory is the same RAM.

#184 · Said "let's rewrite everything", the robot isn't in…

2026-10-03T00:00:00+10:00

Said "let's rewrite everything", the robot isn't in production. First a benchmark on recorded drives. RTAB-Map finds itself on another drive's map in 0.5-2.6 seconds. GEN_3 manages it in 6.7 in one direction and not at all in the other. Out of 40 "switched on anywhere" starts RTAB-Map locked on in 35, once off by 86 cm. cuVSLAM computes odometry in 2.5 ms per frame. But there's no ground truth yet, so I need to stick marks of masking tape on the floor.

#183 · Asked a question: aren't we reinventing the wheel?

2026-10-03T00:00:00+10:00

Asked a question: aren't we reinventing the wheel? The agent did a survey and answered: in navigation no, in localization yes. Nav2 from ROS for example can't do different turning radii left and right, so we keep our own Hybrid A*. But all the wrapping around GEN_3 (timeouts, the millimeter filter, map archives, resets) is patching a black box where one pose is both odometry and global positioning at the same time.

First step: split the pose into odom and map. Navigation now drives on continuous odometry and map jumps don't jerk it. On a 2.8 m drive with a turn the difference is 3 mm.

#182 · Yesterday there was an hour and a half of lowered clocks…

2026-10-03T00:00:00+10:00

Yesterday there was an hour and a half of lowered clocks and I thought it was over-current throttling. Turns out I did it to myself: while turning off GNOME systemctl isolate multi-user.target restarted nvpmodel, and it brought back the default minimum clocks, basically undoing the same jetson_clocks from #164. Now jetson_clocks always runs after nvpmodel. Real over-current is there too, but that's expected: 1-2 times a second the clock drops by half for about a millisecond, around 0.5% of performance in total.

#181 · Turned the robot off for the night, but before that dumped…

2026-10-02T00:00:00+10:00

Turned the robot off for the night, but before that dumped all the day's data into Parquet, and the agent dug through it overnight without the robot. It found a great one: for a few seconds after relocalization the ZED SDK returns coordinates in millimeters, even though the settings say meters. 21 576 such poses in a day. That's where the 4 kilometer "jumps" in the logs came from, and a broken map with obstacles exactly where the robot had just driven. Also the KNOWN_MAP status flips back to INITIALIZING 5-8 seconds later, so "found itself" now only counts after 10 seconds of stable status.

The filter turned out simple: if the pose jumps further than the robot could have driven, it goes into quarantine for 4 seconds. If it comes back, it was a glitch. If it stays at the new place, it's a real correction.

def update(self, now, position):
    if self.accepted is None or self.reachable(self.accepted, now, position):
        self.accepted = (now, position)
        self.pending = None
        return True
    if self.pending is not None and self.reachable(self.pending[1:], now, position):
        if now - self.pending[0] >= self.settle_s:
            self.accepted = (now, position)
            return True
        return False
    self.pending = (now, now, position)
    return False

#180 · "Drive up to the fridge" didn't work before because the…

2026-10-02T00:00:00+10:00

"Drive up to the fridge" didn't work before because the chat agent had no planner and sent the robot blind. Now a separate node builds a Hybrid A* route on a traversability grid that nvblox computes on the GPU in 1-5 ms (before, on the CPU, up to 3 seconds). A route is planned in 5-30 ms and replanned if something shows up on the way.

The fridge was fun: first the robot stopped in front of a wall corner, then 2.26 m from the fridge, and in the successful trip it changed its mind 6 times between the short route and a detour through the whole flat. In the end 5.35 m in 52 seconds and a stop 30-40 cm from the door.

#179 · Found a bug in ZED SDK 5.5: GEN_3 tracking keeps adding…

2026-10-02T00:00:00+10:00

Found a bug in ZED SDK 5.5: GEN_3 tracking keeps adding keyframes even when the robot just stands still. 265 of them in 9 minutes, CPU from 31% to 135%, the map file swelled to 70-80 MB. Workaround: by default the camera only localizes in the existing map and extends it only for a new map or during exploration. At rest the camera process is now 17-45% CPU instead of 134-185%.

#178 · Put the robot on a table and watched it fall through the…

2026-10-02T00:00:00+10:00

Put the robot on a table and watched it fall through the floor in the UI. The floor estimate assumed the wheels are always on the floor and moved up 69 cm together with the camera. Plus the glossy parquet and glass gave reflections, there was a whole mirror room under the floor. Now a new floor level is accepted only after driving a meter on it, and depth rays are clipped at the floor plane right in CUDA.

Also the map is now made of 5 cm voxels, just like the minecraft world I wrote about in #161. At first the robot was driving over bricks sticking out of the floor: the floor sat exactly on a voxel boundary and ±1 cm of noise pushed it into the row above. Shifted the grid by half a voxel and bricks went from 14% to 2%.

#177 · During the day three agents worked in one repo and on one…

2026-10-02T00:00:00+10:00

During the day three agents worked in one repo and on one robot at the same time: one moved the map to nvblox, the second did the detector, the third did voice. They coordinated with messages like "taking the camera", "releasing the camera" and "don't deploy, my image is still building". One of them still ran a local vite build, someone else's deploy shipped it to the robot and the public UI broke with CORS. Just like a real team.

#176 · Added voice

2026-10-02T00:00:00+10:00

Added voice. OpenAI Realtime right in the UI, all the same tools as the chat, charts show up in the same feed. At first it only answered in English because the shared prompt said "always write in English".

You can confirm a mission by voice, but the decision is made by code from the transcript of what I said, not by the model: only short phrases, and any negation wins, so "yes no, cancel" is a no.

#175 · Added object detection

2026-10-02T00:00:00+10:00

Added object detection. We went through a bunch of options from cloud models to plain YOLO on COCO, but the camera sits 13 cm above the floor and models have barely seen this angle: the TV stand was called a bench and the TV a blackboard. In the end it's YOLOE with an open vocabulary, 58 classes baked into the ONNX, and the ZED SDK builds TensorRT itself, computes 3D boxes and tracks objects. Had to add a separate "low wooden cabinet" class, otherwise it couldn't find the TV stand. The first TensorRT engine build took 9 minutes.

#174 · Yesterday I threw out the old nvblox (#160) completely…

2026-10-02T00:00:00+10:00

Yesterday I threw out the old nvblox (#160) completely, together with the self-built torch, in favor of the built-in spatial mapping from the ZED SDK. Today it's back, but done right this time.

The camera process was eating 185% CPU and 3.4 GB of memory. Profiling showed Python had nothing to do with it: almost all the time is inside the SDK, in the tracking optimizer. Along the way the agent attached gdb to the live camera process and froze it for 2 minutes, so gdb on the camera is now banned. Then we recorded an SVO and ran the same drive through different configs: the ZED map is +2.1 GB of memory, while nvblox builds a fuller map in 184 MB and ~2 ms per frame on the GPU. The agent built nvblox on the Jetson and wrote a small C++/CUDA extension with nanobind that takes depth from the camera and outputs mesh blocks in the same format as before. The camera process went from 3.4 GB down to 1.4 GB.

#173 · The robot is now reachable from the internet through a…

2026-10-02T00:00:00+10:00

The robot is now reachable from the internet through a Cloudflare tunnel: live.rabbit0.dev. Then I realized anyone with this link can drive the robot around and reset the map, so I made personal links. I give a link to a friend, they play around, then I delete it and 5 seconds later everything drops for them.

#172 · In the morning the map disappeared from the UI

2026-10-02T00:00:00+10:00

In the morning the map disappeared from the UI. Tracking ran away to -138 meters, and I was sending mesh vertices as int16 in millimeters, so ±32 meters max, and everything overflowed. Now every chunk has its own origin. Never figured out why tracking ran away.

#171 · Went to sleep and left agents working overnight: code…

2026-10-02T00:00:00+10:00

Went to sleep and left agents working overnight: code review, UI performance, Forge in docker compose. In the morning there were 12 commits. The reviewer btw found that after any rejected mission all the safety trips (collision, stall) were silently turned off. Good that the reviewer found it and not a wall.

#170 · Asked for a command so the robot builds a map of the flat…

2026-10-01T00:00:00+10:00

Asked for a command so the robot builds a map of the flat by itself. All the ready-made packages for this are tied to ROS, so our own again: frontier exploration and Hybrid A* for Ackermann with forward and reverse.

At first the robot didn't go anywhere at all. I had tuned the depth thresholds for a pretty map, and in front of a white wall in dim light the camera started throwing away 80-90% of points. The robot thought it was blind. I reverted the thresholds and it drove 6 meters in 46 seconds. A clean map and dense obstacles are fed by the same depth and you have to pick.

#169 · The robot crashed into a wall

2026-10-01T00:00:00+10:00

The robot crashed into a wall. Twice. The safety guard was supposed to stop it at 15 cm, but the camera can't see depth closer than ~30 cm (I wrote about the blind zone back in #154). The wall just disappeared from the scan right before contact and the robot decided the path was clear. Now it stops at 30 cm and obstacles are remembered in world coordinates for a few seconds.

#168 · At some point the Jetson rebooted by itself right in the…

2026-10-01T00:00:00+10:00

At some point the Jetson rebooted by itself right in the middle of maneuvers. After that the RoboClaw moved from /dev/ttyACM0 to /dev/ttyACM1 and the node crashed, and the camera got stuck looking for itself in the saved map. The best part came next: the motors pulled 6 A each and the robot "didn't move", I already thought something was jammed. But it was moving, the camera pose was just frozen.

#167 · First autonomous drives

2026-10-01T00:00:00+10:00

First autonomous drives. The mission queue lives on the robot: "turn around and drive one meter forward" works, it turns 180° within a degree. You click in the 3D scene and the robot drives there, and if the point is behind, it turns around back and forth in a few moves.

#166 · The UI is completely rewritten: now it's one 3D scene with…

2026-10-01T00:00:00+10:00

The UI is completely rewritten: now it's one 3D scene with the robot in Crysis style, telemetry panels, a third-person camera and an AI chat right in the interface. In the chat you can ask about the data and get a chart, or ask the robot to drive, but any motion needs an APPROVE click. The bunny model with ears was also drawn by the agent.

#165 · In parallel I started Forge

2026-10-01T00:00:00+10:00

In parallel I started Forge. The idea is simple: write absolutely everything the robot publishes into ClickHouse on one clock and add an agent that answers questions about this data. For common questions there are pre-checked queries (I called them slabs), for everything else the agent writes SQL itself, but it goes through a static check and EXPLAIN before it runs. Plus anomaly detection with Chronos-2, a foundation model for time series: it predicts what a signal should look like and an anomaly is when the signal leaves the forecast band. The next day Forge moved right onto the Jetson, so recording no longer depends on whether my mac is on.

#164 · I haven't written anything for over a year

2026-10-01T00:00:00+10:00

I haven't written anything for over a year. The robot mostly just sat there: when I turned it on, docker said all the containers had exited 5 months ago. I decided to come back to it with a new approach: almost all the code is now written by Claude Code agents, and I set the tasks, watch what comes out, put the robot on the floor and carry it from place to place.

The first finding was a funny one right away: the ZED couldn't hold 30 fps because the CPU was sitting at 730 MHz out of 1728 and the GPU at 306 out of 1020. Nobody had run jetson_clocks. One command and frame capture went from 34 ms to 14 ms.

#163 · Got to the differential

2025-08-30T00:00:00+10:00

Got to the differential. Implemented and calibrated it. Now the turning radius is minimal and the wheels don’t slip when turning.

#162 · Next, you can build graphs based on this map

2025-08-16T00:00:00+10:00

Next, you can build graphs based on this map. Nodes are voxels with weights, knowing the cost you can use it for route planning, avoiding expensive sections, for example so the robot doesn't get too close to walls or crash into them. Then dijkstra or a* and the robot can search for a path from its position to the goal using pure pursuit.

Right now it's super rough and not accurate, needs calibration and bugfixing. But the fact that the e2e pipeline already works at all is awesome, even though it took so much pain and time and I've probably only moved like 5% towards the goal of implementing local navigation.

#161 · The next step was hooking up the vision pipeline to the…

2025-08-16T00:00:00+10:00

The next step was hooking up the vision pipeline to the robot. At first, I thought about writing the depth map and frames into a shared file but NATS easily handles a low-res stream. So in the end the camera sends rgb + depth to a topic and the nvblox node subscribes to them. For high resolution I’ll have to figure something else out but for now, it works. The camera intrinsics and pose get updated in the kv store instead of just sending them as messages to topics, which turned out to be much more convenient (even though under the hood it’s basically the same thing)

nvblox takes the depth and position and builds the tsdf (distance field to the surface) on the GPU in real time. It smoothes out noise, merges a lot of frames and gives you a map with free and occupied areas. From this you can get a voxel map just like a Minecraft world.

#160 · Built nvblox on jetson

2025-08-16T00:00:00+10:00

Built nvblox on jetson. There's no ready torch package so I had to build it myself. It crashed three times for different reasons, in the end enabling swap helped since jetson doesn't have enough memory to build the package. Building C++ with cuda and python is also painful, even with a docker container where you can build everything, it's not hermetic, it falls apart.

Then I struggled with a minimal dockerfile. I managed to run the tests, but the docker image is huge with a ton of dev dependencies I don't understand so I wanted to get the bare minimum runtime. In the end, after a few days of cleanup, I built a compact image with binaries and python bindings.

#159 · Spatial mapping + three.js

2025-08-09T00:00:00+10:00

Spatial mapping + three.js

#158 · More and more I’m using NATS in the project

2025-08-08T00:00:00+10:00

More and more I’m using NATS in the project. Threw together a simple UI to control camera settings from the UI. For this I decided to step away a bit from the idea of using pub sub for everything and take a look at jetstream. By default NATS core doesn’t have persistence but you can turn it on by enabling jetstream. Basically it’s just one flag and now you get a persistent KV store, object store and some delivery guarantees. All of this is super useful and really easy to use. For example camera settings are just a KV. Both the client and the robot subscribe to this key (in jetstream everything’s still just subjects) and can react to changes, both of them can also change the value and all clients instantly react to it and after restart the state is restored.

#157 · Decided to quickly throw together a blog in English, just…

2025-07-30T00:00:00+10:00

Decided to quickly throw together a blog in English, just to have one. I'll copy everything from robotrabbit0 here. https://rabbit0.dev

#156 · I haven't written anything for a while but that doesn't…

2025-08-08T00:00:00+10:00

I haven't written anything for a while but that doesn't mean I wasn't doing stuff. While I was in Byron Bay I didn’t have the robot around so I worked on a content generator for the blog.

All posts from Telegram go into a markdown file, with metadata for each post.
Every post gets translated through the ChatGPT API
All images get compressed to webp
A preview is generated for every video using ffmpeg and it also gets compressed to webp
Wanted to do it all without any javascript at all
Deployed on cloudflare.
Source code is here rabbit0.dev

#155 · I'm really annoyed that NVIDIA removed the hardware h264…

2025-07-22T00:00:00+10:00

I'm really annoyed that NVIDIA removed the hardware h264 encoder from the Jetson even though they market it as an edge AI device with two CSI ports for cameras. With a software codec, CPU load at 1080p@30fps reaches up to 40% and power consumption shoots up a lot.

After looking into the topic a bit, it feels like the Raspberry Pi 5 also lost the hardware encoder because of a chip shortage, but the rpi4 still has it. By the way, I have one and used it in the robot until I rebuilt it with a Jetson. At the same resolution and frame rate, the rpi4 pulls a bit over 5W and CPU load isn’t more than 5% handling the stream without any hiccups.

Now I’m starting to think maybe moving all the mechanics control and camera streaming from the Jetson to the rpi and merging them into a cluster isn’t such a bad idea. It seems like the extra 5W won’t matter but will free up all resources for Huang.

Now there's the question of how to connect them together since a NATS server needs to run somewhere. Ethernet + a mini switch or ethernet over usb?

#154 · Because the baseline of the stereo pair is pretty wide it…

2025-07-22T00:00:00+10:00

Because the baseline of the stereo pair is pretty wide it can only measure depth from 30 cm. This creates a blind spot in front of the robot. Tolerable but not very nice. Maybe it's worth thinking about a lidar or a proximity sensor for obstacle avoidance.

#153 · I couldn’t resist and decided to play around with points…

2025-07-22T00:00:00+10:00

I couldn’t resist and decided to play around with points cloud and depth measurement. For now I tried the lightest model.

What you see at the bottom of the screen isn’t SLAM yet, it’s just a stateless point cloud that gets published from the robot to nats at 30hz and drawn using treejs in the browser. The further a point is, the greener it gets. That’s just for convenience but in code it’s just a 3 or 4 dimensional matrix (4 if you want to keep the original camera color)

Turns out the tricky part is that 1280x720 is way too heavy and just the raw byte blob of this matrix is 70+Mb. Pumping such an array 30 times a second is impossible because of the 64Mb per message limit and bandwidth restrictions. After dropping the resolution to 320p and compressing it with zlib, I managed to get a couple hundred kilobytes which seems ok. Plus this matrix is sparse so you could also cut out the zeros. Maybe it doesn’t even make sense if you compress anyway.

#152 · Now I have to squash everything back into a single node…

2025-07-22T00:00:00+10:00

Now I have to squash everything back into a single node that exclusively has access to the camera and does everything.

#151 · Because of this, the idea brings a few other problems

2025-07-22T00:00:00+10:00

Because of this, the idea brings a few other problems. At first, I thought about making several nodes that work with the camera. Since only one process can access the camera, I thought about local streaming through the ZED SDK. So, one process connects to the camera and streams a compressed h264 stream. The SDK also allows you to read other parameters through streaming as if you’re using the camera locally. But the problem is the streaming in the SDK really depends on hardware encoding and without it everything crashes and doesn’t work.

#150 · Another disappointment: the NVIDIA Jetson Orin nano…

2025-07-22T00:00:00+10:00

Another disappointment: the NVIDIA Jetson Orin nano doesn't have hardware h264 encoding.

#149 · Today I caught myself thinking that there are so many cool…

2025-07-22T00:00:00+10:00

Today I caught myself thinking that there are so many cool ideas I want to try out but already a ton of technical debt I probably should deal with first.

At the very least, I really want to finish the mechanics right now so I can move on to perception.

redo the steering
reinforce the second floor (laser cut an acrylic plate and rebuild on a new platform)
change the rear motors or raise the axles a bit and buy bigger wheels
tweak the acceleration or distribute the weight better (right now the rabbit jumps a bit when accelerating hard)

#148 · Big milestone

2025-07-21T00:00:00+10:00

Big milestone. It can move and be controlled remotely.

runs on battery
video streams in real time at 720p@30
controller lag is minimal

Problems:

steering is super twitchy
gear ratio is way too high and the robot is really slow and noisy (bldc?)
video isn’t compressed with h264, I just take every frame and compress it as jpeg so tons of gigabytes are flying around, which won’t work over 4g
steering angles are very small, need to redo the hub and links

#147 · Today I wired up all the cables and components

2025-07-21T00:00:00+10:00

Today I wired up all the cables and components. Had to take it apart and put it back together five times because every time some new crap came up.

On the bright side, the battery is awesome. First, it can pass current when it’s plugged into the charger and it can charge and power the robot at the same time.

#146 · Wired up the rear motor controller, voltage sensor, 2 step…

2025-07-20T00:00:00+10:00

Wired up the rear motor controller, voltage sensor, 2 step down regulators - one for the servo steering, the other for the motors and controller - and started it up from the battery. Looks like nothing smoked, that's a good sign.