Gemm

The problem

A refinery has a lot of furnaces. Every three hours, somebody has to walk to each one, read a gauge, and turn a valve based on what the gauge says.

It is not complicated work. It is work that has to happen at four in the morning, in an area where being present is itself the risk, on a loop that never stops. And its value is entirely in the doing: a reading taken three hours late is not a reading, it is a guess.

That round is the job. Not "inspection" in the abstract — that specific loop, walked by a person today.

What we are building

A humanoid that walks the round and operates the valve.

That last part is the hard half, and it is the point. A robot that reads the gauge and radios a number back still needs a person to walk out and turn the valve, which means the trip still happens. Closing the loop is what removes it.

We are not there yet. What works today is the layer underneath, and it is substantial.

Where the idea came from

The name is the four of us: Guido, Eitan, Martín, Matías. It started as the research project for our final year at school, in a programme that wanted the year to produce research rather than another app.

We had a Go2 quadruped before we had a use case, and went looking for one in meetings with the energy companies that run sites like these — people who send humans into those areas and could tell us which round they would most like to stop walking. That is a very different answer from the one you write at a desk. Then the G1 humanoid arrived, and the round became something a robot could plausibly do rather than something to talk about.

What works today

A live 3D and 2D map of the environment, built from the robot's own sensors. All of it comes out of the same pipeline: a 3D map generated in real time while the robot walks, a fully processed 3D pass afterwards, and a 2D floor plan derived from it. The 2D map is what navigation runs against; the 3D one is what a human reads.

Teleoperation through VR. An operator in a headset moves their hands and the robot mirrors them, with minimal delay. That is Guido's, and it is what makes valve operation tractable — before a robot can do the motion on its own, a person has to be able to do it through the robot.

A dashboard that puts the map, the robot's state and the live feed in one place, so whoever is running it is not reading a terminal.

What we are building now

Getting the robot from point A to point B on its own, against the 2D map. The map exists and relocalisation works; routing is the current piece.

The constraints

• The robots are not ATEX certified. They are not rated for an explosive atmosphere, which means no proof of concept can happen in a classified zone. Everything has to be demonstrated in an unclassified area and argued from there.
• The onboard compute is shared with another subsystem, so the perception budget is not the whole machine.
• The link to the operator is WiFi, inside a refinery, through steel. And teleoperation is the one thing on this list where latency is not a quality problem but a safety one.

What I decided

Our own SLAM, not the vendor's

FAST-LIVO2 on ROS2, running onboard. The vendor stack is a black box: it hands you a map and no way to reason about how it was built, so when the map is wrong there is nothing to do about it.

Owning the pipeline is the difference between a demo and a system. It is also what made every finding below possible — you cannot profile a black box.

A density render, not an occupancy grid

For a human deciding where to send a robot, an occupancy grid throws away exactly the information that makes a scene legible. A density render of the same point cloud keeps the structure a person recognises.

The map carries an explicit spatial contract

With autolevelling, so "where the robot thinks it is" and "where the dashboard draws it" cannot drift apart silently. The alternative is a system that looks right until the day it is confidently wrong.

Mapping on Metal MPS for the macOS side

So the processed pass can run on the machines we actually have.

What actually took the time

Not the algorithms — the bridge, which is mine. Each of these is written up properly in the notes, because the findings are more useful than the decisions.

The adapter was the bottleneck, not the SLAM. We spent weeks optimising a pipeline whose real limit was the thin-looking layer translating between two message formats, copying and reallocating on every frame.

The SLAM path was eating the WiFi link. Publishing the full trajectory at full rate saturated the link the live view needed.

The relay's rate ceiling was biting its own input. A throttle applied at the wrong point in the graph fed back into the thing it was throttling.

The onboard compute is shared. Obvious once you know it, invisible while you are measuring.

The team

• Me — the bridge that pulls data off the robot at the lowest delay I can get, the real-time 3D map, the processed 3D pass and the 2D derivation, and the dashboard.
• Guido Jacofsky — the VR teleoperation system.
• Matías Tenenbaum — research.
• Martín Durán — design.

With a mentor who pushed the project to be research in the first place, and industrial mentors who tell us when an idea would not survive a real refinery.

Where it is

In development, and this page says which part is which, because overstating a robot's autonomy is the fastest way to lose a partner who actually works in the field:

• Perception and mapping — working. 3D live, 3D processed, 2D.
• VR teleoperation — working.
• Autonomous A-to-B navigation — in progress.
• Valve operation — the target.

What I would do differently

Profile before optimising, in that order, once. We had two problems in the same area and only one was the one we were working on — and fixing the real one made us more confident in the wrong model, not less. That is the worst possible shape for debugging and I walked straight into it.