How well does AI really work in 3D?
The Harpoon 3D Bench gives models real work from real fields and measures what they build. Two benchmarks today: mechanical parts, and scenes in Blender.
Graded on the result, not the code
Any program that builds the right thing passes. Code that only looks right fails.
Measured, not guessed
Sizes, volumes, contacts and keyframes are measured by software. No human or AI judge decides the score.
The grader is tested too
Every reference answer must pass and every deliberately broken version must fail, on every run.
Proof kept for every answer
The prompt, the raw reply, the file it produced and the graded result are saved for each task.
Can an AI write the code for a mechanical part that is actually right?
Each task describes a part in words, the way an engineer would on a drawing. The model answers with a program. We build the part and measure it against a reference.
The grader is checked on every run: all 50 reference parts pass, and 159 deliberately wrong versions (moved 3 mm, scaled 5%, holes 1 mm smaller) all fail.
- Renders
- The program builds without error, within 60 seconds.
- One valid solid
- Closed, valid, and the right number of separate pieces.
- Size
- Bounding box within 0.5 mm of the reference on every side.
- Holes
- The same number of through holes.
- Volume
- Within 2% of the reference.
- Shape
- Overlap with the reference part of at least 90%.
- Details
- Largest local difference of 0.25 mm or less. A hole 1 mm too small fails.
Can an AI actually get work done in Blender?
The model writes a Blender script. It runs in a fresh, empty Blender scene, and the scene it leaves behind is measured by Blender itself: geometry with modifiers and animation applied, materials, cameras, lights and render settings. Nothing is judged by eye.
Every reference answer passes and every broken variant fails: three legs instead of four, an applied modifier, a bounce on the wrong frame.
Want your model on the bench?
We run it on both benchmarks, or build a private one from your own field: robotics, chips, architecture, film. Your model never sees the test before it takes it.
Get in touch