LocateAnything-3B on a 12 GB: ten production scenes, one consumer GPU

TL;DR

NVIDIA shipped LocateAnything-3B on 2026-05-26 — an open-vocab vision-language grounding model built on the Eagle VLM stack (MoonViT vision encoder + Qwen2.5-3B language backbone + a parallel box decoder). We ran it through ten production-shaped scenes on a 12 GB RTX 4000 Ada Generation GPU, the kind of card that lives in a developer's bag, not a datacenter.

The headline numbers:

• Peak VRAM 9.7–11.1 GB across every scene. The 12 GB consumer budget holds.
• Per-scene latency 7.4–19 seconds for dense multi-class detection on 1600-pixel Wikimedia Commons photos. Referring expression queries: 7.4–13.5 s.
• One outlier at 51 s on a warehouse box-packing scene where the model entered a textbook autoregressive failure mode. Explanation + fix below.
• 25 individually boxed cows in a single forward pass on a pasture photo — exactly the livestock-counting use case the model card advertises, running on a laptop.

This is the SDPA-attention floor. Compiling NVIDIA's la_flash kernel (their claimed 2.5× speedup parallel box decoder) would push every scene under 10 seconds on the same hardware. We didn't compile it for this run; that's a future post.

Who this is for

Four readers, one post — skip to the section that fits you.

• Buying / evaluating — "How does this compare to commercial detection APIs, and what's the licensing reality?"
• Deploying / operating — "Will it fit on the hardware I have, and what's the latency / VRAM budget?"
• Research / methodology — "What's the architecture, and where does it break?"
• Integrating / coding — "Show me the install line and the first working API call."

What LocateAnything-3B actually is

LocateAnything-3B is a 3-billion parameter vision-language model that produces structured grounding output — text labels plus bounding boxes — for arbitrary open-vocabulary queries. You hand it an image plus a list of categories (or a referring expression like "the worker in the red helmet"), and it returns a JSON-ish structured response with one box per detected instance.

It sits at the intersection of three older lineages:

• Open-vocab object detection (OWL-ViT, GroundingDINO, OWLv2) — but multimodal-aware, so it can take prose queries, not just label lists.
• Vision-language models (LLaVA, InternVL, Qwen-VL) — but trained specifically to emit clean box geometry, not just to describe what it sees.
• Referring expression segmentation (RefCOCO-style models) — but at a scale and quality where it's actually usable on real-world images, not just curated benchmarks.

Architecturally it's the Eagle VLM family: a MoonViT vision encoder feeds visual tokens into a Qwen2.5-3B language model, which is fine-tuned with a parallel box decoder head. The parallel-decoder bit is the part that's supposed to make this fast — it emits all the boxes in one step rather than autoregressively generating box tokens one at a time. (Spoiler: when the parallel kernel isn't compiled and you fall back to SDPA + autoregressive emission, you can watch this fail in real time. We did. It's instructive.)

For the buyer: how it compares + the licensing reality

The cost story is interesting because nothing about this model maps cleanly to "API tokens per million." Open-vocab detection is usually priced as per-image inference, and the dominant alternatives are:

• Google Cloud Vision (Object Localization) — ~$1.50 per 1000 images, ~600 categories from a fixed taxonomy, no referring expressions.
• AWS Rekognition (DetectLabels) — similar pricing, similar fixed taxonomy.
• Roboflow Universe Hosted Inference — pay per call, custom datasets, you train the categories.
• Self-hosted LocateAnything-3B on the laptop you already own — $0 marginal cost, any category you can describe in English, ~10–20 s per image at SDPA. Tens of cents of electricity an hour.

At low volume the API path wins on latency and ops simplicity (~300 ms per call versus 10 s for our laptop run). At high volume — say, processing 100k images a day — the per-image API cost crosses $150/day and a single H100 running this model will outprice it on day one. The break-even for a self-hosted batch pipeline is somewhere around 10–50k images/day, depending on whether you can afford to wait 10 s per image or you need the la_flash speedup.

Licensing: read this before you ship it

LocateAnything-3B is released under NVIDIA's non-commercial research license. That is a non-trivial constraint:

• You can build internal demos, research prototypes, blog posts (like this one), and academic work freely.
• You cannot ship it in a product you sell, embed it in a SaaS, or use its output for direct commercial revenue without an NVIDIA commercial agreement.
• If your use case is "show a customer their warehouse photo and count the boxes," you need to either negotiate a license, fine-tune a permissively-licensed base, or stay on the API path.

This is the standard trap for open-weight vision-language models in 2026: the weights are public, the license isn't permissive. Plan for it in your build/buy analysis.

For the operator: VRAM, latency, and what scales

We ran ten use cases — autonomous-driving, surveillance, retail, robotics, agriculture, logistics, surveillance, document AI, the standard set called out in the LocateAnything tech report. Source images are 1600-pixel Wikimedia Commons photos chosen to match the model card's advertised production scenarios. Hardware was NVIDIA RTX 4000 Ada Generation Laptop GPU, 12 GB VRAM, CUDA 13, transformers 4.57.6, BF16, SDPA attention.

Here's the per-scene table — the numbers a sizing exercise needs:

use case	multi-class boxes	top categories	multi-class p50	referring p50
traffic (historical street)	32	19 cars, 2 trucks, bus, 2 traffic lights, 5 people	19.2 s	12.2 s
factory (assembly line)	5	4 people, machine	13.1 s	10.8 s
cattle_field	26	25 cows, person, fence	16.2 s	13.5 s
construction_site	20	6 people, 4 helmets, 4 vests, crane, 2 excavators	16.4 s	12.5 s
warehouse ⚠	339 ⚠	10 people, box(328)	51.7 s	7.8 s
retail_aisle	20	14 bottles, 6 boxes, can	16.9 s	11.4 s
airport_terminal	14	6 backpacks, 4 handbags, suitcase	11.3 s	7.4 s
crop_field_aerial	9	8 trees, tractor, building	14.3 s	11.4 s
port_containers	1	crane	12.5 s	11.8 s
crowd_event	8	stage, 2 screens, 2 speakers, person	8.8 s	7.5 s

Three operational patterns fall out of the data, and they matter for capacity planning:

1. Latency is proportional to output box count, not input image size. Bigger box counts mean more autoregressive emission. The 32-box traffic scene took 19 s; the 1-box port scene took 12 s. Image resolution moves peak VRAM, but it doesn't dominate inference time.
2. Peak VRAM scales with input resolution. COCO 640-pixel inputs landed at 8.5 GB peak in earlier runs; the 1600-pixel Commons photos push to 11.1 GB. The 12 GB consumer budget holds, but barely. A 4K input would not fit; you'd need to downscale or move to a larger card.
3. The la_flash parallel decoder is the lever for production latency. Per the LocateAnything paper, the parallel box decoder gives ~2.5× speedup on dense outputs. That would put the 19 s traffic scene at ~7.6 s and the 16 s cattle scene at ~6.5 s — under the "perceived-instant" threshold for an operator-facing tool. The cost is a build-time investment in compiling flash-attn (we didn't have that working at run time and that's why these are SDPA-floor numbers).

Throughput math for an operator running this in batch

Take the median scene at 14 seconds per image on a 12 GB laptop. One laptop processes ~257 images per hour, or ~6,200 per day if you keep it warm 24/7. That's a small-shop number. Move to a single H100 — same model, more VRAM headroom and more compute — and per-image latency drops to ~3–5 s with SDPA, ~1.5–2 s with the la_flash kernel compiled. One H100 processes ~30,000–50,000 images/day at SDPA, breaking even with the cloud API path at the upper end and pulling ahead from there.

The right architecture is: image-queue → worker pool of N×H100 → write structured boxes to your data store. No router, no failover, no LLM-style streaming complications. Detection workloads are embarrassingly parallel and the model is stateless.

For the researcher: where does it actually break?

The most interesting finding isn't where the model worked — it's where it didn't. Let's walk the failure modes, because they tell you something about how the model is structured.

Warehouse box-loop: 328 boxes for ~15 actual boxes

The food-bank packing scene has maybe 15 visible cardboard boxes. The model emitted 328 box boxes over 51.7 seconds of generation time. Looking at the visualization, the boxes overlap in the same regions: the model correctly identified the first ~15-20 boxes, then got into an autoregressive loop emitting redundant box tokens until it hit --max-new-tokens=2048 and was cut off.

This is the canonical failure mode for sequence-generation detection models. The parallel box decoder (the la_flash kernel) is supposed to eliminate this by emitting all boxes in one parallel step, with the count determined by a learned head rather than an autoregressive stop token. We weren't able to compile flash-attn for this run; the SDPA fallback is exactly what the paper says the parallel decoder is built to replace. Useful to know if you're planning to deploy without the kernel — clip your max_new_tokens aggressively and accept missed detections beyond a certain density.

A second tell: the referring query "the forklift" on the same image returned zero boxes. The model had exhausted its generation budget on the box-loop and couldn't recover for the referring pass. The forklift is clearly visible. The model wasn't wrong about the forklift; it just ran out of budget to think about it.

No hallucination on absent categories

The flip side is the cleanest behavior we saw: across all ten scenes, when a queried category genuinely wasn't visually present, the model returned <box>None</box> rather than drawing a low-confidence guess. The factory image returned only 5 of 11 queried categories because 6 of them — conveyor belt, helmet, safety vest, tool, screen — were either absent from this specific image or visually ambiguous. The model didn't fill the silence with false positives.

For research purposes, this is the more important property than density. A model that hallucinates politely on dense scenes can still be useful with NMS and confidence thresholding. A model that hallucinates on absent categories is structurally unsafe for any audit workflow. LocateAnything-3B passes the second test.

Referring queries: hit or miss on dense scenes

Referring expression grounding worked cleanly on ~5/10 scenes (port_containers, crowd_event, airport_terminal, retail_aisle, crop_field_aerial). It returned zero boxes on the harder ones:

• "the bus" on the traffic scene → 0 boxes despite a visible bus.
• "the worker with a helmet" on the factory scene → 0 boxes, consistent with multi-class also returning no helmet for that image.
• "the forklift" on the warehouse scene → 0 boxes (budget exhausted, as above).

These are honest failures — the model refusing rather than guessing — but in a production referring system you'd want a confidence-thresholded fallback (e.g., re-prompt with a wider category list when the referring head returns empty). For research it's a clean signal that the referring head and the multi-class head are tightly coupled to the language backbone's understanding of the input.

Architecture footprint

From model/config.json:

• Vision encoder: MoonViT, dynamic resolution (hence the VRAM scaling with input size).
• Language backbone: Qwen2.5-3B with a learned auto-mapped configuration — this is why grounding is multimodal-aware rather than vision-token-only.
• Decoder: parallel box head as an extension on top of the LM. With la_flash compiled, boxes emit in one step. Without it, you get autoregressive token-level emission (and the warehouse pathology).
• Precision: bf16 throughout. fp8 would likely fit COCO-resolution inputs in 6 GB but isn't tested here.

Ten use cases, ten images

Cattle counting on a pasture photo

26 boxes total: 25 cows individually identified, plus the foreground person, the wire fence, and a foreground tree. Each cow gets its own tight box despite uniform colour and pose. The referring query "the cow in the foreground" returned 13 boxes — the model interpreted "foreground" as "the front row of the herd" and boxed the whole front line. Permissive but defensible.

Construction site PPE detection

A tall blue tower crane on a residential build. The model detected 6 people, 4 helmets, 4 safety vests, the crane, 2 excavators, a truck, scaffolding. Open-vocab PPE detection from a zero-shot text prompt — the exact thing construction-safety vision systems pay $100k/yr for. The 16 s latency here would drop to ~6 s with la_flash compiled.

Traffic on a black-and-white historical photo

The model detected 32 boxes: 19 cars, 2 trucks, 1 bus, 1 motorcycle, 5 people, 2 traffic lights, 2 traffic signs. Useful evidence that the model generalises beyond modern colour datacenter-camera footage. The referring query "the bus" returned zero boxes — one of the few cases where multi-class hit but referring missed.

Retail aisle: bottle-level inventory

14 individual bottles, 6 boxes, 1 can, 1 price tag, 1 shopping cart, 1 shelf — 20 boxes from a Commons "REMA 1000" cleaning-products aisle. Direct shelf-inventory use case. Latency 17 s; would drop to ~7 s with the kernel.

Airport luggage discrimination

6 backpacks, 4 handbags, 1 suitcase, monitor, signage — 14 boxes from a passenger-with-luggage scene. The model distinguishes backpack vs handbag vs suitcase consistently. Useful for airport-surveillance / lost-baggage applications. This was the fastest dense scene at 11.3 s.

Aerial crop field

9 boxes total: 8 trees lining the field edge, plus a tractor on the road and a distant farm building. Less dense than the cattle scene but still produces a useful structured output for agricultural monitoring use cases.

Port container crane

Only 1 box on the multi-class pass: the crane itself. The image is dominated by a single large crane structure against sky — not actually that dense, so the sparse output is honest. The referring query "the crane" returned a clean single box around the crane and ran in 11.8 s.

Stadium crowd event

8 boxes: the stage, two large display screens, two speaker stacks, and one person in the foreground. The crowd itself wasn't individually boxed — the model treated the audience as a single visual mass, which is the right call for an "event infrastructure" categorisation rather than a "count attendees" task. Latency 8.8 s — the fastest dense scene we ran.

Factory assembly line

Featured in the failure-modes section above. 5 boxes (4 people, 1 machine), 6 categories correctly returned None. Sparse output, honest behaviour.

Warehouse box packing

Also featured in the failure-modes section above. 328 boxes from ~15 actual boxes, 51.7 s of autoregressive box-token loop. The cleanest demonstration in this run of why the la_flash kernel exists.

For the developer: how to actually run it

Three commands to install + run:

cd ML-Models/locate-anything-3b
uv venv .venv --python 3.12
source .venv/bin/activate
uv pip install torch transformers==4.57.6 pillow huggingface_hub accelerate \
               opencv-python-headless decord lmdb requests torchvision peft

# default: sdpa attention, 1 warmup + 3 reps per task
python bench_grounding.py

# pull 10 Commons images + bench all 10 use cases
python bench_production.py
python visualize.py --in results/production.jsonl --out results/visualizations

The minimum API call from Python:

from transformers import AutoModelForCausalLM, AutoProcessor
from PIL import Image

proc = AutoProcessor.from_pretrained("nvidia/LocateAnything-3B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "nvidia/LocateAnything-3B",
    trust_remote_code=True,
    torch_dtype="bfloat16",
    device_map="auto",
)

image = Image.open("warehouse.jpg").convert("RGB")
prompt = "Detect: person, box, forklift, pallet, shelving."

inputs = proc(images=image, text=prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(proc.decode(out[0], skip_special_tokens=True))

Three real-world gotchas:

1. Cap max_new_tokens aggressively on dense scenes (256–512 is usually enough). Anything higher invites the box-loop pathology we saw on the warehouse scene.
2. Use trust_remote_code=True — the model ships custom processor and modeling code in the repo, not stock Transformers.
3. Parse the structured output before drawing boxes. The model emits a structured response with category labels and box coordinates; the visualize script in the ML-Models repo has the parser if you need a reference.

Patterns the production suite confirmed

Four signals worth carrying out of this experiment:

1. Latency scales with output count. Dense scenes are slow; sparse scenes are fast. The per-image budget for a production pipeline should be set against your p95 expected box count, not the average.
2. VRAM scales with input resolution. 1600-pixel inputs use ~11 GB on bf16; pre-downscale aggressively if you're on a 12 GB card and want headroom.
3. la_flash changes the production story. Without it you're SDPA-bound and the warehouse pathology is real. With it, sub-10-second laptop latency is plausible and the failure mode goes away.
4. No false positives on absent categories. The model returns None rather than guessing — useful for audit workflows where you'd rather have a miss than a hallucination.

Bottom line

LocateAnything-3B is a genuinely useful open-vocab grounding model that runs on a 12 GB laptop GPU. For internal demos, research, prototyping, and any use case where the NVIDIA non-commercial license is acceptable, it's the new baseline for open-weight detection. For commercial deployment you need a license or a different model; for production latency you need to compile la_flash. Neither of those is a deal-breaker — they're known knobs.

What we'd want to see next: a head-to-head against OWLv2, GroundingDINO, and Florence-2 on the same 10 scenes with the same hardware, plus a la_flash-on run to confirm the 2.5× speedup claim holds in practice. Both are queued.

The reference dataset (twenty signed JSON results plus the twenty annotated PNGs) is reproducible from the script in the ML-Models repo. If you want to verify any number in this post, the command is at the top of the developer section.