Let's be honest: physical AI and embodied robotics are undergoing a massive shake-up, reshaping the industry, how we work, and what we work on. For decades, robots operating in public spaces, autonomous sidewalk delivery rovers, and last-mile bots like our R-kiwis have run on highly compartmentalized, rule-based software. In these traditional modular stacks, perception, mapping, localization, and control are treated as separate engineering problems, each solved with hand-crafted algorithms, deterministic state machines, and isolated neural networks. But foundation models, specifically Vision-Language-Action (VLA) models, are sparking a paradigm shift that mirrors how natural language processing moved from rigid, rule-based grammar to LLMs.

This transition is driven by necessity. Classical robotics hits a wall in unstructured, dynamic urban environments, and sidewalks are about as unstructured as it gets: unpredictable pedestrian flows, unregistered obstacles, wild weather, and complex social-compliance demands. The street is the most erratic social lab you can find. Traditional robotics asks engineers to manually encode rules for everything, producing systems that excel within their designed operational domain but fail catastrophically the moment a novel edge case arises. Foundation models take a different path: by leveraging massive datasets and multimodal architectures, they replace hand-built rules with learned representations, giving robots a generalized physical intuition of the world.

This is where Robot.com comes in. We're obsessed with automating the physical world to unlock real business efficiency. Our core focus is advanced robotic manipulation and autonomous last-mile delivery, and our north star is making radically low-cost delivery accessible to everyone, everywhere. My pride and joy is the R-kiwi rover, and the streets are its proving ground. We didn't build a prototype and call it a day; we've spent years relentlessly iterating on the hardware and software to scale it up. Today, we operate a commercial fleet of more than 400 active rovers across multiple locations.

This large-scale operation has allowed us to log thousands of autonomous miles and hours of real-world driving. To put that in perspective, these little bots have racked up more than 250,000 autonomous miles and completed over 300,000 deliveries. That fleet does more than deliver orders. It has been generating data on exactly the chaotic, real-world conditions that break classical stacks. Every puddle crossed, every unusual pedestrian interaction, and every chaotic urban edge case has been captured in high fidelity, offering behavioral diversity you can't fake in simulation. All data is strictly anonymized, ensuring no sensitive information is exposed.

Over the past few years, we've been putting that data to work by exploring Robot Foundation Models (RFMs) for autonomous navigation. Everything that follows is grounded in that operational reality, not in a simulator. That is what gives us a launchpad to help the community push AI navigation models well beyond the limits of traditional, rigid stacks.

I'm not here to debate whether modular stacks or RFMs are better, safer, or more efficient. That is a post for another day, and we definitely have our opinions. Today is about something more useful: putting real tools in your hands.

So here's what we're sharing. First, the R-KNav dataset, an extensive, diverse collection of real sidewalk navigation data drawn directly from our deployed fleet. Second, the FoMo Challenge, where you can put that data to work and benchmark your own navigation models in the field. And there are real rewards on the table. The top performers do not just earn bragging rights; they get access to our robot infrastructure so they can put their models on actual hardware, on real sidewalks. If you're an individual contributor, a programmer, a member of a research group, an academic, an industry professional, or anyone interested in training or fine-tuning autonomous navigation models, this is your arena. Let's get into it.

To know more, please visit www.robot.com

Under the Hood: The 10,000-Hour R-KNav Dataset

By treating our daily sidewalk operations as a real-world data lake, we collected a massive 10,000-hour multimodal dataset directly from our R-kiwi sidewalk rovers.

This data was captured at multiple locations while the robots performed real-world delivery tasks in both autonomous and teleoperated modes. We're talking about pure, unfiltered behavioral diversity: severe weather conditions, varying surface types, and chaotic urban scenarios such as crowded pedestrian crossings, narrow sidewalks, and tricky driveways. To ensure we captured a wide variety of environments, the routes were generated using the OSRM (OpenStreetMap Routing Machine) system, incorporating enriched metadata to handle different road types and risk levels.

Every episode in the R-KNav dataset is structured in the Lerobot format, mapping raw sensor data to control commands to help you train models that move from pixels to motion. Each recording includes a comprehensive 360° multimodal view captured from four RGB cameras (synchronized in post-processing using a nearest-neighbor policy), centimeter-level localization via RTK-GPS paired with IMU data for precise physical dynamics, and detailed action and feedback loops that include linear and angular velocity setpoints. Additionally, each episode provides VLM context, including metadata on road type, surface, and weather, as well as natural-language driving instructions generated by our VLM pipeline.

A quick note: You might know that our production R-Kiwis are equipped with 3D LiDAR, but we intentionally removed that data from this release. Why? Because we want this dataset to push the boundaries of strictly vision-based navigation. We want to see what your models can do with pixels, not lasers.

Want to get your hands dirty and dig into the specifics? Head over to our Hugging Face page, where we've dropped 30 minutes of data completely public, with no strings attached. Download it and check out the format for yourself.

If you need more to work with, you can request a 300-hour chunk. Just submit a quick form telling us a bit about your intended use case, and we'll get you access. And for the heavy hitters who need the full firehose of data, it's best to reach out to us directly. Our engineering team will get you sorted.

So What Exactly Is the FoMo Challenge?

Let's clear one thing up: FoMo isn't about the fear of missing out. It comes from our drive to challenge ourselves and dive deep into Foundation Models. Now that you know what the R-KNav dataset is all about, here's the real invitation to the community. Robot.com is opening the door for you to take your fine-tuned or foundation navigation models, built for platforms like our R-kiwi, and put them to the test in a real environment, but in a controlled way.

We're serious about this. We're ready to partner with anyone who can demonstrate standout results in training and testing their models. And yes, we have a rewards pool for exceptional performers. We'll share the details on our official page soon.

So how do we judge the field? We benchmark your model against our current autonomy stack's safety and efficiency KPIs, including interventions or incidents per mile driven and average navigation speed from point A to point B. The evaluation will be conducted on a set of 5–10 navigation tasks on sidewalks that are unseen in the dataset, or on a mix of approximately 30% seen and 70% unseen scenarios. If you're confident that your model's benchmark or training step is worth showing off, submit your interest through our official site (coming soon). We'll evaluate your results, and the next step is to put that solution to work on our infrastructure and our robots, in a real environment, because robots are made for the world, not for the lab.

Here's the threshold. If your model clears 50% of these KPIs during our controlled tests, congratulations. You're officially in the running. From there, it comes down to how you stack up against other participants at the semiannual cutoff each year. This challenge stays open indefinitely, until our webpage says otherwise.

The Road Ahead

So that's the deal. The R-KNav dataset and the FoMo Challenge aren't just another release. They're an open invitation to build the future of sidewalk navigation with us. We've spent years logging the messy, unpredictable reality of the street so you don't have to start from scratch. Now the data is in your hands, the benchmark is set, and the path from your model to real hardware on real sidewalks is wide open.

This is bigger than a leaderboard. It's about proving that foundation models can handle the chaos of the real world, and we'd rather figure that out together, as a community, than behind closed doors. Whether you're a solo developer hacking on weekends, a research group chasing a breakthrough, or a team looking for a serious partnership, there's a place for you here.

Download the data. Train your model. Show us what it can do. The streets are waiting.

Ready to jump in? Grab the free sample on Hugging Face, keep an eye on our official page for challenge submissions, and let's push AI navigation past the limits of the lab, together.

About the author | LinkedIn / GitHub