The Case for On-Device AI
Why we invested in a Korea-rooted local AI lab
There was a post trending on Hacker News last week titled “Local AI Needs to be the Norm,” written by Cyrus. It resonated with a surprising number of people.
And honestly, this shift feels inevitable.
The primary driver? Token costs. Despite the industry’s promise that “AI and intelligence will become commodities,” the reality has been a bit different. Frontier AI products continue getting more expensive, usage limits are becoming more noticeable, and people are actually consuming more tokens than before as workflows shift toward agentic, multi-step tasks.
At the same time, open-source models continue improving at a fast pace. Not just in raw capability, but also in compatibility and tooling for actually running models locally and on-device.
The conclusion is obvious:
The value of on-device AI will continue to grow, especially for specific categories of applications.
Will there be a future where cloud-based models are completely replaced by on-device models or SLMs? Probably not.
But will on-device AI become dramatically better and more important than it is today? Absolutely.
Because there are simply too many places where it makes structural sense.
As Cyrus puts it in the essay:
“Most app features don’t need a model that can write Shakespeare, explain quantum mechanics, and pass the bar exam. They need a model that can do one of these reliably: summarize, classify, extract, rewrite, or normalize.
And for those tasks, local models can be truly excellent.”
(…)
“Local AI shines when the model’s job is transforming user-owned data, not acting as a search engine for the universe.”
And beyond that, there are entire categories where local AI is not merely preferable, but necessary.
Privacy-sensitive environments. Robotics. Vehicles. Consumer hardware. Games and interactive media. Any environment where ultra-low latency, offline reliability, or tight hardware integration matters.
“Ok, I see that on-device AI is cool and useful. But is investing in AI model companies still a viable venture bet?”
Fair question.
For cloud-based frontier models, many investors expect the market to ultimately consolidate around a relatively small number of giant labs like Anthropic and OpenAI. Therefore, they believe the largest venture outcomes will now emerge from the orchestration, infrastructure, and application layers surrounding them.
But on-device AI may evolve differently.
The on-device AI ecosystem is unlikely to become a winner-take-all market dominated by two or three general-purpose models.
Instead, it will become far more fragmented and specialized, with many category leaders optimized around specific deployment environments and constraints.
Because small models operate under tight compute constraints, and those limitations force specialization.
A voice model optimized for ultra-low latency on consumer hardware.
A gaming-native model built for NPC interaction.
A robotics model optimized for edge inference.
A lightweight automotive assistant tuned for in-vehicle systems.
Each category comes with different tradeoffs around latency, memory, power consumption, privacy, and hardware compatibility.
Ironically, those constraints will create more winners, not fewer.
Which naturally brings us to one of our latest portfolio companies.
TheVentures is excited to announce our seed investment into Out-of-Set, an on-device voice AI model lab founded by researchers with deep expertise in low-latency voice AI.
Following our investment into ZETIC.ai in 2024, a team of ex-Qualcomm engineers building the deployment layer for on-device AI, we saw Out-of-Set as an expansion of our broader thesis around the local AI stack.
Out-of-Set: The 🐐 team
The Out-of-Set team consists of researchers who are among the best in the world at building small, fast voice AI models.
Co-founders Hyeongju Kim and Hyeonseung Lee hold MS and PhDs in Computer Science and voice AI, and have deep technical expertise across the broader voice AI stack.
Hyeongju, also a well-published researcher with hundreds of citations, spent nearly five years building TTS systems across multiple Korean AI labs, including Supertone, a HYBE subsidiary (the company behind BTS). One of the projects he is still most proud of from Supertone is Supertonic 2, an on-device TTS model optimized for speed.
At one point, Supertonic 2 reached #1 trending globally on Hugging Face, accumulated thousands of GitHub stars, and became widely recognized as one of the fastest on-device TTS models in the world.
Building on this experience, Out-of-Set is now preparing to launch a new on-device generative voice model with significantly improved performance. Their initial focus is naturally voice, before gradually expanding into broader categories of on-device AI models and tooling over time, including local language models.
And honestly, one of the things we liked most about this strategy was precisely the fact that they started with voice.
Not simply because voice is a large market, but because it is an unusually subjective category with many different tradeoffs.
LLMs today are largely evaluated on intelligence-heavy tasks: coding, reasoning, research, problem solving. Performance in those domains is comparatively easy to benchmark, and users naturally converge toward the models that simply get the job done best.
In those categories, weaker performance immediately feels like “this model is dumb.”
Voice behaves differently.
Unlike reasoning models, there is rarely a single objectively correct answer for what “best” sounds like. A voice model may generate speech slightly less naturally, but still win because it is dramatically faster. Or cheaper. Or more emotionally expressive. Or more customizable. Or easier to self-host locally.
More importantly, imperfect voice quality is often far more tolerable than imperfect reasoning quality. Slightly unnatural speech rarely breaks the product experience in the same way that incorrect reasoning or failed task completion does.
As a result, the spectrum of acceptable tradeoffs becomes much wider, creating more room for specialized players to compete through speed, deployment flexibility, emotional quality, cost structure, or hardware optimization.
Voice, in particular, feels like a category where Davids can challenge Goliaths. Which is exactly why we think this market becomes so interesting, and why we believe highly specialized teams like Out-of-Set can emerge surprisingly quickly and carve out categories of their own.
Why Korea?
Beyond the above mentioned strengths, there is another reason why we think Out-of-Set has a particularly strong shot: they are a Korea-rooted team.
To understand why that matters, you first have to understand where on-device AI becomes most valuable.
We believe many of the highest-value opportunities may emerge not in pure consumer software, but in enterprise and hardware-driven environments.
This is because the core advantages of on-device AI — latency, privacy, reliability, offline capability, and hardware-level optimization — become even more important inside enterprises.
And in many cases, those deployments will not simply involve software APIs.
They will involve actual hardware and closed systems owned, manufactured, or operated by enterprises.
Cars.
Robots.
Consumer electronics.
Gaming systems.
Defense tech.
In these environments, the critical capability becomes co-design.
Not just serving a model through an API, but deeply integrating models into customer hardware and internal systems in the most seamless way possible.
Just as Forward Deployed Engineers at Palantir or legal engineers at Legora work side-by-side with clients to calibrate workflows around real operational constraints, the best local AI model companies will likely operate similarly.
They will not simply ship APIs. They will work closely with customers to ensure models run optimally within real-world hardware and security constraints.
And this is where geographic proximity starts to matter.
The closer model labs are to their customers and industrial ecosystems, the easier it becomes to build deeply integrated systems tailored to real deployment environments.
Now think about the industries mentioned above.
Automobiles.
Consumer electronics.
Semiconductors.
Gaming.
Robotics.
Defense.
These are all industries where Korea is globally competitive.
Hyundai Motors.
Samsung Electronics.
SK Hynix.
Krafton, the company behind PUBG: Battlegrounds.
The AI labs and infrastructure companies with the closest proximity to these ecosystems will have a structural advantage in forming partnerships, securing deployments, and iterating directly alongside these industry giants.
More importantly, those relationships do not simply create revenue opportunities. They can also become distribution channels and expansion pathways into global markets.
Which is why we believe Korea-rooted teams may end up capturing a disproportionate amount of opportunity in this new local AI stack.
We first met the Out-of-Set team while they were still operating in stealth, and over time we watched them iterate with unusual speed, conviction, and technical clarity. The more we got to know the team, the more we became convinced that they represented exactly the kind of technically ambitious, deeply specialized company this market will reward.
Once again, we are incredibly excited to be the very first institutional check into Out-of-Set, and even more excited to see what kind of models they build next.
At TheVentures, we are constantly looking for exceptional technical teams early, often before anyone else notices them. If you are a fellow investor interested in getting connected, feel free to reach out at info@theventures.vc or submit the form below.
GP? If you’d like to discover more companies like Out-of-Set before they become obvious, request access to TheVentures Deal Flow Network.
korea proximity to hardware customers angle is something most US investors are sleeping on - hyundai, samsung, krafton next door changes the co-design math completely. excited to watch out-of-set 🎙️