Why Open Models Win and Closed APIs Still Matter: The Search Paradox at the Heart of Modern AI

Jul 20, 2026

10 min read

2 views

84%

The strange split in modern AI

What if the biggest mistake in AI right now is treating model quality and system usefulness as the same problem?

On one side, we have a powerful story of openness: large language models can now be trained on publicly available data, scaled into competitive systems, and released for everyone to use. On the other side, a quieter but equally important story is unfolding in retrieval and search: the best embedding API is not always the one that looks best in isolation, but the one that fits the messiness of real workflows, language coverage, and budget constraints.

Put differently, the frontier is no longer only about who can build the strongest model. It is about who can turn a model into a reliable component inside a larger system. That is a very different game.

The tension is easy to miss because both stories seem to celebrate capability. But one celebrates access to the model itself, while the other celebrates access to a practical service. Together they reveal a deeper truth: in modern AI, power is shifting from monolithic intelligence to composable infrastructure.

The old fantasy: one model to rule them all

For years, the default imagination in AI was simple. Build a bigger model, feed it more data, and it will eventually do everything better. Search, summarization, question answering, translation, ranking, reasoning, all of it. If a model is strong enough, why bother with anything else?

That fantasy has a certain elegance, but real systems rarely reward elegance. Search is a good example. In practice, retrieval is not one problem, but many problems stacked together: matching short queries, handling domain vocabulary, coping with multilingual text, respecting latency budgets, and staying affordable at scale. A model that looks excellent in a benchmark can still be awkward in production if it is expensive, slow, or brittle outside its comfort zone.

This is why the embedding API question matters. Dense retrieval sounds like a clean technical upgrade, but operationally it is often a bet about the whole search stack. Should embeddings replace lexical retrieval, or merely refine it? Should they be the first stage, or the reranker? Should they serve English well, or support many languages adequately? The answer depends on where the system lives, not just how the model scores.

The same lesson appears on the model-training side. Open foundation models demonstrated that state of the art is not inherently locked behind proprietary data or impossible infrastructure. Publicly available datasets can produce highly capable systems. This matters not only as a philosophical victory for openness, but as a practical shift in what organizations can build, inspect, adapt, and deploy.

But openness alone does not solve usefulness. A released model is not yet a product. It is a material. The real question becomes: how do we shape that material into something robust enough for the world?

Search is where abstraction meets friction

Search is a revealing place to think about AI because it refuses purity. Users do not arrive with neatly labeled tasks. They type misspellings, half remembered phrases, code names, slang, and multilingual queries. They expect relevance, but they also expect speed, coverage, and cost discipline. Search is therefore an excellent stress test for any AI system that claims general intelligence.

This is where the distinction between first stage retrieval and reranking becomes more than an implementation detail. A first stage retriever must cast a wide net. A reranker must make sharper judgments on a smaller candidate set. When embeddings are used as the first stage, they are asked to do both breadth and precision at once. That is ambitious, but not always wise.

A better analogy is hiring. The first stage is the recruiter scanning thousands of resumes. The reranker is the hiring manager reviewing the top candidates in detail. If you ask the hiring manager to read every resume from scratch, the process becomes expensive and slow. If you ask the recruiter to make the final decision alone, subtle distinctions are lost. Good systems, like good organizations, separate candidate generation from final judgment.

That separation explains a broader pattern in retrieval. BM25, the classic lexical method, is not glamorous, but it is cheap, surprisingly sturdy, and often strong at exact matching. Embeddings, by contrast, are better at semantic generalization, meaning, and paraphrase. The most effective system is often not the one that picks a side, but the one that stages a conversation between them.

For English retrieval, reranking BM25 results with embeddings can be a particularly budget friendly approach. This makes intuitive sense: lexical retrieval provides a strong candidate set, and semantic reranking corrects the obvious misses. But multilingual retrieval complicates the picture. Language diversity changes tokenization, surface form overlap, and semantic distance in ways that make lexical and semantic methods interact differently. In those cases, a hybrid approach may work best, even if it costs more.

The lesson is not that embeddings beat lexical search, or that lexical search beats embeddings. The lesson is that retrieval is an orchestration problem, not a single-model problem.

Openness changes the unit of innovation

The release of strong open models matters for a reason deeper than price or licensing. It changes the unit at which innovation happens.

When only a few actors can access a model, most users must treat intelligence as a remote utility. They can call an API, but they cannot inspect, modify, or repurpose the core model behavior. Innovation shifts upward into prompts, wrappers, and product design. That has value, but it also constrains experimentation.

When capable models are released openly, the center of gravity moves. Teams can fine tune, evaluate, distill, combine, and specialize. They can build domain specific systems that are not merely consumers of intelligence but participants in shaping it. A medical retrieval system, a legal search engine, or a multilingual support assistant can be engineered around the actual demands of the environment rather than the average case imagined by a centralized provider.

This is where the relationship between open models and embedding APIs becomes unexpectedly intimate. Open models lower the barrier to experimentation across the stack. APIs, meanwhile, package model behavior into usable services. One democratizes ownership, the other democratizes access. Both matter, but for different layers of the system.

A search team using an open foundation model may still choose a commercial embedding API for one reason: operational leverage. The team may not want to retrain everything from scratch when the real bottleneck is retrieval quality under multilingual constraints. Conversely, a startup may use an open model to avoid dependency risk, while still benchmarking multiple APIs to find the best embedding service for their actual use case.

This reveals a subtle but important shift. In the past, the question was whether a model was good enough. Now the question is whether the interface between models and tasks is well designed enough.

The real bottleneck is not intelligence, it is fit

We often speak about AI as though the central challenge is producing more intelligence. But in many real deployments, the bottleneck is not raw capability. It is fit: fit to language, fit to latency, fit to budget, fit to domain, fit to user expectations.

Consider two search systems. One uses a top tier embedding model as its only retriever. Another uses BM25 to gather candidates, then a semantic reranker to refine them. On paper, the first sounds cleaner and more modern. In practice, the second may be more accurate, more transparent, and less expensive. That is not a contradiction. It is a reminder that performance is contextual.

The same is true for model openness. A 65 billion parameter model trained on public data is impressive not just because of its benchmark scores, but because it demonstrates a new route to capability. Yet once that model is in the wild, what matters is how it behaves inside a real application. Does it summarize? Does it retrieve? Does it answer in multiple languages? Can it be hosted affordably? Can it be improved by local data?

If intelligence is the engine, fit is the drivetrain. A more powerful engine does not help if the wheels slip on the road. Modern AI systems fail less often because they are too weak than because they are badly integrated.

This is why the most valuable practitioners are increasingly system thinkers. They do not ask only, “Which model is best?” They ask:

What is the retrieval architecture?
Where does the semantic signal help most?
What should remain lexical?
Where does multilingual variation break assumptions?
What level of cost is acceptable per query?

That is a more demanding set of questions, but also a more honest one.

A useful mental model: the intelligence stack

One way to unify these ideas is to think in terms of an intelligence stack.

At the bottom are the raw capabilities: foundation models, embeddings, and lexical methods. These are not products yet. They are ingredients.

In the middle are the orchestration choices: candidate generation, reranking, hybrid retrieval, specialization, and multilingual handling. This layer determines whether the ingredients actually work together.

At the top are the user facing outcomes: search relevance, answer quality, trust, speed, and cost.

Most AI discourse focuses on the bottom layer and talks as if it determines the entire stack. But the highest leverage often lives in the middle. A modest model arranged well can outperform a strong model arranged badly. This is especially true in retrieval, where the difference between first stage retrieval and reranking can completely alter system behavior.

The open model story strengthens this framework because it makes the middle layer more accessible. When core models are available to everyone, more teams can experiment with composition rather than accept a single vendor's default workflow. The API story strengthens it from another direction, because it allows teams to test capabilities quickly and plug them into existing pipelines without upfront infrastructure costs.

The real opportunity lies in using both forms of access wisely. Open models give you agency. APIs give you speed. Search systems need both.

Key Takeaways

Stop asking which model is best in the abstract. Ask which model fits your retrieval architecture, language mix, and budget.
Use embeddings strategically, not dogmatically. In many English retrieval setups, reranking BM25 results is more effective and cheaper than using embeddings as the only first stage retriever.
Treat multilingual search as a different problem class. Techniques that work well in English may degrade when surface forms, morphology, and language coverage change.
Think in stacks, not components. Foundation models, embeddings, lexical retrieval, reranking, and product constraints are parts of one system, not separate contests.
Use openness to gain control, not just ideology. Open models are valuable because they let you specialize, inspect, and adapt the system to real needs.

The future belongs to systems that know when not to use the biggest model

The deepest connection between open foundation models and embedding APIs is not that both are forms of AI infrastructure. It is that both point toward a new discipline: judgment about when capability should be centralized, and when it should be decomposed.

In the old era, progress often meant making one model bigger and hoping it absorbed the whole task. In the new era, progress increasingly means arranging multiple tools so each does the narrow job it is best at. Open models expand what is possible. APIs reduce the cost of trying. Retrieval systems reveal the truth about both: intelligence becomes valuable only when it is placed in the right part of the pipeline.

That reframes the competition. The winning system is not necessarily the one with the most impressive single number. It is the one that knows where to spend semantic power, where to rely on lexical precision, and where to preserve flexibility for a multilingual, resource constrained, real world environment.

The next major breakthrough in AI may not come from a model that can do everything. It may come from a system humble enough to do different things in different ways, and wise enough to know the difference.