The No-Hype Enterprise Shortlist: Best Open-Weight LLMs by RAG Workload

Leaderboard rankings are the wrong abstraction for enterprise RAG

Generic MMLU- and MTEB-driven rankings ignore the only three things that matter at deployment: faithfulness under conflicting chunks, citation behavior, and whether the license survives legal review. A model scoring 88 on a public reasoning benchmark and 91 on a hosted RAG suite still fails the procurement meeting if its acceptable-use policy carves out your industry, or if its training-data provenance triggers a sectoral compliance review. Public top-10 lists never weight for this, because the people writing them aren't the ones sitting across from a legal team.

RAG is also a system, not a model. Performance is determined by two models working together — an embedding model that decides whether the system retrieves the right chunks, and a generator that decides whether those chunks become accurate answers.^[3] Ranking generators in isolation, with no statement about the embedding pipeline or retrieval configuration, produces numbers that don't transfer to your corpus. Even with very large context windows, RAG remains the dominant grounding technique for enterprise data regardless of model size.^[1] The leaderboard abstraction quietly assumes the model is the system. It isn't.

Four RAG workload archetypes, not one universal winner

Enterprise RAG splits cleanly into four archetypes, and each rewards a different generator. First, high-volume knowledge assistants — internal helpdesks, customer service, policy lookups — where cost-per-completion and first-token latency dominate. Second, long-document synthesis: legal review, financial filings, technical specifications, where the system reconciles contradictory passages across hundreds of pages. Third, multi-hop reasoning over regulated data, where the answer requires chaining evidence across separately-retrieved chunks and an auditor will ask which chunk justified which clause. Fourth, air-gapped and low-resource deployment, where the constraint isn't accuracy — it's the GPU envelope and the absolute prohibition on shipping internal text to a hosted API.

Each archetype has a different failure mode. A high-QPS assistant fails on cost and latency long before it fails on reasoning depth. A long-document synthesizer fails on faithfulness when retrieved chunks contradict each other. A multi-hop system fails on intermediate-step hallucination. An air-gapped deployment fails the moment somebody quietly proxies an embedding call out to a cloud endpoint. Any single ranking is structurally wrong, because the dimensions it optimizes for aren't shared across archetypes.

Enterprise Bot's BASIC benchmark found Qwen 2.5 72B matching GPT-4o's 86.6% accuracy on customer-service, finance, and healthcare questions at $0.0004 per completion versus GPT-4o's $0.003.^[4] That's a workload-specific result — high-volume, short-answer, citation-heavy — and it does not generalize to multi-hop legal synthesis. Treating it as if it did is the mistake we're trying to prevent.

Shortlist 1: high-volume knowledge assistants — gpt-oss-120b as the legal-clean default

For citation-heavy assistants where latency and cost per completion dominate, mixture-of-experts architectures win. Qwen3-30B-A3B reports a 262K context, RAGAS faithfulness of 0.91, answer relevancy of 0.88, a 98% needle-in-haystack pass rate at 128K, and 1.2-second first-token latency on an A10G.^[3] gpt-oss-120b takes a different route to the same envelope: roughly 117B parameters with 5.1B active, MoE plus MXFP4 quantization, runs on a single 80GB GPU, ships with tool-use support, and is released under Apache 2.0.^[2]

Licensing decides this one. Qwen's weights are excellent and the inference economics are hard to beat, but for regulated sectors the data-governance posture around Chinese-origin training corpora is a legal-review item, not a footnote. gpt-oss-120b clears Apache 2.0 cleanly: no acceptable-use carve-outs, no MAU thresholds, no awkward conversation with general counsel about whether your bank, hospital, or ministry counts as a prohibited use case. Default to gpt-oss-120b for the high-QPS archetype. Use Qwen3-30B-A3B as the cost-optimized alternative only when the sectoral profile permits it.

Shortlist 2: multi-hop reasoning and long-document synthesis — DeepSeek-R1, with caveats

For legal, financial, or technical synthesis across contradictory sources, reasoning-tuned models pull ahead. DeepSeek-R1 reports a RAGAS faithfulness of 0.89, multi-hop QA accuracy of 94%, a 96% needle-in-haystack pass rate at 128K context, and 2.1-second first-token latency on an A10G.^[3] That latency profile is fine for synthesis — nobody is generating a legal memo in 200ms — and multi-hop accuracy is the metric that actually predicts behavior on the chained-evidence questions auditors ask.

The trap is the context-window spec sheet. The same DeepSeek-R1 is described with 128K context by one inference provider and a substantially larger window by another, depending on deployment configuration. The deployed window depends on KV-cache budget, batch size, and quantization — not on the model card. Validate the configuration you actually run, on your hardware, with your corpus, before committing any procurement language to a context-window number. Treat published windows as upper bounds under unspecified conditions, not as guarantees.

DeepSeek-R1 carries the same sectoral data-governance question as Qwen for regulated industries. For European public-sector and healthcare deployments, do the hardware envelope and licensing review before benchmarking — not after.

Shortlist 3: air-gapped deployments — Llama 3.1, with the embedding model in the same rack

For on-premise or air-gapped enterprises the right model is the largest one that fits your GPU budget while keeping the entire pipeline — embedding, retrieval, generation — local. Llama 3.1 8B runs on a single 16GB GPU; the 70B variant needs roughly 80GB of GPU memory.^[6] Quantized Qwen variants give you more points on that curve. The decision is not which model is smartest in absolute terms; it's which model preserves enough accuracy at the hardware envelope your facility can actually power, cool, and physically secure.

The failure mode here is silent and expensive. Teams pick a strong local generator and then quietly send their documents to a hosted embedding API because the embedding model they wanted wasn't available locally. That single decision destroys the data-sovereignty posture the entire on-premise project was meant to protect.^[6] If your generator runs in your rack and your embeddings leave the building, you do not have an air-gapped RAG system. You have a cloud RAG system with extra steps.

Llama 3.1 carries its own licensing constraint — the 700M monthly active user threshold — which matters less for internal deployments and matters a great deal for customer-facing products at scale.

Disqualify on license and provenance before you benchmark

Disqualify models on three grounds before any benchmark runs. First, restrictive acceptable-use clauses: Llama's 700M MAU threshold matters for large consumer products, and Gemma's prohibited-use policy carries domain carve-outs that legal will read carefully. Second, training-data provenance and sectoral data-governance exposure: Qwen and DeepSeek are technically strong and operationally attractive, but for regulated European sectors the question of where the training data came from and which jurisdiction's norms shaped it is a real procurement gate. Third, indemnification gaps: most open-weight releases offer no IP indemnification, and that has to be priced into the deployment decision or covered by the integrator.

Apply this filter first and most public top-10 lists collapse to two or three viable candidates per workload archetype. The shortlist isn't shorter because the other models are worse — it's shorter because the other models cannot survive legal review for your specific deployment context. Running benchmarks on disqualified models is a waste of GPU hours and procurement attention.

Add one more axis: whether the weights, license, and ecosystem are stable enough that you will still be running this model in three years. Community-maintained open-weight directories are useful as a sanity check on commercial-use status before you commit.^[7]

The decision matrix: workload × license × hardware envelope

The defensible procurement output is not a ranked list. It is a three-axis matrix: workload archetype, license tolerance, GPU envelope. A high-volume assistant with Apache-2.0-only license tolerance and an 80GB single-GPU envelope points cleanly at gpt-oss-120b. A multi-hop synthesis workload with permissive sectoral license tolerance and a multi-GPU envelope points at DeepSeek-R1 with a validated deployed context window. An air-gapped 16GB-class deployment for an internal helpdesk points at Llama 3.1 8B with a fully local embedding model. Same matrix, different cells, different defaults.

This is the matrix Wavenetic uses to pre-configure on-premise deployments. WaveNode ships the hardware, the runtime, the model, the RAG application with citation tracking and audit trail, and a European support contract as one GDPR-aligned stack on the customer's own infrastructure, air-gapped where required. That's how customers reach production in under 30 days without re-platforming a year later when the next open-weight generation arrives.

Pick your model by leaderboard and you will rebuild your RAG stack within a year. Pick it by workload archetype and license survivability and you will still be running the same system when the next generation of open weights arrives.