The Best 7B and 8B Code Models in 2026, and Which Ones Security Will Approve

Your head of platform forwards a one-line email from the CISO: “Can we run a code model ourselves instead of sending source code to a vendor?” You already know the shortlist. You’ve read the benchmarks, pulled a couple of models into Ollama on your workstation, and watched one of them write a decent Terraform module in four seconds.

Then you start drafting the answer and realize the benchmarks cover the part nobody on the approval committee will ask about. They’ll ask who trained the model, under what license, where the weights came from, what happens to the code you feed it, and who can reach the server it runs on. “It scored well on HumanEval” doesn’t answer any of those.

This is the comparison with both halves: which 7B–8B code models are actually good in 2026, and which ones you can get through a security and risk review.

Why the 8B Class Is the One to Evaluate

At Q4_K_M quantization, an 8B-class model fits in roughly 5–6GB of VRAM. That’s the size that runs on a single mid-range GPU, which matters less for a developer’s laptop than for your cost model: one modest GPU server can serve a team, and you can run several models side by side for different jobs.

Below 7B, models degrade noticeably on multi-step logic. Above about 14B, you’re into 12GB+ of VRAM per model and the hardware conversation gets more expensive. The 7B–9B range is where useful capability and reasonable hosting cost meet.

What Actually Matters in Practice

HumanEval and similar benchmarks measure whether a model can write one function from a docstring. That’s a spelling test, not a development workflow. What matters daily:

• Instruction following: Does it do what you asked, or creatively re-interpret the prompt?
• Modern syntax awareness: Does it know your framework and provider versions, or generate patterns from three years ago?
• Context adherence: Does it stay consistent with the files you gave it, or invent variables and imports?
• Diff quality: Do its patches apply cleanly?

Test models on your own code and your own language versions. Benchmark scores tell you very little about how a model handles your particular mix of frameworks, and nothing about the approval questions below.

The 2026 Contenders

Qwen3.5 9B: The Strongest All-Rounder

Qwen3.5 9B, from Alibaba’s Qwen team, is the most capable general coding model in this size class. It has a 262,144-token native context window (Ollama’s tag lists 256K), accepts images as well as text, and handles multi-language projects well. In practice you’ll run it at 32K context or less to keep VRAM in check, which is still enough for a module and the files around it.

The weights are released under Apache 2.0.

Add explicit version constraints to the system prompt so it targets the syntax you actually use:

FROM qwen3.5:9b

SYSTEM """You are an expert developer. Always use the latest stable APIs.
For Python, target 3.12+ features. For TypeScript, use 5.x syntax.
If you are uncertain about a specific framework version, say so explicitly
rather than generating code that may be outdated."""

PARAMETER num_ctx 32768

ollama pull qwen3.5:9b

Gemma 4 E4B: The Efficient One

Google’s Gemma 4 E4B has 4.5B effective parameters (8B including embeddings), a 128K context window, and accepts text, image and audio input. It’s the lightest model here and runs well on constrained hardware, which makes it a good second model on a shared server: cheap to keep loaded for quick completions while a larger model handles heavier work.

Gemma 4 is the first Gemma generation released under Apache 2.0. Earlier Gemma versions shipped under Google’s own terms of use, so if your legal team reviewed Gemma before, the answer may have changed.

FROM gemma4:e4b

SYSTEM """You are a concise assistant for code completion and small,
well-scoped code changes."""

PARAMETER num_ctx 8192

ollama pull gemma4:e4b

DeepSeek-R1-Distill-Qwen-7B: The Reasoning Specialist

DeepSeek-R1-Distill-Qwen-7B is a Qwen2.5 base model fine-tuned on reasoning traces generated by DeepSeek-R1. It’s strongest when a task needs step-by-step reasoning: tricky algorithms, debugging logic, explaining why a piece of code behaves the way it does. For routine generation it’s slower and wordier than Qwen3.5.

The weights are MIT-licensed, built on a base model originally released under Apache 2.0.

ollama pull deepseek-r1:7b

IBM Granite 4.1 8B: The Conservative Fallback

Granite 4.1 8B (released April 29, 2026) trails both Qwen3.5 and Gemma 4 on real coding work. It has a 131,072-token context window, generates predictable but unremarkable code, and is released under Apache 2.0. It’s on this list because it shows up in every enterprise shortlist, not because I’d pick it.

What keeps it on those shortlists is IBM’s governance story. IBM announced that the Granite 4.0 family was the first open model family certified under ISO/IEC 42001, the AI management system standard, after an external audit of IBM’s training, testing and data governance processes. IBM also says the Granite 4.0 checkpoints on Hugging Face are cryptographically signed. Two caveats belong in your approval memo: the certification covers IBM’s management system rather than the weights, and you should confirm that the 4.1 checkpoints you deploy carry the same signing before you rely on it.

ollama pull granite4.1:8b

Getting a Model Approved

Here’s what your third-party risk, legal and security reviewers will actually ask about each model:

One point to make early, because it clears up most of the fear in the room: open weights running on your own hardware don’t send anything back to the developer. No prompts, no code, no telemetry from the model itself. The risk with any of these models is about what went into the model (training data, licensing, possible tampering) and about the software around it, not about your code leaving the building.

That reframes the approval. Your reviewers aren’t evaluating a data processor. They’re evaluating a software supply-chain component, and you can treat it like one: an approved source, pinned versions, recorded hashes, an owner.

In a bank, that review usually runs through model risk management as well. US banks working under the SR 11-7 guidance on model risk, for example, will want the model inventoried, its intended use documented and its limitations tested before anyone relies on it. Plan for that paperwork from the start.

Which One to Take to the Committee First

I’d submit Qwen3.5 9B and Gemma 4 E4B together, as a pair. They’re the two best models in this class, and they cover each other’s gaps: Qwen3.5 as the primary coder for generation, refactoring and multi-file work, and Gemma 4 as the fast, light model you keep loaded for completions and small changes. One request, one evaluation, one intake process.

Expect them to move at different speeds. Gemma 4 should clear quickly: a US developer, a standard Apache 2.0 license, and nothing unusual for legal to review once they’ve confirmed the license change from earlier Gemma versions. Qwen3.5 is the one that tests your third-party risk team. Some will clear it on the strength of “the weights run offline and send nothing anywhere.” Others have blanket restrictions on models from certain jurisdictions. Find out which kind yours is before you build the pilot, and bring your evaluation showing where Qwen3.5 is materially better on your own code. That’s the evidence that turns a jurisdiction question into a risk decision instead of a reflex.

If Qwen3.5 is blocked outright, run Gemma 4 alone and treat Granite 4.1 as the fallback for heavier work. You give up capability, but you keep the program moving while the Qwen decision is escalated.

What I’d rule out is leading with DeepSeek-R1-Distill. The weights don’t phone home any more than the others do. But DeepSeek’s consumer app has been restricted by several governments and agencies, the name alone tends to stall the conversation, and its strength (reasoning) usually isn’t the bottleneck in day-to-day platform work.

Where It Runs Matters More Than Which Model

You have four realistic options, and the governance picture changes far more between them than between models:

1. Developer workstations. Easiest to demo, hardest to govern. Every machine has its own model versions, nothing logs centrally, and Ollama’s API has no authentication, so a misconfigured OLLAMA_HOST=0.0.0.0 exposes it to the whole network.
2. A shared inference server in your network. One GPU host or AKS node pool serving the team, with an authenticating gateway in front (Ollama has no built-in auth, so the gateway has to do it), central logging, and one place to pin and update models.
3. Open models hosted in your Azure subscription. The same as option 2, but on a GPU VM or AKS in a landing zone spoke, behind Private Link, with the identity and policy controls you already run.
4. Azure OpenAI with Private Link and Zero Data Retention. Not local, but frontier-model capability with network isolation, keyless auth and no retained prompts. See the Privacy-First AI DevOps blueprint.

My recommendation: if your data can legally go to Azure OpenAI under your contracts and classification policy, option 4 gives your developers a much better model for less operational work, and the 8B models become a complement rather than the plan. If it can’t, because a client contract, a data classification rule or a regulator says the code doesn’t leave your infrastructure, choose option 3, or option 2 if you have no suitable Azure footprint.

What I’d rule out is option 1 as the production pattern. Workstations are fine for the evaluation. They’re not something you can show an auditor.

The Decision Matrix

Hands-On: Evaluate on Your Own Code Before You Ask

Your approval request is stronger with evidence from your own repositories than with a leaderboard screenshot. This script runs the same set of prompts against each candidate through Ollama’s REST API and saves the outputs side by side for review.

Put one prompt per line in prompts.txt, drawn from real work: “Write a Pulumi TypeScript component for a storage account with public access disabled,” “Add input validation to this function,” and so on.

#!/usr/bin/env bash
# Run each prompt against each candidate model and save the outputs for review.
set -euo pipefail

MODELS=("qwen3.5:9b" "gemma4:e4b")
OUT_DIR="eval-$(date +%Y%m%d)"
mkdir -p "$OUT_DIR"

n=0
while IFS= read -r prompt; do
  [ -z "$prompt" ] && continue
  n=$((n + 1))
  for model in "${MODELS[@]}"; do
    jq -n --arg m "$model" --arg p "$prompt" '{model: $m, prompt: $p, stream: false}' \
      | curl -s http://127.0.0.1:11434/api/generate -d @- \
      | jq -r '.response' > "$OUT_DIR/prompt${n}_${model//[:\/]/_}.md"
  done
done < prompts.txt

echo "Wrote $(ls "$OUT_DIR" | wc -l) outputs to $OUT_DIR"

Have two engineers score the outputs blind, without knowing which model produced which file. Record whether each output compiles, follows your standards and needed edits. That scoring sheet becomes the “limitations tested” evidence your model risk reviewers will ask for.

Best Practices

Mirror and pin. Pull approved models once, store them in an internal registry or artifact store, and record the model ID that ollama list shows. Production servers pull from your mirror, never straight from the public registry.

Give each model an owner. Someone decides when to move to a new version, re-runs the evaluation, and updates the inventory. “Latest” is not a version.

Keep two or three models, each with a role. One model for everything means picking the wrong trade-off for most tasks.

Verify before you trust a new language or framework. Training cutoffs leave blind spots, and models don’t always know what they don’t know. Send a few representative prompts and confirm the output compiles.

Troubleshooting

Problem: The model consistently generates outdated syntax.

Cause: The model’s training data predates the framework version you’re using.

Fix: Add explicit version constraints to the Modelfile system prompt, for example: "You must use React 19 Server Components syntax. Do not use class components or legacy lifecycle methods." Models follow explicit constraints more reliably than they infer them.

Problem: CUDA error: out of memory when switching models.

Cause: Ollama allocated VRAM for the new model before fully evicting the previous one.

Fix: Unload explicitly with ollama stop qwen3.5:9b, wait for ollama ps to show no active models, then load the next one. On a shared server, size the GPU for the models you keep loaded at once, not for one at a time.

Key Takeaways

1. Qwen3.5 9B and Gemma 4 E4B are the best models in the class. Qwen3.5 is the strongest coder; Gemma 4 is the most efficient. DeepSeek-R1-Distill reasons well but stalls approvals; Granite 4.1 is a fallback, not a first choice.
2. Open weights don’t send data back to the developer. The approval question is software supply chain and training provenance, not data egress.
3. Submit Qwen3.5 and Gemma 4 as a pair. Gemma 4 should clear quickly; Qwen3.5 tests your third-party risk team, so bring evidence from your own code.
4. Where it runs matters more than which model. A shared, authenticated server in your own subscription beats workstations. Azure OpenAI with Private Link and Zero Data Retention beats both when your data is allowed to use it.
5. Bring evidence from your own code. A blind evaluation on real prompts is what turns a pilot into an approval.

Next Steps:

• Read The Ultimate Guide to Secure AI DevOps on Azure to compare this with Azure OpenAI hardened with Private Link, keyless identity and Zero Data Retention.
• Read Mastering Zero Data Retention: The Guide to Modified Abuse Monitoring in Azure OpenAI if the question behind the CISO’s email is really prompt retention.

Sources

• Hugging Face: Qwen3.5-9B model card
• Hugging Face: Gemma 4 E4B model card
• Hugging Face: DeepSeek-R1-Distill-Qwen-7B model card
• Hugging Face: Granite 4.1 8B model card
• IBM: Granite 4.0 announcement (ISO 42001 and signed checkpoints)