You’re three files deep into a proprietary codebase, staring at a login flow you need to build, and your hand is already halfway to Ctrl+C to paste the surrounding logic into a cloud AI chat window. Then you remember the email from legal. Nothing containing core business logic leaves the network — full stop, no exceptions, no “just this once for a boilerplate function.” So you close the tab and open a blank file instead, back to typing every line of a JWT auth route by hand, the AI agent that could’ve scaffolded it in thirty seconds sitting completely off-limits.
That trade-off used to be real: skip the AI entirely, or spend the budget of a small GPU cluster to run something capable enough to be worth the trouble. It isn’t anymore. As of mid-2026, an 8GB consumer GPU — the kind already sitting in an RTX 3060 or 4060 — runs a genuinely capable agentic coding workflow, entirely offline, with nothing crossing your network boundary. The stack is three pieces: Ollama as the inference engine, Qwen3.5 9B as the model doing the reasoning, and OpenCode as the terminal agent that reads your files, writes code, and executes commands on your behalf.
This is the setup guide for that stack, end to end: the hardware math that makes 8GB enough, wiring OpenCode to a local Ollama instance, and an actual scaffolding run — frontend, backend, and a working JWT authentication route — built by a model that never once touched the internet.
The 8GB GPU Reality Check
Choosing Your Local AI Stack
Here’s the number that decides everything else: an unquantized, FP16 version of a 9B parameter model needs roughly 14-16GB of VRAM just to load. An 8GB card doesn’t get close. Quantization is what closes that gap — specifically 4-bit Q4_K_M quantization, which compresses Qwen3.5 9B’s weight footprint down to roughly 4.5GB. That leaves 3.5GB of VRAM for the KV cache, the working memory the model uses to track the current conversation and generation. Qwen3.5 natively supports context windows up to 262,144 tokens, but that cache ceiling is what actually governs you on 8GB hardware — practically, you’re working with around 60,000 tokens before the cache spills into system RAM.
That spillover is the failure mode to actually worry about, more than any hard crash. When VRAM runs out and the OS falls back to shared system memory, token generation speed collapses from a usable ~40 tokens/sec down to under 2 tokens/sec (depending on your system memory bandwidth) — the difference between a component generating in five seconds and the same component taking four minutes while your GPU fan spins up for nothing.
Why Qwen3.5 9B specifically, instead of reaching for a bigger model? Because a 30B+ model doesn’t fit this hardware at all without aggressive quantization that guts its output quality, and a 9B model at Q4_K_M hits a real sweet spot: small enough to leave headroom for context, strong enough at tool-calling and code generation to run OpenCode’s agentic loop reliably. OpenCode is the orchestrator here, not the brain — it’s the layer that reads your project, decides which files to touch, and executes shell commands. Qwen3.5 9B is the reasoning underneath it, deciding what those files should actually contain.
Local Agentic Architecture Diagram
Visualizes the local agentic architecture where OpenCode orchestrates file and shell operations while delegating reasoning to a locally hosted Qwen3.5 model via Ollama.
Visual Notes:
- The network boundary strictly ends at
localhost, meaning no proprietary code ever leaves the machine. - OpenCode handles project context and tool execution, while Qwen3.5 provides the intelligence to decide which actions to take.
Installing the Offline Engine
Pull the model:
ollama run qwen3.5:9b
Ollama resolves that tag to the Q4_K_M quantization by default — the sweet spot for 8GB cards. Confirm it with ollama list before you go any further; if you see a multi-gigabyte discrepancy from the ~4.5GB figure above, you’ve pulled a different quantization and should re-check the tag.
Configuring OpenCode for Local Execution
Pointing the Agent Home
Install the CLI globally:
npm install -g opencode-ai
Then point it at your local Ollama instance instead of any cloud provider. Edit ~/.config/opencode/opencode.json:
{
"provider": {
"ollama": {
"npm": "@ai-sdk/openai-compatible",
"name": "Ollama",
"options": {
"baseURL": "http://localhost:11434/v1"
},
"models": {
"qwen3.5": { "name": "qwen3.5:9b" }
}
}
}
}
That baseURL pointing at localhost:11434 is the entire trust boundary. As long as it stays pointed there, nothing you write is capable of leaving your machine — there’s no cloud fallback silently kicking in if Ollama hiccups. Launch OpenCode from your project root, and a successful connection shows the model responding to a basic prompt within a second or two. If it hangs instead, Ollama isn’t running — start it before OpenCode, not after.
One more thing stated plainly: never leave the agent unsupervised when it runs npm install or any system command, even fully offline. Local doesn’t mean risk-free, it just means the risk is scoped to your own machine instead of a third party’s.
Establishing Memory with AGENTS.md
A 9B model with a 60K-token practical context window forgets your architectural preferences fast if you re-explain them every session. AGENTS.md, placed at the project root, fixes that — it’s a standing reference OpenCode loads fresh each time, not conversation history that ages out. A minimal starting point:
# AGENTS.md
## Stack
Next.js 15 (App Router) frontend, Express.js REST backend.
## CRITICAL RULES
- Never use `var`. Use `const` or `let`.
- Never hardcode secrets. Use `process.env`.
- Tailwind CSS utility classes only — no inline styles.
This isn’t just a style preference — it directly conserves the resource you’re shortest on. Every prompt where the model would otherwise guess at your stack and generate the wrong pattern is a prompt that burns tokens on a rewrite instead of new code. AGENTS.md heads that off before it starts. The next article in this series explains how to build this file out properly.
Architecting the Application
Scaffolding the Frontend
Don’t ask for “a dashboard.” Ask for one piece: “Generate a Next.js 15 App Router project with TypeScript and Tailwind using create-next-app, then create a single Sidebar.tsx component in src/components/.” One component per prompt keeps the output inside the 150-200 line range where a 9B model’s syntax stays reliable — past that, closing tags and imports start drifting.
Generating the Backend API
Move OpenCode into the backend directory and scope the request the same way: “Initialize an Express server with Helmet, CORS, and express-rate-limit.” Then, separately, ask for routes. The 9B model handles this well specifically because CRUD logic and standard middleware are exactly the kind of well-represented, structurally predictable code a smaller model reasons about reliably — it’s novel architecture and ambiguous requirements where local models start to struggle, not routine REST scaffolding.
Practical Example: Building an Authentication Flow
Step 1 — the prompt. With Express already scaffolded and bcrypt, jsonwebtoken, and zod installed:
“Write a
POST /loginroute inroutes/authRoutes.js. Validatepasswordwith Zod, look up the user with Prisma, compare the password with bcrypt, and return a signed JWT on success. Useprocess.env.JWT_SECRETfor signing. Wrap the logic in a try-catch returning 500 on internal errors, and return 401 on invalid credentials.”
Step 2 — let the agent write it and install what’s missing. OpenCode installs any dependency gaps it finds and produces the route directly.
Step 3 — review before you trust it.
// routes/authRoutes.js
const express = require('express');
const bcrypt = require('bcrypt');
const jwt = require('jsonwebtoken');
const { z } = require('zod');
const { PrismaClient } = require('@prisma/client');
const router = express.Router();
// Note: For a production app, export a singleton Prisma instance from a separate module
// instead of instantiating it directly in the route file to prevent connection exhaustion.
const prisma = new PrismaClient();
const loginSchema = z.object({
email: z.string().email(),
password: z.string().min(8)
});
router.post('/login', async (req, res) => {
try {
const parsed = loginSchema.safeParse(req.body);
if (!parsed.success) {
return res.status(400).json({ error: parsed.error.flatten() });
}
const { email, password } = parsed.data;
const user = await prisma.user.findUnique({ where: { email } });
if (!user) {
return res.status(401).json({ error: 'Invalid credentials' });
}
const validPassword = await bcrypt.compare(password, user.password);
if (!validPassword) {
return res.status(401).json({ error: 'Invalid credentials' });
}
const token = jwt.sign(
{ userId: user.id },
process.env.JWT_SECRET,
{ expiresIn: '1h' }
);
// Note: For a production web app, you should set this token in an HttpOnly
// cookie rather than returning it in the body to prevent XSS attacks.
return res.status(200).json({ token });
} catch (error) {
console.error('login error:', error);
return res.status(500).json({ error: 'Internal server error' });
}
});
module.exports = router;
A working POST /login route, generated in seconds, with the password comparison and JWT signing both correctly in place — and not a single request made outside localhost.
Best Practices and Tips
Do:
- Confirm your
qwen3.5:9bpull resolved toQ4_K_Mwithollama listbefore you start building. - Write narrow, single-component prompts. A 9B model’s output quality holds up far better at 40 lines than at 400.
- Put architectural boundaries in
AGENTS.md, not in a prompt you retype every session.
Don’t:
- Ask the agent to build the entire full-stack app in one prompt. You’ll exhaust the practical 60K-token context window and get truncated files with broken imports.
- Walk away while the agent runs
npm installor any shell command, even locally. - Skip reviewing generated code for basic issues like SQL injection just because it runs offline. Local execution protects your IP, not your code quality.
Performance tip: restart OpenCode after finishing a major milestone — the frontend scaffold, say — instead of continuing the same session into backend work. A fresh session means a flushed context window instead of a slow accumulation toward that 60K ceiling.
Troubleshooting Common Issues
OpenCode hangs or generation crawls below 5 tokens/sec.
The KV cache spilled from VRAM into system RAM. Restart Ollama, close any other GPU-accelerated app (hardware-accelerated browser tabs and some VS Code extensions both eat VRAM quietly), and confirm ollama list shows qwen3.5:9b resolved to Q4_K_M, not an unquantized version.
The model hallucinates file paths or forgets files it already created.
This is context overload — older messages describing your directory tree aged out of the model’s effective attention. Stop the agent with /exit, clear the chat, and restate the exact absolute file path in your next prompt rather than assuming it remembers.
Conclusion
You don’t need a cloud AI budget or a rack of 24GB GPUs to run an agentic coding workflow — 8GB of VRAM, a Q4_K_M-quantized Qwen3.5 9B, and OpenCode get you there, with your proprietary code never leaving your machine. The discipline that makes it work is the same one that makes any AI collaboration better: narrow prompts, explicit architectural rules in AGENTS.md, and reviewing what comes back instead of trusting it blind.
Next steps: install Ollama and OpenCode today and confirm your hardware handles the Q4_K_M model at a usable speed before you build anything on top of it. The next article in this series covers frontend generation in depth — breaking a Next.js dashboard into the atomic components a local model can actually hold in its head.
Related Articles in This Series:
- Scaffolding a Modern Frontend with OpenCode and Next.js
- Building Secure Backend APIs using Qwen3.5 9B