Maker's Notes: Context Engineering as the Core of LLM Application Development
The Pragmatic Path to LLM Simplicity
Lately, I’ve been thinking about how easy it is to overcomplicate things—especially when the tools at your disposal are powerful, new, and evolving fast.
Over the past year, I’ve worked on a handful of Gen-AI/LLM-driven projects—some shipped to users, some built for internal use. And looking back, the things that worked best weren’t the most sophisticated or novel. They were the things that made sense. The things that stayed out of the way.
In the rush to build with AI, it’s tempting to start stacking frameworks, databases, and agent flows before the product even has users. I’ve made that mistake. We all have. But what I’ve learned is this: simplicity isn’t the opposite of ambition—it’s what gives your work the space to grow.
This piece is about that lesson. It’s about resisting the pressure to impress, and choosing instead to build things that are clear, fast, and purposeful—especially when you're still figuring out what works.
At Makers & Breakers, I explore the art of creation and reinvention, building systems that matter, breaking down barriers, and reflecting on the lessons we uncover along the way.
Maker’s Notes is where I share personal insights and stories from the frontlines of product development, leadership, and tech. These reflections tackle the messy realities of building, breaking, and learning—designed to challenge assumptions and spark new ways of thinking.
Simple First, Always
There are two techniques I return to again and again. Neither of them will win you likes on LinkedIn, but they work:
Direct Context Injection:
With context windows getting bigger, sometimes the best move is the obvious one: collect the relevant snippets and stick them in the prompt. It’s fast, transparent, and doesn’t require setting up a vector database. In many cases, that’s all you need.
Basic Keyword Search:
Tools like
grep/ripgrepor SQL queries still do a great job—especially when paired with good structure and naming conventions. For internal knowledge bases, docs, and databases, a basic keyword match often outperforms more complex semantic search. It’s predictable, debuggable, and fast to implement.
I’ve seen teams burn weeks wiring up RAG pipelines before they even knew what kind of questions users would ask. And when something broke, nobody could explain why. Simpler systems give you more confidence—and that confidence buys you speed.
Of course, there are exceptions. If you’re working in a regulated space, or you already have scale and varied input, complexity might be necessary. But let that need emerge—don’t assume it from day one.
A Path You Can Walk One Step at a Time
Here’s a progression I’ve seen work well. It’s not a framework—just a sensible path that keeps things proportional to the problem:
Start with basic keyword matching.
Search documents with grep or a similar tool. Manually curate where needed. Use heuristics if that gets you 80% of the way there.
Add semantic search when keyword matching breaks down.
Long-tail queries, ambiguous inputs, or growing datasets might push you there. If so, bring in vector search with something lightweight like PGVector.
Introduce hybrid or metadata filtering.
Once you have a vector store, combine it with keyword filters or structured metadata—dates, tags, categories. Keep your retrieval observable and understandable.
Only reach for agents or orchestration if your domain demands it.
Multi-step reasoning, tool use, or planning logic should be a response to a real product need—not something you add “just in case.”
This progression keeps you grounded. It lets you grow your system without overcommitting too early.
Build What’s Needed. Nothing More.
Context engineering isn’t about finding the fanciest technique—it’s about finding the right level of effort for the moment you're in. The tools are impressive, yes. But they’re only useful if they move you closer to a product that works.
Most of the best systems I’ve seen are quiet. They do the job. They make sense. They’re built by people who know when to stop adding.
So when you're standing at the next architectural crossroads, pause and ask: “Is this helping—or just making me feel clever?”
Let’s keep building things that are honest, useful, and easy to understand. Let’s keep things simple—until complexity earns its keep.
— Luka