luismori.
AI Research Agents Claude Code Workflows Engineering

AI Agents as Research Tools: From Curiosity to Prototype in Hours

How AI agents are collapsing the gap between 'I wonder how X works' and 'I have a working implementation of X' — and the research workflows that actually hold up under pressure.

10 min read

There is a particular kind of engineering research loop that used to take days. You hit an unfamiliar problem — a new protocol, an unfamiliar framework, a domain you’ve never touched — and the path from “I don’t understand this” to “I have working code” involved reading docs, hunting Stack Overflow, reading papers, experimenting locally, reading more docs, and slowly assembling a mental model.

AI agents have collapsed that loop. Not by making research easier, but by changing the fundamental unit of work. The question is no longer “how long will it take me to understand this?” — it’s “how do I structure the conversation to get to the right answer faster?”

What Changed (and What Didn’t)

The most important thing AI agents changed about research is iteration speed. You can now explore five approaches to a problem in the time it used to take to read one documentation page carefully.

What didn’t change: the need for judgment. Agents hallucinate confidently. They’ll describe APIs that don’t exist, cite papers with wrong results, or produce code that looks correct but fails at the edges. The research skill that matters most in an AI-augmented world is knowing when to trust and when to verify.

The productive mental model: treat the agent as an extremely well-read colleague who sometimes misremembers details. Their synthesis is usually good. Their specifics need checking.

The Research Stack

Most technical research now flows through a small set of tools:

Claude (Sonnet/Opus) — conversational depth, long context for reading full documents and codebases, strong reasoning for comparing approaches.

Perplexity / Search-augmented agents — for current information. Claude’s knowledge has a cutoff; Perplexity pulls live sources. Use this when you need what happened in the last 6 months.

Claude Code — for research that requires running code. This is the game changer. Agents that can write, execute, debug, and iterate code in your actual environment collapse the gap between reading about something and understanding how it actually behaves.

Dedicated MCP servers — for domain-specific research. A database MCP lets the agent query your schema. A GitHub MCP lets it read real codebases. A docs MCP lets it search indexed documentation. Research quality scales with tool access.

Workflow 1: The Architecture Exploration

You’ve been asked to evaluate three approaches to a problem you don’t fully understand yet. Classic example: “should we use event sourcing, CQRS, or a simpler denormalized approach for our audit log requirements?”

The wrong move: open three browser tabs and start reading.

The right move: start with a structured prompt.

I need to evaluate three architectural approaches for an audit log system:
event sourcing, CQRS, and a denormalized append-only table.

Context:
- ~500 write events/second at peak
- Need to replay history for compliance queries
- Team has no existing experience with event sourcing
- Postgres is our current DB, team is comfortable with it

For each approach, give me:
1. How it works (3-sentence summary)
2. What breaks at our scale
3. The hidden complexity that kills teams new to this pattern
4. A concrete example of the trade-off in our specific context

Structure forces the agent to go deep on each option instead of giving you a generic “it depends” answer. The “hidden complexity” ask is especially valuable — that’s where agents surface knowledge that would otherwise take you weeks to discover.

After this initial exploration, follow up with the sharpest tradeoff you identified:

You mentioned that event sourcing at 500 writes/second creates snapshot
management complexity. Walk me through exactly what that looks like — what
does the code look like at the point where it becomes a problem, and what's
the standard mitigation?

Go three to four levels deep before you accept a conclusion.

Workflow 2: The Codebase Archaeology

You’ve inherited a system. There’s no documentation. You need to understand how it works well enough to change it safely.

This is where Claude Code changes the research paradigm entirely.

# Give Claude Code the full codebase context
> Explore the codebase and build a mental model of the data flow.
  Start from the HTTP entry points and trace how a user authentication
  request flows through the system. Note any non-obvious patterns,
  hidden dependencies, or places where the architecture seems inconsistent.

Let it explore for several minutes. Then:

> Based on your exploration, what are the three riskiest places to make
  changes without causing unintended side effects? Why?

The output of this session isn’t just information — it’s a working map of the system. You can then ask targeted questions:

> Where does session invalidation happen? Is it consistent across all
  auth paths, or are there edge cases where sessions don't get cleaned up?

This workflow does in an hour what a documentation sprint takes a week to accomplish. The agent reads every file, traces every import, and synthesizes patterns that a human skimming the codebase would miss.

Workflow 3: The “Build It to Understand It” Pattern

For some topics, reading about them is fundamentally insufficient. You need to run the thing. This is where AI agents + Claude Code create a research workflow with no good prior analog.

The pattern: describe what you want to explore, let the agent build a minimal working version, break it deliberately, and observe what happens.

Example: understanding how PostgreSQL advisory locks actually behave under concurrent load.

Build a minimal test harness that demonstrates PostgreSQL advisory locks:
- Two competing processes trying to acquire the same lock
- Show what happens when the lock holder crashes mid-transaction
- Show the difference between session-level and transaction-level advisory locks
- Make the behavior observable (log each step, timestamps, connection IDs)

Claude Code writes the test, runs it, and shows you the output. You then push on the edges:

Now modify the test to simulate what happens if we hold the advisory lock
while doing a long-running query that sometimes times out.
Does the lock release on timeout? Show me.

In 20 minutes you have empirical understanding of how advisory locks behave in failure scenarios — understanding that would have taken hours of reading docs and Stack Overflow, with no guarantee you’d correctly internalize the subtleties.

Workflow 4: The Literature Synthesis

You’re evaluating a new technique — say, using columnar storage for your analytics workload. You want to understand the academic and practitioner consensus, not just the vendor marketing.

For this, the agent works best as a structured synthesizer, not a search engine:

I'm evaluating columnar storage (specifically DuckDB and ClickHouse) for
analytics on structured event data. I need to understand:

1. What does the systems research literature say about the fundamental
   performance characteristics of columnar vs row-oriented storage?
2. What are the known failure modes that practitioners report but vendors
   don't highlight?
3. What workload shapes is columnar storage *not* good for?

Don't give me marketing claims. Focus on documented behavior, known trade-offs,
and cases where the theory doesn't match practice.

Then follow the thread. When the agent mentions “late materialization as a key columnar optimization,” ask it to explain exactly how late materialization works and where it breaks down. Each interesting claim is a branch you can explore.

The synthesis that takes a week of reading to build manually — reading the foundational papers, the practitioner blog posts, the “we migrated away from X” war stories — can be assembled in a focused two-hour session.

When to Trust, When to Verify

The hardest skill in AI-augmented research is calibration. Some heuristics that hold up:

Trust the agent on:

  • Conceptual explanations of well-documented topics
  • Synthesizing trade-offs across approaches
  • Generating initial implementations for experimentation
  • Identifying questions you should be asking but aren’t

Verify before relying on:

  • Specific API signatures and parameters
  • Performance benchmarks and numbers
  • Claims about “what most teams do” or “the industry standard”
  • Anything where being wrong would be expensive to discover later

Always verify:

  • Security-sensitive code
  • Anything involving cryptography, auth, or data handling
  • Claims about how third-party services behave (they change)
  • Anything the agent seems unusually confident about

A useful forcing function: after a research session, write a two-paragraph summary of what you learned and what you’d still want to verify. If you can’t identify anything to verify, your research isn’t done.

Structuring Long Research Sessions

Research with AI agents works best when it’s structured, not free-form. A few practices that help:

Start with scope. Tell the agent what you’re trying to decide, not just what you want to know. “I’m trying to decide whether to use Redis Streams or Kafka for a 10K msg/sec workload” is more productive than “explain Kafka.”

Use the five-layer rule. For any important claim, go five questions deep before accepting it. “Why?” four times after every answer. This surfaces the underlying assumptions, which is where the real understanding lives.

Write as you go. Ask the agent to help you write down what you’ve learned in a structured format after every major insight. Long conversations drift; structured notes capture the useful parts.

End with the disagreement. Finish every research session by asking the agent: “What’s the strongest argument against the conclusion we’ve reached? What would make this analysis wrong?” The agent is good at steelmanning positions it just argued against.

Practical Example: A Real Research Session

Here’s what a research session on a real problem looks like, compressed:

Problem: Should we migrate our background job queue from Sidekiq (Redis-backed) to a Postgres-based queue like Solid Queue?

Session structure:

  1. Architecture exploration: “Compare Sidekiq and Solid Queue on reliability guarantees, performance characteristics, and operational complexity. Be specific about what ‘at-least-once delivery’ means in practice for each.”

  2. Failure mode investigation: “What happens to jobs in Sidekiq when Redis runs out of memory? What happens to jobs in Solid Queue when the Postgres connection pool is exhausted? Give me the exact failure behavior.”

  3. Build-it-to-understand-it: “Build a minimal test that demonstrates Sidekiq’s behavior when a worker crashes mid-job. Does the job get retried? From where in the execution?”

  4. Team-specific analysis: “We have a team of 4 engineers, a Heroku Postgres setup already in place, and no Redis expertise. We currently have ~200 jobs/minute. Which system reduces our operational risk over the next 2 years?”

  5. Steelman: “Make the strongest possible argument for not migrating from Sidekiq, given our context.”

Total session time: about 90 minutes. Output: a decision with documented reasoning, a list of things to verify with the Solid Queue maintainers, and a proof-of-concept migration script.

The Compounding Effect

The most underrated aspect of AI-augmented research is how it compounds. Each research session produces artifacts — structured notes, decision logs, comparison tables, proof-of-concept code — that become context for future sessions.

A team that runs structured research sessions and captures the outputs builds a knowledge base that agents can use to accelerate every subsequent decision. The CLAUDE.md file in a Claude Code project is the beginning of this — architectural decisions, chosen patterns, known tradeoffs — but the model extends to every research-intensive decision the team makes.

The teams winning with AI aren’t the ones who use agents for the most tasks. They’re the ones who’ve figured out how to use agents to build organizational knowledge faster than their competitors.

Sources