Skip to content
Docs

Build a durable web research agent with Workflow SDK

Build a durable web research agent using the Workflow SDK and AI SDK. Learn to use the search tools and collect references from the web. Each search, read, and finding becomes a journaled step that resumes after a crash or mid-run deploy.

Knowledge Base/AI SDK
14 min read
Last updated June 17, 2026

Build a research agent that searches the web, extracts findings from each source, and returns a cited report, without losing progress when a crash, restart, or deployment interrupts a run that can loop 20 to 50 times. AI SDK's WorkflowAgent runs the reasoning loop to search a query, read a source, record a finding, decide what to search next, and Workflow SDK runs that loop not as one long request but as a sequence of durable, journaled steps, so the search results and findings already recorded survive any interruption.

You'll build a Next.js app that hands a user's question to a WorkflowAgent running inside Vercel Workflows, performs a web search, which searches source pages, and extracts findings from each. Along the way, you'll learn to stream the agent's progress to the browser in real time and return a final report with citations.

To get started, clone this template’s GitHub repository, configure credentials, and run the local app:

Terminal
pnpm install
pnpm dev

Open http://localhost:3000 and submit:

Prompt
When to use Vercel Workflows?

You can also deploy it to Vercel with one click:

Durable web research agent with Workflow SDK

Ask a question and the app researches the web, fetches source pages, extracts cited findings, and synthesizes a grounded research brief while streaming each step to the browser in real time.

Deploy Template

If you are working with an AI coding agent, hand it the project and this prompt:

AI Prompt
I want to deploy and customize the durable web research agent template.


Clone the repo at https://github.com/vercel-labs/durable-web-research-agent, then read AGENTS.md and ARCHITECTURE.md to understand how it's built and the conventions to follow.


Install dependencies with pnpm, then help me run it locally with `pnpm dev` and deploy with `vercel deploy`. When searching for information, check for applicable skill(s) first and then Vercel documentation.

Turn your agent into a Vercel expert with this plugin. It gives your coding agent current knowledge of the Vercel products this template uses, including Vercel Connect, Vercel Workflows, Vercel Cron, AI Gateway, and Chat SDK. The plugin is optional; it is not required to use this template or for this guide.

Terminal
npx plugins add vercel/vercel-plugin

The request flow looks like this:

API structure
POST /api/research
  -> start(researchWorkflow, [question])
  -> searchWeb step using AI Gateway search
  -> for each source page, extractFindings step
  -> synthesizeReport step
  -> return ResearchReport


GET /api/readable/:runId
  -> read workflow stream
  -> transform workflow chunks into UI chunks
  -> send SSE events to the browser


GET /api/run/:runId
  -> read status and returnValue

Before you begin, make sure you have:

For local development, link the Vercel project and pull environment variables:

vercel link
vercel env pull

AI Gateway authenticates requests using Vercel OIDC tokens, which Vercel generates and links to your project automatically.

Create a new Next.js app with create-next-app :

Terminal
pnpm create next-app@latest durable-web-research-agent --yes
cd durable-web-research-agent

Then install the Workflow and AI SDK packages from the current canary release line:

Terminal
pnpm add workflow @ai-sdk/workflow@canary ai@canary zod @vercel/oidc ajv

Wrap your Next.js config with withWorkflow:

next.config.ts
import type { NextConfig } from "next";
import { withWorkflow } from "workflow/next";


const nextConfig: NextConfig = {
  serverExternalPackages: ["@vercel/oidc", "ajv"],
};


export default withWorkflow(nextConfig);

Keep the workflow state compact. Store findings, URLs, snippets, and short page text. Avoid writing the entire crawled websites into the workflow event log.

workflows/research-types.ts
export type Finding = {
  claim: string;
  sourceUrl: string;
  snippet: string;
};


export type ResearchReport = {
  sections: Array<{
    heading: string;
    body: string;
  }>;
  citations: Array<{
    claim: string;
    sourceUrl: string;
  }>;
};


export type SearchWebResult = {
  query: string;
  answer: string;
  sources: Array<{
    title?: string;
    url: string;
    text?: string;
  }>;
};


export type SourcePage = {
  title?: string;
  url: string;
  text: string;
  error?: string;
};


export type ExtractFindingsResult = {
  findings: Finding[];
};

The Workflow SDK gives the app durable control flow and the AI SDK gives the app model calls, AI Gateway tools, and structured outputs. Implement the two branches, as follows:

  • Put generateText(), fetch(), Gateway search tools, and any other side effects inside "use step" functions.
  • Keep the "use workflow" function focused on orchestration, loops over already-returned data, and calls to step functions.
  • Return compact typed objects from steps so Workflow can journal them and replay the run without calling the model again.

The research steps do the work with side effects:

  • searchWeb() calls an AI Gateway search tool.
  • extractFindingsFromPages() calls an AI Gateway model with Output.object.
  • fetchSourcePage() helps when your search tool returns URLs without enough page text.

The example below uses AI Gateway's parallelSearch tool. Search belongs in a step because it is external I/O: it can fail, retry, rate-limit, or return different results over time. Once the step completes, Workflow records the selected sources, and replay can continue from those recorded sources instead of searching again.

parallelSearch returns source-oriented search results with excerpts that are already useful to an LLM. If your project prefers Perplexity-backed search, swap in gateway.tools.perplexitySearch({ maxResults: 5 }).

workflows/research-steps.ts
import { generateText, gateway, Output } from "ai";
import { z } from "zod";
import type {
  ExtractFindingsResult,
  SearchWebResult,
  SourcePage,
} from "./research-types";


const MAX_SOURCE_PAGES = 5;
const MAX_PAGE_CHARS = 12000;


const extractedFindingsSchema = z.object({
  findings: z.array(
    z.object({
      claim: z.string(),
      sourceUrl: z.string().describe("One of the source URLs provided."),
      snippet: z.string(),
    }),
  ),
});


function isRecord(value: unknown): value is Record<string, unknown> {
  return typeof value === "object" && value !== null;
}


function firstString(...values: unknown[]) {
  return values.find((value): value is string => typeof value === "string");
}


function normalizeGatewaySearchOutput(output: unknown): SourcePage[] {
  if (!isRecord(output)) {
    return [];
  }


  const candidates =
    Array.isArray(output.results)
      ? output.results
      : Array.isArray(output.sources)
        ? output.sources
        : [];


  return candidates
    .filter(isRecord)
    .map((result) => {
      const url = firstString(result.url, result.link, result.sourceUrl);
      const text = firstString(
        result.excerpt,
        result.snippet,
        result.text,
        result.content,
      );


      if (!url) {
        return null;
      }


      return {
        title: firstString(result.title),
        url,
        text: (text ?? "").slice(0, MAX_PAGE_CHARS),
      };
    })
    .filter((page): page is SourcePage => page !== null)
    .slice(0, MAX_SOURCE_PAGES);
}


export async function searchWeb({ query }: { query: string }) {
  "use step";


  const result = await generateText({
    model: "openai/gpt-5-nano",
    prompt: `
Find reliable source pages for this research question:


${query}


Return sources that contain concrete technical details, tradeoffs, and
implementation facts. Prefer primary docs and high-signal technical articles.
`.trim(),
    tools: {
      parallel_search: gateway.tools.parallelSearch({
        mode: "agentic",
        maxResults: MAX_SOURCE_PAGES,
        excerpts: {
          maxCharsPerResult: MAX_PAGE_CHARS,
        },
      }),
    },
    toolChoice: { type: "tool", toolName: "parallel_search" },
  });


  const sources = result.toolResults
    .filter((toolResult) => toolResult.toolName === "parallel_search")
    .flatMap((toolResult) => normalizeGatewaySearchOutput(toolResult.output));


  return {
    query,
    answer: result.text,
    sources,
  } satisfies SearchWebResult;
}

Some search tools return URLs and snippets but not enough source text. In that case, add a fetchSourcePage step and call it before extraction. Keep it as a workflow step because page fetches are side effects and can fail or need retries.

workflows/research-steps.ts
function decodeHtmlEntities(value: string) {
  return value
    .replaceAll("&nbsp;", " ")
    .replaceAll("&amp;", "&")
    .replaceAll("&lt;", "<")
    .replaceAll("&gt;", ">")
    .replaceAll("&quot;", "\"")
    .replaceAll("&#39;", "'");
}


function htmlToText(html: string) {
  return decodeHtmlEntities(
    html
      .replace(/<script[\s\S]*?<\/script>/gi, " ")
      .replace(/<style[\s\S]*?<\/style>/gi, " ")
      .replace(/<noscript[\s\S]*?<\/noscript>/gi, " ")
      .replace(/<svg[\s\S]*?<\/svg>/gi, " ")
      .replace(/<[^>]+>/g, " ")
      .replace(/\s+/g, " ")
      .trim(),
  );
}


export async function fetchSourcePage({
  title,
  url,
}: {
  title?: string;
  url: string;
}): Promise<SourcePage> {
  "use step";


  try {
    const response = await fetch(url, {
      headers: {
        accept: "text/html,text/plain;q=0.9,*/*;q=0.8",
        "user-agent": "Mozilla/5.0 (compatible; DurableResearchAgent/1.0)",
      },
    });


    if (!response.ok) {
      return {
        title,
        url,
        text: "",
        error: `Fetch failed with status ${response.status}.`,
      };
    }


    const contentType = response.headers.get("content-type") ?? "";
    const body = await response.text();
    const text = contentType.includes("html")
      ? htmlToText(body)
      : body.replace(/\s+/g, " ").trim();


    return {
      title,
      url,
      text: text.slice(0, MAX_PAGE_CHARS),
    };
  } catch (error) {
    return {
      title,
      url,
      text: "",
      error: error instanceof Error ? error.message : "Unknown fetch error.",
    };
  }
}

Extraction is separate from search on purpose: search finds candidate source material and extraction decides which claims are useful enough to keep and turns messy page text into compact, typed research artifacts.

Keeping extraction separate gives you three benefits:

  • The final report can depend on a stable Finding[] instead of raw search prose.
  • The UI can stream findings as soon as each page is processed.
  • If synthesis fails, Workflow reuses the already-extracted findings instead of asking the search model to recreate them.

Use Output.object() so the extraction step returns predictable data.

workflows/research-steps.ts
export async function extractFindingsFromPages({
  pages,
  question,
}: {
  pages: SourcePage[];
  question: string;
}): Promise<ExtractFindingsResult> {
  "use step";


  const readablePages = pages.filter((page) => page.text.trim().length > 0);
  if (readablePages.length === 0) {
    return { findings: [] };
  }


  const allowedUrls = new Set(readablePages.map((page) => page.url));
  const { output } = await generateText({
    model: "anthropic/claude-haiku-4.5",
    output: Output.object({
      schema: extractedFindingsSchema,
    }),
    prompt: `
Question:
${question}


Fetched source pages:
${JSON.stringify(
  readablePages.map((page) => ({
    title: page.title,
    url: page.url,
    text: page.text,
  })),
  null,
  2,
)}


Extract 8 to 12 concise findings that answer the question. Each finding must
use one of the provided page URLs as sourceUrl and include a short snippet from
that page text. Prefer concrete technical differences, tradeoffs, use cases,
ecosystem facts, deployment details, and limitations.
`.trim(),
  });


  return {
    findings: output.findings.filter((finding) =>
      allowedUrls.has(finding.sourceUrl),
    ),
  };
}

The URL filter after generation prevents the model from inventing citations that were not in the fetched source set.

The workflow body should mostly coordinate steps. It can loop over the bounded search results, but it should not perform network requests or model calls directly.

This workflow also writes progress events to getWritable(). Those events look like AI SDK model-call stream parts, so the UI can render tool starts and finishes.

workflows/research-workflow.ts
import type { ModelCallStreamPart } from "@ai-sdk/workflow";
import { getWritable } from "workflow";
import {
  extractFindingsFromPages,
  fetchSourcePage,
  searchWeb,
} from "./research-steps";
import type { Finding } from "./research-types";
import { synthesizeReport } from "./synthesize";


const MAX_SOURCE_PAGES = 5;


type WorkflowStreamPart =
  | ModelCallStreamPart
  | { type: "finish-step" }
  | { type: "start-step" };


function findingKey(finding: Finding) {
  return `${finding.claim}\n${finding.sourceUrl}`;
}


function dedupeFindings(findings: Finding[]) {
  return [
    ...findings.reduce((deduped, finding) => {
      deduped.set(findingKey(finding), finding);
      return deduped;
    }, new Map<string, Finding>()).values(),
  ];
}


async function writeToolCall(
  writable: WritableStream<WorkflowStreamPart>,
  toolCallId: string,
  toolName: string,
  input: unknown,
) {
  "use step";


  const writer = writable.getWriter();
  try {
    await writer.write({
      type: "tool-call",
      toolCallId,
      toolName,
      input,
    } as WorkflowStreamPart);
  } finally {
    writer.releaseLock();
  }
}


async function writeToolResult(
  writable: WritableStream<WorkflowStreamPart>,
  toolCallId: string,
  toolName: string,
  output: unknown,
) {
  "use step";


  const writer = writable.getWriter();
  try {
    await writer.write({
      type: "tool-result",
      toolCallId,
      toolName,
      output,
    } as WorkflowStreamPart);
    await writer.write({ type: "finish-step" });
    await writer.write({ type: "start-step" });
  } finally {
    writer.releaseLock();
  }
}


export async function researchWorkflow(question: string) {
  "use workflow";


  const writable = getWritable<WorkflowStreamPart>();
  const normalizedQuestion = question.trim();
  const findings: Finding[] = [];


  await writeToolCall(writable, "search-0", "searchWeb", {
    query: normalizedQuestion,
    maxSources: MAX_SOURCE_PAGES,
  });
  const searchResult = await searchWeb({ query: normalizedQuestion });
  await writeToolResult(writable, "search-0", "searchWeb", searchResult);


  for (const [index, source] of searchResult.sources
    .slice(0, MAX_SOURCE_PAGES)
    .entries()) {
    const fetchCallId = `fetch-${index}`;
    await writeToolCall(writable, fetchCallId, "fetchPage", source);
    const page = source.text
      ? {
          title: source.title,
          url: source.url,
          text: source.text,
      }
      : await fetchSourcePage(source);


    await writeToolResult(writable, fetchCallId, "fetchPage", page);


    const extractCallId = `extract-${index}`;
    await writeToolCall(writable, extractCallId, "extractFindings", {
      sourceUrl: page.url,
      title: page.title,
      hasText: page.text.trim().length > 0,
    });
    const extracted = await extractFindingsFromPages({
      pages: [page],
      question: normalizedQuestion,
    });
    findings.push(...extracted.findings);
    await writeToolResult(writable, extractCallId, "extractFindings", extracted);
  }


  return synthesizeReport(normalizedQuestion, dedupeFindings(findings));
}

To keep the run predictable even if a search provider returns more results than requested, this adds the following guardrails:

  • the Gateway search tool receives maxResults: 5
  • the workflow slices searchResult.sources to MAX_SOURCE_PAGES

Synthesis should read only the extracted findings. This keeps the report grounded in durable evidence instead of the model's memory of earlier search text.

That design also makes failure recovery cheaper. If the final report step retries, it receives the same Finding[] from the event log and does not repeat search or extraction. If there are no findings, return an explicit empty-state report instead of asking the model to improvise.

workflows/synthesize.ts
import { generateText, Output } from "ai";
import { z } from "zod";
import type { Finding, ResearchReport } from "./research-types";


const reportSchema = z.object({
  sections: z.array(
    z.object({
      heading: z.string(),
      body: z.string(),
    }),
  ),
  citations: z.array(
    z.object({
      claim: z.string(),
      sourceUrl: z.string().describe("The source URL for the cited claim."),
    }),
  ),
});


export async function synthesizeReport(
  question: string,
  findings: Finding[],
): Promise<ResearchReport> {
  "use step";


  if (findings.length === 0) {
    return {
      sections: [
        {
          heading: "Insufficient sourced findings",
          body:
            "The research run completed without any extracted findings, so a cited report cannot be generated from the workflow state.",
        },
      ],
      citations: [],
    };
  }


  const { output } = await generateText({
    model: "anthropic/claude-sonnet-4.6",
    output: Output.object({
      schema: reportSchema,
    }),
    prompt: `
Question:
${question}


Findings:
${JSON.stringify(findings, null, 2)}


Write a deep, practical research brief that answers the question using only
these findings. For comparison questions, include sections for positioning,
core capabilities, ecosystem and language/runtime fit, agent/orchestration
support, deployment tradeoffs, and a recommendation matrix. Every section must
ground its claims in the findings. Include citations for the most important
claims using the exact sourceUrl values from the findings.
`.trim(),
  });


  return {
    sections: output.sections,
    citations:
      output.citations.length > 0
        ? output.citations
        : findings.slice(0, 10).map((finding) => ({
            claim: finding.claim,
            sourceUrl: finding.sourceUrl,
          })),
  };
}

The deterministic citation fallback matters. Structured output improves shape, but it does not guarantee the model will populate every optional-looking field the way your UI expects.

Start the workflow with a POST request and return the run ID. This route returns immediately. The browser can then subscribe to the readable stream and poll status by runId.

app/api/research/route.ts
import { NextResponse } from "next/server";
import { start } from "workflow/api";
import { researchWorkflow } from "@/workflows/research-workflow";


export async function POST(req: Request) {
  const { question } = (await req.json()) as { question?: string };


  if (!question?.trim()) {
    return NextResponse.json(
      { error: "Question is required." },
      { status: 400 },
    );
  }


  const run = await start(researchWorkflow, [question.trim()]);


  return NextResponse.json({
    runId: run.runId,
  });
}

Workflow streams are read by run ID. The code below converts the workflow stream into AI SDK UI chunks and sends them as Server-Sent Events.

app/api/readable/[runId]/route.ts
import { toUIMessageChunk } from "@ai-sdk/workflow";
import type { UIMessageChunk } from "ai";
import { getRun } from "workflow/api";


type RouteContext = {
  params: Promise<{ runId: string }>;
};


type ModelCallStreamPartWithToolName = {
  type: string;
  toolName?: string;
};


function createResearchUIChunkTransform() {
  return new TransformStream<unknown, UIMessageChunk>({
    start(controller) {
      controller.enqueue({ type: "start" });
      controller.enqueue({ type: "start-step" });
    },
    transform(part, controller) {
      const chunk = toUIMessageChunk(part as never);
      if (!chunk) {
        return;
      }


      if (
        chunk.type === "tool-output-available" &&
        typeof part === "object" &&
        part !== null &&
        "toolName" in part &&
        typeof (part as ModelCallStreamPartWithToolName).toolName === "string"
      ) {
        controller.enqueue({
          ...chunk,
          toolName: (part as ModelCallStreamPartWithToolName).toolName,
        } as UIMessageChunk);
        return;
      }


      controller.enqueue(chunk);
    },
    flush(controller) {
      controller.enqueue({ type: "finish-step" });
      controller.enqueue({ type: "finish" });
    },
  });
}


export async function GET(_request: Request, { params }: RouteContext) {
  const { runId } = await params;


  let run;
  try {
    run = await getRun(runId);
    await run.status;
  } catch {
    return Response.json(
      { error: `Run ${runId} was not found.` },
      { status: 404 },
    );
  }


  const readable = (
    run.getReadable() as unknown as ReadableStream<unknown>
  ).pipeThrough(createResearchUIChunkTransform());


  const encoder = new TextEncoder();
  const stream = readable.pipeThrough(
    new TransformStream<unknown, Uint8Array>({
      transform(chunk, controller) {
        const data = typeof chunk === "string" ? chunk : JSON.stringify(chunk);
        controller.enqueue(encoder.encode(`data: ${data}\n\n`));
      },
    }),
  );


  return new Response(stream, {
    headers: {
      "Content-Type": "text/event-stream; charset=utf-8",
      "Cache-Control": "no-cache, no-transform",
      Connection: "keep-alive",
      "X-Accel-Buffering": "no",
    },
  });
}

The custom transform preserves toolName on output chunks. The UI needs that metadata to decide which tool outputs contain findings.

The final report is the workflow return value.

app/api/run/[runId]/route.ts
import { NextResponse } from "next/server";
import { getRun } from "workflow/api";


type RouteContext = {
  params: Promise<{ runId: string }>;
};


function toIso(value: Date | null | undefined) {
  return value ? value.toISOString() : null;
}


export async function GET(_request: Request, { params }: RouteContext) {
  const { runId } = await params;


  try {
    const run = await getRun(runId);


    const [status, workflowName, createdAt, startedAt, completedAt, returnValue] =
      await Promise.all([
        run.status,
        run.workflowName,
        run.createdAt,
        run.startedAt,
        run.completedAt,
        run.returnValue.catch(() => null),
      ]);


    return NextResponse.json({
      runId,
      status,
      workflowName,
      createdAt: toIso(createdAt),
      startedAt: toIso(startedAt),
      completedAt: toIso(completedAt),
      returnValue,
    });
  } catch {
    return NextResponse.json(
      { error: `Run ${runId} was not found.` },
      { status: 404 },
    );
  }
}

The backend is intentionally UI-agnostic. A React console, a CLI, or an internal dashboard can all use the same three contracts:

EndpointClient responsibility
POST /api/researchSend { question }, receive { runId }
GET /api/readable/:runIdSubscribe to streamed UI chunks
GET /api/run/:runIdPoll status and read the final returnValue

The stream route already converts workflow events into AI SDK UI chunks. A client does not need to know how Workflow stores the run. It only needs to interpret these events:

  • tool-input-available means a workflow phase started, such as searchWeb, fetchPage, or extractFindings.
  • tool-output-available means that phase finished.
  • tool-output-available with toolName: "extractFindings" contains { findings: Finding[] }.
  • finish means the readable stream ended.

Dedupe streamed findings by toolCallId and finding index. A reconnect or retry can replay chunks that the browser has already seen.

You now have a web research agent ready to answer your questions. The complete template includes a polished frontend with real-time streaming for findings and workflow steps.

Inspect result.toolResults from the Gateway search call. Gateway tools can differ in output shape. Normalize results, sources, excerpts, and URLs before passing data to extraction.

Use a Gateway-executed search tool that is enabled for your team. If parallelSearch is not available, try perplexitySearch, or configure provider access in AI Gateway.

Make sure synthesis receives extracted findings, not raw search text. Add the deterministic citation fallback shown above so the UI always has source URLs when findings exist.

Check that the stream route preserves toolName on tool-output-available chunks and that the UI reads findings from extractFindings outputs.

Move side effects out of the workflow body and into "use step" functions. Native fetch, provider calls, timestamps, random values, and writable stream writes should be inside steps.

FAQ

No. The guide uses AI Gateway model strings and Gateway search tools, so authentication goes through the linked Vercel project and OIDC. You do not need to add OpenAI, Anthropic, or Perplexity keys to the app.

Not for this example. Findings are step outputs in the workflow run, so they are durable for the life of that run. Add a database only if you want long-term storage, cross-run search, or product analytics.

Completed steps are already recorded in the Workflow event log. When the app resumes, Workflow can continue from the last completed step instead of starting the whole research run over.

A pipeline gives the backend a predictable budget and failure model. The model still performs search, extraction, and synthesis, but Workflow owns the order of operations: search once, process up to five sources, extract findings, then write the report.

Search finds candidate sources; extraction turns source text into durable findings. Keeping them separate lets the UI stream findings as they are recorded, and lets retries reuse extracted findings without repeating search.

Was this helpful?

supported.

Read related documentation

Explore more Vercel Workflows guides