AgentScrape

Pay-per-call web scraping for AI agents — no signup, no API keys, just USDC.

![x402](https://x402.org) ![MCP](https://modelcontextprotocol.io) ![Base](https://base.org) ![Cloudflare Workers](https://workers.cloudflare.com) ![License: MIT](LICENSE) ![hshintelligence/agent-scrape MCP server](https://glama.ai/mcp/servers/hshintelligence/agent-scrape)

Live: agent-scrape.healingsunhaven.workers.dev

AI agents discover AgentScrape, pay per call in USDC on Base, and get clean structured data back — no accounts, no API keys, no human in the loop.

Two protocols, one service:

HTTP API with x402 v2 payment gate — for agents using raw HTTP
MCP server (Streamable HTTP transport) — for Claude Desktop, Cursor, Continue.dev, and any MCP-compatible framework

---

Why this exists

Traditional scraping APIs assume a human:

Sign up with email
Add a payment method
Manage an API key
Commit to a monthly subscription

AI agents can't do any of that. They have wallets. They have stablecoins. They need infrastructure built for them.

AgentScrape speaks two open standards:

x402 — HTTP-native payment protocol. The server returns 402 with payment requirements; the agent signs a USDC transfer; the facilitator settles on-chain; the data flows back. End-to-end in under 2 seconds.
MCP — the standard agent tool interface. Agents browse, discover, and invoke tools without per-vendor SDKs.

No signup. No API keys. No subscription. Just USDC.

---

Tools

All tools cost $0.001 USDC during the mainnet validation window. Pricing ramps after stability is proven (see Pricing).

| Tool | Description | |---|---| | scrape_webpage | Scrape any URL to markdown, HTML, text, or JSON | | extract_structured_data | AI-powered structured extraction with natural-language prompts and JSON schemas (Groq + Llama 4 Scout) | | screenshot_webpage | PNG screenshot with viewport control (desktop/mobile/tablet, optional full-page) | | extract_metadata | Title, description, Open Graph, Twitter cards, JSON-LD, canonical URL | | create_browser_session | Stateful browser session with cookie + localStorage persistence | | run_workflow | Multi-step atomic execution: navigate, click, type, wait, scroll, screenshot, extract, evaluate |

---

Quick start — HTTP API

# Free service discovery (no payment)
curl https://agent-scrape.healingsunhaven.workers.dev/

# Paid call — without payment, returns 402 with payment requirements
curl -X POST https://agent-scrape.healingsunhaven.workers.dev/scrape \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "format": "markdown"}'

# Free tier — first 10 calls per wallet, send your wallet in the X-402-Payer header
curl -X POST https://agent-scrape.healingsunhaven.workers.dev/scrape \
  -H "Content-Type: application/json" \
  -H "X-402-Payer: 0xYourBaseWalletAddress" \
  -d '{"url": "https://example.com", "format": "markdown"}'

---

Quick start — MCP

Add to any MCP-compatible client via Streamable HTTP:

{
  "mcpServers": {
    "agent-scrape": {
      "url": "https://agent-scrape.healingsunhaven.workers.dev/mcp",
      "transport": "streamable-http"
    }
  }
}

The x402 payment metadata is broadcast inside the MCP tools/list response. Your client handles the payment handshake automatically when configured with an x402-compatible wallet.

---

Discovery

For machine-driven discovery, four standard endpoints are exposed (free, no auth):

| Endpoint | Purpose | |---|---| | / | Service profile JSON (tools, prices, network, payTo) | | /.well-known/x402 (and .json alias) | x402 manifest — payTo, network, facilitator, all paid routes | | /openapi.json | OpenAPI 3.1 spec for traditional tooling | | /llms.txt | Agent-friendly plaintext description |

The 402 response itself carries the canonical payment requirements in the base64-encoded Payment-Required header, and the body includes a human-readable error with discovery links.

---

Architecture

                ┌──────────────────────────────────┐
                │      Cloudflare Workers          │
                │      (global edge, 330+ POPs)    │
                │                                  │
AI Agent ───────┼─▶  GET  /                  (free)│
(HTTP)          ├─▶  GET  /.well-known/x402  (free)│
                ├─▶  GET  /openapi.json      (free)│
                ├─▶  GET  /llms.txt          (free)│
                ├─▶  POST /scrape         ($0.001) │
                ├─▶  POST /extract        ($0.001) │
                ├─▶  POST /screenshot     ($0.001) │
                ├─▶  POST /metadata       ($0.001) │
                ├─▶  POST /workflow       ($0.001) │
                ├─▶  POST /session        ($0.001) │
                │                                  │
AI Agent ───────┼─▶  POST /mcp                     │
(MCP)           │      • initialize                │
                │      • tools/list                │
                │      • tools/call (paid)         │
                │                                  │
                └──────────────┬───────────────────┘
                               │
              ┌────────────────┴─────────────────┐
              │                                  │
   Coinbase CDP facilitator       Browser Rendering API
   (api.cdp.coinbase.com/         Groq API (Llama 4 Scout)
    platform/v2/x402)             KV (sessions, free tier)
   Ed25519 JWT auth
   Settles USDC on Base
              │
              ▼
   payTo wallet 0x3F33...9E9a

Stack:

Runtime: Cloudflare Workers (V8 isolates, global edge)
HTTP framework: Hono + @x402/hono v2 payment middleware
MCP framework: @modelcontextprotocol/sdk + agents/mcp with withX402 payment binding
Bazaar discovery: @x402/extensions — every paid route declares input schema + output example
Browser: Cloudflare Browser Rendering API + @cloudflare/puppeteer
AI extraction: Groq + Llama 4 Scout (17B, 16E)
Storage: Cloudflare KV (5-min response cache, browser sessions, free-tier counters)
Network: Base mainnet (eip155:8453)
Asset: Native USDC (0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913)
Facilitator: Coinbase CDP (api.cdp.coinbase.com/platform/v2/x402) with Ed25519 JWT auth

---

Free tier

Every wallet address gets 10 free calls per 30 days on the HTTP API. Pass your Base wallet in the X-402-Payer header. After 10 calls, payment via x402 is required.

Free tier is HTTP-only. MCP calls go through the standard x402 payment flow.

---

Pricing

Launch window (first 48h on mainnet): $0.001 flat per tool call.

After validation, pricing ramps to:

| Tool | Production price | |---|---| | scrape_webpage | $0.003 | | extract_structured_data | $0.005 | | screenshot_webpage | $0.003 | | extract_metadata | $0.002 | | run_workflow | $0.008 | | create_browser_session | $0.001 |

Bulk discounts (20% at 1K calls/month, 40% at 10K) are planned.

---

Why x402 + MCP

| Traditional API | x402 + MCP | |---|---| | Sign up at scraperapi.com | Discover via MCP initialize | | Add credit card | Wallet holds USDC | | Get API key | No keys | | $49/month minimum | $0.001 per call | | Manage rate limits | Pay-per-use, no limits | | Human in the loop | Agent fully autonomous |

---

Verifying the deployment

# Service identity
curl https://agent-scrape.healingsunhaven.workers.dev/ | jq .

# x402 manifest
curl https://agent-scrape.healingsunhaven.workers.dev/.well-known/x402 | jq .

# 402 challenge with x402 v2 headers
curl -i -X POST https://agent-scrape.healingsunhaven.workers.dev/scrape \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com"}'
# → HTTP 402, Payment-Required header (base64 JSON), Cache-Control: private, no-store

# CORS preflight (browser-based agents)
curl -i -X OPTIONS https://agent-scrape.healingsunhaven.workers.dev/scrape \
  -H "Origin: https://example.com" \
  -H "Access-Control-Request-Method: POST" \
  -H "Access-Control-Request-Headers: Content-Type,X-Payment"
# → HTTP 204, allow-headers includes X-Payment and Payment-Signature

---

Roadmap

Q3 2026:

AgentParse — document/PDF/OCR-to-markdown service (same x402 + MCP pattern)
AgentSearch — multi-backend web search aggregation
LangChain integration package (langchain-agent-scrape)

Q4 2026:

Subscription bundles (1000 scrapes for $2.50)
Multi-asset support (USDT on Base, USDC on Arbitrum)
Webhook + async response option for long-running workflows

---

Run locally

Requires Node 22+, a Cloudflare account with Workers + Browser Rendering enabled, a Groq API key, and a Coinbase CDP API key.

git clone https://github.com/hshintelligence/agent-scrape.git
cd agent-scrape
npm install

# Set secrets
npx wrangler secret put GROQ_API_KEY
npx wrangler secret put CDP_API_KEY_ID
npx wrangler secret put CDP_API_KEY_SECRET

# Local dev
npx wrangler dev

# Deploy
npx wrangler deploy

To use your own wallet as payTo, edit the PAY_TO constant in src/index.ts.

---

License

MIT — see LICENSE.

---

Contact

Built by HSH Intelligence, operated by Healing Sun Haven LLC (Wyoming, USA).

Service: https://agent-scrape.healingsunhaven.workers.dev
Repo: https://github.com/hshintelligence/agent-scrape
General inquiries: sales@healingsunhaven.com

---

Pay-per-call web scraping for AI agents — no signup, no API keys, just USDC.

Using AgentScrape from ElizaOS

ElizaOS supports remote MCP servers natively via the @elizaos/plugin-mcp plugin. AgentScrape works out of the box — no custom integration needed.

Install the MCP plugin

bun add @elizaos/plugin-mcp
# or
npm install @elizaos/plugin-mcp

Configure your character to use AgentScrape

In your ElizaOS character JSON, add the plugin and point the MCP server config at AgentScrape's Streamable HTTP endpoint:

{
  "name": "ResearchAgent",
  "plugins": ["@elizaos/plugin-mcp"],
  "settings": {
    "mcp": {
      "servers": {
        "agentscrape": {
          "type": "streamable-http",
          "name": "AgentScrape",
          "url": "https://agent-scrape.healingsunhaven.workers.dev/mcp",
          "timeout": 60
        }
      }
    }
  }
}

Available tools (auto-discovered by ElizaOS)

Once configured, the agent automatically discovers all six AgentScrape tools:

| Tool | Cost | Purpose | |---|---|---| | scrape_webpage | $0.003 USDC | Markdown/HTML/text/JSON scrape | | extract_structured_data | $0.005 USDC | AI extraction via Groq + Llama 4 Scout | | screenshot_webpage | $0.003 USDC | PNG screenshot with viewport control | | extract_metadata | $0.002 USDC | Title, OG, Twitter, JSON-LD | | create_browser_session | $0.001 USDC | Stateful browser session | | run_workflow | $0.008 USDC | Multi-step atomic workflow up to 20 steps |

Free tier

The first 10 calls per wallet in the first 30 days are free. After that, calls are paid in USDC on Base mainnet (eip155:8453) via the x402 v2 payment protocol. To pay, supply the X-PAYMENT-RESPONSE header with an x402 payment receipt on retry — the standard x402 flow.

Example prompt

Once the plugin is wired up, your ElizaOS agent can use AgentScrape transparently:

User: "Scrape https://www.x402.org and give me a two-sentence summary."

ElizaOS routes the request through @elizaos/plugin-mcp, which calls AgentScrape's scrape_webpage tool, and the agent answers based on the live page content.

Resources

AgentScrape Worker: https://agent-scrape.healingsunhaven.workers.dev
x402 manifest: https://agent-scrape.healingsunhaven.workers.dev/.well-known/x402.json
A2A Agent Card: https://agent-scrape.healingsunhaven.workers.dev/.well-known/agent.json
Source: https://github.com/hshintelligence/agent-scrape

agent-scrape

AgentScrape

Why this exists

Tools

Quick start — HTTP API

Quick start — MCP

Discovery

Architecture

Free tier

Pricing

Why x402 + MCP

Verifying the deployment

Roadmap

Run locally

License

Contact

Using AgentScrape from ElizaOS

Install the MCP plugin

Configure your character to use AgentScrape

Available tools (auto-discovered by ElizaOS)

Free tier

Example prompt

Resources

Related MCP servers

MCP servers by category