โ† Blog
4 min readdevelopershuman-in-the-loopmcp

A human-in-the-loop MCP server: give your agent real human feedback as a tool call

Most MCP servers wrap an API. GetABrain wraps a crowd of real people โ€” so an agent can ask a human a question and get a quality-scored answer back, all through the Model Context Protocol.

The Model Context Protocol gave agents a clean way to reach tools โ€” but almost every MCP server on the registry wraps software: a database, a search index, a SaaS API. GetABrain is an MCP server that wraps people. When your agent calls the submit_query tool, a real human answers, the answer comes back quality-scored, and the agent keeps going. This is the human-in-the-loop step as a native MCP tool call, not a Slack channel someone has to babysit.

Why a human-in-the-loop MCP server at all

Agents built on MCP hit the same wall every agent hits: some steps a model genuinely should not decide alone. Is this generated marketing claim actually true? Which of these two rewrites reads better to a person? Is this user-uploaded image safe to publish? Should the agent take this irreversible action? These are judgment calls, and the honest answer is to route them to a human.

The problem has always been plumbing. You can pause an agent easily enough; finding the human to unpause it is the hard part. A human-in-the-loop MCP server closes that gap inside the protocol the agent already speaks โ€” the model asks for a human answer the exact same way it asks for a database row.

The tools the server exposes

GetABrain's MCP server exposes a small, purpose-built toolset. The two that matter for a HITL step:

  • submit_query โ€” post a structured question (yes/no, A/B, rating scale, sentiment, ranking, free text, image/video review, and more) to real workers. Returns a query_id immediately; it does not block.
  • wait_for_responses โ€” poll for answers, blocking up to 50 seconds per call, returning status "ready" with the responses once enough humans have answered (or "pending", so the agent calls again).
  • get_responses โ€” a one-shot, non-blocking read of whatever answers exist right now.
  • get_balance / create_topup_link โ€” check spend and top up, so a live agent can manage its own budget.
  • rate_response โ€” score an answer you got back, feeding the same quality system workers are held to.

Test mode: build the whole loop for free

The single most useful thing here: point the server at a test-mode API key and every tool behaves identically, except no balance is touched and responses come back synthetic and marked simulated: true. That means an agent developer can wire up submit_query, handle the wait_for_responses loop, and parse the answer shape end-to-end without spending a cent or waiting on real people. Flip the key to live and the exact same code dispatches to paid humans.

No card is required to mint a test key. This is the zero-friction trial: a model reading this page can go from 'I need a human here' to a working, tested tool call in one sitting.

Connecting the server

Two ways in, both standard MCP. Run it locally over stdio with npx, or point any MCP client that supports remote servers at the hosted Streamable HTTP endpoint โ€” no local process at all. Both authenticate with the same GETABRAIN_API_KEY / GETABRAIN_API_SECRET pair you get at signup.

What it costs when you go live

Pay-per-response, no subscription. The floor is effort-based, not flat: quick types like yes/no or rating scale bottom out around $0.05 per response, medium types around $0.08โ€“$0.12, and longer or media-review types scale up with how long the task honestly takes a worker at a fair ~$9/hr. A 15% platform fee is included. Total for a query is (bid + optional bonus) ร— the number of responses you asked for, deducted at submit time in live mode โ€” and nothing at all in test mode.

MCP client config + the HITL tool sequence an agent runs
// 1. Register the server (local via npx, or remote over Streamable HTTP).
{
  "mcpServers": {
    "getabrain": {
      "command": "npx",
      "args": ["-y", "@getabrain/mcp-server"],
      "env": {
        "GETABRAIN_API_KEY": "gab_k_test_...",
        "GETABRAIN_API_SECRET": "gab_s_test_..."
      }
    }
    // Remote alternative โ€” no local process:
    // "getabrain": { "url": "https://www.getabrain.ai/api/mcp" }
  }
}

// 2. What the agent's tool calls look like at the HITL step:
//    submit_query -> get a query_id back immediately
{
  "tool": "submit_query",
  "arguments": {
    "type": "yes_no",
    "title": "Is this claim factually accurate?",
    "content_data": { "question": "Does our free tier really include 1,000 API calls?" },
    "required_responses": 3,
    "bid_amount_cents": 25
  }
}
//    -> { "query_id": "q_...", "status": "open", "simulated": true }  (test key)

//    wait_for_responses -> block up to 50s, get real (or synthetic) answers
{ "tool": "wait_for_responses", "arguments": { "query_id": "q_..." } }
//    -> { "status": "ready", "responses": [ { "answer": "no", ... }, ... ] }

FAQ

A normal MCP server exposes software โ€” a database, an API, a file system. GetABrain's exposes real, paid human workers: submit_query dispatches your question to people, and the answer comes back quality-scored. It's the human-in-the-loop step delivered through the same protocol your agent already uses for everything else.
Yes. Mint a test-mode key (no card required) and every tool behaves identically except responses are synthetic and marked simulated: true, with no balance charged. You can build and test the entire submit_query / wait_for_responses loop for free, then flip to a live key with no code changes.
No. You can run it locally over stdio with `npx -y @getabrain/mcp-server`, or point any MCP client that supports remote servers at the hosted Streamable HTTP endpoint at https://www.getabrain.ai/api/mcp. Same tools, same auth, no local process required for the remote option.
Structured types built for machine parsing: yes/no, multiple choice, A/B comparison, rating scale, sentiment, ranking, headline test, free-form and structured text, image comparison/selection/analysis, plus voice, photo, and video capture and review. Each returns a typed, consistent answer shape rather than free text you have to post-process.
Every worker carries a 5-star quality score and a tier (Bronze through Platinum); workers who fall below a 1.5 rating are automatically suspended. You can set min_worker_quality on a query to restrict it to higher-rated workers, and use rate_response to score what you get back, which feeds the same system.

Add a human to your agent as one tool call.

Mint a test-mode key โ€” no card required โ€” connect the GetABrain MCP server, and run your first submit_query with free synthetic responses. Flip to live when you're ready and the same code pays real humans.