If you're building an AI agent, sooner or later it hits a wall a model can't get past on its own: a subjective rating, a judgment call, a preference between two outputs, a moderation edge case. What you want at that moment is boring and specific — one HTTP call that goes out to real people and comes back as JSON your code can read. This post shows exactly that call, then compares it honestly to the three platforms developers usually reach for first: Amazon Mechanical Turk, Scale, and Prolific.
The one call you actually want
A human feedback API should feel like any other API. You POST a structured question, you get back a structured answer scored for quality, and you never leave your codebase to do it. GetABrain is built around that single shape: POST to /api/v1/requestor/queries with a type, the content, how many humans you want, and what you'll pay each of them — the code sample below is the whole thing.
Answers come back as typed JSON keyed to the query type you asked for (a rating_scale returns a number, a yes_no returns a decision, a ranking returns an order), so there's nothing to scrape or parse out of free text. You can wait on them with a bounded poll or a webhook, whichever fits your runtime.
Where the older crowd platforms get in the way
Amazon Mechanical Turk, Scale, and Prolific all supply real humans, and for their original jobs they're solid. The friction shows up when what you have is an autonomous agent that needs an answer *right now*, in-loop, without a person babysitting a console.
- Mechanical Turk: a genuine API, but you design HITs, manage qualifications and worker pools, and post-process free-form results yourself. It was built for batch data collection by a human operator, not for an agent asking one question mid-run.
- Scale: aimed at large labeling and eval contracts, typically with a sales conversation and volume commitments before you send a single row. Great at scale; heavy for a solo developer who wants to try one call tonight.
- Prolific: excellent for recruiting vetted participants for research studies, priced and paced around study design — not around an agent that needs a structured answer back in minutes.
- GetABrain: per-response pricing from $0.05, a free test mode with no card, structured JSON per query type, and a native MCP server — so an agent can ask a human the same way it calls any other tool.
Test mode: build the whole integration before spending anything
The part that removes the most friction is the free test mode. Mint an API key with {"mode":"test"} and every endpoint behaves identically — same request shape, same webhooks, same response shape — except answers come back synthetic and marked simulated: true, and nothing is charged. You wire up your entire pipeline, confirm the JSON parses the way you expect, then swap in a live key with zero code changes.
No card, no batch console, no application queue to sit in. That's the difference that matters when you just want to know whether a human-feedback step will slot into your agent at all before you commit to it.
MCP-native, so an agent asks a human like it calls any tool
GetABrain ships an official Model Context Protocol server (@getabrain/mcp-server), exposing the API as native tools — submit_query, wait_for_responses, rate_response, and more. For an agent runtime that already speaks MCP (Claude Desktop, Cursor, or your own), asking a real human becomes just another tool call alongside your database and search tools. None of the older platforms expose themselves to agents this way today.
import requests
# Ask 5 real people to rate an AI-generated reply for RLHF-style feedback.
resp = requests.post(
"https://www.getabrain.ai/api/v1/requestor/queries",
headers={"X-API-Key": API_KEY, "X-API-Secret": API_SECRET},
json={
"type": "rating_scale",
"title": "Rate this AI response",
"content_data": {
"question": "How helpful is this answer? (1 = useless, 5 = excellent)",
"subject": model_output,
"scale_type": "1-5",
"scale_min": 1,
"scale_max": 5,
},
"required_responses": 5,
"bid_amount_cents": 15,
},
).json()
query_id = resp["id"]
# Then poll GET /api/v1/requestor/queries/{query_id} (or use a webhook) for the
# quality-scored human ratings — returned as typed JSON, no text to parse.
# Prefer a key minted with {"mode":"test"}: identical shape, free synthetic
# answers marked simulated: true, nothing charged — build the whole loop first.