Notes on human-in-the-loop APIs, AI agent workflows, and honest comparisons to the rater and crowdwork platforms — written from what's actually true of the live product.
LLM-as-a-judge is fast and cheap, but it inherits the same blind spots as the model it's grading. Here's where it holds up, where it doesn't, and how to drop real human judgment into the same pipeline.
Read the postPairwise preference, rating scale, or free-form critique — collect the human judgment RLHF pipelines need with one API call instead of standing up your own labeling ops.
Read the postYour agent can call a search API, a weather API, a payments API. Here's the one for when it needs an actual person's judgment.
Read the postYou want one HTTP call that returns a real human answer in a shape your agent can parse. Here is how that call looks — and where the older crowd platforms get in the way.
Read the postMost MCP servers wrap an API. GetABrain wraps a crowd of real people — so an agent can ask a human a question and get a quality-scored answer back, all through the Model Context Protocol.
Read the postAn honest look at the rater-platform waitlist problem — and what changes when signup and pay are instant.
Read the postLangGraph's interrupt() pauses your graph and waits for input. Here's a pattern for making GetABrain the human on the other end of that wait.
Read the post