If you’re here, you will have seen dozens of posts claiming that Outlier has paid the poster thousands of dollars. You would have also seen posters claiming that it is a scam and a dumpster fire that is on the verge of collapse.
Naturally, before you give them your time, you want to know whether Outlier is legit or whether Outlier is a scam.
Unlike other reviews, I’ve spent a bit of time on the platform, and can tell you from first hand experience, and not just from a quick glance followed by an AI prompt.
I’m currently on Project Blackbeard, and have previously tasked on Project Melvin’s Mansion. I’ve had invites to the now reduced Project Aether/Project Multimango. I onboarded for Project Blueberry Bagels, but opted not to task. I’ve also spent a lot of time staring at an empty queue, wondering whether I messed up my application.
I hope this review will give you an idea of what to expect.
What is Outlier?
Outlier is an AI training platform that connects contractors with AI companies needing human feedback on language model outputs.
As a contractor, you carry out tasks on their online platform. The data from the tasks is used to train AI in new techniques or on topics and skills that it needs to improve.
You are not a direct employee of Outlier. You are also not an employee or contractor of their clients who want AI training data.
Tasks are fairly standard for AI platforms and generally involve ranking answers to a prompt that you were provided or you wrote. The exact methods change from one project to the next, but the general concept is the same. You will also write rubrics (like a marking sheet to assess whether an AI answer is correct).
How Does Outlier Onboarding Work?
Outlier has a two-stage onboarding process, with no AI interview (like those Mercor, Micro1, or Alignerr).
First you apply for the platform, providing your CV and listing your experience, language ability, specialist, and your coding level.
Second, you will wait to be matched with projects that may be suitable to you. Upon matching, you will be given the project details, onboarding guides, and pay terms. You can proceed to reading the onboarding documents and taking assessments after accepting the pay terms.
Your eligibility for the project is then assessed based on your performance in the onboarding test. Typically this is a mixture of multichoice and open-ended answers based on the onboarding material.
This second stage can take from 30 minutes to a few hours.
My experience so far has been:
Project Melvin’s Mansion (generalist): Onboarding took around one to two hours and was unpaid. The tasks required a lot of reading and careful attention to the rubric. It’s not a quick skim — if you rush it, you’ll struggle with the actual work too, or, worse you’ll find yourself completing hours of onboarding and not being allowed to task.
Project Blackbeard (public service specialist): Onboarding took around 40 minutes and was considerably more straightforward.
Project Blueberry Bagels: I completed the onboarding in about an hour. At the time, there were issues with assessment tasks asking questions based on information which you could only see in subsequent chapters. I passed the assessments, but the task wasn’t immediately clear to me, so I stepped back. This happens — not every Outlier project will suit every contractor, and it’s better to acknowledge that than to push through and underperform.
As a general rule, there are no universal rules for onboarding. Your experience will vary significantly based on your skills, the project managers, and the onboarding material. Please be aware that projects change a lot, so my experience with onboarding to Melvin’s Mansion or Blueberry Bagels may not longer apply.
Project Blackbeard — Public Service Specialist Work
This is the highest-paying project I’ve worked on at Outlier.
What the work involves: Side-by-side comparison tasks for public service-related queries. You’re evaluating AI responses using your domain knowledge, comparing outputs, and identifying which response is more accurate or better suited for a professional context.
Pay rate: $75 per hour during the active tasking time, dropping to $21 per hour once the task time limit is exceeded.
Task length: Tasks started with an estimated tasking time of 1.5 hours, and a maximum task cap of a bit longer than 2 hours. The task structure was soon revised, and the tasking time dropped to 35 minutes.
Consistency: This is where Blackbeard falls short. The pay is excellent when tasks are available, but they come in waves. There will be stretches with nothing in the queue, and you have no way to predict when work will appear. If you’re treating this as a primary income source, the inconsistency will frustrate you. If you’re treating it as high-value supplementary work, it’s very good when it’s on.
In the two weeks that I have been on the project, it has been updated and rescoped at least three times. Before each major change, there have been Empty Queues.
Verdict on Blackbeard: Best project I’ve encountered on Outlier. Easy to onboard, well-paid, and the work is relatively straightforward if you have the relevant background. The inconsistency is the only real drawback.
Project Melvin’s Mansion — Generalist Rubric and Golden Answer Work
This is one of the standard generalist tracks and a commonly discussed Outlier project.
What the work involves: You provide a prompt to an AI system, along with a detailed description of the role that the system takes (e.g. you are a SEO expert, etc.). You are given a set of parameters that the system wants you to test, which are incorporated into the prompt and description.
You are shown two responses. If you are satisfied with the responses and their ability to match your input and the system requirements, you can start the real work.
First is creating a rubric, which is essentially a detailed marking sheet that leaves no room for ambiguity. The you mark the AI responses against the rubric. You then provide a golden answer, or the most ideal response to the prompt.
The difficulty here is the intermediary checks that are automatically carried out by the system, called linters. These linters apply for most Outlier projects, but I found them to be particularly problematic here. Linters are automated checks, which will read your input and provide feedback on everything from grammar to content. Some are dismissible, but some require attention before proceeding.
Pay rate: $25 per hour during active tasking time, dropping to $18 per hour after that.
Difficulty: This project was genuinely hard. The rubric is detailed and the tasks require sustained concentration. The gap between the effort required and the pay rate is noticeable — this is not easy money. If you go in expecting to breeze through it, you’ll either produce poor work or find it exhausting.
Verdict on Melvin’s Mansion: Worth doing if you’re a strong writer and enjoy detailed analytical work, but go in with realistic expectations. The $25/hr rate is fair but the linter overhead and UX friction (see below) mean your effective rate will often be lower than the headline figure.
Projects I Was Invited to But Didn’t Take
Project Aether: I received an invite but didn’t participate. Aether was a large-scale generalist project that ran for an extended period — it had significant headcount but has since wound down. If you see references to it in older reviews, that context is relevant.
The Queue Problem — What Nobody Tells You
This is the single most important thing to understand about Outlier, and most reviews skip it entirely.
You have very little control over what appears in your queue. Projects open and close based on client demand, and your access to any given project depends on your qualifications, your onboarding performance, and factors you can’t see. Even if you’ve onboarded for a project, work may not appear for days or weeks.
If you ever look at the Outlier Community pages, you will see endless questions from people about why they were moved from one project to another or asking to be put back on a different project.
For me, I signed up to Outlier a long time ago and didn’t take it seriously. I saw Mission Invites, Project notices, and Webinar invites, and ignored them all. By the time I came back, I was on Empty Queue with no idea how to get a gig.
In practice, unless you are lucky or have in-demand skills, Outlier will not be a constant source of income for you. The pay rates can be good, and there is something to be said for not applying for new projects. However, relying exclusively on Outlier is liable to leave you with periods of no activity.
You can alleviate the impact of these dry patches by treating Outlier as one element in your platform stack and by maintaining a presence on the site, even if it’s doing onboardings and an occasional task.
The Linter Problem — Outlier’s Biggest Frustration
This deserves its own section because it’s the thing that will affect your earnings, and almost no review mentions it.
Outlier uses automated linters to assess your task submissions before they’re accepted. In theory, this maintains quality. In practice, it creates a significant and genuinely frustrating workflow problem.
The linters are pedantic. They will flag minor grammar and spelling issues that don’t affect the quality of your answer. More problematically, they produce contradictory feedback. A common situation for me was a linter identifying a problem and providing a potential fix with suggested new wording. Implementing that wording exactly would lead to a new linter issue with the exact opposite warning and suggested wording identical to my original. Sometimes the linter would contradict project guidelines. In most cases, the linters are dismissible and allow you to provide your justifications. However, they use up time, bringing you closer to the task limit and the reduced pay rate that applies when you exceed active tasking time limits.
This is compounded by the platform’s web-based UX, which requires a lot of scrolling to navigate tasks. Combined with linter back-and-forth, you can spend a disproportionate amount of time on a single task that should take half as long. The interface also has stability issues — crashes are reported, and they happen at the worst possible moment.
The practical effect on Project Melvin’s Mansion is significant. The work is already cognitively demanding. Add linter friction and UX overhead on top, and the effective hourly rate drops below what the headline figure suggests. On Project Blackbeard the tasks are more contained, which limits the damage — but the linter issue is present across the platform.
This isn’t a reason to avoid Outlier. The pay at the specialist level is still among the best available. But it is a reason to go in with realistic expectations about how much of your time will be productive versus spent navigating the system.
Who is Outlier Best Suited For?
Outlier works best for people who have specialist domain knowledge that qualifies them for higher-paying projects. If you have a background in law, medicine, engineering, finance, or public service, you’re likely to access better-paid, more consistent work than the generalist track offers.
For generalists, Outlier is a legitimate option but not an exceptional one. The work is harder than comparable platforms, and the pay reflects that only partially. It’s worth having in your stack, but probably not worth prioritising over platforms with more consistent availability. Having said that, there is no harm in making an application, and seeing if anything surprising pops up over time.
For new entrants into AI training, Outlier is definitely a platform you should join. If you are lucky enough to land a good entry gig, you will learn a lot. The experience could lead to more Outlier gigs, and it will look good when listed prominently on a machine readable/AI optimised CV.
Does Outlier pay?
Outlier is the platform which has never had any issues paying the right amount on time. Every other platform I have worked on has had delays, missed payments, or wrong amounts, except for Outlier.
They pay weekly in USD.
The payments options are PayPal or AirTM (AirTM referral for a small bonus payment for qualifying sign ups)
Verdict
Outlier is legitimate, mostly well-run, and offers some decent headline pay rates ( particularly at the specialist level). But the gap between headline rate and effective rate is real. The automated linter system creates friction that costs time, the UX adds overhead, and the queue inconsistency means you can’t rely on it as a primary income source.
If you have specialist expertise, Outlier is worth prioritising and the frustrations are manageable. If you’re coming in as a generalist, be honest with yourself about whether the effort-to-pay ratio works for you — and make sure you have other platforms running alongside
Applying
Outlier only opens referrals to existing taskers with a track record.
You can apply through my Outlier referral here. Outlier only allows for a limited number of referrals per month.
If you miss out on a referral or want you want to apply directly, you can see the latest AI training jobs on our main site. The listings are collected daily from AI training and data annotation platforms across the web.