Getting the right workers is critically important for most managed crowdsourcing projects. Success depends on a pool of workers who understand and can do the tasks involved. So, while it’s certainly possible to open a project up to the entire crowd and then start blocking poor performers along the way, gradually thinning the worker pool down to the best of the best, it’s important to recognize that qualifying crowd workers for a campaign is far preferable to blocking them.
Putting a “block” on a worker has negative effects on his or her account and ability to work, so blocking a worker can be seen as an aggressive act, comparable to the firing of an employee. There’s bad feeling on both sides, and the worker will have trouble picking up a new job, even one they’ve already shown they’re suited for. This can also lead to bad social chatter about your firm from the crowdworker community.
Conversely, qualifying (and disqualifying) workers is a more equitable way of putting together an excellent worker pool that avoids negatively affecting workers. By qualifying workers—making sure that they can do tasks with known answers before allowing them to work on live records—it’s possible to pinpoint those people who are not well-suited to a certain task as well as those who are exceptionally adept.
It’s also important to communicate with workers who seem to have trouble hitting gold standards or qualifications. For example, when a worker with a very high gold accuracy rating suddenly hits zero on a specific campaign run, something is clearly wrong. A word from the project manager often helps the crowd worker become aware of a simple copy and paste error and allows him or her to correct the mistake. A manager could, of course, simply look at the numbers and boot the offender right out of the pool. In many cases, though, little errors are made by capable, useful workers who can benefit the project and deserve a second chance.
Qualifying crowd workers, means, essentially, treating them as you would an in-house employee: making sure that they’re able to do the work and monitoring them to ensure they continue to meet the standard you’ve set. This leads to higher overall quality and the likelihood that these highly qualified resources will participate in future projects.