Apr 5, 201208:00 AMPoint of View
The METROPOLIS Blog
Lab Report XXVI
Communitysourcing vending machine, image from bid.berkeley.eduIn this test case the researchers set out to investigate the accuracy of community grade exams. They then compared the results with grading by a single individual and with Amazon’s Mechanical Turk crowdsourcing. The results were clear. Communitysourcing produced a 2% higher grading accuracy than individual grading. As the researchers state, “Mechanical Turk workers had no success grading the same exams.” So much for crowdsourcing work that requires specific expertise. Of course, there are some caveats to this initial study. For one, the length of the study was admittedly short: only one week. Another issue is that the participants, in addition to being self-identified experts in the subject they were grading, may also have been motivated by the novelty of the project. There is no data on the psychological impetus for targeted audiences in public settings and how they might respond to participating in a communitysourcing project. That would depend on the type of project that requires communitysourcing expertise, as well as on the placement of the kiosk, and the type of rewards offered. The potential for harnessing expertise for short-term, anonymous tasks is clear. We’ve seen, for example, that the input of crowds corrects for the biases of a few individuals. Another application to pursue further is making statistical research more accurate. Normally, statistical research at universities is heavily biased towards a young, educated population because the participants are overwhelmingly undergrads from certain cultural, racial, and economic demographics. Students of color, as well as graduate students are less likely to volunteer for university studies, a fact that skews the results and their relevance. Communitysourcing promises to be an important corrective. Sherin Wing writes on social issues as well as topics in architecture, urbanism, and design. She is a frequent contributor to Archinect, Architect Magazine and other publications. She is also co-author of The Real Architect’s Handbook. She received her PhD from UCLA. Follow Sherin on Twitter at@xiaying For Previous Lab Reports follow this link.