Efficient Crowd-sourcing Techniques for Large Tasks - Essay Example

Most studies done on crowdsourcing have focused more on the quality of results and throughput with very or no attention to efficiency or the tradeoff therein. This is because; the number of tasks to be performed is usually very small. Additionally, the tasks performed by users are also small, and as a result, the techniques employed to guarantee result quality and scale the tasks that arise do not tackle the aspect of efficiency. This paper aims at evaluating existing result-improvement mechanisms and proposes a novel technique that is independent of task complexities and size but ensures efficiency and a tradeoff between the quality of results and efficiency.
Keywords: Efficiency; Throughput; Result Quality; Crowdsourcing; Data

INTRODUCTION
Crowdsourcing is very popular and has become very important for data collection-labeled data- that are dependent on human input to ensure the quality of data as demonstrated by reCAPTCHA, Amazon Mechanical Turk, ESP game, and Netflix among others. Crowdsourcing is the distribution of small chunks of large, complicated tasks to a large community of users over the internet who offers small contributions towards solving or offering a solution (Brabham 2008). In addition to data labeling, crowdsourcing has been utilized in double keying individual words, image labeling, word disambiguation, and grading of word pairs that depict relatedness (Sautter & Böhm 2011).

Several challenges arise from crowdsourcing including a lack of guarantees for the correctness of human inputs. This can be a result of accidental mistakes resulting from misjudgment or sloppiness. Sometimes, users may cheat to increase their chances of receiving a reward by cheating to reduce their effort. These challenges require the development of crowdsourcing techniques that can ensure data quality, and efficiency, counter errors, work with very large data sets, and yield high throughput. Previous studies and crowdsourcing mechanisms such as r-Redundancy have focused more on working with large tasks, countering errors, yielding high throughput, and ensuring data quality (Eckert et al. 2010). This study, however, proposes novel techniques that are independent of task complexities and size but ensure efficiency and enable tradeoffs between the quality of results and efficiency.

THE NEED FOR EFFICIENT CROWDSOURCING TECHNIQUES FOR COMPLEX TASKS
Crowdsourcing mechanisms such as r-Redundancy, have focused more on working with large tasks, countering errors, yielding high throughput, and ensuring data quality. They have assumed that most users offer meaningful and correct inputs. These mechanisms have ensured data quality, even in the presence of dishonesty, cheating, and inability but do not discourage such a behavior (Wauthier & Jordan 2011). Therefore, a mechanism that probes users is required, one that would not only ensure quality and high throughput but ensure efficiency, as well.

OVERVIEW OF EFFICIENT CROWDSOURCING TECHNIQUES FOR COMPLEX TASKS
There are crowdsourcing techniques that combine error countering, data quality enforcement, and large tasks handling. v-Voting is one such mechanism that mostly focuses on countering errors (Alonso et al. 2008). It reduces the required number of inputs, unlike r-Redundancy by obtaining inputs from different users and combining them to the final result. There is also Vote Boosting, which is a mechanism that builds on v-Voting to increase throughput (Sautter & Böhm 2011; Brabham 2008).

TOWARDS EFFICIENT CROWDSOURCING TECHNIQUES FOR COMPLEX TASKS

The proposed techniques would be an improvement on Vote Boosting that would, in addition to increasing throughput, it would be independent of the nature of tasks, counter errors, cheating and dishonesty while ensuring efficiency. This will thus enable a tradeoff between result quality, throughput, and efficiency.
CONCLUSION

Crowdsourcing is very popular and has become very important for data collection-labeled data- that are dependent on human input to ensure the quality of data. However, new challenges have resulted from working with large numbers of users. Particularly, dishonesty, possible misjudgment, and potential inability have become threats to results quality. This study has examined various mechanisms used to ensure data quality and high throughput and their relationship to task efficiency. The study has observed that a technique such as v-Voting is better than early redundancy approaches since it reduces throughput by utilizing capable users. Additionally, appears from the study that results of higher quality are obtained through v-Voting as opposed to r-Redundancy. On the other hand, Vote Boosting offers a tradeoff between result quality, throughput, and efficiency in a desirable way. Read More