000118149 001__ 118149
000118149 005__ 20230519145538.0
000118149 0247_ $$2doi$$a10.1371/journal.pone.0252604
000118149 0248_ $$2sideral$$a126898
000118149 037__ $$aART-2021-126898
000118149 041__ $$aeng
000118149 100__ $$aChristoforou E.
000118149 245__ $$aAn experimental characterization of workers'' behavior and accuracy in crowdsourced tasks
000118149 260__ $$c2021
000118149 5060_ $$aAccess copy available to the general public$$fUnrestricted
000118149 5203_ $$aCrowdsourcing systems are evolving into a powerful tool of choice to deal with repetitive or lengthy human-based tasks. Prominent among those is Amazon Mechanical Turk, in which Human Intelligence Tasks, are posted by requesters, and afterwards selected and executed by subscribed (human) workers in the platform. Many times these HITs serve for research purposes. In this context, a very important question is how reliable the results obtained through these platforms are, in view of the limited control a requester has on the workers'' actions. Various control techniques are currently proposed but they are not free from shortcomings, and their use must be accompanied by a deeper understanding of the workers'' behavior. In this work, we attempt to interpret the workers'' behavior and reliability level in the absence of control techniques. To do so, we perform a series of experiments with 600 distinct MTurk workers, specifically designed to elicit the worker''s level of dedication to a task, according to the task''s nature and difficulty. We show that the time required by a worker to carry out a task correlates with its difficulty, and also with the quality of the outcome. We find that there are different types of workers. While some of them are willing to invest a significant amount of time to arrive at the correct answer, at the same time we observe a significant fraction of workers that reply with a wrong answer. For the latter, the difficulty of the task and the very short time they took to reply suggest that they, intentionally, did not even attempt to solve the task. © 2021 Christoforou et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
000118149 540__ $$9info:eu-repo/semantics/openAccess$$aby$$uhttp://creativecommons.org/licenses/by/3.0/es/
000118149 592__ $$a0.852$$b2021
000118149 590__ $$a3.752$$b2021
000118149 593__ $$aMultidisciplinary$$c2021$$dQ1
000118149 591__ $$aMULTIDISCIPLINARY SCIENCES$$b29 / 74 = 0.392$$c2021$$dQ2$$eT2
000118149 655_4 $$ainfo:eu-repo/semantics/article$$vinfo:eu-repo/semantics/publishedVersion
000118149 700__ $$aFernández Anta A.
000118149 700__ $$0(orcid)0000-0003-1874-2881$$aSánchez A.
000118149 773__ $$g16, 6 (2021), [14 pp]$$pPLoS One$$tPLoS ONE$$x1932-6203
000118149 8564_ $$s905645$$uhttps://zaguan.unizar.es/record/118149/files/texto_completo.pdf$$yVersión publicada
000118149 8564_ $$s2455846$$uhttps://zaguan.unizar.es/record/118149/files/texto_completo.jpg?subformat=icon$$xicon$$yVersión publicada
000118149 909CO $$ooai:zaguan.unizar.es:118149$$particulos$$pdriver
000118149 951__ $$a2023-05-18-15:37:53
000118149 980__ $$aARTICLE