for Information Systems
and Digital Innovation
Photo: UHH/Denstorf
12 June 2024
Dear doctoral students and researchers,
we would like to draw your attention to a guest lecture next week by Prof. Christian Wagner from City University Hong Kong. The lecture will discuss the data quality of online crowdsourcing platforms and its impact for empirical research. Prior to the guest lecture postdoctoral fellow Xiaobo Ke will give a short presentation on their study of the impact of experience anticipation for amateur esports participation (Ke, & Wagner, 2024, What explains the next level of gaming? An experience-anticipation model for amateur esports participation. Computers in Human Behavior).
All interested researchers are welcome to attend, we request a short confirmation of interest at isdi-teamassistenz.bwl@uni-hamburg.de.
Date: Wednesday, 12.06.2024
Time: 12.30 – 14.00
Room: VMP5 3017
Guest Lecture:
Prolific Outperforms MTurk on Data Quality—Can Shortfalls Be Mitigated?
Christian Wagner and Xiaobo Ke
City University of Hong Kong
Abstract
Human behaviour studies increasingly engage participants from online crowdsourcing platforms, such as Amazon’s Mechanical Turk (MTurk) or Prolific. The validity of research conducted on these platforms hinges on the quality of participants’ responses. Poor data can lead researchers to falsely reject or accept hypotheses or discard entire models. Data quality suffers when respondents are not paying enough attention, are cognitively incapable of considered responses, or provide willfully wrong answers. Numerous quality assurance (QA) measures have been identified to help researchers obtain better data from participants on the online crowdsourcing platforms. But are they necessary and effective? In this study, we assess threats to the quality of crowdsourced data and the effectiveness of four frequently recommended QA mechanisms, comparing the responses from two online crowdsourcing platforms, MTurk and Prolific. We conclude that respondents on Prolific overall provide significantly better responses than those on MTurk. Without QA interventions, MTurk responses are insufficient in quality to re-validate known models. Surprisingly, only one mechanism effectively identified and eliminated poor responses in our study, namely a response consistency check tied to the use of a reversed questionnaire item. Based upon these findings, we discuss how researchers can improve data quality through proper QA measures, plus corresponding research requirements.
Christian Wagner is Chair Professor of Social Media at the City University of Hong Kong. He started his academic career at the University of Southern California after receiving his PhD in Business Administration from the University of British Columbia in 1989. He has authored or co-authored over 150 publications during the last 30 years, exploring the role of information systems in problem-solving, creativity, and gaming. He is an award-winning author, multiple teaching award winner, experienced administrator, and software entrepreneur. He is most recognized for his research on wikis and their impact on organizational performance.
Google Scholar Link: https://scholar.google.com/citations?user=Q8zA5GgAAAAJ&hl=en
CityU Scholar Link: https://scholars.cityu.edu.hk/en/persons/christian-wagner(4a67e1e8-b34a-49b5-8448-c1cde6171814).html
Xiaobo Ke is currently serving as a Postdoctoral Fellow at the City University of Hong Kong (CityU) and will be joining the faculty at Hong Kong Shue Yan University as an Assistant Professor in the fall of 2024. Additionally, he is an Honorary Research Fellow at the Sau Po Centre on Ageing at The University of Hong Kong (HKU). His academic journey previously included a role as a Postdoctoral Fellow at HKU, after completing his Ph.D. study at CityU. His research interests span esports, social media, gerontechnology, and gamification. His works have been recognized in various international journals and conferences. His esports research can be found in respected outlets such as Communications of the Association for Information Systems, Computers in Human Behavior, Managing Sport and Leisure, and European Conference on Information Systems.
Google Scholar Link: https://scholar.google.com/citations?user=J363HCEAAAAJ&hl=en
CityU Scholar Link: https://scholars.cityu.edu.hk/en/persons/xiaobo-ke(ea175738-e6bb-404e-b4b7-4d50f9e22850).html