Authors:
Carlos Toxtli、Christopher Curtis、Saiph Savage
Paper:
https://arxiv.org/abs/2408.07838
Introduction
Crowdworkers play a crucial role in enhancing AI services, yet they often face poor working conditions, especially those from non-US/European backgrounds. This disparity arises from the assumption that crowdworkers are a homogeneous group, leading to standardized interfaces that neglect cultural diversity. This paper proposes creating culturally-aware workplace tools, specifically designed to adapt to monochronic and polychronic work styles. The proposed tool, “CultureFit,” aims to improve the well-being and productivity of crowdworkers by integrating cultural dimensions into its design.
Related Work
Universal Design
Universal design aims to create products that provide equivalent experiences for a wide range of users, often at the expense of cultural nuances. While this approach can make interfaces more globally approachable, it can also generate harm during the adaptation period, particularly for workers from underrepresented cultures.
Chronemics and Different Cultural Dimensions within Time Management
Chronemics, the study of time perceptions, distinguishes between monochronic and polychronic cultures. Monochronic cultures view time as linear and prioritize schedules, while polychronic cultures see time as flexible and prioritize human interactions. These insights inform the design of culturally-aware tools for crowdworkers.
Culture and Crowd Work
Crowdsourcing platforms attract a global workforce, leading to diverse cultural backgrounds among crowdworkers. Research has shown that polychronic workers often face more hardships and are joining crowdsourcing platforms in increasing numbers. This highlights the need for tools that support diverse cultural backgrounds.
The Tooling Add-on Movement
Recent efforts have focused on augmenting crowdsourcing platforms with tools that improve the experience for workers and requesters without requiring platform buy-in. These tools, often in the form of web add-ons, provide additional functionalities and help address power imbalances.
Worker Community System Designs
Data-driven tools have facilitated collaboration and community-building among workers, improving their collective knowledge and support network. These tools have begun to foster mutual support for workplace challenges, and our paper explores how to enhance support for crowdworkers by recognizing their cultural identities.
CultureFit
CultureFit is a tool designed to improve the experiences of crowdworkers by tailoring job notifications to their cultural backgrounds. It functions as a Chrome plugin, operating independently of crowdsourcing platforms like Toloka. Grounded in culture theory, CultureFit accommodates cultural variations in time perception, specifically monochronic and polychronic orientations.
Polychronic Work Notification Interface
CultureFit caters to polychronic workers by opportunistically notifying them about job opportunities across their computer operating system, web browser, or crowdsourcing marketplace. This supports their multitasking preferences and helps build social connections.
Monochronic Notification Interface
For monochronic workers, CultureFit provides task notifications only when they are on the Toloka platform and have completed their current tasks. This minimizes distractions and helps them manage their workdays in a focused and efficient manner.
User Scenarios
User Scenario for Monochronic Crowdworker: Bob
Bob is a dedicated monochronic crowdworker who prefers structured work. Before CultureFit, he spent significant time searching for tasks and was often distracted by non-work-related notifications. After adopting CultureFit, Bob receives curated task notifications aligned with his schedule, improving his work efficiency and satisfaction.
User Scenario for Polychronic Crowdworker: Alejandro
Alejandro is a dynamic polychronic crowdworker who enjoys multitasking. Before CultureFit, he found traditional crowdsourcing platforms restrictive. After adopting CultureFit, Alejandro receives notifications that support his multitasking nature, allowing him to transition seamlessly between tasks and stay connected to social engagements.
Methods
Our IRB-approved field experiment focused on comparing the experiences of crowdworkers who utilized CultureFit and those who did not. We implemented a between-subjects design with control and intervention groups, further divided into sub-groups based on cultural orientations.
Pre-Test Stage
The pre-test stage involved a 7-day period where all workers completed a pre-survey, installed a web-plugin with telemetry tracking, and did crowd work as usual. This established a baseline understanding of their wages, perceptions, and digital behaviors.
Test Stage
During the 7-day test stage, workers in the intervention group gained access to CultureFit, while control groups continued their usual crowd work activities. This helped identify market fluctuations that could influence the results observed with CultureFit.
Post-Test Stage
At the experiment’s end, workers completed a post-survey about their experiences during the test stage. This provided insights into the impact of CultureFit on their work practices and perceptions.
Data Analysis of Workers’ Survey Data
We conducted both quantitative and qualitative analyses of participant survey responses, identifying common themes and calculating median values for Likert scale questions.
Participants
We recruited 55 participants from a diverse pool across 24 countries, categorizing them into four conditions: Monochronic/CultureFit, Polychronic/CultureFit, Monochronic/Control, and Polychronic/Control.
Results
Overview
Our study included 55 workers, with a median age of 30. We collected over two million telemetry logs, tracking workers’ digital behaviors, wages, and task data.
Pre-Test Stage: Survey Results
Significant cultural differences emerged in workers’ preferences for planning and supervision, consistent with known cultural preferences of monochronic and polychronic individuals.
Pre-Test Stage: Telemetry Logs Results
Monochronic workers finished fewer tasks but earned slightly higher hourly wages than polychronic workers. However, no significant differences were found in the number of tasks completed or wages earned between the two groups.
Test Stage: Telemetry Logs Results
Polychronic workers using CultureFit saw a significant increase in their wages, while monochronic workers did not experience significant changes. This suggests that CultureFit effectively aligned with polychronic workers’ multitasking abilities and preferences.
Post-Test Stage: Post-Survey Quantitative Results
CultureFit users reported significant changes in their experiences and schedules during the test stage. Polychronic workers in the CultureFit condition had a significant shift in how much they felt they utilized their strengths.
Post-Test Stage: Post-Survey Qualitative Results
Workers using CultureFit reported positive changes in their work behavior and appreciated the tool’s ability to help them manage their work schedule more effectively.
Analyzing the Crowdworker Dropouts
We examined the characteristics of workers who dropped out of the study and found no significant associations between cultural traits, geographic regions, or gender and dropout rates.
Discussion
Understanding the Integration of Cultural Insights in Crowd Work Tool Design
Our results suggest that culturally aware tools can have a greater impact on populations traditionally overlooked in the design process. Future research should focus on designing tools that cater specifically to the culture and needs of underrepresented groups.
Understanding the Feasibility and Challenges of Culturally-Aware Tools in Crowd Work
Our approach of designing a tool that co-exists within existing crowdsourcing platforms makes it more feasible to create culturally-aware crowd work tools. However, challenges remain, such as potential dissatisfaction from requesters with different cultural backgrounds.
Designing the Future of Culturally-Aware Tools for Crowd Work
Future crowd work tools should integrate social features, support multiple goals, and facilitate cultural understanding. This can enhance worker engagement and broaden global perspectives.
Conclusion
Our research highlights the importance of incorporating cultural insights into digital labor tool design. The proposed tool, CultureFit, significantly enhanced earnings for culturally diverse workers and provides a novel dataset for future research. Our findings underline the significance of culturally-aware tools in improving the well-being and productivity of crowdworkers.
Illustrations:
- !
- !
- !
- !
- !
- !
- !
- !
- !
- !
- !
- !
- !
- !
