Case study
4000
contractors recruited and
onboarded
50
languages represented
1
week turnaround
“Having LXT here is definitely important for us to be able to adhere to our timelines.”
– Vendor Manager, Top Ten Global Technology Company
Situation
A top ten global technology company wanted to fine tune the foundational Large Language Models that power its Generative AI solution in 50 languages to reach a wide range of global users. Its goal was to ensure that the solution’s output was evaluated by a diverse pool of human contributors for accuracy, clarity and bias.
Given the nature of the competitive landscape for Generative AI, this was a very time-sensitive project. A large crowd of contributors needed to be onboarded very quickly to perform the rating and evaluation tasks, and the company was facing very tight internal deadlines. It needed a reliable partner that could deliver high-quality data with speed and at scale.
Solution
The company chose LXT based on its prior history of delivering similar projects on or ahead of schedule – with high quality – and its ability to quickly adjust to client needs. Within one week, the LXT team had onboarded contributors in all 50 languages. To achieve this, LXT reached out to its global network of contributors and used enhanced language assessments for testing and screening to ensure that contributors had the required level of language proficiency in each target language.
Contributors then performed the following tasks:
- Comparison Data Collection: contributors ranked multiple model responses by quality. For example, for a given user prompt, they might see a few potential completions and rank them from best to worst.
- Rating: Contributors rate model responses for various inputs, providing a clearer picture of performance and potential biases.
- Categorization: To understand the kinds of mistakes the model makes, some outputs were categorized based on their nature, e.g., “unsafe” or “untruthful.”
- Rewriting: Contributors might rewrite a model’s response to be more accurate, clearer, or safer. This feedback could be incorporated to further fine-tune the model.
LXT designed a program to maximize contributor output and ensure that volume targets were completed ahead of schedule as data batches were made available. The Operations team distributed the work across multiple project managers to maximize throughput and collaborated with key internal stakeholders to maximize the program’s quality, improve automated workflows and build customized reporting.
Results
LXT was successful in qualifying and onboarding 4000 contributors across all 50 languages in one week. Resources were closely managed to uncover cost savings for the client; for example, by optimizing CPU resources the LXT was able to achieve a consistent 5% reduction in project costs. Robust reporting and dashboards ensured that the client had full visibility into project progress and timelines so they could effectively manage internal expectations. LXT’s speed in onboarding a large volume of contributors was critical to the client achieving its internal timelines. LXT’s thorough approach and consistent results in the initial phase of the program resulted in a continued partnership with increasing volumes of data across all languages.
