Improving search relevance for a leading global ecommerce provider

Situation

Online shoppers expect a convenient, easy-to-use experience that makes it easy to find the product they are looking for. For an ecommerce company to ensure the relevancy of the results generated by its search engine, it needs to constantly compare customer search queries and query results to identify any irrelevant results. In this particular case, a leading global ecommerce company wanted to process more than 100,000 search relevance tasks across English, German and Japanese to improve search results for users of these languages. Each data batch contained a variety of tasks which required cleaning and organizing the data before processing. 

Complicating matters, the client didn’t have any clear guidelines about how to process the data, its ‘gold’ data set was out of date, and there was no reporting tool or process for evaluating work before submitting it. Completing the project would therefore require that LXT apply a greater degree of oversight with its data processors, and make extensive enhancements to the client’s original dataset. 

Solution

To ensure successful completion of the project, LXT developed an end-to-end managed service program, which included recruiting and onboarding contributors, providing continual support, creating project guidelines, conducting quality reviews, and delivering the data. 

LXT recruited contributors across multiple time zones, and project management representatives were available 24/7 to address any questions or concerns. LXT also enhanced the client’s guidelines to provide contributors with clearer guidance and minimize any obstacles to productivity. 

Furthermore, LXT developed a variety of materials to engage and equip contributors: automated reports informed them about the quality of their work and their progress toward completing the project; dashboards enabled them to follow-up on their progress and quality scores; and continuous announcements were distributed with any client updates or FAQs. 

Finally, as each data batch was completed, LXT conducted a review to ensure that the quality of the work met the client’s standards. 

Results

Despite the very tight timeline, the LXT team delivered ahead of schedule and also exceeded the client’s 90 percent quality score across all three languages. LXT proved itself early on which led the client to extend the project scope and include more complex tasks. The LXT team continued to deliver high-quality data throughout the course of the project even with the increased task complexity.

LXT’s high-quality data equipped its client to make additional enhancements to its search algorithms with the goal of creating a better customer experience, increasing the performance of its search experience, and improving adoption by online shoppers. 

Another output of the project was a new ‘gold’ dataset that LXT developed for its client across all three languages, providing an updated reference point against which to measure any future search relevance projects. The gold dataset was used to ensure calibration between the client and LXT while providing a way to measure the thorough understanding and application of guidelines, as well as the accuracy of labeling. Localizing this gold dataset ensured no instructions were lost in translation and boosted the confidence of ratings in all three languages.