Nov 16, 2022|

JD Cloud Ranks No.1 in World’s Top Dataset for Task-Oriented Dialogue Modeling


by Doris Liu

The model proposed by the team of JD Cloud’s intelligent customer service system, Yanxi (言犀), which provides users with 24/7 service in all scenarios, ranked first among models of MultiWOZ, the world’s top dataset for end-to-end task-oriented dialogue modelling.

Mars, an end-to-end task-oriented dialogue system proposed by the Yanxi team, innovatively models the relationship between dialogue context and dialogue/action state through semantic-aware contrastive learning strategies to ensure a better performance of the system. Empirical results show the proposed Mars system maintains state-of-the-art performance on the MultiWOZ 2.0, CamRest676, and CrossWOZ.

Proposed by the University of Cambridge, MultiWOZ, which is short for the Multi-Domain Wizard-of-Oz dataset, is a fully-labeled collection of human-human written conversations spanning over multiple domains and topics. It is one of the most influential task-based dialogue datasets, containing highly natural conversations between tourists and staff obtained from information centers of tourist cities, in which 70 percent of the dialogue is from 2-5 domains in multiple scenarios, and it is becoming one of the most classic and challenging datasets in the field of natural language processing.

Many prestigious universities and research institutions across the world have participated in the competitive challenge since the release of MultiWOZ, including research groups from Tsinghua University, Hong Kong University of Science and Technology, Microsoft Research, Amazon, DeepMind, and Salesforce.

An example of end-to-end task-oriented dialogue generation

As part of the challenge, Yanxi participated in end-to-end task-oriented dialogue generation, which needs to first identify the user’s intention through the user dialogue and generate dialogue states, query the database based on the generated dialogue states for entity matching, and then provide natural language responses that meet the user’s goals according to the dialogue policies based on the query database results.As the industry’s first large-scale commercial intelligent customer service system, Yanxi has accumulated rich practices in task-based dialogue generation through support of the entire chain and lifecycle of’s customer service, with a generation of 10 million dialogues per day on average, serving JD’s over 580 million customers and 178,000 merchants with a smooth experience.