by Kelly Dawson
The rapid rise of voice and AI technology in recent years has only been accelerated by COVID-19, increasing the appeal of contactless communication that contributes to making consumers feel safe in physical environments, and providing additional means of communicating with customer service during a time when movement is limited.
While most consumers are more familiar with JD.com’s e-commerce business and advanced logistics network, behind the scenes JD is making enormous strides in voice-recognition technology and AI, earning international recognition for cutting-edge research in the field, and already impacting the lives of many, including millions of visually-impaired shoppers who are increasingly going online with the help of JD’s intelligent customer service.
In fact, voice will likely become people’s primary means of interacting with the digital world and the way consumers will interact with brands in the retail space in the future, according to a new report released this month by WPP, “Do you hear what I hear? Why brands have a crucial role to play in our voice-activated digital future？”
“Voice is fast emerging as the dominant computing interface,” the report notes. “It will connect us with the entirety of the digital world—some 1.73 billion websites as of Q4 2019, and more than 26 billion connected devices. Voice will be the way we engage with brands on multi-dimensional levels. Its impact will be profound.”
International recognition
One of the seminal lessons from the early adopters of technologies in retail and brands in recent years has been that experimenting early has given them a commanding lead in capturing the market, the report notes.
JD has been forging forward with its own research in voice-recognition technology, with recent acknowledgement at INTERSPEECH 2020, the world’s largest technical conference focused on speech processing and application. The conference highlighted four papers from JD AI Research Institute, one of which was co-written with the University of Texas at Dallas. Covering topics including speech recognition, speech enhancement, voiceprint recognition and speech synthesis, all four papers represented innovative breakthroughs in the field.
One paper that was honored at the event addresses the problem of how reverberation and noise in a surrounding environment can interfere with voice recognition. JD AI Research Institute proposed a multi-directional sound detection method that uses acoustic signal processing to provide more abundant and accurate information.
Another paper, which was developed in cooperation with the University of Texas at Dallas, proposes a method to improve the ability to express and detect speech under complex conditions by applying multiple networks that are more easily understood by the decoder.
Additionally, JD proposed data enhancement solutions for far-field speaker recognition based on 1,100 hours of data from a total of 120 speakers to transform far-field data to near-field through technologies such as beamforming, channel exchange and de-reverberation.
This research and related technology have been successfully applied by JD in phone interaction scenarios including intelligent chatbots and customer service voice robots, with plans to expand further in the coming years.
Meanwhile, JD is already utilizing voice and AI to enhance the customer service experience, and making possible experiences that up until a few years ago were difficult to imagine for many.
New possibilities with speech
According to the China Disabled Persons’ Federation, there are more than 17 million visually impaired people in China, many of which are now able to enjoy the conveniences of online shopping thanks to JD. What would have been unthinkable in the past—placing an online order, for example—is now easily achieved with JD’s voice-search function, spoken notifications for functions from searching to payment, and even in notifying customers of new promotions.
Since 2016, JD has been continually upgrading its shopping app to serve disabled people, especially the visually impaired. In order to meet their needs, JD’s R&D team conducted in-depth research into the needs of visually impaired consumers, regularly interviewing members of this community for first-hand user feedback to recent changes to the app. Some of JD’s R&D team made a habit of visiting massage parlors staffed by blind masseuses, introducing new features and asking for their opinions and requests for how the app might better serve them.
Additionally, JD collaborated with the Information Accessibility Research Association based in Shenzhen, to accurately understand the needs of visually impaired people based on the organization’s data and research.
In an appreciation letter to JD, one visually impaired customer named Liquan Wu wrote, “Because of JD, I am able to shop freely in a way that people like me have always dreamed of; because of JD, people who can’t see can buy items for our parents, relatives and friends, which we could never do before. It feels so great that I never want to lose it.”
JD is also using voice to serve other communities. For example, JD Health and JD IoT jointly released a smart speaker product called “Family Doctor Guardian Star”, which integrates JD Health’s family doctor telemedicine program. Customers can speak to the speaker to call their designated general practitioners under the family doctor program, and make face-to-face appointments via voice commands. The speaker can also be connected with other health monitoring devices so that patients and their doctors are more easily able to achieve a comprehensive understanding of the patient’s needs.
Voice & AI
In order to hone in on more accurate, satisfying responses over time, voice technology is often enhanced with AI, another area in which JD is making rapid advancements.
For example, China Unicom customers in the Southwestern Chinese province of Sichuan, are already benefiting from the network’s partnership with JD, which integrates AI into its customer service apparatus.
Customers who call the customer service hotline are greeted by a natural-sounding voice that responds to their needs, seamlessly transferring them to a customer service representative if necessary. AI is also integrated into JD’s text messaging customer service communications, ultimately drastically improving the response rate, accuracy level and overall satisfaction level. As of this year, 95% of inquiries received by JD partner West China Medical University Second Hospital’s AI-powered medical care customer service created by JD are answered by AI at a 90% accuracy rate, according to JD.
JD receives millions of customer service inquiries every day, and during JD’s June 18 Grand Promotion, its smart customer service chatbot fielded more than 32 million text and voice inquiries, of which 90% were solved independently.
JD’s chatbot uses deep learning and transfer learning technologies to respond to requests, and is even able to detect the subtlest of human emotions in real-time and provide appropriate responses.
For example, if a customer uses language or a tone that signals frustration, the chatbot will soften its tone, using comforting language while providing proposed solutions to the customer’s complaint.
In fact, this kind of adaptive AI and voice interaction is just the beginning, according to WPP. Voice technology will be able to recognize detailed nuance in speech patterns, and will be able to measure not just actions but intentions, as well as detect emotional states.
“Because online commerce will grow increasingly conversational, this will make it far more human-like,” WPP predicts. “It will become contextual and personal. And this means it can be used across the length of the commerce journey to purchase. It will stretch from search to purchase; to post-sale support, and include opportunities for evaluation and comment. Welcome to the world of ‘true conversational commerce.’”
As JD continues to advance across multiple industries, adapting to customers’ ever-evolving needs will be essential. Voice and AI will be a key tool in meeting those demands.
As Dr. Bowen Zhou, President of JD Ai has said, “AI is not just a technology. It’s a solution to some of the most pressing challenges faced by many industries.”
(kellydawson@jd.com)
