Mar 2, 2022|

JD Explore Academy Excels at World’s Top Robotic Arm Competition


by Yiming Yan

SAPIEN (SimulAted Part-based Interactive ENvironment) ManiSkill (Manipulation Skill) Challenge 2021, the world’s top robotic arm competition, recently announced the results and final rankings, in which the team “Silver-Bullet-3D” from the Vision and Multimedia Lab of JD Explore Academy ranked the top two in all three tracks.

The ManiSkill challenge builds four tasks out of a large and diversified set of articulated objects in order to demonstrate the ability to employ artificial intelligence (AI) technology in real scenarios like robotic arm control.

Silver-Bullet-3D took first place in both the ‘No Interaction Track’ and the ‘No Restriction Track’ of the challenge. No interaction track requires the solution to only use the demonstration trajectories provided. In order to model the complex texture and structural information of the robotic arm and the manipulated object, the participating team proposed a solution based on imitation learning, which consists of two main modules, behavioral cloning with direct imitation behavior and offline augmented learning, as well as a transformer-based relational modeling network.

The no-restriction track allows participants to do whatever they wish, such as labeling new data, creating a new setting, and so on. In response, the team proposed a heuristic rule-based solution (HRM), which decomposes a complex operation task into a series of subtasks. The solution received first place in the official evaluation, with a margin of  23.8 percent over second place.

JD Explore Academy has been awarded 18 patents in the field of computer vision technology, and its core members have published 180 more papers at premier AI conferences. What’s more, JD Explore Academy won the Best Demonstration Award for its multimodal interaction digital human technology and the Best Open-Source Project Award for its cross-modal analysis technology at the 29th ACM International Conference on Multimedia.

The SAPIEN ManiSkill Challenge was organized by UC San Diego, UC Berkeley, and Stanford University at ICLR (International Conference on Learning Representations) 2022, the world’s leading conference on machine learning. Thirty teams from globally renowned organizations and colleges participated in the event.