Modeling robotic operations controlled by natural language
Yu CHENG
Jiatong BAO
Yunyi JIA
Zhihui DENG
Zhiyong SUN
Sheng BI
Congjian LI
Ning XI
摘要：There are multiple ways to control a robotic system.Most of them require the users to have prior knowledge about robots or get trained before using them.Natural language based control attracts increasing attention due to its versatility and less requirements for users.Since natural language instructions from users cannot be understood by the robots directly,the linguistic input has to be processed into a formal representation which captures the task specification and removes the ambiguity inherent in natural language.For most of existing natural language controlled robotic system,they assume the given language instructions are already in correct orders.However,it is very likely for untrained users to give commands in a mixed order based on their direct observation and intuitive thinking.Simply following the order of the commands can lead to failures of tasks.To provide a remedy for the problem,we propose a novel framework named dependency relation matrix (DRM) to model and organize the semantic information extracted from language input,in order to figure out an executable sequence of subtasks for later execution.In addition,the proposed approach projects abstract language input and detailed sensory information into the same space,and uses the difference between the goal specification and temporal status of the task under implementation to monitor the progress of task execution.In this paper,we describe the DRM framework in detail,and illustrate the utility of this approach with experiment results.
机标关键词：natural languagerobotic systemsemantic informationintuitive thinkingrelation matrixprior knowledgesequence of
资助基金：This work was partially supported by the National Science Foundation()This work was partially supported by the National Science Foundation(CNS-1320561)This work was partially supported by the National Science Foundation(IIS-1208390)the U.S.Army Research Laboratory,and the U.S.Army Research Office()the U.S.Army Research Laboratory,and the U.S.Army Research Office(W911NF-11-D-0001)the U.S.Army Research Laboratory,and the U.S.Army Research Office(W911NF-09-1-0321)the U.S.Army Research Laboratory,and the U.S.Army Research Office(W911 NF-10-1-0358)the U.S.Army Research Laboratory,and the U.S.Army Research Office(W911NF-14-1-0327)the U.S.Army Research Laboratory,and the U.S.Army Research Office(W911NF-16-1-0572)
论文发表日期：2017-01-01
在线出版日期：2025-08-15（本平台首次上网日期，不代表文献的发表时间）
页数：9( 258-266 )
英文信息