Shared Mental Models for Human-Robot Teaming

Stefanos Nikolaidis, Przemyslaw Lasota

When humans work in teams, it is crucial for the members to develop fluent team behavior. We believe that the same holds for robot teammates, if they are to perform in a similarly fluent manner as members of a human-robot team. Our collaborative planning and execution algorithms are inspired by studies in human team coordination, and aim to support a mutual adaptation process for humans and robot that produces fluency in joint-action. This goal of the research is to improve the efficiency of manual assembly processes through human-robot collaboration. This work is performed in collaboration with ABB.

Nikolaidis, S., K. Gu, R. Ramakrishnan, and J. A. Shah, "Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks", The 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI) [Best Enabling Technology Paper], Portland, Oregon, USA, 03/2015.
Nikolaidis, S., and J. A. Shah, "Human-Robot Cross-Training: Computational Formulation, Modeling and Evaluation of a Human Team Training Strategy", Proceedings of the IEEE/ACM International Conference on Human-Robot Interaction (25% Acceptance Rate), 03/2013.

Fast Scheduling of Human-Robot Teams with Temporospatial Constraints

Matthew Gombolay, Ronald Wilcox

New uses of robotics in traditionally manual manufacturing processes require the careful choreography of human and robotic agents to support safe and efficient coordinated work. Tasks must be allocated among agents and scheduled to meet temporal deadlines and spatial restrictions on agent proximity. These systems must also be capable of replanning on-the-fly to adapt to disturbances in the schedule and to respond to people working in close physical proximity. We are developing fast, near optimal task assignment and scheduling algorithms that scale to multiagent, factory-size problems and support on-the-fly replanning with temporal and spatial-proximity constraints. We demonstrate that this capability enables human and robotic agents to effectively work together in close proximity to perform manufacturing-relevant tasks, including multi-robot composite material placement, drilling, and robotic assistance in manual tasks. This research is performed in collaboration with Boeing Research and Technology.

Improving the Efficiency of Close Proximity Human-Robot Collaboration

Claudia Pérez-D'Arpino, Przemyslaw Lasota, Vaibhav Unhelkar

Manipulation tasks where humans and robots share a common volume represent new challenges for robotics in manufacturing environments. The robot's motion planning algorithm requires an awareness of the human’s pose and his or her next actions to compute trajectories that synchronize naturally with the human motion. We are developing statistical models and algorithms to predict in real time the behavior of a human working together with a robot, using both human motion and task procedure information. The efficacy of the approach is evaluated through human subject experimentation. This capability is used by the robot to deconflict work assignments, assist in manipulation tasks, and support the human by providing tools and materials at the right time.

Pérez-D'Arpino, C., and J. Shah, "Fast Target Prediction of Human Reaching Motion for Cooperative Human-Robot Manipulation Tasks Using Time Series Classification", IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, 05/2015.
Unhelkar, V. V., C. Pérez-D'Arpino, L. Stirling, and J. A. Shah, "Human-Robot Co-Navigation using Anticipatory Indicators of Human Walking Motion", IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, 05/2015.
Lasota, P. A., and J. A. Shah, "Analyzing the Effects of Human-Aware Motion Planning on Close-Proximity Human–Robot Collaboration", Human Factors: The Journal of the Human Factors and Ergonomics Society [2014 Human Factors Prize Finalist], vol. 57, issue 1, pp. 21-33, 02/2015.
Lasota, P. A., G. F. Rossano, and J. A. Shah, "Toward Safe Close-Proximity Human-Robot Interaction with Standard Industrial Robots", The 10th IEEE International Conference on Automation Science and Engineering (CASE), Taipei, Taiwan, 08/2014.

Intelligent Machine Participation in Human Meetings

Joseph Kim, Been Kim

Inherent human limitations pose significant challenges for team planning in a highly dynamic crisis environment.  Stress, urgency, resource conflict, and confirmation bias may get in the way of establishing a clear shared mental model of what needs to get done and how the response team will proceed. Our research effort is aimed at developing intelligent agents that effectively particpate in human team planning session for disaster response. We are developing algorithms to detect weak agreements among team members at planning time, and track plan formation directly from the human team's planning conversation. This reseach is performed in collaboration with Lincoln Laboratory, and our long term goal is to integrate this technology into the Next Generation Incident Command System (NICS), a web-based tool for distributed planning and shared situational awareness.

Kim, J., C. J. Banks, and J. A. Shah, "Collaborative Planning with Encoding of Users’ High-level Strategies", To appear at the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), San Francisco, California, 02/2017.
Kim, J., and J. A. Shah, "Improving Team's Consistency of Understanding in Meetings", IEEE Transactions on Human-Machine Systems (THMS), vol. 46, issue 5, pp. 625-637, 04/2016.
Kim, B., C. M. Chacha, and J. A. Shah, "Inferring Robot Task Plans from Human Team Meetings: A Generative Modeling Approach with Logic-Based Prior", Twenty-Seventh AAAI Conference on Artificial Intelligence (AAAI-13) (29% Acceptance Rate), 07/2013.

Mobile Robotic Assistants for Manufacturing Environments

Vaibhav Unhelkar, Jim Boerkoel

Human workers in assembly spend a significant portion of their time retrieving and staging tools and parts for each job. A robotic assistant can provide productivity benefit by performing these non-value-added tasks for the worker. In this project, we are developing a mobile robot to assist workers on an automotive assembly line. The robot must navigate a confined and busy space. Motion planning techniques that treat people as obstacles can significantly degrade the efficiency of the system. We're developing planning techniques the incorporate models of human behavior to anticipate human motions and actions. Our aim is to use these models of human behavior to improve the efficiency with which the robot traverses the line and maneuvers around people.

Unhelkar, V. V., J. Perez, J. C. Boerkoel, J. Bix, S. Bartscher, and J. A. Shah, "Towards Control and Sensing for an autonomous Mobile Robotic Assistant navigating Assembly Lines", IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 06/2014.
Unhelkar, V. V., H. Chit Siu, and J. A. Shah, "Comparative Performance of Human and Mobile Robotic Assistants in Collaborative Fetch-and-Deliver Tasks", The 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Bielefeld, Germany, 03/2014.

Research