Dr. Stephen Hart

Senior Scientist

Dr. Hart has over ten years of experience in on developing software for making programming humanoid robots easier.  In his graduate studies at UMass and his postdoctoral work at the Italian Institute of Technology, this work focused on how to create behavioral knowledge representations that are suitable for learning manipulation skills and that be reused in different contexts or on different robots. As a Senior Researcher for General Motors R&D between 2010 and 2012, Dr. Hart was the “Applications Lead” for the NASA Robonaut 2 project, working on-site at NASA Johnson Space Center.  There, he led a team of software developers, researchers, and collaborators in developing the Robot Task Commander (RTC) application programming tool, currently the standard interface for developing applications on the Robonaut 2 unit located on the International Space Station (ISS). RTC provided an improvement over the state of the art application development system for R2 as it allowed the user to quickly incorporate existing skills from a library into new applications and to easily integrate new algorithms, sensors, or control systems without requiring recompilation.  From 2012 to 2014, Dr. Hart was the “Behavior & Application Lead” for the NASA-JSC Valkyrie project, where he developed the Affordance Template framework as a graphical extension of RTC.  Affordance Templates allow operators to create and execute robot tasks in a 3D environment that aggregates virtual representations task goals, an avatar of the robot in its current configuration, and available sensor data.  Affordance Templates provide an intuitive interface for controlling complex robot systems, requiring only a mouse and a computer screen, and are therefore suitable for more general users, such as ISS crew members or workers on a factory floor.  In late 2014, Dr. Hart joined TRACLabs as a Senior Scientist and joined numerous projects.  As “Software Lead” for the TRACLabs DARPA Robotics Challenge team, he led the team of 6 programmers and researchers to a 9th place finish (out of 23 competing teams) at the 2015 DRC finals.   Additionally, he is involved in multiple ongoing NASA projects to enhance the development tools used to program and operate humanoid robots easier.


  • 2002 B.S. Computer Systems Engineering, University of Massachusetts Amherst. Amherst, MA
  • 2005 M.S. Computer Science, University of Massachusetts Amherst. Amherst, MA
  • 2009 Ph.D. Computer Science, University of Massachusetts Amherst. Amherst, MA


  • Stephen Hart, Paul Dinh, and Kimberly Hambuchen. The Affordance Template ROS Package for Robot Task Programming. In IEEE International Conference on Robotics and Automation (ICRA), 2015.
  • Joshua James, Yifan Weng, Stephen Hart, Patrick Beeson, and Robert Burridge. Prophetic Goal-Space Planning for Human-in-the-Loop Mobile Manipulation. In Proceedings of the IEEE RAS Humanoids Conference, 2015.
  • Stephen Hart, John D. Yamokoski, Paul Dinh, Brian Wightman, and Nic Radford. Robot Task Commander: A Framework and IDE for Robot Application Development. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2014.
  • N.A. Radford, P. Strawser, K. Hambuchen, J.S. Mehling, W.K. Verdeyen, S. Donnan, J. Holley, J. Sanchez, V. Nguyen, L. Bridgwater, R. Berka, R. Ambrose, C. McQuin, J.D. Yamokoski, S. Hart, R. Guo, A. Parsons, B. Wightman, P. Dinh, B. Ames, C. Blakely, C. Edmonson, B. Sommers, R. Rea, C. Tobler, H. Bibby, B. Howard, L. Nui, A. Lee, M. Conover, L. Truong, D. Chesney, R. Platt Jr., G. Johnson, C.-L. Fok, N. Paine, L. Sentis, E. Cousineau, R. Sinnet, J. Lack, M. Powell, B. Morris, A. Ames. Valkyrie: NASA’s First Bipedal Humanoid Robot. Journal of Field Robotics, 2015
  • Stephen Hart and Rod Grupen. Intrinsically Motivated Affordance Discovery and Modeling. Intrinsically Motivated Learning in Natural and Artificial Systems. 279-300. G. Baldassarre and M. Mirolli eds. Springer, 2013.
  • Stephen Hart and Rod Grupen. Learning Generalizable Control Programs. IEEE Transactions on Autonomous Mental Development, 3(3):216-231, 2011.
  • Stephen Hart and Rod Grupen. Natural Task Decomposition with Intrinsic Potential Fields. IEEE International Conference on Intelligent Robots and Systems (IROS), 2008.
  • Stephen Hart, Rod Grupen, and David Jensen. A Relational Representation for Procedural Task Knowledge. Proceedings of the American Association for Artificial Intelligence (AAAI) Conference, 2005.