PROWL

Many existing person tracking systems are challenged by non-laboratory scenarios, including variable lighting conditions, rain, smoke, tracking distance, and tracking speed. Under contract for the U.S. Army, TRACLabs developed a combined LWIR/visible-spectrum stereo camera payload called PROWL (Perception for Robotic Operation over Widespread Lighting) that overcomes these limitations. This sensor payload intelligently combines visible spectrum and thermal (LWIR) sensing to provide 24-hour, all-lighting operations. The sensor contains low-power onboard FPGA/Atom processing that runs custom-built autonomous leader-following, object recognition, and gesture recognition software, and it supports robotic tele-operation via display of temperature, range, and raw video feeds in a web-based user interface. In exhaustive tests, we provide evidence that by using a 3D thermal sensor, a person can be tracked in three dimensions with high success using very simple tracking methods, in many of the challenging lighting conditions and other weather conditions that confound other systems. PROWL, using only simple ICP-based point matching algorithms, obtains 100% person tracking success at 20 frames per second out to 13 meters and zero false- positive/false-negative gesture recognition within 7 meters in all tested scenarios, which includes a sunny outdoor environment, a nighttime outdoor environment, a blackout indoor environment, and a whiteout smoke-filled indoor environment. The final PROWL deliverable was manufactured by Carnegie Robotics, who provided an IPP-67 rated payload.