Talk: Human Understanding for HoloLens (Watch)

Abstract: Mixed reality devices such as HoloLens offer new and exciting holographic experiences and capabilities. Underpinning these experiences is a rich understanding of the user, powered by state of the art computer vision algorithms. In this talk we'll explore some of the challenges we faced in bringing those algorithms into HoloLens 2, as well as taking a glimpse at some of the exciting opportunities for computer vision to impact future devices and enable transformative experiences such remote presence.

Bio: Jamie Shotton is Partner Director of Science at Microsoft. He leads the Mixed Reality & AI Labs in Cambridge and Belgrade, where his team incubates transformative new technologies and experiences from early stage research to shipping product. He studied Computer Science at the University of Cambridge, where he remained for his PhD in computer vision and machine learning, before joining Microsoft in 2008. His research focuses at the intersection of computer vision, AI, machine learning, and graphics, with particular emphasis on systems that understand the motion, shape, and appearance of people in 3D. He has explored applications of this work for mixed reality, virtual presence, human-computer interaction, gaming, and healthcare. He has shipped foundational features in multiple products including body tracking for Kinect and the hand- and eye-tracking that enable HoloLens 2’s instinctual interaction model. He has received multiple Best Paper and Best Demo awards at top-tier academic conferences. His work on Kinect was awarded the Royal Academy of Engineering’s gold medal MacRobert Award in 2011, and he shares Microsoft’s Outstanding Technical Achievement Award for 2012 with the Kinect engineering team. In 2014 he received the PAMI Young Researcher Award, and in 2015 the MIT Technology Review Innovator Under 35 Award.