As humans and robots start to cohabit the same environment and interact deeper with each other, the need for robots and other artificial agents to truly understand each other increases. This understanding not only involves the ability for robots to decode human speech, but also the ability to ground words that refer to physical objects, actions, and concepts –  to the perceptual information coming from the sensors embedded on the agent. It also involves the ability for robots to better understand human cues in interaction scenarios. This talk will highlight the research at Örebro University where we look at how we can develop better perceptual systems that can make sense of sensor data and how that can then be used to create conducive conditions for interaction.