The goal is to teach a robot the meaning of objects in its environment. This is achieved by what we call the Architecture Space-Game (ASG), where the robot takes the active part of asking various people about objects and their meaning. A key aspect is the need to disambiguate between different labels and relations used by different persons. In this paper, we show the importance of the structure of the dialogue to achieve useful input to build up object knowledge. From experiments with large audiences, we derive steps that will lead to an improved human-robot understanding.