Do you see what I see?: A discussion regarding the nature of perception in embodied machine learners

Abstract To improve our understanding of what an embodied machine learner is and is not capable of, researchers have recommended moving away from traditional task optimisation methods of assessment in favour of a more general profiling approach. These recommendations propose using multiple tasks with varying demands to obtain a domain-general assessment of an embodied machine learner’s capabilities. While these recommendations are a welcome contribution to machine learning research, we suggest that a greater focus should also be paid to the perceptual mechanisms of embodied machine learners. Specifically, by assessing how an embodied machine learner recognises and perceives an ’object’ in the environment, and building on this, what perceptions embodied machine learners develop regarding the behaviours and relations of objects to one another (i.e., objecthood). In a discussion of this, we first explore how similar questions about perceptions of objects and objecthood have been investigated in philosophical discussions and psychological research. We then narrow our focus to Gestaltian psychology and discuss how Gestaltian principles might be adapted into a machine learning context. In doing this, machine learning researchers may receive an enriched set of criteria to assess the perceptual mechanisms of embodied machine learners. We then suggest that these assessments may provide additional insight into the perceptual mechanisms of an embodied machine learner, with examples from ongoing initiatives provided. If the insights obtained by this approach can be reliably reproduced, researchers may also be able to more accurately predict how the machine learner will perceive and interact with novel objects. This could be accomplished by applying the domain-general knowledge acquired about the machine learner to novel contexts. However, for reproducible results to be acquired, robust and valid experimental methods must be used, which we discuss in the final section of this paper.