Abstract
Data-driven knowledge discovery is becoming a new trend in various scientific fields. In light of this, the goal of the present paper is to introduce a novel framework to study one interesting topic in cognitive and behavioral studies – multimodal communication between human-human and human-robot interaction. We present an overall solution from data capture, through data coding and validation, to data analysis and visualization. In data collection, we have developed a multimodal sensing system to gather fine-grained video, audio and human body movement data. In data analysis, we propose a hybrid solution based on visual data mining and information-theoretic measures. We suggest that this data-driven paradigm will lead not only to breakthroughs in understanding multimodal communication, but will also serve as a successful case study to demonstrate the promise of data-intensive discovery which can be applied in various research topics in cognitive and behavioral studies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bell, G., Hey, T., Szalay, A.: Computer science. Beyond the data deluge. Science 323, 1297–1298 (2009)
Cohen, P.R., Adams, N.: Intelligent Data Analysis in the Twenty First Century. In: Proceedings of the Intelligent Data Analysis Conference, Lyon, France (2009)
Baron-Cohen, S.: Mindblindness: an essay on autism and theory of mind. MIT Press/Bradford Books (1995)
Itti, L., Koch, C., Niebur, E.: A Model of Saliency-Based Visual Attention for Rapid Scene Analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259 (1998)
Yu, C., Smith, L.B., Shen, H., Pereira, A.F., Smith, T.G.: Active Information Selection: Visual Attention Through the Hands. IEEE Transactions on Autonomous Mental Development 2, 141–151 (2009)
Oates, T., Cohen, P.R.: Searching for Structure in Multiple Streams of Data. In: Proceedings of the Thirteenth International Conference on Machine Learning, pp. 346–354 (1996)
Yu, C., Zhong, Y., Smith, T., Park, I., Huang, W.: Visual Data Mining of Multimedia Data for Social and Behavioral Studies. Information Visualization 8, 56–70 (2009)
Lin, J., Keogh, E., Li, W., Lonardi, S.: Experiencing SAX: A Novel Symbolic Representation of Time Series. Data Mining and Knowledge Discovery Journal, 107–144 (2007)
Kantz, H., Schreiber, T.: Nonlinear Time Series Analysis. Cambridge University Press, Cambridge (1997)
Yu, C., Scheutz, M., Schermerhorn, P.: Investigating Multimodal Real-Time Patterns of Joint Attention in an HRI Word Learning Task. In: 5th ACM/IEEE International Conference on Human-Robot Interaction (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yu, C., Smith, T.G., Hidaka, S., Scheutz, M., Smith, L.B. (2010). A Data-Driven Paradigm to Understand Multimodal Communication in Human-Human and Human-Robot Interaction. In: Cohen, P.R., Adams, N.M., Berthold, M.R. (eds) Advances in Intelligent Data Analysis IX. IDA 2010. Lecture Notes in Computer Science, vol 6065. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13062-5_22
Download citation
DOI: https://doi.org/10.1007/978-3-642-13062-5_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13061-8
Online ISBN: 978-3-642-13062-5
eBook Packages: Computer ScienceComputer Science (R0)Springer Nature Proceedings Computer Science
