{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T20:54:36Z","timestamp":1776286476441,"version":"3.50.1"},"reference-count":30,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2016,10,20]],"date-time":"2016-10-20T00:00:00Z","timestamp":1476921600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Bavarian Elite Network","award":["Bavarian Elite Research Scholarship"],"award-info":[{"award-number":["Bavarian Elite Research Scholarship"]}]},{"name":"DFG and TU Munich","award":["Open Accessing Publishing Program"],"award-info":[{"award-number":["Open Accessing Publishing Program"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Biological and technical systems operate in a rich multimodal environment. Due to the diversity of incoming sensory streams a system perceives and the variety of motor capabilities a system exhibits there is no single representation and no singular unambiguous interpretation of such a complex scene. In this work we propose a novel sensory processing architecture, inspired by the distributed macro-architecture of the mammalian cortex. The underlying computation is performed by a network of computational maps, each representing a different sensory quantity. All the different sensory streams enter the system through multiple parallel channels. The system autonomously associates and combines them into a coherent representation, given incoming observations. These processes are adaptive and involve learning. The proposed framework introduces mechanisms for self-creation and learning of the functional relations between the computational maps, encoding sensorimotor streams, directly from the data. Its intrinsic scalability, parallelisation, and automatic adaptation to unforeseen sensory perturbations make our approach a promising candidate for robust multisensory fusion in robotic systems. We demonstrate this by applying our model to a 3D motion estimation on a quadrotor.<\/jats:p>","DOI":"10.3390\/s16101751","type":"journal-article","created":{"date-parts":[[2016,10,20]],"date-time":"2016-10-20T10:15:49Z","timestamp":1476958549000},"page":"1751","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["A Self-Synthesis Approach to Perceptual Learning for Multisensory Fusion in Robotics"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6184-0546","authenticated-orcid":false,"given":"Cristian","family":"Axenie","sequence":"first","affiliation":[{"name":"Neuroscientific System Theory Group, Department of Electrical and Computer Engineering, Technical University of Munich, Arcisstrasse 21, Munich 80333, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6591-1118","authenticated-orcid":false,"given":"Christoph","family":"Richter","sequence":"additional","affiliation":[{"name":"Neuroscientific System Theory Group, Department of Electrical and Computer Engineering, Technical University of Munich, Arcisstrasse 21, Munich 80333, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5998-9640","authenticated-orcid":false,"given":"J\u00f6rg","family":"Conradt","sequence":"additional","affiliation":[{"name":"Neuroscientific System Theory Group, Department of Electrical and Computer Engineering, Technical University of Munich, Arcisstrasse 21, Munich 80333, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2016,10,20]]},"reference":[{"key":"ref_1","unstructured":"Braitenberg, V. (1984). Vehicles\u2014Experiments in Synthetic Psychology, MIT Press."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1212","DOI":"10.1038\/81497","article-title":"Computational principles of movement neuroscience","volume":"3","author":"Wolpert","year":"2000","journal-title":"Nat. Neurosci."},{"key":"ref_3","unstructured":"Shaw, R., and Bransford, J. (1977). Perceiving, Acting, and Knowing: Toward an Ecological Psychology, Lawrence Erlbaum."},{"key":"ref_4","unstructured":"Cook, M., Jug, F., Krautz, C., and Steger, A. (2010). Artificial Neural Networks\u2014ICANN 2010, Springer."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2552","DOI":"10.1016\/j.neucom.2006.05.014","article-title":"A self-organizing map of sigma-pi units","volume":"70","author":"Weber","year":"2007","journal-title":"Neurocomputing"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2788","DOI":"10.3390\/e15072788","article-title":"Non-linear canonical correlation analysis using alpha-beta divergence","volume":"15","author":"Mandal","year":"2013","journal-title":"Entropy"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1095","DOI":"10.1016\/S0893-6080(00)00067-8","article-title":"Nonlinear canonical correlation analysis by neural networks","volume":"13","author":"Hsieh","year":"2000","journal-title":"Neural Netw."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"638","DOI":"10.1186\/s40064-016-2205-6","article-title":"Conflict management based on belief function entropy in sensor fusion","volume":"5","author":"Yuan","year":"2016","journal-title":"Springerplus"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Xie, Z., Tie, Y., and Guan, L. (2015, January 24\u201327). A new audiovisual emotion recognition system using entropy-estimation-based multimodal information fusion. Proceedings of the 2015 IEEE International Symposium on Circuits and Systems (ISCAS), Lisbon, Portugal.","DOI":"10.1109\/ISCAS.2015.7168736"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Liu, L., Wang, S., Liu, D., Zhang, Y., and Peng, Y. (2015, January 5\u20139). Entropy-based sensor selection for condition monitoring and prognostics of aircraft engine. Proceedings of the 26th European Symposium on Reliability of Electron Devices, Failure Physics and Analysis, Toulouse, France.","DOI":"10.1016\/j.microrel.2015.06.076"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1109\/TCST.2014.2324978","article-title":"Minimum-Entropy control for non-Gaussian stochastic networked control systems and its application to a networked DC motor control system","volume":"23","author":"Ren","year":"2014","journal-title":"Trans. Control Syst. Technol."},{"key":"ref_12","unstructured":"Gibson, E.J., and Pick, A.D. (2003). An Ecological Approach to Perceptual Learning and Development, Oxford University Press."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"352","DOI":"10.1016\/j.cub.2009.01.036","article-title":"Motor task variation induces structural learning","volume":"19","author":"Braun","year":"2009","journal-title":"Curr. Biol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1016\/0306-4522(95)00355-X","article-title":"The shape of self-motion perception\u2014II. Framework and principles for simple and complex motion","volume":"2","author":"Holly","year":"1996","journal-title":"Neuroscience"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.robot.2014.11.019","article-title":"Cortically inspired sensor fusion network for mobile robot egomotion estimation","volume":"71","author":"Axenie","year":"2014","journal-title":"Robot. Auton. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1111\/j.1467-7687.2007.00567.x","article-title":"Neuroconstructivism","volume":"10","author":"Westermann","year":"2007","journal-title":"Dev. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kohonen, T. (2001). Self-Organizing Maps, Wiley.","DOI":"10.1007\/978-3-642-56927-2"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chen, Z., Haykin, S., Eggermont, J.J., and Becker, S. (2007). Correlative Learning: A Basis for Brain and Adaptive Systems, Wiley.","DOI":"10.1002\/9780470171455"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2103","DOI":"10.1162\/NECO_a_00638","article-title":"Efficient sensory encoding and bayesian inference with heterogeneous neural populations","volume":"26","author":"Ganguli","year":"2014","journal-title":"Neural Comput."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2262","DOI":"10.1109\/TIM.2012.2187245","article-title":"Estimation of attitude and external acceleration using inertial sensor measurement during various dynamic conditions","volume":"61","author":"Lee","year":"2012","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Villaverde, A.F., Ross, J., Moran, F., and Banga, J.R. (2014). MIDER: Network inference with mutual information distance and entropy reduction. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0096732"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Meyer, P.E., Kontos, K., Lafitte, F., and Bontempi, G. (2007). Information-theoretic inference of large transcriptional regulatory networks. EURASIP J. Bioinform. Syst. Biol., 79879.","DOI":"10.1155\/2007\/79879"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1093\/bioinformatics\/18.suppl_2.S231","article-title":"The mutual information: Detecting and evaluating dependencies between variables","volume":"18","author":"Steuer","year":"2002","journal-title":"Bioinformatics"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1063\/1.1336499","article-title":"On the deduction of chemical reaction pathways from measurements of time series of concentrations","volume":"11","author":"Samoilov","year":"2001","journal-title":"Chaos"},{"key":"ref_25","unstructured":"Samoilov, M. (1997). Reconstruction and Functional Analysis of General Chemical Reactions and Reaction Networks. [Ph.D. Thesis, Stanford University]."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"970","DOI":"10.1021\/j100003a020","article-title":"Statistical construction of chemical reaction mechanisms from measured time-series","volume":"99","author":"Arkin","year":"1995","journal-title":"J. Phys. Chem."},{"key":"ref_27","unstructured":"Lungarella, M., and Sporns, O. (2005, January 19\u201321). Information self-structuring: Key principle for learning and development. Proceedings the 4th International Conference on Development and Learning, Banff, AB, Canada."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1109\/MRA.2012.2205629","article-title":"Build your own quadrotor","volume":"19","author":"Hyon","year":"2012","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_29","unstructured":"Brent, R.P. (2013). Algorithms for Minimization without Derivatives, Dover Publications."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"8849","DOI":"10.3182\/20140824-6-ZA-1003.02673","article-title":"2-Sliding mode trajectory tracking control and EKF estimation for quadrotors","volume":"47","author":"Mercado","year":"2014","journal-title":"IFAC Proc. Vol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/10\/1751\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:33:34Z","timestamp":1760211214000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/10\/1751"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,10,20]]},"references-count":30,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2016,10]]}},"alternative-id":["s16101751"],"URL":"https:\/\/doi.org\/10.3390\/s16101751","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,10,20]]}}}