{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,27]],"date-time":"2026-06-27T11:31:20Z","timestamp":1782559880811,"version":"3.54.5"},"reference-count":55,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2020,3,16]],"date-time":"2020-03-16T00:00:00Z","timestamp":1584316800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2016YFC0802904"],"award-info":[{"award-number":["2016YFC0802904"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61671470"],"award-info":[{"award-number":["61671470"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["BK20161470"],"award-info":[{"award-number":["BK20161470"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]},{"name":"62nd batch of funded projects of China Postdoctoral Science Foundation","award":["2017M623423"],"award-info":[{"award-number":["2017M623423"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Multi-object tracking (MOT) plays a crucial role in various platforms. Occlusion and insertion among targets, complex backgrounds and higher real-time requirements increase the difficulty of MOT problems. Most state-of-the-art MOT approaches adopt the tracking-by-detection strategy, which relies on compute-intensive sliding windows or anchoring schemes to detect matching targets or candidates in each frame. In this work, we introduce a more efficient and effective spatial\u2013temporal attention scheme to track multiple objects in various scenarios. Using a semantic-feature-based spatial attention mechanism and a novel Motion Model, we address the insertion and location of candidates. Some online-learned target-specific convolutional neural networks (CNNs) were used to estimate target occlusion and classify by adapting the appearance model. A temporal attention mechanism was adopted to update the online module by balancing current and history frames. Extensive experiments were performed on Karlsruhe Institute of Technologyand Toyota Technological Institute (KITTI) benchmarks and an Armored Target Tracking Dataset (ATTD) built for ground-armored targets. Experimental results show that the proposed method achieved outstanding tracking performance and met the actual application requirements.<\/jats:p>","DOI":"10.3390\/s20061653","type":"journal-article","created":{"date-parts":[[2020,3,18]],"date-time":"2020-03-18T08:13:27Z","timestamp":1584519207000},"page":"1653","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Spatial\u2013Semantic and Temporal Attention Mechanism-Based Online Multi-Object Tracking"],"prefix":"10.3390","volume":"20","author":[{"given":"Fanjie","family":"Meng","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, College of Field Engineering, Army Engineering University of PLA, Nanjing 210007, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinqing","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, College of Field Engineering, Army Engineering University of PLA, Nanjing 210007, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1511-9499","authenticated-orcid":false,"given":"Dong","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, College of Field Engineering, Army Engineering University of PLA, Nanjing 210007, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6281-2990","authenticated-orcid":false,"given":"Faming","family":"Shao","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, College of Field Engineering, Army Engineering University of PLA, Nanjing 210007, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lei","family":"Fu","sequence":"additional","affiliation":[{"name":"Department of Armament Science and Technology, College of Field Engineering, Army Engineering University of PLA, Nanjing 210007, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,3,16]]},"reference":[{"key":"ref_1","first-page":"312","article-title":"Armored target detection in battlefield environment based on top-down aggregation network and hierarchical scale optimization","volume":"33","author":"Haoze","year":"2019","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"ref_2","first-page":"1681","article-title":"Image detection method for tank and armored targets based on hierarchical multi-scale convolution feature extraction","volume":"38","author":"Haoze","year":"2017","journal-title":"Acta Armamentarii"},{"key":"ref_3","unstructured":"Qi, C., Wanli, O., Hongsheng, L., Xiaogang, W., Liu, B., and Yu, N. (2017, January 22\u201329). Online multi-object tracking using CNN-based single object tracker with spatial-temporal attention mechanism. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2526","DOI":"10.1109\/TIP.2018.2806280","article-title":"Good features to correlate for visual tracking","volume":"27","author":"Gundogdu","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1109\/LSP.2018.2811750","article-title":"Robust fusion for multisensor multiobject tracking","volume":"25","author":"Fantacci","year":"2018","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Jia, B., Lv, J., and Liu, D. (2019). Deep learning-based automatic downbeat tracking: A brief review. Multimedia Systems, 1\u201322.","DOI":"10.1007\/s00530-019-00607-x"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., and Yan, J. (2018). SiamRPN++: Evolution of siamese visual tracking with very deep networks. arXiv.","DOI":"10.1109\/CVPR.2019.00441"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, Q., Zhang, L., Bertinetto, L., Hu, W., and Torr, P.H.S. (2018). Fast online object tracking and segmentation: A unifying approach. arXiv.","DOI":"10.1109\/CVPR.2019.00142"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Melekhov, I., Kannala, J., and Rahtu, E. (2016, January 4\u20138). Siamese network features for image matching. Proceedings of the 2016 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico.","DOI":"10.1109\/ICPR.2016.7899663"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2146","DOI":"10.1109\/TPAMI.2018.2849374","article-title":"On detection, data association and segmentation for multi-target tracking","volume":"41","author":"Yicong","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Dawei, Z., Hao, F., Liang, X., Tao, W., and Bin, D. (2018). Multi-object tracking with correlation filter for autonomous vehicle. Sensors, 18.","DOI":"10.3390\/s18072004"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"5667","DOI":"10.1109\/TIP.2017.2745103","article-title":"A hybrid data association framework for robust online multi-object tracking","volume":"26","author":"Yang","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Bolme, D.S., Beveridge, J.R., Draper, B.A., and Lui, Y.M. (2010, January 13\u201318). Visual object tracking using adaptive correlation filters. Proceedings of the Twenty-Third IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2010, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539960"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1109\/TPAMI.2014.2345390","article-title":"High-speed tracking with kernelized correlation filters","volume":"37","author":"Henriques","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1586","DOI":"10.1049\/iet-ipr.2017.1043","article-title":"Regularisation learning of correlation filters for robust visual tracking","volume":"12","author":"Min","year":"2018","journal-title":"IET Image Process."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1016\/j.jvcir.2018.01.008","article-title":"Learning adaptively windowed correlation filters for robust tracking","volume":"51","author":"Kuai","year":"2018","journal-title":"J. Visual Comm. Image Represent."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Li, F., Tian, C., Zuo, W., Zhang, L., and Yang, M.H. (2018, January 18\u201323). Learning spatial-temporal regularized correlation filters for visual tracking. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00515"},{"key":"ref_18","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G. (2012, January 3\u20136). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the NIPS\u201912 Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, Nevada."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/MCI.2018.2840738","article-title":"Recent trends in deep learning based natural language processing","volume":"13","author":"Tom","year":"2018","journal-title":"IEEE Comput. Intell. Mag."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1109\/MSP.2017.2749125","article-title":"Advanced deep-learning techniques for salient and category-specific object detection: A survey","volume":"35","author":"Han","year":"2018","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1109\/MM.2018.112130335","article-title":"Domain-Specific Approximation for Object Detection","volume":"38","author":"Chin","year":"2018","journal-title":"IEEE Micro"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1109\/MSP.2017.2764116","article-title":"Deep learning for understanding faces: Machines may be just as good, or better, than humans","volume":"35","author":"Ranjan","year":"2018","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Voigtlaender, P., Krause, M., Osep, A., Luiten, J., Sekar, B.B.G., and Geiger, A. (2018). Mots: Multi-object tracking and segmentation. arXiv.","DOI":"10.1109\/CVPR.2019.00813"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Seguin, G., Bojanowski, P., Lajugie, R., and Laptev, I. (2016, January 27\u201330). Instance-Level Video Segmentation from Object Tracks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.400"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sadeghian, A., Alahi, A., and Savarese, S. (2017, January 22\u201329). Tracking the untrackable: Learning to track multiple cues with long-term dependencies. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.41"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Babenko, B., Yang, M.H., and Belongie, S. (2009, January 20\u201325). Visual tracking with online Multiple Instance Learning. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206737"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1442","DOI":"10.1109\/TPAMI.2013.230","article-title":"Visual tracking: An experimental survey","volume":"36","author":"Smeulders","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"50466","DOI":"10.1109\/ACCESS.2019.2910835","article-title":"Comparative object similarity learning-based robust visual tracking","volume":"7","author":"Yang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Son, J., Baek, M., Cho, M., and Han, B. (2017, January 21\u201326). Multi-object Tracking with Quadruplet Convolutional Neural Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.403"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster r-cnn: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1542","DOI":"10.1109\/TMI.2017.2665671","article-title":"Detection and localization of robotic tools in robot-assisted surgery videos using deep neural networks for region proposal and detection","volume":"36","author":"Sarikaya","year":"2017","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1007\/s10032-019-00335-y","article-title":"An anchor-free region proposal network for faster r-cnn based text detection approaches","volume":"22","author":"Zhong","year":"2019","journal-title":"Int. J. Doc. Anal. Recognit."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.neucom.2018.03.030","article-title":"Face detection using deep learning:an improved faster rcnn approach","volume":"299","author":"Sun","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Chen, Y., Li, W., Sakaridis, C., Dai, D., and Van Gool, L. (2018, January 18\u201323). Domain adaptive faster r-cnn for object detection in the wild. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00352"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Giuseppe, S., Massimiliano, G., Antonio, M., and Raffaele, G. (2018). A cnn-based fusion method for feature extraction from sentinel data. Remote Sens., 10.","DOI":"10.3390\/rs10020236"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Wang, J., Chen, K., Yang, S., Loy, C., and Lin, D. (2019). Region proposal by guided anchoring. arXiv.","DOI":"10.1109\/CVPR.2019.00308"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/S0301-5629(97)00281-0","article-title":"Multilevel and motion model-based ultrasonic speckle tracking algorithms","volume":"24","author":"Yeung","year":"1998","journal-title":"Ultrasound Med. Biol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1408","DOI":"10.1109\/ACCESS.2015.2471935","article-title":"Track detection of low observable targets using a motion model","volume":"3","author":"Park","year":"2015","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1109\/TPAMI.2017.2691769","article-title":"Confidence-based data association and discriminative deep appearance learning for robust online multi-object tracking","volume":"40","author":"Bae","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Henschel, R., Leal-Taix\u00e9, L., Cremers, D., and Rosenhahn, B. (2018, January 18\u201322). Fusion of head and full-body detectors for multi-object tracking. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00192"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"4585","DOI":"10.1109\/TIP.2018.2843129","article-title":"Interacting tracklets for multi-object tracking","volume":"27","author":"Long","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1109\/TPAMI.2018.2858826","article-title":"Focal loss for dense object detection","volume":"42","author":"Lin","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_43","unstructured":"Leal-Taixe, L., Milan, A., Reid, I., Roth, S., and Schindler, K. (2015). MOTChallenge 2015: Towards a benchmark for multi-target tracking. arXiv."},{"key":"ref_44","unstructured":"Milan, A., Leal-Taixe, L., Reid, I., Roth, S., and Schindler, K. (2016). Mot16: A benchmark for multi-object tracking. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16\u201321). Are we ready for autonomous driving? The KITTI vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"ref_46","first-page":"246309","article-title":"Evaluating multiple object tracking performance: The clear mot metrics\u2019, eurasip","volume":"1","author":"Keni","year":"2008","journal-title":"EURASIP J. Image Video Proc."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Leal-Taix\u00e9, L., Ferrer, C.C., and Schindler, K. (July, January 26). Learning by tracking: Siamese cnn for robust target association. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, NV, USA.","DOI":"10.1109\/CVPRW.2016.59"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Wang, B., Wang, L., Shuai, B., Zuo, Z., and Wang, G. (July, January 26). Joint Learning of Convolutional Neural Networks and Temporally Constrained Metrics for Tracklet Association. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, NV, USA.","DOI":"10.1109\/CVPRW.2016.55"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"2054","DOI":"10.1109\/TPAMI.2015.2505309","article-title":"Multi-target tracking by discrete-continuous energy minimization","volume":"38","author":"Milan","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_50","first-page":"1","article-title":"Learning optimal parameters for multi-target tracking with contextual interactions","volume":"122","author":"Wang","year":"2016","journal-title":"Int. J. Comput. Vis."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Kieritz, H., Becker, S., Hubner, W., and Arens, M. (2016, January 23\u201326). Online multi-person tracking using Integral Channel Features. Proceedings of the 2016 13th IEEE International Conference on Advanced Video and Signal BasedSurveillance (AVSS), Colorado Springs, CO, USA.","DOI":"10.1109\/AVSS.2016.7738059"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Yoon, J.H., Lee, C.R., Yang, M.H., and Yoon, K.J. (July, January 26). Online multi-object tracking via structural constraint event aggregation. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.155"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Lenz, P., Geiger, A., and Urtasun, R. (2015, January 7\u201313). FollowMe: Efficient Online Min-Cost Flow Tracking with Bounded Memory and Computation. Proceedings of the IEEE International Conference on Computer Vision (ICCV) 2015, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.496"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Zhang, W., Zhou, H., Sun, S., Wang, Z., Shi, J., and Loy, C.C. (November, January 27). Robust multi-modality multi-object tracking. Proceedings of The IEEE International Conference on Computer Vision (ICCV) 2019, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00245"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Sharma, S., Ansari, J.A., Murthy, J.K., and Krishna, K.M. (2018, January 21\u201325). Beyond pixels: Leveraging geometry and shape cues for online multi-object tracking. Proceedings of the IEEE Conference on Robotics and Automation (ICRA) 2018, Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8461018"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/6\/1653\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:07:16Z","timestamp":1760173636000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/6\/1653"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,16]]},"references-count":55,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["s20061653"],"URL":"https:\/\/doi.org\/10.3390\/s20061653","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,16]]}}}