close
Skip to main content
Log in

Fully automated 2D and 3D convolutional neural networks pipeline for video segmentation and myocardial infarction detection in echocardiography

  • 1218: Engineering Tools and Applications in Medical Imaging
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Myocardial infarction (MI) is a life-threatening disorder that occurs due to a prolonged limitation of blood supply to the heart muscles, and which requires an immediate diagnosis to prevent death. To detect MI, cardiologists utilize in particular echocardiography, which is a non-invasive cardiac imaging that generates real-time visualization of the heart chambers and the motion of the heart walls. These videos enable cardiologists to identify almost immediately regional wall motion abnormalities (RWMA) of the left ventricle (LV) chamber, which are highly correlated with MI. However, data acquisition is usually performed during emergency which results in poor-quality and noisy data that can affect the accuracy of the diagnosis. To address the identified problems, we propose in this paper an innovative, real-time and fully automated model based on convolutional neural networks (CNN) to early detect MI in a patient’s echocardiography. Our model is a pipeline consisting of a 2D CNN that performs data preprocessing by segmenting the LV chamber from the apical four-chamber (A4C) view, followed by a 3D CNN that performs a binary classification to detect MI. The pipeline was trained and tested on the HMC-QU dataset consisting of 162 echocardiography. The 2D CNN achieved 97.18% accuracy on data segmentation, and the 3D CNN achieved 90.9% accuracy, 100% precision, 95% recall, and 97.2% F1 score. Our detection results outperformed existing state-of-the-art models that were tested on the HMC-QU dataset for MI detection. This work demonstrates that developing a fully automated system for LV segmentation and MI detection is efficient and propitious.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. FESC MD, Lüscher TF (2015) Myocardial infarction: mechanisms, diagnosis, and complications. European Heart Journal 36(16):947–949

    Article  Google Scholar 

  2. Hubbard J (2003) Complications associated with myocardial infarction. Nursing Times 99(15):28–29

    Google Scholar 

  3. Laslett LJ, Alagona P, Clark BA, Drozda JP, Saldivar F, Wilson SR, Poe C, Hart M (2012) The worldwide environment of cardiovascular disease: Prevalence, diagnosis, therapy, and policy issues. Journal of the American College of Cardiology 60(25–Supplement):S1–S49

    Article  Google Scholar 

  4. Thygesen K, Alpert JS, Jaffe AS, Chaitman BR, Bax JJ, Morrow DA, White HD (2018) Fourth universal definition of myocardial infarction (2018). Journal of the American College of Cardiology 72:2231–2264

    Article  Google Scholar 

  5. Kosmidou I, Redfors B, Selker HP, Thiele H, Patel MR, Udelson JE et al (2017) Infarct size, left ventricular function, and prognosis in women compared to men after primary percutaneous coronary intervention in st-segment elevation myocardial infarction. European Heart Journal 38(21):1656–1663

    Article  Google Scholar 

  6. Hernandez AF, Velazquez EJ, Solomon SD, Kilaru R, Diaz R, O’Connor CM, Ertl G, Maggioni AP, Rouleau J-L, van Gilst W, Pfeffer MA, Califf RM (2005) Left Ventricular Assessment in Myocardial Infarction: The VALIANT Registry. Archives of Internal Medicine 165(18):2162–2169

    Article  Google Scholar 

  7. Palmieri V, Okin PM, Bella JN, Gerdts E, Wachtell K, Gardin J, Papademetriou V, Nieminen MS, Dahlöf B, Devereux RB (2003) Echocardiographic wall motion abnormalities in hypertensive patients with electrocardiographic left ventricular hypertrophy. Hypertension 41(1):75–82

    Article  Google Scholar 

  8. Roberts R, Kleiman N (1994) Earlier diagnosis and treatment of acute myocardial infarction necessitates the need for a new diagnostic mind-set’. Circulation 89(2):872–81

    Article  Google Scholar 

  9. Van de Werf F, Ardissino W, Cokkinos DV, Fox KA, Julian D et al (2003) Management of acute myocardial infarction in patients presenting with st-segment elevation. European Heart Journal 22:28–66

    Article  Google Scholar 

  10. Dwivedi G, Chan KL, Friedrich MG, Beanlands RS (2013) Cardiovascular imaging: New directions in an evolving landscape. Canadian Journal of Cardiology 29(3):257–259

    Article  Google Scholar 

  11. Bax J, Delgado V (2017) Advanced imaging in valvular heart disease. Nature Reviews Cardiology 14:01

    Article  Google Scholar 

  12. Douglas PS, Cerqueira MD, Berman DS, Chinnaiyan K, Cohen MS, Lundbye JB et al (2016) The future of cardiac imaging: Report of a think tank convened by the american college of cardiology. JACC: Cardiovascular Imaging 9(10):1211–1223

    Google Scholar 

  13. Sevakula RK, Au-Yeung WTM, Singh JP, Heist EK, Isselbacher EM, Armoundas AA (2020) State-of-the-art machine learning techniques aiming to improve patient outcomes pertaining to the cardiovascular system. Journal of the American Heart Association 9(4):e013924

    Article  Google Scholar 

  14. Gottdiener JS, Bednarz J, Devereux R, Gardin J, Klein A, Manning WJ, Morehead A, Kitzman D, Oh J, Quinones M, Schiller NB, Stein JH, Weissman NJ (2004) American society of echocardiography recommendations for use of echocardiography in clinical trials. Journal of the American Society of Echocardiography 17(10):1086–1119

    Google Scholar 

  15. Horowitz RS, Morganroth J, Parrotto C, Chen CC, Soffer J, Pauletto FJ (1982) Immediate diagnosis of acute myocardial infarction by two-dimensional echocardiography. Circulation 65(2):323–329

    Article  Google Scholar 

  16. Kiranyaz S, Degerli A, Hamid T, Mazhar R, Ahmed REF, Abouhasera R, Zabihi M, Malik J, Hamila R, Gabbouj M (2020) Left ventricular wall motion estimation by active polynomials for acute myocardial infarction detection. IEEE Access 8:210301–210317

    Article  Google Scholar 

  17. Gahungu N, Trueick R, Bhat S, Sengupta PP, Dwivedi G (2020) Current challenges and recent updates in artificial intelligence and echocardiography. Current Cardiovascular Imaging Reports 13(2):1–2

    Article  Google Scholar 

  18. Cao C, Liu C, Tan H, Song D, Shu W, Li W, Zhou Y, Bo X, Xie Z (2018) Deep learning and its applications in biomedicine. Genomics, Proteomics & Bioinformatics 16:17–32

    Article  Google Scholar 

  19. McBee MP, Awan OA, Colucci AT, Ghobadi CW, Kadom N, Kansagra AP, Tridandapani S, Auffermann WF (2018) Deep learning in radiology. Academic Radiology 25(11):1472–1480

    Article  Google Scholar 

  20. Davenport T, Kalakota R, Cokkinos DV (2019) The potential for artificial intelligence in healthcare. Future Healthcare Journal 6(2):94–98

    Article  Google Scholar 

  21. Chen M, Fang L, Zhuang Q, Liu H (2019) Deep learning assessment of myocardial infarction from mr image sequences. IEEE Access :1–1

  22. Narula S, Shameer K, Salem Omar AM, Dudley JT, Sengupta PP (2016) Machine-learning algorithms to automate morphological and functional assessments in 2d echocardiography. Journal of the American College of Cardiology 68(21):2287–2295

    Article  Google Scholar 

  23. Kiranyaz S, Zabihi M, Rad AB, Ince T, Hamila R, Gabbouj M (2020) Real-time phonocardiogram anomaly detection by adaptive 1d convolutional neural networks. Neurocomputing 411:291–301

    Article  Google Scholar 

  24. Mishra A, Dutta P, Ghosh MK (2003) A ga based approach for boundary detection of left ventricle with echocardiographic image sequences. Image and Vision Computing 21:967–976

    Article  Google Scholar 

  25. Zhu Y, Drangova M, Pelc NJ (1996) Fourier tracking of myocardial motion using cine-pc data. Magnetic Resonance in Medicine 35(4):471–480

    Article  Google Scholar 

  26. Wu CC, Hsu WD, Islam MM, Poly TN, Yang HC, Nguyen PAA, Wang YC, Li YCJ (2019) An artificial intelligence approach to early predict non-st-elevation myocardial infarction patients with chest pain. Computer Methods and Programs in Biomedicine 173:109–117

    Article  Google Scholar 

  27. Upendra KET, Ranaweera GAC, Samaradiwakara NHAP, Munasinghe A, Jayaratne KL, Wickramasinghe MIE (2018) Artificial neural network application in classifying the left ventricular function of the human heart using echocardiography. In: 2018 National Information Technology Conference (NITC). pp 1–6

  28. Klein HO, Tordjman T, Ninio R, Sareli P, Oren V, Lang R, Gefen J, Pauzner C, Di Segni E, David D, Kaplinsky E (1983) The early recognition of right ventricular infarction: diagnostic accuracy of the electrocardiographic v4r lead. Circulation 67(3):558–565

    Article  Google Scholar 

  29. O’Boyle JE, Parisi AF, Nieminen M, Kloner RA, Khuri S (1983) Quantitative detection of regional left ventricular contraction abnormalities by 2-dimensional echocardiography. The American Journal of Cardiology 51(10):1732–1738

    Article  Google Scholar 

  30. Wharton G, Steeds R, Rana B, Wheeler R, Smith N, Oxborough D, Brewerton H, Allen J, Chambers J, Sandoval J, Lloyd G, Kanagala P, Matthew T, Massani N, Jones R (2015) A minimum dataset for a standard transthoracic echocardiogram, from the british society of echocardiography education committee. Echo Research and Practice 2:03

    Article  Google Scholar 

  31. Kurt M, Shaikh K, Peterson L, Kurrelmeyer K, Shah G, Nagueh S, Fromm R, Quinones M, Zoghbi W (2009) Impact of contrast echocardiography on evaluation of ventricular function and clinical management in a large prospective cohort. Journal of the American College of Cardiology 53:802–10

    Article  Google Scholar 

  32. Neskovic AN, Hagendorff A, Lancellotti P, Guarracino F, Varga A, Cosyns B, Flachskampf FA, Popescu BA et al (2013) Emergency echocardiography: the European Association of Cardiovascular Imaging recommendations. European Heart Journal - Cardiovascular Imaging 14(1):1–11

    Article  Google Scholar 

  33. Nagata Y, Kado Y, Onoue T, Otani K, Nakazono K, Otsuji Y, Takeuchi M (2018) Impact of image quality on reliability of the measurements of left ventricular systolic function and global longitudinal strain in 2d echocardiography. Echo Research and Practice 5:27–39

    Article  Google Scholar 

  34. Qazi M, Fung G, Krishnan S, Rosales R, Steck H, Rao RB, Poldermans D, Chandrasekaran D (2007) Automated heart wall motion abnormality detection from ultrasound images using bayesian networks. pp 519–525

  35. Degerli A, Zabihi M, Kiranyaz S, Hamid T, Mazhar R, Hamila R, Gabbouj M Hmc-qu dataset

  36. Hamad medical corporation. https://www.hamad.qa/EN/Pages/default.aspx. Accessed 24 Sept 2020

  37. Wael M, Ibrahim E-S, Fahmy AS (2015) Detection of lv function abnormality using temporal patterns of normalized wall thickness. Journal of Cardiovascular Magnetic Resonance 17:47

    Article  Google Scholar 

  38. Moynihan PF, Parisi AF, Feldman CL (1981) Quantitative detection of regional left ventricular contraction abnormalities by two-dimensional echocardiography. i. analysis of methods. Circulation 63(4):752–760

    Article  Google Scholar 

  39. Ulloa A, Jing L, Good CW, David P. vanMaanen SR, Suever JD, et al. (2018 ) A deep neural network predicts survival after heart imaging better than cardiologists. arXiv:1811.10553

  40. Kusunose K, Abe T, Haga A, Fukuda D, Yamada H, Harada M, Sata M (2019) A deep learning approach for assessment of regional wall motion abnormality from echocardiographic images. JACC: Cardiovascular Imaging 13:374–381

    Google Scholar 

  41. Ohman EM, Casey C, Bengtson JR, Pryor D, Tormey W, Horgan JH (1990) Early detection of acute myocardial infarction: additional diagnostic information from serum concentrations of myoglobin in patients without st elevation. British heart journal 163(6):335–338

    Article  Google Scholar 

  42. Degerli A, Zabihi M, Kiranyaz S, Hamid T, Mazhar R, Hamila R, Gabbouj M (2021) Early detection of myocardial infarction in low-quality echocardiography. IEEE Access 9:34442–34453

    Article  Google Scholar 

  43. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(12):2481–2495

    Article  Google Scholar 

  44. Xing Y, Zhong L, Zhong X (2020) An encoder-decoder network based fcn architecture for semantic segmentation. Wireless Communications and Mobile Computing 2020:1–9

    Google Scholar 

  45. Alhassan V, Henry CJ, Ramanna S, Storie C (2020) A deep learning framework for land-use/land-cover mapping and analysis using multispectral satellite imagery. Neural Computing and Applications 32:8529–8544

    Article  Google Scholar 

  46. Dumoulin V, Visin F (2016) A guide to convolution arithmetic for deep learning. pp 18–23

  47. Refaeilzadeh P, Tang L, Liu H (2009) Cross-validation. pp 532–538

  48. Mannor S, Peleg D, Rubinstein R (2005) The cross entropy method for classification. pp 561–568

  49. Chollet F. et al (2015) Keras. https://keras.io

  50. Madani A, Ong J, Tibrewal A, Mofrad M (2018) Deep echocardiography: data-efficient supervised and semi-supervised deep learning towards automated diagnosis of cardiac disease. npj Digital Medicine, 1

  51. TensorRT cardiovascular diseases. https://developer.nvidia.com/tensorrt-getting-started. Accessed: 16 Feb 2021

Download references

Acknowledgements

The authors wish to acknowledge the valuable contribution of researchers at Medical Research Centre at Hamad Medical Corporation in the State of Qatar for the creation of this work and this publication.

Funding

The work of Sheela Ramanna and Christopher J. Henry was funded by the NSERC Discovery Grants Program (nos. 194376, 418413).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Oumaima Hamila.

Ethics declarations

Conflicts of interest

There is no conflict of interest with the funders.

Consent to participate

All the Authors consent to the content of the manuscript.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hamila, O., Ramanna, S., Henry, C.J. et al. Fully automated 2D and 3D convolutional neural networks pipeline for video segmentation and myocardial infarction detection in echocardiography. Multimed Tools Appl 81, 37417–37439 (2022). https://doi.org/10.1007/s11042-021-11579-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s11042-021-11579-4

Keywords

Profiles

  1. Tahir Hamid