close
Skip to main content
Springer Nature Link
Log in

Enhancing phishing email detection with stylometric features and classifier stacking

  • Regular Contribution
  • Published:
Save article
View saved research
International Journal of Information Security Aims and scope Submit manuscript

Abstract

Phishing is the most common and potentially dangerous cyber attack that organizations are forced to deal with on a constant basis, rendering its automated detection as early as possible a necessity to ensure the security of computer systems. Focusing on the email level, this work improves content-based phishing email detection by integrating stylometric features with the commonly-used vectorization techniques, as well as by utilizing classifier stacking. Leveraging a diverse set of stylometric features, we systematically explore different methods of combining them with vectorized text as well as multiple stacking configurations for the machine learning algorithms. Our findings demonstrate that the proposed methods consistently outperform vectorization-only baselines on an imbalanced dataset, with a smaller improvement to a balanced one. Specifically, we achieved an \(F_1\) measure of 0.9843 on the balanced set and 0.9656 on the imbalanced one by stacking multiple different classifiers that were trained on the content and stylometric features separately, improving baselines by more than 2.2% for the imbalanced dataset. As such, our work contributes to the ongoing efforts in cybersecurity by further enhancing the performance of state-of-the-art phishing email detection systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Data availibility

All of the data used are publicly available at https://monkey.org/~jose/phishing/ and https://www.cs.cmu.edu/~./enron/

Notes

  1. In cases where there are no references for a feature in the table, the feature was proposed by us.

  2. The list used is Python’s string.punctuation which can be referenced in the official documentation: https://docs.python.org/3/library/string.html#string.punctuation

  3. https://languagetool.org/

  4. https://spamassassin.apache.org/

References

  1. Radicati Group: Email Statistics Report 2022-2026 Executive Summary. Tech. rep., https://www.radicati.com/wp/wp-content/uploads/2022/11/Email-Statistics-Report-2022-2026-Executive-Summary.pdf (2022)

  2. Federal Bureau of Investigation. 2022 Internet Crime Report. Tech. rep., (2022). https://www.ic3.gov/Media/PDF/AnnualReport/2022_IC3Report.pdf

  3. Anti-Phishing Working Group. APWG Phishing Activity Trends Report 4th Quarter 2022. Tech. rep., Publications Office of the European Union, Luxembourg (2022). https://docs.apwg.org/reports/apwg_trends_report_q4_2022.pdf

  4. Anti-Phishing Working Group. APWG Phishing Activity Trends Report 2nd Quarter 2023. Tech. rep., Publications Office of the European Union, Luxembourg (2023). https://docs.apwg.org/reports/apwg_trends_report_q2_2023.pdf

  5. Das, A., Baki, S., Aassal, A.E., Verma, R.M., Dunbar, A.: SoK: a comprehensive reexamination of phishing research from the security perspective. IEEE Commun. Surv. Tutor. 22(1), 671–708 (2020). https://doi.org/10.1109/COMST.2019.2957750

    Article  Google Scholar 

  6. Abdillah, R., Shukur, Z., Mohd, M., Murah, M.Z.: Phishing classification techniques: a systematic literature review. IEEE Access 10, 41574–41591 (2022). https://doi.org/10.1109/ACCESS.2022.3166474

    Article  Google Scholar 

  7. Aleroud, A., Karabatis, G.: Bypassing detection of url-based phishing attacks using generative adversarial deep neural networks. In: Verma RM, Khan L, Mohan CK (eds) IWSPA@CODASPY ’20: Proceedings of the 6th International Workshop on Security and Privacy Analytics, New Orleans, LA, USA, March 18, 2020, ACM, pp. 53–60, (2020), https://doi.org/10.1145/3375708.3380315

  8. Almomani, A., Gupta, B.B., Atawneh, S., Meulenberg, A., Almomani, E.: A survey of phishing email filtering techniques. IEEE Commun. Surv. Tutor. 15(4), 2070–2090 (2013). https://doi.org/10.1109/SURV.2013.030713.00020

    Article  MATH  Google Scholar 

  9. Chandrasekaran, M., Narayanan, K., Upadhyaya, S.: Phishing email detection based on structural properties. In: New York State Cyber Security Conference, Albany, NY, USA, (2006)

  10. Fette, I., Sadeh, N.M., Tomasic, A.: Learning to detect phishing emails. In: Williamson CL, Zurko ME, Patel-Schneider PF, Shenoy PJ (eds) Proceedings of the 16th International Conference on World Wide Web, WWW 2007, Banff, Alberta, Canada, May 8-12, 2007, ACM, pp. 649–656, (2007). https://doi.org/10.1145/1242572.1242660

  11. Bergholz, A., Beer, J.D., Glahn, S., Moens, M., Paaß, G., Strobel, S.: New filtering approaches for phishing email. J Comput. Secur. 18(1), 7–35 (2010). https://doi.org/10.3233/JCS-2010-0371

    Article  Google Scholar 

  12. Moradpoor, N., Clavie, B., Buchanan, B.: Employing machine learning techniques for detection and classification of phishing emails. In: 2017 Computing Conference, pp. 149–156, (2017). https://doi.org/10.1109/SAI.2017.8252096

  13. Nguyen, M., Nguyen, T., Nguyen, T.H.: A deep learning model with hierarchical lstms and supervised attention for anti-phishing, (2018). CoRR abs/1805.01554, arXiv:1805.01554

  14. Fang, Y., Zhang, C., Huang, C., Liu, L., Yang, Y.: Phishing email detection using improved RCNN model with multilevel vectors and attention mechanism. IEEE Access 7, 56329–56340 (2019). https://doi.org/10.1109/ACCESS.2019.2913705

    Article  Google Scholar 

  15. Bountakas, P., Koutroumpouchos, K., Xenakis, C.: A comparison of natural language processing and machine learning methods for phishing email detection. In: Reinhardt D, Müller T (eds) ARES 2021: The 16th International Conference on Availability, Reliability and Security, Vienna, Austria, August 17-20, 2021, ACM, pp. 127:1–127:12, (2021). https://doi.org/10.1145/3465481.3469205

  16. Lee, J., Tang, F., Ye, P., Abbasi, F., Hay, P., Divakaran, D.M.: D-fence: A flexible, efficient, and comprehensive phishing email detection system. In: IEEE European Symposium on Security and Privacy, EuroS &P 2021, Vienna, Austria, September 6-10, 2021, IEEE, pp. 578–597, (2021). https://doi.org/10.1109/EUROSP51992.2021.00045

  17. Abdallah, E.E., Abdallah, A.E., Bsoul, M., Otoom, A.F., Al-Daoud, E.: Simplified features for email authorship identification. Int. J. Secur. Netw. 8(2), 72–81 (2013). https://doi.org/10.1504/IJSN.2013.055941

    Article  MATH  Google Scholar 

  18. Shams, R., Mercer, R.E.: Supervised classification of spam emails with natural language stylometry. Neural Comput. Appl. 27(8), 2315–2331 (2016). https://doi.org/10.1007/S00521-015-2069-7

    Article  MATH  Google Scholar 

  19. Dewan, P., Kashyap, A., Kumaraguru, P.: Analyzing social and stylometric features to identify spear phishing emails. In: 2014 APWG Symposium on Electronic Crime Research (eCrime), pp. 1–13, (2014). https://doi.org/10.1109/ECRIME.2014.6963160

  20. Duman, S., Kalkan-Cakmakci, K., Egele, M., Robertson, W.K., Kirda, E.: Emailprofiler: Spearphishing filtering with header and stylometric features of emails. In: 40th IEEE Annual Computer Software and Applications Conference, COMPSAC 2016, Atlanta, GA, USA, June 10-14, 2016, IEEE Computer Society, pp. 408–416, (2016). https://doi.org/10.1109/COMPSAC.2016.105

  21. Islam, M.R., Abawajy, J.H.: A multi-tier phishing detection and filtering approach. J. Netw. Comput. Appl. 36(1), 324–335 (2013). https://doi.org/10.1016/J.JNCA.2012.05.009

    Article  MATH  Google Scholar 

  22. Shyni, C.E., Sarju, S., Swamynathan, S.: A multi-classifier based prediction model for phishing emails detection using topic modelling, named entity recognition and image processing. Circuits Syst. 07(09), 2507–2520 (2016). https://doi.org/10.4236/cs.2016.79217

    Article  Google Scholar 

  23. Bird, S., Loper, E., Klein, E.: Natural language processing with Python. “ O’Reilly Media, Inc.” (2009)

  24. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Bengio Y, LeCun Y (eds) 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings, (2013a). http://arxiv.org/abs/1301.3781

  25. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Burges CJC, Bottou L, Ghahramani Z, Weinberger KQ (eds) Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pp. 3111–3119, (2013b). https://proceedings.neurips.cc/paper/2013/hash/9aa42b31882ec039965f3c4923ce901b-Abstract.html

  26. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Burstein J, Doran C, Solorio T (eds) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), Association for Computational Linguistics, pp. 4171–4186, (2019). https://doi.org/10.18653/V1/N19-1423

  27. Toolan, F., Carthy, J.: Feature selection for spam and phishing detection. In: 2010 eCrime Researchers Summit, eCrime 2010, Dallas, TX, USA, October 18-20, 2010, IEEE, pp. 1–12, (2010). https://doi.org/10.1109/ECRIME.2010.5706696

  28. Fourkioti, O., Symeonidis, S., Arampatzis, A.: Language models and fusion for authorship attribution. Inf. Process Manag. 56(6), 102061 (2019). https://doi.org/10.1016/J.IPM.2019.102061

    Article  MATH  Google Scholar 

  29. McKinney, Wes: Data Structures for Statistical Computing in Python. In: Stéfan van der Walt, Jarrod Millman (eds) Proceedings of the 9th Python in Science Conference, pp. 56–61, (2010). https://doi.org/10.25080/Majora-92bf1922-00a

  30. Buitinck, L., Louppe, G., Blondel, M., Pedregosa, F., Mueller, A., Grisel, O., Niculae, V., Prettenhofer, P., Gramfort, A., Grobler, J., Layton, R., VanderPlas, J., Joly, A., Holt, B., Varoquaux, G.: API design for machine learning software: experiences from the scikit-learn project. In: ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pp. 108–122, (2013)

  31. Řehůřek, R., Sojka, P.: Software Framework for Topic Modelling with Large Corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, ELRA, Valletta, Malta, pp. 45–50, (2010)

  32. Cohen, W.W.: Enron Email Dataset. (2015), https://www.cs.cmu.edu/~./enron/, accessed: 2022-03-16

  33. Nazario, J.: Phishing Email Corpus, (2005-2022). https://monkey.org/~jose/phishing/README.txt, accessed: 2022-03-16

  34. Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput. 10(7), 1895–1923 (1998). https://doi.org/10.1162/089976698300017197

    Article  MATH  Google Scholar 

  35. Halgas, L., Agrafiotis, I., Nurse, J.R.C.: Catching the phish: Detecting phishing attacks using recurrent neural networks (rnns). In: You I (ed) Information Security Applications - 20th International Conference, WISA 2019, Jeju Island, South Korea, August 21-24, 2019, Revised Selected Papers, Springer, Lecture Notes in Computer Science, vol 11897, pp. 219–233, (2019). https://doi.org/10.1007/978-3-030-39303-8_17

Download references

Author information

Authors and Affiliations

  1. Department of Electrical & Computer Engineering, Democritus University of Thrace, Xanthi, Greece

    Ilias Chanis & Avi Arampatzis

Authors
  1. Ilias Chanis
    View author publications

    Search author on:PubMed Google Scholar

  2. Avi Arampatzis
    View author publications

    Search author on:PubMed Google Scholar

Contributions

Ilias Chanis: Conceptualization, Methodology, Data curation, Formal analysis, Investigation, Resources, Software, Validation, Writing - original draft, Writing - review & editing; Avi Arampatzis: Conceptualization, Methodology, Project administration, Supervision, Validation, Writing - review & editing

Corresponding author

Correspondence to Ilias Chanis.

Ethics declarations

Conflict of interest

The authors have no competing financial or non-financial interests to declare that are relevant to the content of this article.

Human participants

No funding was received for conducting this study. The article does not contain any studies that involve human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chanis, I., Arampatzis, A. Enhancing phishing email detection with stylometric features and classifier stacking. Int. J. Inf. Secur. 24, 15 (2025). https://doi.org/10.1007/s10207-024-00928-7

Download citation

  • Published:

  • Version of record:

  • DOI: https://doi.org/10.1007/s10207-024-00928-7

Keywords