close
Skip to main content
Springer Nature Link
Log in

Learning-Based Stock Trending Prediction by Incorporating Technical Indicators and Social Media Sentiment

  • Published:
Save article
View saved research
Cognitive Computation Aims and scope Submit manuscript

Abstract

Stock trending prediction is a challenging task due to its dynamic and nonlinear characteristics. With the development of social platform and artificial intelligence (AI), incorporating timely news and social media information into stock trending models becomes possible. However, most of the existing works focus on classification or regression problems when predicting stock market trending without fully considering the effects of different influence factors in different phases. To address this gap, this research solves stock trending prediction problem utilizing both technical indicators and sentiments of the social media text as influence factors in different situations. A 3-phase hybrid model is proposed where daily sentiment values and technical indicators are considered when predicting the trends of the stocks. The proposed method leverages both traditional learning and deep learning methods as the core predictors in different phases. Accuracy and F1-score are used to evaluate the performance of the proposed method. Incorporating the technical indicators and social media sentiments, the performance of the proposed method with different learning-based methods as core predictors is analyzed and compared in different situations. Specifically, multi-layer perceptron (MLP), naïve bayes (NB), decision tree (DT), logistic regression (LR), random forest (RF), extreme gradient boosting (XGBoost), long short-term memory (LSTM), and convolutional neural networks (CNN) are leveraged as the core learning predictor module, with different combinations of the degree of involvement of technical and sentiment information. The result demonstrates the effectiveness of the proposed method with an accuracy of 73.41% and F1-score of 84.19%. The result also shows that various learning-based methods perform differently for the prediction of different stocks. This research not only demonstrates the merits of the proposed method, it also shows that integrating social opinions with technical indicators is a right direction for enhancing the performance of learning-based stock market trending analysis methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Data Availability

All data generated or analyzed during this study are included in this published article.

References

  1. Bollen J, Mao H, Zeng X. Twitter mood predicts the stock market. J Comput Sci. 2011;2(1):1–8.

    Article  Google Scholar 

  2. Patel J, Shah S, Thakkar P, Kotecha K. Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques. Expert Syst Appl. 2015;42(1):259–68.

    Article  Google Scholar 

  3. Ma Y, Mao R, Lin Q, Wu P, Cambria E. Multi-source aggregated classification for stock price movement prediction. Inf Fusion. 2023;91:515–28.

    Article  Google Scholar 

  4. Maini SS, Govinda K. Stock market prediction using data mining techniques. In: 2017 International Conference on Intelligent Sustainable Systems (ICISS) IEEE. 2017:654–61.

  5. Varfis A, Versino C. Univariate economic time series forecasting by connectionist methods. In: 1990 International Conference on Neural Networks (ICNN) IEEE. 1990:342–5.

  6. Rather AM, Agarwal A, Sastry V. Recurrent neural network and a hybrid model for prediction of stock returns. Expert Syst Appl. 2015;42(6):3234–41.

    Article  Google Scholar 

  7. Hafezi R, Shahrabi J, Hadavandi E. A bat-neural network multi-agent system (BNNMAS) for stock price prediction: case study of DAX stock price. Appl Soft Comput. 2015;29:196–210.

    Article  Google Scholar 

  8. Xiong L, Lu Y. Hybrid ARIMA-BPNN model for time series prediction of the Chinese stock market. In: 2017 3rd International Conference on Information Management (ICIM). IEEE; 2017. p. 93-7.

  9. Lee SW, Um JY. Stock fluctuation prediction method and server. Google Patents; 2019. US Patent 10,185,996.

  10. Kim KJ. Financial time series forecasting using support vector machines. Neurocomputing. 2003;55(1–2):307–19.

  11. Bharathi S, Geetha A. Sentiment analysis for effective stock market prediction. Int J Intell Eng Syst. 2017;10(3):146–54.

    Google Scholar 

  12. Ichinose K, Shimada K. Stock market prediction using keywords from expert articles. In: International Conference on Soft Computing and Data Mining. Springer. 2018:409–17.

  13. Zhang X, Zhang Y, Wang S, Yao Y, Fang B, Philip SY. Improving stock market prediction via heterogeneous information fusion. Knowl Based Syst. 2018;143:236–47.

    Article  Google Scholar 

  14. Si J, Mukherjee A, Liu B, Pan SJ, Li Q, Li H. Exploiting social relations and sentiment for stock prediction. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014:1139–45.

  15. Wang Z, Ho S-B, Lin Z. Stock market prediction analysis by incorporating social and news opinion and sentiment. In: 2018 IEEE International Conference on Data Mining Workshops (ICDMW) IEEE. 2018:1375–80.

  16. Nguyen TH, Shirai K, Velcin J. Sentiment analysis on social media for stock movement prediction. Expert Syst Appl. 2015;42(24):9603–11.

    Article  Google Scholar 

  17. Li B, Chan KC, Ou C, Ruifeng S. Discovering public sentiment in social media for predicting stock movement of publicly listed companies. Inform Syst. 2017;69:81–92.

    Article  Google Scholar 

  18. Hu H, Tang L, Zhang S, Wang H. Predicting the direction of stock markets using optimized neural networks with Google Trends. Neurocomputing. 2018;285:188–95.

    Article  Google Scholar 

  19. Hu Z. Crude oil price prediction using CEEMDAN and LSTM-attention with news sentiment index. Oil & Gas Science and Technology-Revue d’IFP Energies nouvelles. 2021;76:28.

    Article  Google Scholar 

  20. Malandri L, Xing FZ, Orsenigo C, Vercellis C, Cambria E. Public mood-driven asset allocation: the importance of financial sentiment in portfolio management. Cognit Comput. 2018;10(6):1167–76.

    Article  Google Scholar 

  21. Parray IR, Khurana SS, Kumar M, Altalbe AA. Time series data analysis of stock price movement using machine learning techniques. Soft Comput. 2020;24(21):16509–17.

    Article  Google Scholar 

  22. Dey PP, Nahar N, Hossain B. Forecasting stock market trend using machine learning algorithms with technical indicators. Int J Inform Technol Comput Sci. 2020;12(3):32–8.

    Google Scholar 

  23. Agrawal M, Shukla PK, Nair R, Nayyar A, Masud M. Stock prediction based on technical indicators using deep learning model. Comput Mater Continua. 2022;70(1):287–304.

    Article  Google Scholar 

  24. Li Y, Pan Y. A novel ensemble deep learning model for stock prediction based on stock prices and news. Int J Data Sci Anal. 2022;13(2):139–49.

    Article  Google Scholar 

  25. Picasso A, Merello S, Ma Y, Oneto L, Cambria E. Technical analysis and sentiment embeddings for market trend prediction. Expert Syst Appl. 2019;135:60–70.

    Article  Google Scholar 

  26. Fischer T, Krauss C. Deep learning with long short-term memory networks for financial market predictions. Eur J Oper Res. 2018;270(2):654–69.

    Article  MathSciNet  MATH  Google Scholar 

  27. Nelson DM, Pereira AC, de Oliveira RA, Stock market’s price movement prediction with LSTM neural networks. In,. International joint conference on neural networks (IJCNN). IEEE. 2017:1419–26.

  28. Stoean C, Paja W, Stoean R, Sandita A. Deep architectures for long-term stock price prediction with a heuristic-based strategy for trading simulations. PloS one. 2019;14(10).

    Article  Google Scholar 

  29. Kim T, Kim HY. Forecasting stock prices with a feature fusion LSTM-CNN model using different representations of the same data. PloS one. 2019;14(2).

    Article  Google Scholar 

  30. Sezer OB, Ozbayoglu AM. Financial trading model with stock bar chart image time series with deep convolutional neural networks. Intell Autom Soft Comput. 2020;26(2):323–34.

    Google Scholar 

  31. Huang W, Nakamori Y, Wang SY. Forecasting stock market movement direction with support vector machine. Comput Oper Res. 2005;32(10):2513–22.

    Article  MATH  Google Scholar 

  32. Naeini MP, Taremian H, Hashemi HB, Stock market value prediction using neural networks. International conference on computer information systems and industrial management applications (CISIM). IEEE. 2010:132–6.

  33. Huang TT, Chang CH. Intelligent stock selecting via Bayesian naive classifiers on the hybrid use of scientific and humane attributes. In: 2008 Eighth International Conference on Intelligent Systems Design and Applications. vol.1. IEEE. 2008:617–21.

  34. Wang Z, Jiao R, Jiang H. Emotion recognition using WT-SVM in human-computer interaction. J New Media. 2020;2(3):121.

    Article  Google Scholar 

  35. Henrique BM, Sobreiro VA, Kimura H. Stock price prediction using support vector regression on daily and up to the minute prices. J Finance Data Sci. 2018;4(3):183–201.

    Article  Google Scholar 

  36. Marković I, Stojanović M, Stanković J, Stanković M. Stock market trend prediction using AHP and weighted kernel LS-SVM. Soft Comput. 2017;21(18):5387–98.

    Article  Google Scholar 

  37. Anitescu C, Atroshchenko E, Alajlan N, Rabczuk T. Artificial neural network methods for the solution of second order boundary value problems. Comput Mater Continua. 2019;59(1):345–59.

    Article  Google Scholar 

  38. Göçken M, Özçalıcı M, Boru A, Dosdoğru AT. Integrating metaheuristics and artificial neural networks for improved stock price prediction. Expert Syst Appl. 2016;44:320–31.

    Article  Google Scholar 

  39. Zhu K, Zhang N, Ying S, Wang X. Within-project and cross-project software defect prediction based on improved transfer naive Bayes algorithm. Comput Mater Continua. 2020;63(2):891–910.

    Google Scholar 

  40. Khaidem L, Saha S, Dey SR. Predicting the direction of stock market prices using random forest. arXiv preprint http://arxiv.org/abs/1605.00003. 2016.

  41. Chen T, Guestrin C. XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining 2016:785–94.

  42. Wang Z, Chong CS, Lan L, Yang Y, Ho S-B, Tong JC, Fine-grained sentiment analysis of social media with emotion sensing. In Future Technologies Conference (FTC). IEEE. 2016:1361–4.

  43. Wang Z, Ho S-B, Cambria E. A review of emotion sensing: categorization models and algorithms. Multimed Tools Appl. 2020;79:35553–82.

    Article  Google Scholar 

  44. Xing FZ, Cambria E, Welsch RE. Natural language based financial forecasting: a survey. Artif Intell Rev. 2018;50(1):49–73.

    Article  Google Scholar 

  45. Hu Z, Wang Z, Ho S-B, Tan A-H. Stock market trend forecasting based on multiple textual features: a deep learning method. In: 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI). IEEE 2021:1002–7.

  46. Xing FZ, Cambria E, Zhang Y. Sentiment-aware volatility forecasting. Knowledge-Based Syst. 2019;176:68–76.

    Article  Google Scholar 

  47. Merello S, Ratto AP, Oneto L, Cambria E. Ensemble application of transfer learning and sample weighting for stock market prediction. In: 2019 International Joint Conference on Neural Networks (IJCNN). IEEE. 2019:1–8.

  48. Gupta R, Chen M. Sentiment analysis for stock price prediction. In: 2020 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR). IEEE. 2020:213–8.

  49. Khan W, Malik U, Ghazanfar MA, Azam MA, Alyoubi KH, Alfakeeh AS. Predicting stock market trends using machine learning algorithms via public sentiment and political situation analysis. Soft Comput 2019:1–25.

  50. Cambria E, Liu Q, Decherchi S, Xing F, Kwok K. SenticNet 7: a commonsense-based neurosymbolic AI framework for explainable sentiment analysis. In: LREC 2022: 3829–39.

Download references

Author information

Authors and Affiliations

  1. School of Computing and Information Systems, Singapore Management University, 80 Stamford Road, Singapore, 178902, Singapore

    Zhaoxia Wang

  2. School of Information Management and Engineering, Shanghai University of Finance and Economics, 777 Guoding Road, Shanghai, 200433, China

    Zhenda Hu

  3. School of Computer Science and Engineering, Nanyang Technological University, 50 Nanyang Avenue, Singapore, 639798, Singapore

    Fang Li & Erik Cambria

  4. Social and Cognitive Computing Department, Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), 1 Fusionopolis Way, Singapore, 138632, Singapore

    Seng-Beng Ho

Authors
  1. Zhaoxia Wang
    View author publications

    Search author on:PubMed Google Scholar

  2. Zhenda Hu
    View author publications

    Search author on:PubMed Google Scholar

  3. Fang Li
    View author publications

    Search author on:PubMed Google Scholar

  4. Seng-Beng Ho
    View author publications

    Search author on:PubMed Google Scholar

  5. Erik Cambria
    View author publications

    Search author on:PubMed Google Scholar

Contributions

Zhaoxia Wang: conceptualization, methodology, supervision, data curation, software design, visualization, writing — original draft, writing — review and editing. Zhenda Hu: investigation, formal analysis, software testing, visualization, validation, writing — review and editing. Fang LI: investigation, data curation, software development, writing — original draft. Seng-Beng HO: conceptualization, methodology, supervision, writing — original draft, writing — review and editing. Erik Cambria: data curation, investigation, writing — review and editing.

Corresponding author

Correspondence to Zhaoxia Wang.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Hu, Z., Li, F. et al. Learning-Based Stock Trending Prediction by Incorporating Technical Indicators and Social Media Sentiment. Cogn Comput 15, 1092–1102 (2023). https://doi.org/10.1007/s12559-023-10125-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1007/s12559-023-10125-8

Keywords

Profiles

  1. Erik Cambria View author profile