Forecasting time series with long short-term memory networks

Nguyen Quoc Dung * , Phan Nguyet Minh and Ivan Zelinka

* Corresponding author (nqdung@vanlanguni.edu.vn)

Article Sidebar

Full Text: PDF

Received: 04 Mar 2020
Revised: 25 Apr 2020
Accepted: 31 Jul 2020
Published: 31 Jul 2020
DOI: 10.22144/ctu.jen.2020.016
Views
379
Downloads
182
Crossref
Scopus
Google Scholar
Europe PMC
How to Cite
Nguyen, Q. D., Phan, N. M., & Ivan, Z. (2020). Forecasting time series with long short-term memory networks. CTU Journal of Innovation and Sustainable Development , 12(2), 53-59. https://doi.org/10.22144/ctu.jen.2020.016
Issue
Vol. 12 No. 2 (2020)
Section
Natural Sciences

Main Article Content

Abstract

Deep learning methods such as recurrent neural network and long short-term memory have attracted a great amount of attentions recently in many fields including computer vision, natural language processing and finance. Long short-term memory is a special type of recurrent neural network capable of predicting future values of sequential data by taking the past information into account. In this paper, the architectures of various long short-term memory networks are presented and the description of how they are used in sequence prediction is given. The models are evaluated based on the benchmark time series dataset. It is shown that the bidirectional architecture obtains the better results than the single and stacked architectures in both the experiments of different time series data categories and forecasting horizons. The three architectures perform well on the macro and demographic categories, and achieve average mean absolute percentage errors less than 18%. The long short-term memory models also show the better performance than most of the baseline models.
Keywords: Long short-term memory, recurrent neural network, sequence prediction, time series

Article Details

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

References

Box, G. and Jenkins, G., 1970. Time Series Analysis: Forecasting and Control. San Francisco: Holden-Day.

Cho, K., Merrienboer, B., Gulcehre, C., et al., 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. arXiv preprint. https://arxiv.org/abs/1406.1078v3

Gers, F.A., Schmidhuber, J. and Cummins, F., 1999. Learning to forget: continual prediction with LSTM. 9th International Conference on Artificial Neural Networks: ICANN '99, Edinburgh, UK, 850-855. https://doi.org/10.1049/cp:19991218

Gers, F.A., Schmidhuber, J. and Cummins, F., 2000. Learning to Forget: Continual Prediction with LSTM. Neural Computation, 12(10): 2451-2471. https://doi.org/10.1162/089976600300015015

Guo, T., Xu, Z., Yao, X., Chen, H., Aberer, K. and Funaya, K., 2016. Robust online time series prediction with recurrent neural networks. In: 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), Montreal, QC, Canada, pp. 816-825.

Hochreiter, S. and Schmidhuber, J., 1997. Long short-term memory. Neural Computation, 9(8): 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735

Hsu, D., 2017. Time series forecasting based on augmented long short-term memory. arXiv preprint https://arxiv.org/abs/1707.00666

Jurafsky, J. and Martin, J.H., 2019. An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition (third edition draft), accessed on 27 February 2020. Available from https://web.stanford.edu/~jurafsky/slp3/edbook_oct162019.pdf

Karpathy, A., 2015. The Unreasonable Effectiveness of Recurrent Neural Networks, accessed on 27 February 2020. Available from http://karpathy.github.io/2015/05/21/rnn-effectiveness/

Li, X. and Wu, X., 2015. Constructing long short-term memory based deep recurrent neural networks for large vocabulary speech recognition. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, QLD, pp. 4520-4524.

M3-Competition, 2000. The 3003 Time Series of The M3-Competition, accessed on 01 February 2020. Available from https://forecasters.org/resources/time-series-data/m3-competition/

Makridakis, S. and Hibon, M., 2000. The M3-Competition: results, conclusions and implications. International Journal of Forecasting, 16(4): 451-476. https://doi.org/10.1016/S0169-2070(00)00057-1

Makridakis, S., Spiliotis, E. and Assimakopoulos, V., 2018. The M4 Competition: Results, findings, conclusion and way forward. International Journal of Forecasting, 34 (4): 802-808. https://doi.org/10.1016/j.ijforecast.2018.06.001

Malhotra, P., Vig, L., Shroff, G. and Agarwal, P., 2015. Long short-term memory networks for anomaly detection in time series. In: ESANN 2015 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges (Belgium), pp. 89-94.

Olah, C., 2015. Understanding LSTM Networks, accessed on 27 February 2020. Available from https://colah.github.io/posts/2015-08-Understanding-LSTMs/

Rumelhart, D., Hinton, G. and Williams, R., 1986. Learning representations by back-propagating errors. Nature, 323: 533–536. https://doi.org/10.1038/323533a0

Sutskever, I. and Vinyals, O. and Le, Q. V., 2014. Sequence to Sequence Learning with Neural Networks. Advances in Neural Information Processing Systems, 27: 3104-3112.

Ullah, A., Ahmad, J., Muhammad, K., Sajjad, M. and Baik, S.W., 2017. Action Recognition in Video Sequences using Deep Bi-Directional LSTM with CNN Features. IEEE Access, 6: 1155-1166.

Vinyals, O., Toshev, A., Bengio, S. and Erhan, D., 2015. Show and Tell: A Neural Image Caption Generator. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3156-3164.