Nguyen Quoc Dung * , Phan Nguyet Minh and Ivan Zelinka

* Correspondence: Nguyen Quoc Dung (email:

Main Article Content


Deep learning methods such as recurrent neural network and long short-term memory have attracted a great amount of attentions recently in many fields including computer vision, natural language processing and finance. Long short-term memory is a special type of recurrent neural network capable of predicting future values of sequential data by taking the past information into account. In this paper, the architectures of various long short-term memory networks are presented and the description of how they are used in sequence prediction is given. The models are evaluated based on the benchmark time series dataset. It is shown that the bidirectional architecture obtains the better results than the single and stacked architectures in both the experiments of different time series data categories and forecasting horizons. The three architectures perform well on the macro and demographic categories, and achieve average mean absolute percentage errors less than 18%. The long short-term memory models also show the better performance than most of the baseline models.
Keywords: Long short-term memory, recurrent neural network, sequence prediction, time series

Article Details


Box, G. and Jenkins, G., 1970. Time Series Analysis: Forecasting and Control. San Francisco: Holden-Day.

Cho, K., Merrienboer, B., Gulcehre, C., et al., 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. arXiv preprint.

Gers, F.A., Schmidhuber, J. and Cummins, F., 1999. Learning to forget: continual prediction with LSTM. 9th International Conference on Artificial Neural Networks: ICANN '99, Edinburgh, UK, 850-855.

Gers, F.A., Schmidhuber, J. and Cummins, F., 2000. Learning to Forget: Continual Prediction with LSTM. Neural Computation, 12(10): 2451-2471.

Guo, T., Xu, Z., Yao, X., Chen, H., Aberer, K. and Funaya, K., 2016. Robust online time series prediction with recurrent neural networks. In: 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), Montreal, QC, Canada, pp. 816-825.

Hochreiter, S. and Schmidhuber, J., 1997. Long short-term memory. Neural Computation, 9(8): 1735-1780.

Hsu, D., 2017. Time series forecasting based on augmented long short-term memory. arXiv preprint

Jurafsky, J. and Martin, J.H., 2019. An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition (third edition draft), accessed on 27 February 2020. Available from

Karpathy, A., 2015. The Unreasonable Effectiveness of Recurrent Neural Networks, accessed on 27 February 2020. Available from

Li, X. and Wu, X., 2015. Constructing long short-term memory based deep recurrent neural networks for large vocabulary speech recognition. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, QLD, pp. 4520-4524.

M3-Competition, 2000. The 3003 Time Series of The M3-Competition, accessed on 01 February 2020. Available from

Makridakis, S. and Hibon, M., 2000. The M3-Competition: results, conclusions and implications. International Journal of Forecasting, 16(4): 451-476.

Makridakis, S., Spiliotis, E. and Assimakopoulos, V., 2018. The M4 Competition: Results, findings, conclusion and way forward. International Journal of Forecasting, 34 (4): 802-808.

Malhotra, P., Vig, L., Shroff, G. and Agarwal, P., 2015. Long short-term memory networks for anomaly detection in time series. In: ESANN 2015 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges (Belgium), pp. 89-94.

Olah, C., 2015. Understanding LSTM Networks, accessed on 27 February 2020. Available from

Rumelhart, D., Hinton, G. and Williams, R., 1986. Learning representations by back-propagating errors. Nature, 323: 533–536.

Sutskever, I. and Vinyals, O. and Le, Q. V., 2014. Sequence to Sequence Learning with Neural Networks. Advances in Neural Information Processing Systems, 27: 3104-3112.

Ullah, A., Ahmad, J., Muhammad, K., Sajjad, M. and Baik, S.W., 2017. Action Recognition in Video Sequences using Deep Bi-Directional LSTM with CNN Features. IEEE Access, 6: 1155-1166.

Vinyals, O., Toshev, A., Bengio, S. and Erhan, D., 2015. Show and Tell: A Neural Image Caption Generator. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3156-3164.