Exploring the Impact of Magnitude- and Direction-based Loss Function on the Profitability using Predicted Prices from Deep Learning
Researches on predicting prices (as time series) from deep learning models usually use a magnitude-based error measurement (such as ). However, in trading, the error in the predicted direction could affect trading results much more than the magnitude error. Few works consider the impact of ill-predicted trading direction as part of the error measurement.
In this work, we first find parameter sets of LSTM and TCN models with low magnitude-based error measurement, and then calculate the profitability using program trading. Relationships between profitability and error measurements are analyzed.
We also propose a new loss function considering both directional and magnitude error for previous models for re-evaluation. Three commodities are tested: gold, soybean, and crude oil (from GLOBEX). Our findings are: with given parameter sets, if merchandise (gold and soybean) is of low averaged magnitude error, then its profitability is more stable. The proposed loss function can further improve profitability. If it is of larger magnitude error (crude oil), then its profitability is unstable, and the proposed loss function cannot improve nor stabilize the profitability.
Furthermore, the relationship between profitability and error measurement for models of LSTM and TCN with or without customized loss function is not, as commonly believed, highly positively correlated (i.e., the more precise the predicted value, the more trading profit) since the correlation coefficients are rarely higher than 0.5 in all our experiments. However, the customized loss functions perform better in TCN than in LSTM.
Bai, S., Kolter, J.Z., & Koltun, V. (2018). An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. Available at: https://arxiv.org/pdf/1803.01271.pdf.
Brownlee, J. (2019). Better deep learning. Australia: Machine learning Mastery.
Chollet, F. (2017). Deep learning with Python. USA: Manning Publications.
Davidson-Pilon, C. (2013). Computes the MEAN-ABSOLUTE SCALED ERROR forecast error for univariate time series prediction. Available at: https://github.com/CamDavidsonPilon/Python-Numerics/blob/master/TimeSeries/MASE.py. Accessed on 18 January 2019.
Greff, K., Srivastava, R. K., Koutnik, J., Steunebring, B. R., & Schmidhuber, J. (2017). LSTM: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems, 28(10), 2222–2232.
Hochreiter, S., Klambauer, G., Unterthiner, T., & Mayr, A. (2017). Self-normalizing neural networks. Proceedings of the NIPS 2017, Advances in Neural Information Processing Systems 30. Availabel at: https://papers.nips.cc/paper/6698-self-normalizing-neural-networks.pdf.
Hyndman, R.J., Koehler, A.B. (2006). Another look at measures of forecast accuracy. Proceedings of the International Journal of Forecasting, 22(4), 679-688. https://doi.org/10.1016/j.ijforecast.2006.03.001.
Krizhevsky, A., Sutskever, I., & Hinton, G.E. (2012). ImageNet classification with deep convolutional neural network. NIPS2012: Proceedings of the 25th International Conference on Neural Information Processing Systems, 1, 1097-1105.
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436-444.
Leinweber, D.J. (2007). Stupid data miner tricks: Overfitting the S&P 500. The Journal of Investing, 16(1), 15-22.
Li, H., Shen, Y., & Zhu, Y. (2018). Stock price prediction using attention-based multi-input LSTM. Proceedings of the 10th Asian Conference on Machine Learning (PMLR 95), pp. 454-469.
Multicharts. (2019). MultiCharts12. Available at: https://www.multicharts.com/. Accessed on 18 June 2019.
NVDIA. (2018). cuDNN developer guide. Available at: https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html. Aaccessed on 2 October 2019.
Pant, N. (2017). A guide for time series prediction using recurrent neural networks(LSTMS). Available at: https://blog.statsbot.co/time-series-prediction-using-recurrent-neural-networks-lstms-807fa6ca7f. Accessed on 6 September 2018.
Pardo, R. (2008). The evaluation and optimization of trading strategies. (2nd ed.) USA: John Wiley.
Schoneburg, E. (1990). Stock price prediction using neural networks: A project report. Proceedings of the Neurocomputing 2, 17-27.
Walter, J., Ritter, H., & Schulten, K. (1990). Non-linear Prediction with Self-organizing Maps. Proceedings of the IJCNN International Joint Conference on Neural Networks, pp. 17-21.
Copyright (c) 2020 International Journal of Engineering and Management Research
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.