It has been a hot and challenging topic to predict the chaotic time series in the medium-to-long term. We combine autoencoders and convolutional neural networks (AE-CNN) to capture the intrinsic certainty of chaotic time series. We utilize the transfer learning (TL) theory to improve the prediction performance in medium-to-long term. Thus, we develop a prediction scheme for chaotic time series-based AE-CNN and TL named AE-CNN-TL. Our experimental results show that the proposed AE-CNN-TL has much better prediction performance than any one of the following: AE-CNN, ARMA, and LSTM.

Chaos is the unity of extrinsic randomness and intrinsic certainty, which widely exists in social fields and natural systems. However, in many cases, we cannot represent chaotic systems by establishing analytical mathematical models but by observing the time series [

The most significant characteristic of a chaotic system is its initial sensitivity; that is, a slight change in input will produce random and unpredictable difference after long enough evolution, which determines the difficulty in medium-to-long-term prediction for a chaotic system. Similarly, the error in chaotic time series may be amplified to an unacceptable degree due to its initial sensitivity, which will lead to the failure of prediction. However, the orbit of the chaotic system also has intrinsic certainty such as self-similarity, which indicates that a chaotic system can be predicted to a certain extent, even in the medium-to-long term by using fractal geometry and other related theories. But the self-similarity and other intrinsic certainty of a chaotic system are very difficult to be measured accurately. So far, there is no commonly accepted theory that can be employed to accurately measure the intrinsic certainty such as self-similarity and realize the medium-to-long-term prediction of a chaotic system. Inspired by Xia et al. [

To solve abovementioned difficulties, we will mainly implement the following contributions:

Employ a convolutional neural network (CNN) to extract high-order features of a chaotic time series for medium-to-long-term prediction. The CNN has a strong representation learning ability, so we can use it to classify the input information according to its hierarchical structure and extract high-order features from the input information.

Utilize the transfer learning (TL) theory to improve the medium-to-long-term prediction performance. We extract the intrinsic high-order features from the chaotic time series and use the transfer learning theory to overlay them with their high-order features to enhance its prediction performance.

The remainder of this article is organized as follows: in Section

According to the step number of prediction, we can divide the prediction methods into the single-step prediction [

Taking accurate multistep predictions with a novel recurrent neural network (RNN), Min et al. [

As we know, we can employ an autoencoder (AE) to learn a representation (encoding) for a time series by training the network to ignore signal “noise,” i.e., denoising. Considering CNN’s advantages of shift invariant or space invariant, the CNN is most commonly applied to classify images, cluster images by similarity, and perform object recognition within scenes. In this article, we will employ CNN to extract the intrinsic high-order features including similarity from the chaotic time series to implement predictions. By combining the advantages of AE and CNN, we can use AE-CNN to predict a chaotic time series, as shown in Figure

The architecture of AE-CNN.

If there are a chaotic time series

The architecture of AE-CNN-TL.

We assume that both time series

From Figure

The optimization of AE-CNN-TL is to minimize the reconstruction error of AE and the training error of the whole model. At the denoising stage, the output of AE is an approximate copy of input. Therefore, we have to minimize the reconstruction error between input and output, which could maintain the direct significance of temperature. To obtain a sound model, it is necessary to minimize the training error of the whole model. We formulate the objective functions as

It is necessary to obtain a sound model to minimize the training error of the whole model. We formulate the objective function of the whole model as

In this article, we use the Beijing PM2.5 dataset (

Descriptive statistics for PM2.5 concentration and temperature.

PM2.5 concentration | Temperature | |
---|---|---|

Mean | 98.61 | 12.40 |

Median | 72.00 | 14.00 |

Standard deviation | 92.05 | 12.18 |

Sample variance | 8,473.27 | 148.24 |

Kurtosis | 7.77 | 1.88 |

Skewness | 1.80 | −0.14 |

Range | 994.00 | 61.00 |

Minimum | 0.00 | −19.00 |

Maximum | 994.00 | 42.00 |

Count | 41,757 | 41,757 |

To obtain a desirable model, we divide the experimental data into three parts: 70% training dataset, 10% validation dataset, and 20% test dataset. The training dataset is to reach a sound model, the validation dataset is to further determine the parameters of the whole network, and the test dataset is to test the generalization ability of the model.

The root mean square error (RMSE) is a widely used evaluation index for continuous type forecasting. Generally speaking, the smaller the RMSE value is, the better its prediction performance is. The RMSE is defined as follows:

According to the 0-1 test algorithm for chaos, we sample ^{th} data point and set a random number

Then, we calculate the following test value of chaos:

Figure

Plots of

Figure

Plots in the new coordinate p-s plane for temperature.

Similar to Figure

Similar to Figure

Phase in

Figure

Phase portrait of temperature vs. PM2.5 concentration.

In short, Figures

In this part, we test the prediction performance of the proposed AE-CNN-TL for chaotic time series.

To compare the prediction performance of AE-CNN and AE-CNN-TL, we firstly employ AE-CNN to predict PM2.5 concentration without TL from temperature, as shown in Figure

PM2.5 concentration prediction without TL from temperature.

PM2.5 concentration prediction with TL from temperature.

Temperature prediction without TL from PM2.5 concentration.

Temperature prediction with TL from PM2.5 concentration.

RMSEs of prediction.

Prediction task | Prediction approach | RMSE |

PM2.5 concentration prediction | ARMA | 79.21 |

LSTM | 43.50 | |

AE-CNN | 89.55 | |

AE-CNN-TL | 21.53 | |

Temperature prediction | ARMA | 72.47 |

LSTM | 37.68 | |

AE-CNN | 12.39 | |

AE-CNN-TL | 1.61 |

From Table

In this article, we develop a prediction scheme for chaotic time series of medium-to-long-term-based AE-CNN and TL, named AE-CNN-TL, which has much better prediction performance than any one of the following: AE-CNN, ARMA, and LSTM. This prediction scheme can be used in chaotic time series of medium-to-long term of many natural fields and social systems in the real world.

In fact, AE-CNN-TL can also be used to reveal the dynamic link between PM2.5 concentration and temperature. What is more, AE-CNN-TL can be used to explore the Granger causality [

We know that the Beijing PM2.5 dataset originates from the nature field. In fact, we can try to employ the proposed AE-CNN-TL to predict a chaotic time series of medium-to-long term which comes from some artificial systems, such as the game system [

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation, to any qualified researcher.

The authors declare that there are no conflicts of interest regarding the publication of this article.

This work was supported by the Natural Science Foundation of Shandong Province (Grant no. ZR2016FM26) and the National Social Science Foundation of China (Grant no. 16FJY008).