Machine learning for short-term weather forecasting
In research between the University of Leeds and the Met Office, variational autoencoders have been explored for use in precipitation nowcasting. Nowcasting is short term (0-6 hour) weather forecasting which makes use of weather observations. In this application autoencoders have been used as a supervised learning technique, to predict the temporal evolution of rainfall patterns. A sequence of three consecutive timeframes of rainfall radar data are used as inputs into the VAE, with the aim of then predicting the next frame in the sequence. Using a variational autoencoder makes it possible to generate ensemble (i.e. multiple different but equally likely) weather predictions, by sampling many times from the latent distribution, which helps to account for the uncertainty that is inherent in the system.
An autoencoder is an unsupervised machine learning technique that performs data encoding and decoding using feed forward neural networks. Autoencoders are made of two components: the encoder translates input data into lower dimensional space; and the decoder tries to reconstruct the original data from the encoded data. The lower dimensional encoding is often referred to as the latent space representation. The output from the decoder can then be compared to the original input to give a so-called reconstruction loss. During training, the autoencoder will aim to minimise this reconstruction loss.
There are a variety of applications of autoencoders, including for image de-noising and anomaly detection. Variational autoencoders are autoencoders where the input data is encoded into a mean and standard deviation to give a latent distribution. A latent vector can then be sampled from this distribution and fed into the decoder to generate an output.
Image: Variational autoencoder predictions of rain rate 30 minutes in advance, compared to the radar truth in the top left.
Visit the auto encoders notebook
Other useful links...