Our EGU 2021 contribution is a small tutorial about the intuition for using mixture density networks in rainfall-runoff modelling.
Authors: Daniel Klotz, Frederik Kratzert, Martin Gauch, Alden K. Sampson, Günter Klambauer, Johannes Brandstetter, Sepp Hochreiter, and Grey Nearing
------------------------
# Corresponding Discussion Paper:
doi.org/10.5194/hess-2021-154
# Abstract:
Uncertainty is a central part of hydrological inquiry. Deep Learning provides us with new tools for estimating these inherent uncertainties. The currently best performing rainfall-runoff models are based on Long Short-Term Memory (LSTM) networks. However, most LSTM-based modelling studies focus on point estimates.
Building on the success of LSTMs for estimating point predictions, this contribution explores different extensions to directly provide uncertainty estimations. We find that the resulting models provide excellent estimates in our benchmark for daily rainfall-runoff across hundreds basins. We provide an intuitive overview of these strong results, the benchmarking procedure, and the approaches used for obtaining them.
In short, we extend the LSTMs in two ways to obtain uncertainty estimations. First, we parametrize LSTMs so that they directly provide uncertainty estimates in the form of mixture densities. This is possible because it is a general function approximation approach. It requires minimal a-priori knowledge of the sampling distribution and provides us with an estimation technique for the aleatoric uncertainty of the given setup. Second, we use Monte Carlo Dropout to randomly mask out random connections of the network. This enforces an implicit approximation to a Gaussian Process and therefore provides us with a tool to estimate a form of epistemic uncertainty. In the benchmark the mixture density based approaches provide better estimates, especially the ones that use Asymmetric Laplacians as components.