Comparing machine learning methods and selecting a final model is a common operation in applied machine learning.
Models are commonly evaluated using resampling methods like k-fold cross-validation from which mean skill scores are calculated and compared directly. Although simple, this approach …