Deep ensembles perform better than a single network thanks to the diversity\namong their members. Recent approaches regularize predictions to increase\ndiversity; however, they also drastically decrease individual members'\nperformances. In this paper, we argue that learning strategies for deep\nensembles need to tackle the trade-off between ensemble diversity and\nindividual accuracies. Motivated by arguments from information theory and\nleveraging recent advances in neural estimation of conditional mutual\ninfor…