Yoshua Bengio: What is the most exciting machine learning research paper you read in 2015?

No single paper stands out, and I realize talking to people that different researchers are impressed by different contributions, so the choice of the advances below is very subjective:

* the Batch Normalization paper is exciting because of the impact it already had in training numerous architectures, and it has been adopted as a standard

* the Ladder Networks paper is exciting because it is bringing back unsupervised learning ideas (here some particularly interesting stack of denoising autoencoders) into the competition with straight supervised learning, especially in a semi-supervised context

* this year's papers on generative adversarial networks (GAN), the LAPGAN and DCGAN, have really raised the bar on generative modelling of images in impressive ways, suddenly making this approach the leader and contributing to the spirit of rapid progress in unsupervised learning over the last year; they compete with another big advance in deep generative modelling based on variational autoencoders, including the very impressive DRAW paper from early last year.

* the papers that use content-based attention mechanisms have been numerous over the past year; I saw it start with our neural machine translation with attention, followed by the neural Turing machine (and later the end-to-end memory networks), and many exciting uses of this type of processing for things like caption generation and manipulating data structures (I liked in particular the Pointer Networks and the other papers on differentiable data structure operations with stacks, queues, Teaching Machines to Read and Comprehend, etc.). So this architectural device is here to stay...