AMNet: Memorability Estimation With Attention

In this paper we present the design and evaluation of an end to end trainable, deep neural network with a visual attention mechanism for memorability estimation in still images. We analyze the suitability of transfer learning of deep models from image classification to the memorability task. Further on we study the impact of the attention mechanism on the memorability estimation and evaluate our network on the SUN Memorability and the LaMem dataset, the only large dataset with memorability labels to this date. Our network outperforms the existing state of the art models on both, the LaMem and SUN datasets in the term of the Spearman’s rank correlation as well as mean squared error, approaching human consistency.