We illustrate experimentatlly the benefits of truncating the CNNs earlier, at the level of the convolutional layers.In this manner, one obtains powerful local image descriptors that, combined with traditional pooling encoders, like Fisher Vector, result in a boost in recognition accuracy in a diverse set of visual domains.