SimCLR is able to achieve the same (~76.5% top-1 ImageNet accuracy) as a ResNet-50 trained with Supervised Learning. SimCLR has also set a new high for Semi-Supervised Learning on 1% and 10% of the data, and it performs as well as Transfer Learning from models pre-trained on ImageNet classification! This paper explains the details of the algorithm such as the composition of data augmentations, separate projection from representation to contrastive loss, and the role of scaling up in unsupervised learning!

Links:
SimCLR: https://arxiv.org/pdf/2002.05709.pdf
CPC: https://arxiv.org/pdf/1905.09272.pdf
ImageBERT: https://arxiv.org/pdf/2001.07966.pdf
Google AI Blog: Revisiting the unreasonable effectiveness of data: https://ai.googleblog.com/2017/07/rev...

Thanks for watching! Please Subscribe!