Adapting CLIP Without Training It: A Hands-On Study of CLIP, Tip-Adapter, and Tip-Adapter-F on EuroSAT
All Science Academy, Konya, 2026
- Yayın Türü: Kitap / Araştırma Kitabı
- Basım Tarihi: 2026
- Yayınevi: All Science Academy
- Basıldığı Şehir: Konya
- Isparta Uygulamalı Bilimler Üniversitesi Adresli: Evet
Özet
Large vision–language models can recognise new classes without a single labelled example, but their accuracy drops sharply in domains that look unlike their training data. This book examines how far CLIP can be adapted to such a domain with only a handful of labelled images and without retraining it. It explains how CLIP's encoders produce vectors, how a zero-shot classifier is built from class names, and how Tip-Adapter, a training-free key–value cache, and Tip-Adapter-F, which fine-tunes the cache keys, build on them, and it places both among the other ways of adapting CLIP. The methods are compared on the EuroSAT satellite image dataset with three CLIP backbones, one to 64 labelled images per class and five random support sets, under a protocol in which every setting is chosen on validation data only. Zero-shot CLIP classifies at most 50.6 percent of the test images correctly. With one labelled image per class, Tip-Adapter raises accuracy to 64.5 percent and Tip-Adapter-F to 68.5 percent; Tip-Adapter-F reaches 87.3 percent with sixteen and 91.8 percent with 64 images per class and is the best method in all twenty-one experimental conditions. The training-free cache beats a linear probe when labels are scarcest but then falls behind and produces over-confident probabilities that temperature scaling cannot repair; it is, however, the most robust method when support labels are wrong. The domain gap lies on the language side rather than the visual side. All experiments can be reproduced with the companion Colab notebook.