NegCosIC: Negative Cosine Similarity-Invariance- Covariance Regularization for Few-Shot Learning

NegCosIC: Negative Cosine Similarity-Invariance- Covariance Regularization for Few-Shot Learning

Blog Article

Few-shot learning continues to pose a challenge as it is inherently difficult for visual recognition models to generalize with limited labeled examples.When the training data is limited, the demonhorns edge process of training and fine-tuning the model will be unstable and inefficient due to overfitting.In this paper, we introduce NegCosIC: Negative Cosine Similarity-Invariance-Covariance Regularization, a method that aims to improve the mean accuracy from the perspective of stabilizing the fine-tuning process and regularizing variance.

NegCosIC incorporates a negative simple cosine similarity loss to stabilize the parameters of the feature extractor during fine-tuning.In addition, NegCosIC integrates invariance loss and covariance loss to regularize the embeddings in order to reduce overfitting.Experimental results demonstrate that NegCosIC is able to bring substantial improvements over the current state-of-the-art methods.

An in-depth worse case analysis is also conducted and shows that NegCosIC is able to outperform state-of-the-art methods on worst case accuracy.The proposed NegCosIC achieved 2.15% and 2.

13% higher accuracy on miniImageNet 1-shot and 5-shot tasks, 3.22% and 2.67% higher accuracy on CUB orly impressions 1-shot and 5-shot tasks, and 2.

13% and 7.74% higher accuracy on CIFAR-FS 1-shot and 5-shot tasks in terms of worst-case accuracies.