Introduction

• Recent successful Deep Learning networks rely on fully supervised learning with large amount of labeled training data
  • Scalability and generalizability of model decrease in data scarcity scenarios
    • Few-Shot learning to solve
• This paper is specifically about the fine-tuning method of few-shot
• Many other fine-tuning methods have difficulty discriminating novel and base feature space
  • Propose new two-step fine-tuning method
    • Step 1: Association, Step 2: Discrimination

Proposed Strategy

Association Step

• Pairs each novel class to most similar base class and aligns the feature distribution to paired
  • pairs with semantic similarity
  • aligns with feature distribution alignment
• Similarity Measurement
  • WordNet, English vocabulary graph network, to get class-to-class Lin Similarity
• Feature Distribution Alignment
  • Learns to align the feature distribution of the novel class to the associated base class
    • assigning pseudo labels and freezing the classifier
  • Freeze all layers except single fc layer
• Novel classes are distinguishable from each other
  • aligned to different associated base class

Discrimination Step

• Disentangles the classification branches for base & novel class and enlarge inter-class separability by set-specialized margin loss

• Disentangling

  

  • $f(\cdot;W^B_{cls}),f(\cdot;W^N_{cls}):$ classifier for base and novel classes
  • $g(\cdot,\tilde W^B_{origin}),g(\cdot,\tilde W^N_{asso}):$ last fc layer with frozen weights for base and novel classes
  • $FC_2$ load original weights that are pretrained from base classes and $FC_2'$ load weights after association
    • suitable to deal with base and novel classes
    • background class also added to novel classifier

• Set-Specialized Margin Loss

  

  • margin loss as an auxiliary loss for classifier

  

  

  • cosine similarity to formulate logits prediction
  • $W:$ Weight of classifier
  • $X:$ input feature
  • $\tau:$ Temperature factor
  • maximize margin of decision boundary between classes

  

  • few-shot learning scenario, inherent bias exist
    • higher scores on base classes, background sample dominates
    • adopted set-specialized handling of different set