Introduction

• First proposing fine-tuning method to few-shot object detection
  • only the last layer of existing detectors on rare classes
  • two-stage training scheme
    1. train entire object detector with all base classes
    2. fine-tune last layer with small balanced training set (novel & base)
       • instance-level feature normalization
• Also propose improved methods to evaluate few-shot object detection
  • previous protocol problems
    1. high variance accuracy measurements
       • published comparisons unreliable
    2. only report accuracy on novel classes
       • fail to evaluate knowledge retention on base classes
  • revised protocol
    • adopt multiple runs
    • new evaluations to check both base and novel
• Focus on the training schedule and the instance-level feature normalization

Proposed Approach

• backbone and RPN features are class-agnostic
  • features learned from base classes are likely to transfer to novel classes without updates
• separate feature representation learning and box predictor learning to two stages
• outperforms previous meta-learning approaches and more memory efficient
• able to address severe data imbalance issue without repeated sampling
  • checked from results on LVIS

First Stage

• train feature extractor and box predictor with only base classes
• use common joint loss

Second Stage

• create small balanced training set with K shots per class

  • novel & base classes

• assign random initialized weights to box prediction networks for novel classes

• fine-tune only the box classification and regression networks

  • feature extractor all freeze

• used smaller learning rate (reduced by 20)

• Cosine similarity

  

  • used cosine similarity classifier in second fine-tuning stage
  • also adopted instance-level feature normalization
    • help reduce the intra-class variance and reduce novel and base trade-off