Benchmark task: Object Detection¶
Object Detection is a very common benchmark for evaluating feature representation quality. In VISSL, we use Detectron2 for the object detection benchmark.
This benchmark involves 2 steps:
Step1: Converting the self-supervised model weights so they are compatible with
Detectron2.Step2: Using the converted weights in Step1, run the benchmark.
Converting weights to Detectron2¶
VISSL provides a script to convert the weight of VISSL compatible models to Detectron2. We recommend users to adapt this script to suit their needs (different model architecture etc).
To run the script, follow the command:
python extra_scripts/convert_vissl_to_detectron2.py \
--input_model_file <input_model_path>.torch \
--output_model <converted_d2_model_path>.torch \
--weights_type torch \
--state_dict_key_name classy_state_dict
The script above converts ResNe(X)ts models in VISSL to the models compatible with ResNe(X)ts in Detectron2.
Benchmark: Faster R-CNN on VOC07¶
VISSL provides the YAML configuration files for Detectron2 for the benchmark task of Object detection using Faster R-CNN on VOC07.
The configuration files are available here.
To run the benchmark, VISSL provides a python script that closely follows MoCo object detection.
Please make sure to install Detectron2 following the Detectron2 Installation instructions.
To run the benchmark:
python tools/object_detection_benchmark.py \
--config-file ../configs/config/benchmark/object_detection/voc07/rn50_transfer_voc07_detectron2_e2e.yaml \
--num-gpus 8 MODEL.WEIGHTS <converted_d2_model_path>.torch
Note
We recommend users to consult Detectron2 documentation for how to use the configuration files and how to run the trainings.
PIRL object detection¶
To reproduce the object detection benchmark, the LR and warmup iterations are different. Use the following command:
python tools/object_detection_benchmark.py \
--config-file ../configs/config/benchmark/object_detection/voc07/pirl_npid/rn50_transfer_voc07_pirl_npid.yaml \
--num-gpus 8 MODEL.WEIGHTS <converted_d2_model_path>.torch
Benchmark: Faster R-CNN on VOC07+12¶
VISSL provides the YAML configuration files for Detectron2 for the benchmark task of Object detection using Faster R-CNN on VOC07+12.
The configuration files are available here.
To run the benchmark, VISSL provides a python script that closely follows MoCo object detection.
Please make sure to install Detectron2 following the Detectron2 Installation instructions.
For the VOC07+12 benchmark, most self-supervised approaches use their set of hyperparams. VISSL provides the settings used in
Scaling and Benchmarking Self-Supervised Visual Representation Learning¶
python tools/object_detection_benchmark.py \
--config-file ../configs/config/benchmark/object_detection/voc0712/iccv19/rn50_transfer_voc0712_detectron2_e2e.yaml \
--num-gpus 8 MODEL.WEIGHTS <converted_d2_model_path>.torch
MoCoV2¶
python tools/object_detection_benchmark.py \
--config-file ../configs/config/benchmark/object_detection/voc0712/mocoV2/rn50_transfer_voc0712_detectron2_e2e.yaml \
--num-gpus 8 MODEL.WEIGHTS <converted_d2_model_path>.torch
Benchmark: Mask R-CNN on COCO¶
Benchmarking on COCO introduces many variants (model architecture, FPN or not, C4). We provide config files for all the variants here and encourage users to pick the settings most suitable for their needs.
Benchmarking on COCO is not as widely adopted (compared to VOC07 and voc0712 evaluation) in self-supervision literature. This benchmark has been demonstrated extensively in MoCoV2 paper and we encourage users to refer to the paper.
An example run:
python tools/object_detection_benchmark.py \
--config-file ../configs/config/benchmark/object_detection/COCOInstance/sbnExtraNorm_precBN_r50_c4_coco.yaml \
--num-gpus 8 MODEL.WEIGHTS <converted_d2_model_path>.torch