VISSL documentation¶
VISSL is a computer vision library for state-of-the-art Self-Supervised Learning research with PyTorch. VISSL aims to accelerate research cycle in self-supervised learning: from designing a new self-supervised task to evaluating the learned representations.
Notes
Flowcharts for VISSL execution
Training resource setup
Self-supervision approaches
Evaluation tasks
Using VISSL Modules
Extending VISSL Modules
Large Scale Training
- Activation checkpointing to reduce model memory
- LARC for Large batch size training
- Handling invalid images in dataloader
- Resume training from iteration: Stateful data sampler
- Mixed precision training (fp16)
- Train on multiple-gpus
- Train on multiple machines
- Using SLURM
- ZeRO: Optimizer state and gradient sharding