v0.1.5

Index

What is VISSL?
Installation
Getting Started with VISSL

Notes

YAML Configuration system
Using Tensorboard in VISSL
- How to use Tensorboard in VISSL
- Example usage
Compatibility with Other Libraries
Contributing to VISSL
Contact

Flowcharts for VISSL execution

Training: Step-by-step execution
Feature Extraction: Step-by-step execution
Benchmark on VOC07: Step-by-step execution
Nearest Neighbor Benchmark: Step-by-step execution

Training resource setup

Train models on CPU
Train anything on 1-gpu
Train on SLURM cluster

Self-supervision approaches

Train RotNet model
Train Jigsaw model
Train NPID (and NPID++) model
Train ClusterFit model
Train PIRL model
Train SimCLR model
Train MoCo model
Train DeepCluster V2 model
Train SwAV model

Evaluation tasks

Benchmark: Linear Image Classification
Benchmark task: Full-finetuning
- Benchmark: ImageNet-1k
- Benchmark: Places205
Benchmark: Nearest Neighbor k-means
Benchmark task: Full finetuning on Imagenet 1% , 10% subsets
Benchmark task: Object Detection
How to Extract Features
- Config File for Feature Extraction
- How to extract features
Summary: Feature Eval Config Settings
How to Load Pretrained Models

Using VISSL Modules

Training
Building Models
- Trunks
- Heads
  - Types of Heads one can specify
  - Applying heads on multiple trunk features
Using Optimizers
- Creating Optimizers
  - Using different LR for Head and trunk
  - Using LARC
- Creating LR Schedulers
  - How to set learning rate
  - Auto-scaling of Learning Rate
Using PyTorch and VISSL Losses
Using Meters
Hooks
- How to enable certain hooks in VISSL
Using Data

Extending VISSL Modules

Add custom Train loop
Add new Hooks
Add new Optimizers
Add new LR schedulers
Add new Losses to VISSL
Add new Meters
Add new Models
- Adding New Heads
- Adding New Trunks
Using Custom Datasets
Add new Data Source
Add new Dataloader
Add new Data Transforms
Add new Data Collators

Large Scale Training

Activation checkpointing to reduce model memory
- How it works?
- How to use activation checkpointing in VISSL?
LARC for Large batch size training
- What is LARC
- How to enable LARC
  - Using Apex
Handling invalid images in dataloader
- How VISSL solves it
- Enabling QueueDataset
  - Tuning the queue size of QueueDataset
Resume training from iteration: Stateful data sampler
- Issue with PyTorch DataSampler for large data training
- How to use VISSL custom DataSampler
Mixed precision training (fp16)
- Using Apex
Train on multiple-gpus
Train on multiple machines
Using SLURM
ZeRO: Optimizer state and gradient sharding
- How ZeRO works?
- How to use ZeRO in VISSL?

API Documentation

API Documentation

VISSL

Docs »
Training: Step-by-step execution
Edit on GitHub

Training: Step-by-step execution¶

We demonstrate step-by-step execution of how training works in VISSL in the following flowchart.

../_images/train_flowchart.png

Next Previous

© Copyright 2021, VISSL contributors Revision 21467ad3.

Built with Sphinx using a theme provided by Read the Docs.