vissl.hooks package

class vissl.hooks.SSLClassyHookFunctions(value)

Bases: enum.Enum

Enumeration of all the hook functions in the ClassyHook class.

on_start = 1

on_phase_start = 2

on_forward = 3

on_loss_and_meter = 4

on_backward = 5

on_update = 6

on_step = 7

on_phase_end = 8

on_end = 9

vissl.hooks.default_hook_generator(cfg: vissl.utils.hydra_config.AttrDict) → List[classy_vision.hooks.classy_hook.ClassyHook]

The utility function that prepares all the hoooks that will be used in training based on user selection. Some basic hooks are used by default.

Optional hooks:

• Tensorboard hook,

• loss specific hooks (swav loss, deepcluster loss, moco loss) used only when the loss is being used

• model complexity hook (if user wants to compute model flops, activations, params) enable the hook via MODEL.MODEL_COMPLEXITY.COMPUTE_COMPLEXITY = True

Returns

hooks (List(functions)) – list containing the hook functions that will be used

vissl.hooks.deepclusterv2_hooks module

class vissl.hooks.deepclusterv2_hooks.InitMemoryHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

Initialize the memory banks. Valid only for DeepClusterV2 training

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_start(task) → None

At the begining of the training, initialize the memory banks

class vissl.hooks.deepclusterv2_hooks.ClusterMemoryHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

Cluster the memory banks with distributed k-means. Valid only for DeepClusterV2 trainings.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(task) → None

At the beginning of each epochs, cluster the memory banks with distributed k-means

vissl.hooks.log_hooks module

All the hooks involved in human-readable logging

class vissl.hooks.log_hooks.LogGpuStatsHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

Hook executed at the start of training and after every training iteration is done. Logs Gpu nvidia-smi stats to logger streams: at the start of training and after 50 training iterations.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_start(task: classy_vision.tasks.classy_task.ClassyTask) → None

Logs Gpu nvidia-smi stats to logger streams.

on_step(task: classy_vision.tasks.classy_task.ClassyTask) → None

Print the nvidia-smi stats again to get more accurate nvidia-smi useful for monitoring memory usage.

class vissl.hooks.log_hooks.LogLossLrEtaHook(btime_freq: Optional[int] = None)

Bases: classy_vision.hooks.classy_hook.ClassyHook

Hook executed after every parameters update step. Logs training stats like: LR, iteration, ETA, batch time etc to logger streams.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

__init__(btime_freq: Optional[int] = None) → None

Parameters

btime_freq – if specified, logs average batch time of rolling_freq batches also.

on_update(task: classy_vision.tasks.classy_task.ClassyTask) → None

Executed after after parameter update. If the current phase is training, and it’s a logging iteration, we compute and log several helpul training stats to keep track of ongoing training.

For monitoring the batch size (average training iteration time), we allow monitoring the stats (optionally) for every N iterations to get better idea about the batch time and training eta.

Set the btime_freq input using cfg.PERF_STAT_FREQUENCY=N ensuring that cfg.MONITOR_PERF_STATS = True.

class vissl.hooks.log_hooks.LogLossMetricsCheckpointHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

Hook called after every forward pass (to check training doesn’t give NaN), after every step and at the end of epoch (to check if the model should be checkpointed) and print the meters values at the end of every phase.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called each time a model forward is done and make sure that the model forward output is not NaN. If we encounter NaN as the model output, we checkpoint the model to enable debugging and also checkpoint the model input sample, model output.

on_step(task: classy_vision.tasks.classy_task.ClassyTask) → None

In some cases, we might want to checkpoint after certain number of iterations. If we want to checkpoint after every N iterations, check the checkpoint frequency matches and checkpoint if it does.

on_phase_end(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called at the end of each phase and forward. We log the metrics and also save the checkpoint. We pass the mode: phase or iteration

class vissl.hooks.log_hooks.LogPerfTimeMetricsHook(log_freq: Optional[int] = None)

Bases: classy_vision.hooks.classy_hook.ClassyHook

Computes and prints performance metrics. Logs at the end of a phase or every log_freq if specified by user.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

__init__(log_freq: Optional[int] = None) → None

Parameters

log_freq – if specified, logs every log_freq batches also.

on_phase_start(task: classy_vision.tasks.classy_task.ClassyTask) → None

Initialize start time and reset perf stats

on_loss_and_meter(task: classy_vision.tasks.classy_task.ClassyTask) → None

Log performance metrics every log_freq batches, if log_freq is not None.

on_phase_end(task: classy_vision.tasks.classy_task.ClassyTask) → None

Log performance metrics at the end of a phase if log_freq is None.

vissl.hooks.moco_hooks module

class vissl.hooks.moco_hooks.MoCoHook(momentum: float, shuffle_batch: bool = True)

Bases: classy_vision.hooks.classy_hook.ClassyHook

This hook corresponds to the loss proposed in the “Momentum Contrast for Unsupervised Visual Representation Learning” paper, from Kaiming He et al. See http://arxiv.org/abs/1911.05722 for details and https://github.com/facebookresearch/moco for a reference implementation, reused here.

Called after every forward pass to update the momentum encoder. At the beginning of training i.e. after 1st forward call, the encoder is contructed and updated.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(task: classy_vision.tasks.classy_task.ClassyTask) → None

• Update the momentum encoder.

• Compute the key reusing the updated moco-encoder. If we use the batch shuffling, the perform global shuffling of the batch and then run the moco encoder to compute the features. We unshuffle the computer features and use the features as “key” in computing the moco loss.

vissl.hooks.state_update_hooks module

class vissl.hooks.state_update_hooks.SSLModelComplexityHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

Logs the number of paramaters, forward pass FLOPs and activations of the model. Adapted from: https://github.com/facebookresearch/ClassyVision/blob/master/classy_vision/hooks/model_complexity_hook.py#L20 # NOQA

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_start(task) → None

Before the training starts, run one forward only pass of the model on the dummy input of shape specified by user in MODEL.MODEL_COMPLEXITY.INPUT_SHAPE We calculate the flops, activations and number of params in the model.

class vissl.hooks.state_update_hooks.SetDataSamplerEpochHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

We use DistributedDataSampler for sampling the data. At the beginnning of each training epoch/phase, we need to set the epoch for the sampler.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called at the start of each epoch or phase to set the data sampler epoch. This is important to ensure the data is shuffled and the shuffling can be reproduced deterministically if the training is resumed from a checkpoint.

class vissl.hooks.state_update_hooks.UpdateBatchesSeenHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

Book-keeping only hook. Tracks how many forward passes have been done. aka how many batches have been seen by the trainer irrespective of the train or test phase. updates task.batches

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called each time forward pass is triggered. We update the number of batches we have seen. This is useful for debugging.

class vissl.hooks.state_update_hooks.UpdateTrainIterationNumHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

Book-keeping hook: updates the training iteration number (only updated if it’s a training phase). task.iteration is updated.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called each time forward pass is triggered. We update the number of batches we have seen. This is useful for debugging.

class vissl.hooks.state_update_hooks.UpdateTrainBatchTimeHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

After after parameters update step (training phase), we update the batch time aka the training time for the current iteration.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called each time forward pass is triggered. We update the number of batches we have seen. This is useful for debugging.

class vissl.hooks.state_update_hooks.UpdateTestBatchTimeHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

Include the batch time for test phase as well and called every time loss has been computed. Only updates task.batch_time if it’s a test phase and train phase is already updated by UpdateTrainBatchTimeHook hook.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called each time a loss has been computed. Append the batch time for test phase.

class vissl.hooks.state_update_hooks.CheckNanLossHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

After every loss computation, verify the loss is not infinite. Called for both training/test phase.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called each time a loss has been computed and checks that loss is not None.

class vissl.hooks.state_update_hooks.FreezeParametersHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

Hook that helps to freeze some specified model parameters for certain number of training iterations. The parameters are specified in a dictionary containing {param_name: frozen_iterations}. Used in SwAV training.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(task: classy_vision.tasks.classy_task.ClassyTask) → None

After every backward pass and before updating the parameters, check if there are parameters that should stay frozen. Set the grad to None for those params.

vissl.hooks.swav_hooks module

class vissl.hooks.swav_hooks.SwAVUpdateQueueScoresHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

Update queue scores, useful with small batches and helps getting meaningful gradients.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(task) → None

If we want to use queue in SwAV training, update the queue scores after every forward.

class vissl.hooks.swav_hooks.NormalizePrototypesHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

L2 Normalize the prototypes in swav training. Optional.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(task: classy_vision.tasks.classy_task.ClassyTask) → None

Optionally normalize prototypes

vissl.hooks.swav_momentum_hooks module

class vissl.hooks.swav_momentum_hooks.SwAVMomentumHook(momentum: float, momentum_eval_mode_iter_start: int, crops_for_assign: List[int])

Bases: classy_vision.hooks.classy_hook.ClassyHook

This hook is for the extension of the SwAV loss proposed in paper https://arxiv.org/abs/2006.09882 by Caron et al. The loss combines the benefits of using the SwAV approach with the momentum encoder as used in MoCo.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

__init__(momentum: float, momentum_eval_mode_iter_start: int, crops_for_assign: List[int])

Parameters

• momentum (float) – for the momentum encoder

• momentum_eval_mode_iter_start (int) – from what iteration should the momentum encoder network be in eval mode

• crops_for_assign (List[int]) – what crops to use for assignment

on_forward(task: classy_vision.tasks.classy_task.ClassyTask) → None

Forward pass with momentum network. We forward momentum encoder only on the single resolution crops that are used for assignment in the swav loss.

class vissl.hooks.swav_momentum_hooks.SwAVMomentumNormalizePrototypesHook

Bases: classy_vision.hooks.classy_hook.ClassyHook

L2 Normalize the prototypes in swav training. Optional. We normalize the momentum_encoder output prototypes as well additionally.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_forward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_phase_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_update(task: classy_vision.tasks.classy_task.ClassyTask) → None

Optionally normalize prototypes

vissl.hooks.tensorboard_hooks module

class vissl.hooks.tensorboard_hook.SSLTensorboardHook(tb_writer: torch.utils.tensorboard.writer.SummaryWriter, log_params: bool = False, log_params_every_n_iterations: int = - 1, log_params_gradients: bool = False)

Bases: classy_vision.hooks.classy_hook.ClassyHook

SSL Specific variant of the Classy Vision tensorboard hook

on_loss_and_meter(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_backward(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_start(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_end(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

on_step(*args, **kwargs) → None

Derived classes can set their hook functions to this.

This is useful if they want those hook functions to not do anything.

__init__(tb_writer: torch.utils.tensorboard.writer.SummaryWriter, log_params: bool = False, log_params_every_n_iterations: int = - 1, log_params_gradients: bool = False) → None

The constructor method of SSLTensorboardHook.

Parameters

• tb_writer – Tensorboard SummaryWriter instance

• log_params (bool) – whether to log model params to tensorboard

• log_params_every_n_iterations (int) – frequency at which parameters should be logged to tensorboard

• log_params_gradients (bool) – whether to log params gradients as well to tensorboard.

on_forward(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called after every forward if tensorboard hook is enabled. Logs the model parameters if the training iteration matches the logging frequency.

on_phase_start(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called at the start of every epoch if the tensorboard hook is enabled. Logs the model parameters once at the beginning of training only.

on_phase_end(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called at the end of every epoch if the tensorboard hook is enabled. Log model parameters and/or parameter gradients as set by user in the tensorboard configuration. Also resents the CUDA memory counter.

on_update(task: classy_vision.tasks.classy_task.ClassyTask) → None

Called after every parameters update if tensorboard hook is enabled. Logs the parameter gradients if they are being set to log, log the scalars like training loss, learning rate, average training iteration time, batch size per gpu, img/sec/gpu, ETA, gpu memory used, peak gpu memory used.