Add new Meters

VISSL allows adding new meters easily. Follow the steps below to add a new loss:

  • Step1: Create a new loss my_new_meter in vissl/meters/my_new_meter.py following the template

from classy_vision.meters import ClassyMeter, register_meter

@register_meter("my_new_meter")
class MyNewMeter(ClassyMeter):
    """
    Add docuementation on what this meter does

    Args:
        add documentation about each meter parameter
    """

    def __init__(self, meters_config: AttrDict):
        # implement what the init method should do like
        # setting variable to update etc.
        self.reset()

    @classmethod
    def from_config(cls, meters_config: AttrDict):
        """
        Get the MyNewMeter instance from the user defined config
        """
        return cls(meters_config)

    @property
    def name(self):
        """
        Name of the meter
        """
        return "my_new_meter"

    @property
    def value(self):
        """
        Value of the meter which has been globally synced. This is the value printed and
        recorded by user.
        """
        # implement how the value should be calculated/finalized/returned to user
        ....

        return {"my_metric_name": value, ....}

    def sync_state(self):
        """
        Globally syncing the state of each meter across all the trainers.
        Should perform distributed communications like all_gather etc
        to correctly gather the global values to compute the metric
        """
        # implement what Communications should be done to globally sync the state
        ...

        # update the meter variables to store these global gathered values
        ...

    def reset(self):
        """
        Reset the meter. Should reset all the meter variables, values.
        """
        self._scores = torch.zeros(0, self.num_classes, dtype=torch.float32)
        self._targets = torch.zeros(0, self.num_classes, dtype=torch.int8)
        self._total_sample_count = torch.zeros(1)
        self._curr_sample_count = torch.zeros(1)

    def __repr__(self):
        # implement what information about meter params should be
        # printed by print(meter). This is helpful for debugging
        return repr({"name": self.name, "value": self.value})

    def set_classy_state(self, state):
        """
        Set the state of meter. This is the state loaded from a checkpoint when the model
        is resumed
        """
        # implement how to set the state of the meter
        ....

    def get_classy_state(self):
        """
        Returns the states of meter that will be checkpointed. This should include
        the variables that are global, updated and affect meter value.
        """
        return {
            "name": self.name,
            ...
        }

    def update(self, model_output, target):
        """
        Update the meter every time meter is calculated
        """
        # implement how to update the meter values
        ...

    def validate(self, model_output, target):
        """
        Validate that the input to meter is valid
        """
        # implement how to enforce the validity of the meter inputs
        ....
  • Step2: Register the meter and meter params with VISSL Configuration. Add the params that the meter takes in VISSL defaults.yaml as follows:

METERS:
  my_new_meter:
    param1: value1
    param2: value2
    ...
  • Step3: Meter is ready to use. Simply set the configuration param METERS.name=my_new_meter or if you want to use multiple meters METERS.names=[meter_one, my_new_meter].