virtex.factories

---

This module is a collection of factories for creating objects of datasets, models, optimizers and other useful components. For example, a ResNet-50 visual backbone can be created as:

>>> # Explicitly by name, args and kwargs:
>>> backbone = VisualBackboneFactory.create(
...     "torchvision::resnet50", pretrained=False
... )
>>> # Directly from a config object:
>>> _C = Config(override_list=["MODEL.VISUAL.NAME", "torchvision::resnet50"])
>>> backbone = VisualBackboneFactory.from_config(_C)

Creating directly from Config is fast and simple, and ensures minimal changes throughout the codebase upon any change in the call signature of underlying class; or config hierarchy. Refer description of specific factories for more details.

class virtex.factories.Factory

Bases: object

Base class for all factories. All factories must inherit this base class and follow these guidelines for a consistent behavior:

• Factory objects cannot be instantiated, doing factory = SomeFactory() is illegal. Child classes should not implement __init__ methods.

• All factories must have an attribute named PRODUCTS of type Dict[str, Callable], which associates each class with a unique string name which can be used to create it.

• All factories must implement one classmethod, from_config() which contains logic for creating an object directly by taking name and other arguments directly from Config. They can use create() already implemented in this base class.

• from_config() should not use too many extra arguments than the config itself, unless necessary (such as model parameters for optimizer).

classmethod create(name: str, *args, **kwargs) → Any

Create an object by its name, args and kwargs.

classmethod from_config(config: virtex.config.Config) → Any

Create an object directly from config.

---

Dataloading-related Factories

class virtex.factories.TokenizerFactory

Bases: virtex.factories.Factory

Factory to create text tokenizers. This codebase ony supports one tokenizer for now, but having a dedicated factory makes it easy to add more if needed.

Possible choices: {"SentencePieceBPETokenizer"}.

classmethod from_config(config: virtex.config.Config) → virtex.data.tokenizers.SentencePieceBPETokenizer

Create a tokenizer directly from config.

Parameters

config – Config object with all the parameters.

class virtex.factories.ImageTransformsFactory

Bases: virtex.factories.Factory

Factory to create image transformations for common preprocessing and data augmentations. These are a mix of default transformations from albumentations and some extended ones defined in virtex.data.transforms.

It uses sensible default values, however they can be provided with the name in dict syntax. Example: random_resized_crop::{'scale': (0.08, 1.0)}

Note

This factory does not implement from_config() method. It is only used by PretrainingDatasetFactory and DownstreamDatasetFactory.

Possible choices: {"center_crop", "horizontal_flip", "random_resized_crop", "normalize", "global_resize", "color_jitter", "smallest_resize"}.

classmethod create(name: str, *args, **kwargs) → Any

Create an object by its name, args and kwargs.

classmethod from_config(config: virtex.config.Config)

Augmentations cannot be created from config, only create().

class virtex.factories.PretrainingDatasetFactory

Bases: virtex.factories.Factory

Factory to create Dataset s for pretraining VirTex models. Datasets are created depending on pretraining task used. Typically these datasets either provide image-caption pairs, or only images from COCO Captions dataset (serialized to an LMDB file).

As an exception, the dataset for multilabel_classification provides COCO images and labels of their bounding box annotations.

Possible choices: {"bicaptioning", "captioning", "masked_lm", "token_classification", "multilabel_classification"}.

classmethod from_config(config: virtex.config.Config, split: str = 'train')

Create a dataset directly from config. Names in this factory match with names in PretrainingModelFactory because both use same config parameter MODEL.NAME to create objects.

Parameters

• config – Config object with all the parameters.

• split – Which dataset split to load. One of {"train", "val"}.

class virtex.factories.DownstreamDatasetFactory

Bases: virtex.factories.Factory

Factory to create Dataset s for evaluating VirTex models on downstream tasks.

Possible choices: {"datasets/VOC2007", "datasets/imagenet"}.

classmethod from_config(config: virtex.config.Config, split: str = 'train')

Create a dataset directly from config. Names in this factory are paths of dataset directories (relative to the project directory), because config parameter DATA.ROOT is used to create objects.

Parameters

• config – Config object with all the parameters.

• split – Which dataset split to load. One of {"trainval", "test"} for VOC2007, or one of {"train", "val"} for ImageNet.

---

Modeling-related Factories

class virtex.factories.VisualBackboneFactory

Bases: virtex.factories.Factory

Factory to create visual_backbones. This factory supports any ResNet-like model from Torchvision. Use the method name for model as in torchvision, for example, torchvision::resnet50, torchvision::wide_resnet50_2 etc.

Possible choices: {"torchvision"}.

classmethod from_config(config: virtex.config.Config) → virtex.modules.visual_backbones.VisualBackbone

Create a visual backbone directly from config.

Parameters

config – Config object with all the parameters.

class virtex.factories.TextualHeadFactory

Bases: virtex.factories.Factory

Factory to create textual_heads. Architectural hyperparameters for transformers can be specified as name::*. For example, transdec_postnorm::L1_H1024_A16_F4096 would create a transformer textual head with L = 1 layers, H = 1024 hidden size, A = 16 attention heads, and F = 4096 size of feedforward layers.

Textual head should be "none" for pretraining tasks which do not involve language modeling, such as "token_classification".

Possible choices: {"transdec_postnorm", "transdec_prenorm", "none"}.

classmethod from_config(config: virtex.config.Config) → torch.nn.modules.module.Module

Create a textual head directly from config.

Parameters

config – Config object with all the parameters.

class virtex.factories.PretrainingModelFactory

Bases: virtex.factories.Factory

Factory to create models for different pretraining tasks.

Possible choices: {"bicaptioning", "captioning", "masked_lm", "token_classification", "multilabel_classification"}.

classmethod from_config(config: virtex.config.Config) → torch.nn.modules.module.Module

Create a model directly from config.

Parameters

config – Config object with all the parameters.

---

Optimization-related Factories

class virtex.factories.OptimizerFactory

Bases: virtex.factories.Factory

Factory to create optimizers. Possible choices: {"sgd", "adamw"}.

classmethod from_config(config: virtex.config.Config, named_parameters: Iterable[Any]) → torch.optim.optimizer.Optimizer

Create an optimizer directly from config.

Parameters

• config – Config object with all the parameters.

• named_parameters – Named parameters of model (retrieved by model.named_parameters()) for the optimizer. We use named parameters to set different LR and turn off weight decay for certain parameters based on their names.

class virtex.factories.LRSchedulerFactory

Bases: virtex.factories.Factory

Factory to create LR schedulers. All schedulers have a built-in LR warmup schedule before actual LR scheduling (decay) starts.

Possible choices: {"none", "multistep", "linear", "cosine"}.

classmethod from_config(config: virtex.config.Config, optimizer: torch.optim.optimizer.Optimizer) → torch.optim.lr_scheduler.LambdaLR

Create an LR scheduler directly from config.

Parameters

• config – Config object with all the parameters.

• optimizer – Optimizer on which LR scheduling would be performed.