Configurations

Core Configuration

pytorch_tabular.config.DataConfig dataclass

Data configuration.

PARAMETER	DESCRIPTION
target	A list of strings with the names of the target column(s). It is mandatory for all except SSL tasks. TYPE: Optional[List[str]] DEFAULT: field(default=None, metadata={'help': 'A list of strings with the names of the target column(s). It is mandatory for all except SSL tasks.'})
continuous_cols	Column names of the numeric fields. Defaults to [] TYPE: List DEFAULT: field(default_factory=list, metadata={'help': 'Column names of the numeric fields. Defaults to []'})
categorical_cols	Column names of the categorical fields to treat differently. Defaults to [] TYPE: List DEFAULT: field(default_factory=list, metadata={'help': 'Column names of the categorical fields to treat differently. Defaults to []'})
date_columns	(Column names, Freq) tuples of the date fields. For eg. a field named introduction_date and with a monthly frequency should have an entry ('intro_date','M'} TYPE: List DEFAULT: field(default_factory=list, metadata={'help': "(Column names, Freq) tuples of the date fields. For eg. a field named `introduction_date` and with a monthly frequency should have an entry ('intro_date','M'}"})
encode_date_columns	Whether or not to encode the derived variables from date TYPE: bool DEFAULT: field(default=True, metadata={'help': 'Whether or not to encode the derived variables from date'})
validation_split	Percentage of Training rows to keep aside as validation. Used only if Validation Data is not given separately TYPE: Optional[float] DEFAULT: field(default=0.2, metadata={'help': 'Percentage of Training rows to keep aside as validation. Used only if Validation Data is not given separately'})
continuous_feature_transform	Whether or not to transform the features before modelling. By default it is turned off.. Choices are: [None,yeo-johnson,box- cox,quantile_normal,quantile_uniform]. TYPE: Optional[str] DEFAULT: field(default=None, metadata={'help': 'Whether or not to transform the features before modelling. By default it is turned off.', 'choices': [None, 'yeo-johnson', 'box-cox', 'quantile_normal', 'quantile_uniform']})
normalize_continuous_features	Flag to normalize the input features(continuous) TYPE: bool DEFAULT: field(default=True, metadata={'help': 'Flag to normalize the input features(continuous)'})
quantile_noise	NOT IMPLEMENTED. If specified fits QuantileTransformer on data with added gaussian noise with std = :quantile_noise: * data.std ; this will cause discrete values to be more separable. Please not that this transformation does NOT apply gaussian noise to the resulting data, the noise is only applied for QuantileTransformer TYPE: int DEFAULT: field(default=0, metadata={'help': 'NOT IMPLEMENTED. If specified fits QuantileTransformer on data with added gaussian noise with std = :quantile_noise: * data.std ; this will cause discrete values to be more separable. Please not that this transformation does NOT apply gaussian noise to the resulting data, the noise is only applied for QuantileTransformer'})
num_workers	The number of workers used for data loading. For windows always set to 0 TYPE: Optional[int] DEFAULT: field(default=0, metadata={'help': 'The number of workers used for data loading. For windows always set to 0'})
pin_memory	Whether or not to pin memory for data loading. TYPE: bool DEFAULT: field(default=True, metadata={'help': 'Whether or not to pin memory for data loading.'})
handle_unknown_categories	Whether or not to handle unknown or new values in categorical columns as unknown TYPE: bool DEFAULT: field(default=True, metadata={'help': 'Whether or not to handle unknown or new values in categorical columns as unknown'})
handle_missing_values	Whether or not to handle missing values in categorical columns as unknown TYPE: bool DEFAULT: field(default=True, metadata={'help': 'Whether or not to handle missing values in categorical columns as unknown'})

pytorch_tabular.config.ModelConfig dataclass

Base Model configuration

PARAMETER	DESCRIPTION
task	Specify whether the problem is regression or classification. backbone is a task which considers the model as a backbone to generate features. Mostly used internally for SSL and related tasks.. Choices are: [regression,classification,backbone]. TYPE: str DEFAULT: field(metadata={'help': 'Specify whether the problem is regression or classification. `backbone` is a task which considers the model as a backbone to generate features. Mostly used internally for SSL and related tasks.', 'choices': ['regression', 'classification', 'backbone']})
head	The head to be used for the model. Should be one of the heads defined in pytorch_tabular.models.common.heads. Defaults to LinearHead. Choices are: [None,LinearHead,MixtureDensityHead]. TYPE: Optional[str] DEFAULT: field(default='LinearHead', metadata={'help': 'The head to be used for the model. Should be one of the heads defined in `pytorch_tabular.models.common.heads`. Defaults to LinearHead', 'choices': [None, 'LinearHead', 'MixtureDensityHead']})
head_config	The config as a dict which defines the head. If left empty, will be initialized as default linear head. TYPE: Optional[Dict] DEFAULT: field(default_factory=lambda : {'layers': ''}, metadata={'help': 'The config as a dict which defines the head. If left empty, will be initialized as default linear head.'})
embedding_dims	The dimensions of the embedding for each categorical column as a list of tuples (cardinality, embedding_dim). If left empty, will infer using the cardinality of the categorical column using the rule min(50, (x + 1) // 2) TYPE: Optional[List] DEFAULT: field(default=None, metadata={'help': 'The dimensions of the embedding for each categorical column as a list of tuples (cardinality, embedding_dim). If left empty, will infer using the cardinality of the categorical column using the rule min(50, (x + 1) // 2)'})
embedding_dropout	Dropout to be applied to the Categorical Embedding. Defaults to 0.0 TYPE: float DEFAULT: field(default=0.0, metadata={'help': 'Dropout to be applied to the Categorical Embedding. Defaults to 0.0'})
batch_norm_continuous_input	If True, we will normalize the continuous layer by passing it through a BatchNorm layer. TYPE: bool DEFAULT: field(default=True, metadata={'help': 'If True, we will normalize the continuous layer by passing it through a BatchNorm layer.'})
learning_rate	The learning rate of the model. Defaults to 1e-3. TYPE: float DEFAULT: field(default=0.001, metadata={'help': 'The learning rate of the model. Defaults to 1e-3.'})
loss	The loss function to be applied. By Default it is MSELoss for regression and CrossEntropyLoss for classification. Unless you are sure what you are doing, leave it at MSELoss or L1Loss for regression and CrossEntropyLoss for classification TYPE: Optional[str] DEFAULT: field(default=None, metadata={'help': 'The loss function to be applied. By Default it is MSELoss for regression and CrossEntropyLoss for classification. Unless you are sure what you are doing, leave it at MSELoss or L1Loss for regression and CrossEntropyLoss for classification'})
metrics	the list of metrics you need to track during training. The metrics should be one of the functional metrics implemented in torchmetrics. By default, it is accuracy if classification and mean_squared_error for regression TYPE: Optional[List[str]] DEFAULT: field(default=None, metadata={'help': 'the list of metrics you need to track during training. The metrics should be one of the functional metrics implemented in ``torchmetrics``. To use your own metric, please use the `metric` param in the `fit` method By default, it is accuracy if classification and mean_squared_error for regression'})
metrics_prob_input	Is a mandatory parameter for classification metrics defined in the config. This defines whether the input to the metric function is the probability or the class. Length should be same as the number of metrics. Defaults to None. TYPE: Optional[bool] DEFAULT: field(default=None, metadata={'help': 'Is a mandatory parameter for classification metrics defined in the config. This defines whether the input to the metric function is the probability or the class. Length should be same as the number of metrics. Defaults to None.'})
metrics_params	The parameters to be passed to the metrics function. task is forced to be multiclass because the multiclass version can handle binary as well and for simplicity we are only using multiclass. TYPE: Optional[List] DEFAULT: field(default=None, metadata={'help': 'The parameters to be passed to the metrics function. `task` is forced to be `multiclass`` because the multiclass version can handle binary as well and for simplicity we are only using `multiclass`.'})
target_range	The range in which we should limit the output variable. Currently ignored for multi-target regression. Typically used for Regression problems. If left empty, will not apply any restrictions TYPE: Optional[List] DEFAULT: field(default=None, metadata={'help': 'The range in which we should limit the output variable. Currently ignored for multi-target regression. Typically used for Regression problems. If left empty, will not apply any restrictions'})
seed	The seed for reproducibility. Defaults to 42 TYPE: int DEFAULT: field(default=42, metadata={'help': 'The seed for reproducibility. Defaults to 42'})

pytorch_tabular.config.SSLModelConfig dataclass

Base SSLModel Configuration

PARAMETER	DESCRIPTION
encoder_config	The config of the encoder to be used for the model. Should be one of the model configs defined in PyTorch Tabular TYPE: Optional[ModelConfig] DEFAULT: field(default=None, metadata={'help': 'The config of the encoder to be used for the model. Should be one of the model configs defined in PyTorch Tabular'})
decoder_config	The config of decoder to be used for the model. Should be one of the model configs defined in PyTorch Tabular. Defaults to nn.Identity TYPE: Optional[ModelConfig] DEFAULT: field(default=None, metadata={'help': 'The config of decoder to be used for the model. Should be one of the model configs defined in PyTorch Tabular. Defaults to nn.Identity'})
embedding_dims	The dimensions of the embedding for each categorical column as a list of tuples (cardinality, embedding_dim). If left empty, will infer using the cardinality of the categorical column using the rule min(50, (x + 1) // 2) TYPE: Optional[List] DEFAULT: field(default=None, metadata={'help': 'The dimensions of the embedding for each categorical column as a list of tuples (cardinality, embedding_dim). If left empty, will infer using the cardinality of the categorical column using the rule min(50, (x + 1) // 2)'})
embedding_dropout	Dropout to be applied to the Categorical Embedding. Defaults to 0.1 TYPE: float DEFAULT: field(default=0.1, metadata={'help': 'Dropout to be applied to the Categorical Embedding. Defaults to 0.1'})
batch_norm_continuous_input	If True, we will normalize the continuous layer by passing it through a BatchNorm layer. DEPRECATED - Use head and head_config instead TYPE: bool DEFAULT: field(default=True, metadata={'help': 'If True, we will normalize the continuous layer by passing it through a BatchNorm layer. DEPRECATED - Use head and head_config instead'})
learning_rate	The learning rate of the model. Defaults to 1e-3 TYPE: float DEFAULT: field(default=0.001, metadata={'help': 'The learning rate of the model. Defaults to 1e-3'})
seed	The seed for reproducibility. Defaults to 42 TYPE: int DEFAULT: field(default=42, metadata={'help': 'The seed for reproducibility. Defaults to 42'})

pytorch_tabular.config.TrainerConfig dataclass

Trainer configuration

PARAMETER	DESCRIPTION
batch_size	Number of samples in each batch of training TYPE: int DEFAULT: field(default=64, metadata={'help': 'Number of samples in each batch of training'})
data_aware_init_batch_size	Number of samples in each batch of training for the data-aware initialization, when applicable. Defaults to 2000 TYPE: int DEFAULT: field(default=2000, metadata={'help': 'Number of samples in each batch of training for the data-aware initialization, when applicable. Defaults to 2000'})
fast_dev_run	runs n if set to n (int) else 1 if set to True batch(es) of train, val and test to find any bugs (ie: a sort of unit test). TYPE: bool DEFAULT: field(default=False, metadata={'help': 'runs n if set to ``n`` (int) else 1 if set to ``True`` batch(es) of train, val and test to find any bugs (ie: a sort of unit test).'})
max_epochs	Maximum number of epochs to be run TYPE: int DEFAULT: field(default=10, metadata={'help': 'Maximum number of epochs to be run'})
min_epochs	Force training for at least these many epochs. 1 by default TYPE: Optional[int] DEFAULT: field(default=1, metadata={'help': 'Force training for at least these many epochs. 1 by default'})
max_time	Stop training after this amount of time has passed. Disabled by default (None) TYPE: Optional[int] DEFAULT: field(default=None, metadata={'help': 'Stop training after this amount of time has passed. Disabled by default (None)'})
gpus	DEPRECATED: Number of gpus to train on (int). -1 uses all available GPUs. By default uses CPU (None) TYPE: Optional[int] DEFAULT: field(default=None, metadata={'help': 'DEPRECATED: Number of gpus to train on (int). -1 uses all available GPUs. By default uses CPU (None)'})
accelerator	The accelerator to use for training. Can be one of 'cpu','gpu','tpu','ipu', 'mps', 'auto'. Defaults to 'auto'. Choices are: [cpu,gpu,tpu,ipu,'mps',auto]. TYPE: Optional[str] DEFAULT: field(default='auto', metadata={'help': "The accelerator to use for training. Can be one of 'cpu','gpu','tpu','ipu','auto'. Defaults to 'auto'", 'choices': ['cpu', 'gpu', 'tpu', 'ipu', 'mps', 'auto']})
devices	Number of devices to train on (int). -1 uses all available devices. By default uses all available devices (-1) TYPE: Optional[int] DEFAULT: field(default=None, metadata={'help': 'Number of devices to train on (int). -1 uses all available devices. By default uses all available devices (-1)'})
devices_list	List of devices to train on (list). If specified, takes precedence over devices argument. Defaults to None TYPE: Optional[List[int]] DEFAULT: field(default=None, metadata={'help': 'List of devices to train on (list). If specified, takes precedence over `devices` argument. Defaults to None'})
accumulate_grad_batches	Accumulates grads every k batches or as set up in the dict. Trainer also calls optimizer.step() for the last indivisible step number. TYPE: int DEFAULT: field(default=1, metadata={'help': 'Accumulates grads every k batches or as set up in the dict. Trainer also calls optimizer.step() for the last indivisible step number.'})
auto_lr_find	Runs a learning rate finder algorithm (see this paper) when calling trainer.tune(), to find optimal initial learning rate. TYPE: bool DEFAULT: field(default=False, metadata={'help': 'Runs a learning rate finder algorithm (see this paper) when calling trainer.tune(), to find optimal initial learning rate.'})
auto_select_gpus	If enabled and devices is an integer, pick available gpus automatically. This is especially useful when GPUs are configured to be in 'exclusive mode', such that only one process at a time can access them. TYPE: bool DEFAULT: field(default=True, metadata={'help': "If enabled and `devices` is an integer, pick available gpus automatically. This is especially useful when GPUs are configured to be in 'exclusive mode', such that only one process at a time can access them."})
check_val_every_n_epoch	Check val every n train epochs. TYPE: int DEFAULT: field(default=1, metadata={'help': 'Check val every n train epochs.'})
gradient_clip_val	Gradient clipping value TYPE: float DEFAULT: field(default=0.0, metadata={'help': 'Gradient clipping value'})
overfit_batches	Uses this much data of the training set. If nonzero, will use the same training set for validation and testing. If the training dataloaders have shuffle=True, Lightning will automatically disable it. Useful for quickly debugging or trying to overfit on purpose. TYPE: float DEFAULT: field(default=0.0, metadata={'help': 'Uses this much data of the training set. If nonzero, will use the same training set for validation and testing. If the training dataloaders have shuffle=True, Lightning will automatically disable it. Useful for quickly debugging or trying to overfit on purpose.'})
deterministic	If true enables cudnn.deterministic. Might make your system slower, but ensures reproducibility. TYPE: bool DEFAULT: field(default=False, metadata={'help': 'If true enables cudnn.deterministic. Might make your system slower, but ensures reproducibility.'})
profiler	To profile individual steps during training and assist in identifying bottlenecks. None, simple or advanced, pytorch. Choices are: [None,simple,advanced,pytorch]. TYPE: Optional[str] DEFAULT: field(default=None, metadata={'help': 'To profile individual steps during training and assist in identifying bottlenecks. None, simple or advanced, pytorch', 'choices': [None, 'simple', 'advanced', 'pytorch']})
early_stopping	The loss/metric that needed to be monitored for early stopping. If None, there will be no early stopping TYPE: Optional[str] DEFAULT: field(default='valid_loss', metadata={'help': 'The loss/metric that needed to be monitored for early stopping. If None, there will be no early stopping'})
early_stopping_min_delta	The minimum delta in the loss/metric which qualifies as an improvement in early stopping TYPE: float DEFAULT: field(default=0.001, metadata={'help': 'The minimum delta in the loss/metric which qualifies as an improvement in early stopping'})
early_stopping_mode	The direction in which the loss/metric should be optimized. Choices are: [max,min]. TYPE: str DEFAULT: field(default='min', metadata={'help': 'The direction in which the loss/metric should be optimized', 'choices': ['max', 'min']})
early_stopping_patience	The number of epochs to wait until there is no further improvements in loss/metric TYPE: int DEFAULT: field(default=3, metadata={'help': 'The number of epochs to wait until there is no further improvements in loss/metric'})
early_stopping_kwargs	Additional keyword arguments for the early stopping callback. See the documentation for the PyTorch Lightning EarlyStopping callback for more details. TYPE: Optional[Dict] DEFAULT: field(default_factory=lambda : {}, metadata={'help': 'Additional keyword arguments for the early stopping callback. See the documentation for the PyTorch Lightning EarlyStopping callback for more details.'})
checkpoints	The loss/metric that needed to be monitored for checkpoints. If None, there will be no checkpoints TYPE: Optional[str] DEFAULT: field(default='valid_loss', metadata={'help': 'The loss/metric that needed to be monitored for checkpoints. If None, there will be no checkpoints'})
checkpoints_path	The path where the saved models will be TYPE: str DEFAULT: field(default='saved_models', metadata={'help': 'The path where the saved models will be'})
checkpoints_every_n_epochs	Number of training steps between checkpoints TYPE: int DEFAULT: field(default=1, metadata={'help': 'Number of training steps between checkpoints'})
checkpoints_name	The name under which the models will be saved. If left blank, first it will look for run_name in experiment_config and if that is also None then it will use a generic name like task_version. TYPE: Optional[str] DEFAULT: field(default=None, metadata={'help': 'The name under which the models will be saved. If left blank, first it will look for `run_name` in experiment_config and if that is also None then it will use a generic name like task_version.'})
checkpoints_mode	The direction in which the loss/metric should be optimized TYPE: str DEFAULT: field(default='min', metadata={'help': 'The direction in which the loss/metric should be optimized'})
checkpoints_save_top_k	The number of best models to save TYPE: int DEFAULT: field(default=1, metadata={'help': 'The number of best models to save'})
checkpoints_kwargs	Additional keyword arguments for the checkpoints callback. See the documentation for the PyTorch Lightning ModelCheckpoint callback for more details. TYPE: Optional[Dict] DEFAULT: field(default_factory=lambda : {}, metadata={'help': 'Additional keyword arguments for the checkpoints callback. See the documentation for the PyTorch Lightning ModelCheckpoint callback for more details.'})
load_best	Flag to load the best model saved during training TYPE: bool DEFAULT: field(default=True, metadata={'help': 'Flag to load the best model saved during training'})
track_grad_norm	Track and Log Gradient Norms in the logger. -1 by default means no tracking. 1 for the L1 norm, 2 for L2 norm, etc. TYPE: int DEFAULT: field(default=-1, metadata={'help': 'Track and Log Gradient Norms in the logger. -1 by default means no tracking. 1 for the L1 norm, 2 for L2 norm, etc.'})
progress_bar	Progress bar type. Can be one of: none, simple, rich. Defaults to rich. TYPE: str DEFAULT: field(default='rich', metadata={'help': 'Progress bar type. Can be one of: `none`, `simple`, `rich`. Defaults to `rich`.'})
precision	Precision of the model. Can be one of: 32, 16, 64. Defaults to 32.. Choices are: [32,16,64]. TYPE: int DEFAULT: field(default=32, metadata={'help': 'Precision of the model. Can be one of: `32`, `16`, `64`. Defaults to `32`.', 'choices': [32, 16, 64]})
seed	Seed for random number generators. Defaults to 42 TYPE: int DEFAULT: field(default=42, metadata={'help': 'Seed for random number generators. Defaults to 42'})
trainer_kwargs	Additional kwargs to be passed to PyTorch Lightning Trainer. See https://pytorch-lightning.readthedocs.io/en/latest/api/pytorch_lightning.trainer.html#pytorch_lightning.trainer.Trainer TYPE: Dict[str, Any] DEFAULT: field(default_factory=dict, metadata={'help': 'Additional kwargs to be passed to PyTorch Lightning Trainer.'})

pytorch_tabular.config.ExperimentConfig dataclass

Experiment configuration. Experiment Tracking with WandB and Tensorboard

PARAMETER	DESCRIPTION
project_name	The name of the project under which all runs will be logged. For Tensorboard this defines the folder under which the logs will be saved and for W&B it defines the project name TYPE: str DEFAULT: field(default=MISSING, metadata={'help': 'The name of the project under which all runs will be logged. For Tensorboard this defines the folder under which the logs will be saved and for W&B it defines the project name'})
run_name	The name of the run; a specific identifier to recognize the run. If left blank, will be assigned a auto-generated name TYPE: Optional[str] DEFAULT: field(default=None, metadata={'help': 'The name of the run; a specific identifier to recognize the run. If left blank, will be assigned a auto-generated name'})
exp_watch	The level of logging required. Can be gradients, parameters, all or None. Defaults to None. Choices are: [gradients,parameters,all,None]. TYPE: Optional[str] DEFAULT: field(default=None, metadata={'help': 'The level of logging required. Can be `gradients`, `parameters`, `all` or `None`. Defaults to None', 'choices': ['gradients', 'parameters', 'all', None]})
log_target	Determines where logging happens - Tensorboard or W&B. Choices are: [wandb,tensorboard]. TYPE: str DEFAULT: field(default='tensorboard', metadata={'help': 'Determines where logging happens - Tensorboard or W&B', 'choices': ['wandb', 'tensorboard']})
log_logits	Turn this on to log the logits as a histogram in W&B TYPE: bool DEFAULT: field(default=False, metadata={'help': 'Turn this on to log the logits as a histogram in W&B'})
exp_log_freq	step count between logging of gradients and parameters. TYPE: int DEFAULT: field(default=100, metadata={'help': 'step count between logging of gradients and parameters.'})

pytorch_tabular.config.OptimizerConfig dataclass

Optimizer and Learning Rate Scheduler configuration.

PARAMETER	DESCRIPTION
optimizer	Any of the standard optimizers from torch.optim. TYPE: str DEFAULT: field(default='Adam', metadata={'help': 'Any of the standard optimizers from [torch.optim](https://pytorch.org/docs/stable/optim.html#algorithms).'})
optimizer_params	The parameters for the optimizer. If left blank, will use default parameters. TYPE: Dict DEFAULT: field(default_factory=lambda : {}, metadata={'help': 'The parameters for the optimizer. If left blank, will use default parameters.'})
lr_scheduler	The name of the LearningRateScheduler to use, if any, from torch.optim.lr_scheduler. If None, will not use any scheduler. Defaults to None TYPE: Optional[str] DEFAULT: field(default=None, metadata={'help': 'The name of the LearningRateScheduler to use, if any, from https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate. If None, will not use any scheduler. Defaults to `None`'})
lr_scheduler_params	The parameters for the LearningRateScheduler. If left blank, will use default parameters. TYPE: Optional[Dict] DEFAULT: field(default_factory=lambda : {}, metadata={'help': 'The parameters for the LearningRateScheduler. If left blank, will use default parameters.'})
lr_scheduler_monitor_metric	Used with ReduceLROnPlateau, where the plateau is decided based on this metric TYPE: Optional[str] DEFAULT: field(default='valid_loss', metadata={'help': 'Used with ReduceLROnPlateau, where the plateau is decided based on this metric'})

pytorch_tabular.config.ExperimentRunManager(exp_version_manager='.pt_tmp/exp_version_manager.yml')

The manages the versions of the experiments based on the name. It is a simple dictionary(yaml) based lookup. Primary purpose is to avoid overwriting of saved models while runing the training without changing the experiment name.

PARAMETER	DESCRIPTION
exp_version_manager	The path of the yml file which acts as version control. Defaults to ".pt_tmp/exp_version_manager.yml". TYPE: str DEFAULT: '.pt_tmp/exp_version_manager.yml'

Source code in src/pytorch_tabular/config/config.py

def __init__(
    self,
    exp_version_manager: str = ".pt_tmp/exp_version_manager.yml",
) -> None:
    """The manages the versions of the experiments based on the name. It is a simple dictionary(yaml) based lookup.
    Primary purpose is to avoid overwriting of saved models while runing the training without changing the
    experiment name.

    Args:
        exp_version_manager (str, optional): The path of the yml file which acts as version control.
            Defaults to ".pt_tmp/exp_version_manager.yml".
    """
    super().__init__()
    self._exp_version_manager = exp_version_manager
    if os.path.exists(exp_version_manager):
        self.exp_version_manager = OmegaConf.load(exp_version_manager)
    else:
        self.exp_version_manager = OmegaConf.create({})
        os.makedirs(os.path.split(exp_version_manager)[0], exist_ok=True)
        with open(self._exp_version_manager, "w") as file:
            OmegaConf.save(config=self.exp_version_manager, f=file)

Head Configuration

In addition to these core classes, we also have config classes for heads

pytorch_tabular.models.common.heads.LinearHeadConfig dataclass

A model class for Linear Head configuration; serves as a template and documentation. The models take a dictionary as input, but if there are keys which are not present in this model class, it'll throw an exception.

PARAMETER	DESCRIPTION
layers	Hyphen-separated number of layers and units in the classification/regression head. eg. 32-64-32. Default is just a mapping from intput dimension to output dimension TYPE: str DEFAULT: field(default='', metadata={'help': 'Hyphen-separated number of layers and units in the classification/regression head. eg. 32-64-32. Default is just a mapping from intput dimension to output dimension'})
activation	The activation type in the classification head. The default activaion in PyTorch like ReLU, TanH, LeakyReLU, etc. https://pytorch.org/docs/stable/nn.html#non-linear-activations- weighted-sum-nonlinearity TYPE: str DEFAULT: field(default='ReLU', metadata={'help': 'The activation type in the classification head. The default activaion in PyTorch like ReLU, TanH, LeakyReLU, etc. https://pytorch.org/docs/stable/nn.html#non-linear-activations-weighted-sum-nonlinearity'})
dropout	probability of an classification element to be zeroed. TYPE: float DEFAULT: field(default=0.0, metadata={'help': 'probability of an classification element to be zeroed.'})
use_batch_norm	Flag to include a BatchNorm layer after each Linear Layer+DropOut TYPE: bool DEFAULT: field(default=False, metadata={'help': 'Flag to include a BatchNorm layer after each Linear Layer+DropOut'})
initialization	Initialization scheme for the linear layers. Defaults to kaiming. Choices are: [kaiming,xavier,random]. TYPE: str DEFAULT: field(default='kaiming', metadata={'help': 'Initialization scheme for the linear layers. Defaults to `kaiming`', 'choices': ['kaiming', 'xavier', 'random']})

pytorch_tabular.models.common.heads.MixtureDensityHeadConfig dataclass

MixtureDensityHead configuration

PARAMETER	DESCRIPTION
num_gaussian	Number of Gaussian Distributions in the mixture model. Defaults to 1 TYPE: int DEFAULT: field(default=1, metadata={'help': 'Number of Gaussian Distributions in the mixture model. Defaults to 1'})
sigma_bias_flag	Whether to have a bias term in the sigma layer. Defaults to False TYPE: bool DEFAULT: field(default=False, metadata={'help': 'Whether to have a bias term in the sigma layer. Defaults to False'})
mu_bias_init	To initialize the bias parameter of the mu layer to predefined cluster centers. Should be a list with the same length as number of gaussians in the mixture model. It is highly recommended to set the parameter to combat mode collapse. Defaults to None TYPE: Optional[List] DEFAULT: field(default=None, metadata={'help': 'To initialize the bias parameter of the mu layer to predefined cluster centers. Should be a list with the same length as number of gaussians in the mixture model. It is highly recommended to set the parameter to combat mode collapse. Defaults to None'})
weight_regularization	Whether to apply L1 or L2 Norm to the MDN layers. Defaults to L2. Choices are: [1,2]. TYPE: Optional[int] DEFAULT: field(default=2, metadata={'help': 'Whether to apply L1 or L2 Norm to the MDN layers. Defaults to L2', 'choices': [1, 2]})
lambda_sigma	The regularization constant for weight regularization of sigma layer. Defaults to 0.1 TYPE: Optional[float] DEFAULT: field(default=0.1, metadata={'help': 'The regularization constant for weight regularization of sigma layer. Defaults to 0.1'})
lambda_pi	The regularization constant for weight regularization of pi layer. Defaults to 0.1 TYPE: Optional[float] DEFAULT: field(default=0.1, metadata={'help': 'The regularization constant for weight regularization of pi layer. Defaults to 0.1'})
lambda_mu	The regularization constant for weight regularization of mu layer. Defaults to 0 TYPE: Optional[float] DEFAULT: field(default=0, metadata={'help': 'The regularization constant for weight regularization of mu layer. Defaults to 0'})
softmax_temperature	The temperature to be used in the gumbel softmax of the mixing coefficients. Values less than one leads to sharper transition between the multiple components. Defaults to 1 TYPE: Optional[float] DEFAULT: field(default=1, metadata={'help': 'The temperature to be used in the gumbel softmax of the mixing coefficients. Values less than one leads to sharper transition between the multiple components. Defaults to 1'})
n_samples	Number of samples to draw from the posterior to get prediction. Defaults to 100 TYPE: int DEFAULT: field(default=100, metadata={'help': 'Number of samples to draw from the posterior to get prediction. Defaults to 100'})
central_tendency	Which measure to use to get the point prediction. Defaults to mean. Choices are: [mean,median]. TYPE: str DEFAULT: field(default='mean', metadata={'help': 'Which measure to use to get the point prediction. Defaults to mean', 'choices': ['mean', 'median']})
speedup_training	Turning on this parameter does away with sampling during training which speeds up training, but also doesn't give you visibility on train metrics. Defaults to False TYPE: bool DEFAULT: field(default=False, metadata={'help': "Turning on this parameter does away with sampling during training which speeds up training, but also doesn't give you visibility on train metrics. Defaults to False"})
log_debug_plot	Turning on this parameter plots histograms of the mu, sigma, and pi layers in addition to the logits(if log_logits is turned on in experment config). Defaults to False TYPE: bool DEFAULT: field(default=False, metadata={'help': 'Turning on this parameter plots histograms of the mu, sigma, and pi layers in addition to the logits(if log_logits is turned on in experment config). Defaults to False'})
input_dim	The input dimensions to the head. This will be automatically filled in while initializing from the backbone.output_dim TYPE: int DEFAULT: field(default=None, metadata={'help': 'The input dimensions to the head. This will be automatically filled in while initializing from the `backbone.output_dim`'})