The Global Configuration#

The main configuration of the framework is located inside vibe/config.toml. There are several parameters that can be set that will alter how VIBE runs. This includes the actual list of modes that are run. The settings that have the be configured are:

base_path

The location where all the output files of VIBE will be written. Best practice is to make this path point outside of VIBE, one directory above where you have installed it.

b2monitoring_path

The path where the b2monitoring repository was cloned. This is only for needed for run_quality modes, so in most cases you can set it to an empty string.

modes_to_run

In this section one can choose which modes to use when running VIBE.

[modes_to_run.analysis_validation]
# Uncomment the following lines if needed:
# all = true
# by_name = ["mc_compare"]
by_group = ["tracking", "physics"]

[modes_to_run.skim_production]
# Uncomment the following lines if needed:
# all = true
# by_name = ["fei_compare"]
by_group = ["slme"]

There is a bit going on here which needs to be understood. We will break it down level by level

Top Level: the level corresponds to the variants of the ModeType enum found in vibe/core/mode.py.
- analysis_validation
- skim_production
- run_quality
Mode Type configuration: Underneath the variant there are 3 different options available. only one can be chosen.
- all: If chosen, all modes of this ModeType will be run.
- by_name: If chosen then the user must list the names of the modes that need to be run.
- by_group: If chosen, the user must create a list containing the groups they wish to run for that specific mode.

Note

All available analysis validation groups are contained in the DPGroup enum, whilst skim production groups are contained in the SkimWorkingGroup enum.

analysis_validation_light_release

The basf2 light release used for the processing of the steering files on the grid.

Warning

The light release chosen here is only affecting the processing of the datesets on the grid and not the ones run offline. The datasets defined with the additional parameter offline=true are processed with the release that is currently set up locally.

gbasf2_submission_campaign

This string defines an additional identifier for the projects submitted to the grid. This identifier has to be changed in case one wants to submit a new project to the grid. The identifier defined here can be overwritten for each dataset in the mode configuration.

use_belle_dataprod_proxy_group

Set this parameter to true to use the special belle_dataprod proxy group. Note that you have to be part of this group to be able to use it, so in most cases this parameter has to be set to false.

luigi_workers

This parameter controls the number of luigi tasks that are run in parallel.

Note

The config.toml uses Pydantic models defined in `vibe/config_validation/pydantic_models, which will check that the config file is formatted correctly and all data that has been inputted is as expected. For a more detailed look at the models used to validate the VIBE config, see Global VIBE Configuration Validator Models

Global VIBE Configuration Validator Models#

pydantic model VibeConfigModel[source]#

Bases: BaseModel

The Pydantic Model for the VIBE config.toml file. Here we can define any configuration variables along with

default behaviour
regex string pattern matching for paths and light releases

We can also define additional model validation intop of the type hints when defining attributes. In this VibeConfig model we use the @model_validator(mode=’after’) decorator to tell pydantic to run the decorated method after the VibeConfig model has been initially validated. For example:

@model_validator(mode='after')
def check_modes_to_run(self) -> Self:
    ModesToRunConfig(**self.modes_to_run)
    return self

Here we are validating the modes_to_run attribute by using the after decorator. The validation is actually just calling a second Pydantic Model specifically made to validate the modes_to_run structure. If the ModesToRun model is successful, then we return self.

Show JSON schema

{
   "title": "VibeConfigModel",
   "description": "The Pydantic Model for the VIBE config.toml file. Here we can define\nany configuration variables along with\n\n- default behaviour\n- regex string pattern matching for paths and light releases\n\nWe can also define additional model validation intop of the type hints\nwhen defining attributes. In this VibeConfig model we\nuse the @model_validator(mode='after') decorator to tell pydantic\nto run the decorated method after the VibeConfig model has been initially\nvalidated. For example:\n\n.. code-block:: python\n\n    @model_validator(mode='after')\n    def check_modes_to_run(self) -> Self:\n        ModesToRunConfig(**self.modes_to_run)\n        return self\n\nHere we are validating the `modes_to_run` attribute by using the after decorator.\nThe validation is actually just calling a second Pydantic Model specifically made\nto validate the `modes_to_run` structure. If the ModesToRun model is successful,\nthen we return self.",
   "type": "object",
   "properties": {
      "base_path": {
         "description": "Base path for all outputs of VIBE.",
         "pattern": "^\\/(?:[^\\/]+\\/)*[^\\/]+$",
         "title": "Base Path",
         "type": "string"
      },
      "b2monitoring_path": {
         "description": "Mirabelle database path for MirabelleUploadTask.",
         "pattern": "^\\/(?:[^\\/]+\\/)*[^\\/]+$",
         "title": "B2Monitoring Path",
         "type": "string"
      },
      "analysis_validation_light_release": {
         "default": "light-2503-ceres",
         "description": "basf2 light release name, must contain 'light'.",
         "title": "Analysis Validation Light Release",
         "type": "string"
      },
      "gbasf2_submission_campaign": {
         "description": "Submission campaign suffix for gbasf2 project name.",
         "pattern": ".*v.*",
         "title": "Gbasf2 Submission Campaign",
         "type": "string"
      },
      "use_belle_dataprod_proxy_group": {
         "default": false,
         "description": "If True, use the dataprod group proxy for grid submission.",
         "title": "Use Belle Dataprod Proxy Group",
         "type": "boolean"
      },
      "luigi_workers": {
         "description": "Number of Luigi workers to run.",
         "exclusiveMinimum": 0,
         "title": "Luigi Workers",
         "type": "integer"
      },
      "modes_to_run": {
         "additionalProperties": {
            "additionalProperties": true,
            "type": "object"
         },
         "description": "Structure defining the modes which need to be run.",
         "title": "Modes To Run",
         "type": "object"
      }
   },
   "required": [
      "base_path",
      "b2monitoring_path",
      "gbasf2_submission_campaign",
      "luigi_workers",
      "modes_to_run"
   ]
}

Fields:

analysis_validation_light_release (str)
b2monitoring_path (str)
base_path (str)
gbasf2_submission_campaign (str)
luigi_workers (int)
modes_to_run (dict[str, dict])
use_belle_dataprod_proxy_group (bool)

Validators:

check_modes_to_run » all fields

field analysis_validation_light_release: str = 'light-2503-ceres'#

basf2 light release name, must contain ‘light’.

Validated by:

check_modes_to_run

field b2monitoring_path: str [Required]#

Mirabelle database path for MirabelleUploadTask.

Constraints:

pattern = ^/(?:[^/]+/)*[^/]+$

Validated by:

check_modes_to_run

field base_path: str [Required]#

Base path for all outputs of VIBE.

Constraints:

pattern = ^/(?:[^/]+/)*[^/]+$

Validated by:

check_modes_to_run

field gbasf2_submission_campaign: str [Required]#

Submission campaign suffix for gbasf2 project name.

Constraints:

pattern = .*v.*

Validated by:

check_modes_to_run

field luigi_workers: int [Required]#

Number of Luigi workers to run.

Constraints:

gt = 0

Validated by:

check_modes_to_run

field modes_to_run: dict[str, dict] [Required]#

Structure defining the modes which need to be run.

Validated by:

check_modes_to_run

field use_belle_dataprod_proxy_group: bool = False#

If True, use the dataprod group proxy for grid submission.

Validated by:

check_modes_to_run

validator check_modes_to_run » all fields[source]#

pydantic model ModesToRunConfig[source]#

Bases: BaseModel

Model for validating the modes_to_run variables structure defined in the config.toml. Each attribute maps to one of the ModeTypes defined in the mode.py module and all have identical value structures:

class ModesToRunConfig(BaseModel):
    analysis_validation : dict[str, bool | list[str]] = Field(default={})
    skim_production : dict[str, bool | list[str]]  = Field(default={})
    run_quality : dict[str, bool | list[str]]  = Field(default={})

In this model we make use of the @field_validator decorator which we use to validate each attribute that is not an empty dictionary. We are checking if the dictionary has of the following keys

any
by_group
by_name

If none of these are found inside the non-empty dictionary then an assertion error is raised. How this is done is as follows:

@field_validator('*')
@classmethod
def check_values_of_dictionaries(cls, modes_to_run_dict:dict) -> dict:
    if modes_to_run_dict:
        assert any(f for f in ['all', 'by_group', 'by_name'] if f in modes_to_run_dict.keys()), f'ModeType variant defined for modes_to_run in the config.toml does not contain a any of the following: all, by_name, by_group'
    return modes_to_run_dict

Note in the field_validator decorator we use the ‘*’, this is short-hand to tell pydantic every field of this model should be validated using this method.

Note that this model does not allow any unknown fields. A ValidationError is raised if an unknown field is encountered. This would occur if a new ModeType is added to the ModeType Enum without reflecting that here in the model.

Show JSON schema

{
   "title": "ModesToRunConfig",
   "description": "Model for validating the `modes_to_run` variables structure defined in the config.toml.\nEach attribute maps to one of the ModeTypes defined in the mode.py module and all have identical\nvalue structures:\n\n.. code-block:: python\n\n    class ModesToRunConfig(BaseModel):\n        analysis_validation : dict[str, bool | list[str]] = Field(default={})\n        skim_production : dict[str, bool | list[str]]  = Field(default={})\n        run_quality : dict[str, bool | list[str]]  = Field(default={})\n\n\nIn this model we make use of the @field_validator decorator which we use to validate each\nattribute that is not an empty dictionary. We are checking if the dictionary has of the following keys\n\n- any\n- by_group\n- by_name\n\nIf none of these are found inside the non-empty dictionary then an assertion error is raised. How this is done is as follows:\n\n.. code-block:: python\n\n    @field_validator('*')\n    @classmethod\n    def check_values_of_dictionaries(cls, modes_to_run_dict:dict) -> dict:\n        if modes_to_run_dict:\n            assert any(f for f in ['all', 'by_group', 'by_name'] if f in modes_to_run_dict.keys()), f'ModeType variant defined for modes_to_run in the config.toml does not contain a any of the following: all, by_name, by_group'\n        return modes_to_run_dict\n\nNote in the field_validator decorator we use the '*', this is short-hand to tell pydantic every field of\nthis model should be validated using this method.\n\nNote that this model does not allow any unknown fields. A ValidationError is raised if an unknown field is\nencountered. This would occur if a new ModeType is added to the ModeType Enum without reflecting that\nhere in the model.",
   "type": "object",
   "properties": {
      "analysis_validation": {
         "additionalProperties": {
            "anyOf": [
               {
                  "type": "boolean"
               },
               {
                  "items": {
                     "type": "string"
                  },
                  "type": "array"
               }
            ]
         },
         "default": {},
         "title": "Analysis Validation",
         "type": "object"
      },
      "skim_production": {
         "additionalProperties": {
            "anyOf": [
               {
                  "type": "boolean"
               },
               {
                  "items": {
                     "type": "string"
                  },
                  "type": "array"
               }
            ]
         },
         "default": {},
         "title": "Skim Production",
         "type": "object"
      },
      "run_quality": {
         "additionalProperties": {
            "anyOf": [
               {
                  "type": "boolean"
               },
               {
                  "items": {
                     "type": "string"
                  },
                  "type": "array"
               }
            ]
         },
         "default": {},
         "title": "Run Quality",
         "type": "object"
      }
   },
   "additionalProperties": false
}

Config:

extra: str = forbid

Fields:

analysis_validation (dict[str, bool | list[str]])
run_quality (dict[str, bool | list[str]])
skim_production (dict[str, bool | list[str]])

Validators:

check_values_of_dictionaries » all fields

field analysis_validation: dict[str, bool | list[str]] = {}#

Validated by:

check_values_of_dictionaries

field run_quality: dict[str, bool | list[str]] = {}#

Validated by:

check_values_of_dictionaries

field skim_production: dict[str, bool | list[str]] = {}#

Validated by:

check_values_of_dictionaries

validator check_values_of_dictionaries » all fields[source]#

The Global Configuration

Contents

The Global Configuration#

Global VIBE Configuration Validator Models#