database¶

Subpackages¶

repository

backend¶

class SQLAlchemyBackend(session: 'AsyncSession', options: 'db.Options', serializer: 'ExternalSerializationStrategy', workspace_id: 'UUID | None' = None, scenario_id: 'UUID | None' = None, single_scenario_mode: 'bool' = False, single_workspace_mode: 'bool' = False, tmpfile_dir: 'pathlib.Path | None' = None, workspace_service_cls: 't.Type[WorkspaceService]' = <class 'movici_data_core.services.general.WorkspaceService'>, dataset_type_service_cls: 't.Type[DatasetTypeService]' = <class 'movici_data_core.services.general.DatasetTypeService'>, entity_type_service_cls: 't.Type[EntityTypeService]' = <class 'movici_data_core.services.general.EntityTypeService'>, attribute_type_service_cls: 't.Type[AttributeTypeService]' = <class 'movici_data_core.services.general.AttributeTypeService'>, model_type_service_cls: 't.Type[ModelTypeService]' = <class 'movici_data_core.services.general.ModelTypeService'>, dataset_service_cls: 't.Type[DatasetService]' = <class 'movici_data_core.services.dataset.DatasetService'>, scenario_service_cls: 't.Type[ScenarioService]' = <class 'movici_data_core.services.scenario.ScenarioService'>, update_service_cls: 't.Type[UpdateService]' = <class 'movici_data_core.services.update.UpdateService'>)¶

Bases: object

attribute_type_service_cls¶: alias of AttributeTypeService

property attribute_types¶

dataset_service_cls¶: alias of DatasetService

dataset_type_service_cls¶: alias of DatasetTypeService

property dataset_types¶

property datasets¶

entity_type_service_cls¶: alias of EntityTypeService

property entity_types¶

for_scenario(scenario_id: UUID)¶

for_workspace(workspace_id: UUID)¶

model_type_service_cls¶: alias of ModelTypeService

property model_types¶

options: Options¶

property repository¶

scenario_id: UUID | None = None¶

scenario_service_cls¶: alias of ScenarioService

property scenarios¶

serializer: ExternalSerializationStrategy¶

session: sqlalchemy.ext.asyncio.AsyncSession¶

async set_database_mode(new_mode: DatabaseMode)¶

Change the mode of this database. Upgrading is always possible along the path SINGLE_SCENARIO -> SINGLE_WORKSPACE -> MULTIPLE_WORKSPACES. Downgrading is only possible if a single workspace and/or scenario currently exists in the database

Parameters:: new_mode – the new database mode

Set various database options for the database

Parameters:

strict_dataset_types – set/unset the STRICT_DATASET_TYPES option, which governs whether to automatically create non-existing dataset types when they are encountered in an uploaded dataset
strict_entity_types – set/unset the STRICT_ENTITY_TYPES option, which governs whether to automatically create non-existing entity types when they are encountered in an uploaded dataset
strict_attribute_types – set/unset the STRICT_ATTRIBUTE_TYPES option, which governs whether to automatically create non-existing attribute types when they are encountered in an uploaded dataset
strict_model_types – set/unset the STRICT_MODEL_TYPES option, which governs whether to automatically create non-existing model types when they are encountered in an uploaded scenario config
strict_scenario_datasets – set/unset the STRICT_SCENARIO_DATASETS option, which governs whether to automatically create stubs for non-existing datasets when they are encountered in an uploaded scenario config

single_scenario_mode: bool = False¶

single_workspace_mode: bool = False¶

tmpfile_dir: Path | None = None¶

async update_schema()¶

update_service_cls¶: alias of UpdateService

property updates¶

workspace_id: UUID | None = None¶

workspace_service_cls¶: alias of WorkspaceService

property workspaces¶

class SQLAlchemyServer(dbapi_url: str, tmpfile_dir: Path, serializer: ExternalSerializationStrategy | None = None)¶

Bases: object

This class is responsible for building a SQLAlchemyBackend. When running inside an FastAPI application, this class has the same lifetime as the application. From this class, request-scoped SQLAlchemyBackend instances are created.

Parameters:

dbapi_url – a DB API url string
serializer – a class for instantiating an ExternalSerializationStrategy. Default: EntityInitDataFormat
tmpfile_dir – a path to a directory that may be used to store temporary files

attribute_type_service_cls¶: alias of AttributeTypeService

begin(**engine_kwargs)¶

dataset_service_cls¶: alias of DatasetService

dataset_type_service_cls¶: alias of DatasetTypeService

dbapi_url: str¶

engine: sqlalchemy.ext.asyncio.AsyncEngine¶

entity_type_service_cls¶: alias of EntityTypeService

get_backend(session_kwargs: dict[str, Any] | None = None)¶

get_session(**session_kwargs)¶

model_type_service_cls¶: alias of ModelTypeService

scenario_service_cls¶: alias of ScenarioService

serializer: ExternalSerializationStrategy¶

session_factory: sqlalchemy.ext.asyncio.async_sessionmaker.sqlalchemy.ext.asyncio.AsyncSession¶

async setup_db(mode: DatabaseMode = DatabaseMode.SINGLE_SCENARIO)¶

tmpfile_dir: Path¶

update_service_cls¶: alias of UpdateService

workspace_service_cls¶: alias of WorkspaceService

db_types¶

class GUID(*args: Any, **kwargs: Any)¶

Bases: TypeDecorator

Platform-independent GUID type.

Uses PostgreSQL’s UUID type or MSSQL’s UNIQUEIDENTIFIER, otherwise uses CHAR(32), storing as stringified hex values.

source: https://docs.sqlalchemy.org/en/20/core/custom_types.html#backend-agnostic-guid-type

cache_ok = True¶

load_dialect_impl(dialect)¶

process_bind_param(value, dialect)¶

process_result_value(value, dialect)¶

class JSONTuple(*args: Any, **kwargs: Any)¶

Bases: TypeDecorator

cache_ok = True¶

process_bind_param(value, dialect)¶

process_result_value(value, dialect)¶

class TZDateTime(*args: Any, **kwargs: Any)¶

Bases: TypeDecorator

cache_ok = True¶

process_bind_param(value, dialect)¶

process_result_value(value, dialect)¶

general¶

async create_default_scenario(session: sqlalchemy.ext.asyncio.AsyncSession, workspace_id: UUID, name='default_scenario', display_name='default_scenario') → UUID¶

async create_default_workspace(session: sqlalchemy.ext.asyncio.AsyncSession, name='__default__', display_name='__default__') → UUID¶

get_engine(dbapi_url: str, **kwargs)¶

async get_options(session: sqlalchemy.ext.asyncio.AsyncSession)¶

async get_version(session: sqlalchemy.ext.asyncio.AsyncSession)¶

async initialize_database(session: sqlalchemy.ext.asyncio.AsyncSession, mode: DatabaseMode)¶

async set_options(session: sqlalchemy.ext.asyncio.AsyncSession, **options)¶

model¶

class Attribute(*args: Any, **kwargs: Any)¶

Bases: Base

length: Mapped[int]¶

class AttributeDataType(*values)¶

Bases: Enum

BOOL = 'bool'¶

FLOAT = 'float'¶

INT = 'int'¶

STR = 'str'¶

class AttributeType(*args: Any, **kwargs: Any)¶

Bases: Base

property data_type¶

description: Mapped[str]¶

has_rowptr: Mapped[bool]¶

to_domain() → AttributeType¶

unit_shape¶: alias of JSONTuple

unit_type: Mapped[AttributeDataType]¶

class Base(*args: Any, **kwargs: Any)¶

Bases: DeclarativeBase

type_annotation_map = {<class 'datetime.datetime'>: <class 'movici_data_core.database.db_types.TZDateTime'>, <class 'dict'>: sqlalchemy.JSON, <class 'tuple'>: <class 'movici_data_core.database.db_types.JSONTuple'>, <class 'uuid.UUID'>: <class 'movici_data_core.database.db_types.GUID'>}¶

class DataArray(*args: Any, **kwargs: Any)¶

Bases: Base

data: Mapped[bytes]¶

max_val: Mapped[float | None]¶

min_val: Mapped[float | None]¶

to_numpy() → ndarray¶

Reconstruct numpy array from stored data.

Returns:: Reconstructed NumPy array

class DatabaseMode(*values)¶

Bases: Enum

Mode in which this database (file) runs.

SINGLE_SCENARIO: This database contains a single scenario including init data and possibly updates
SINGLE_WORKSPACE: This database may contain multiple scenarios but all scenarios belong to a single workspace
MULTIPLE_WORKSPACES: This database may contain multiple workspace, each containing their own scenarios, datasets and simulation results

MULTIPLE_WORKSPACES = 'multiple_workspaces'¶

SINGLE_SCENARIO = 'single_scenario'¶

SINGLE_WORKSPACE = 'single_workspace'¶

class Dataset(*args: Any, **kwargs: Any)¶

Bases: Base

dataset_type¶: alias of DatasetType

epsg_code: Mapped[int | None]¶

to_domain(has_raw_data: bool = False, has_attributes: bool = False) → Dataset¶

class DatasetAttribute(*args: Any, **kwargs: Any)¶: Bases: Base

class DatasetType(*args: Any, **kwargs: Any)¶

Bases: Base

format: Mapped[DatasetFormat]¶

mimetype: Mapped[str | None]¶

to_domain() → DatasetType¶

class EntityType(*args: Any, **kwargs: Any)¶

Bases: Base

to_domain() → EntityType¶

class Metadata(*args: Any, **kwargs: Any)¶

Bases: Base

A table that should contain a single entry with the schema version of this database. Future iterations of the data-core may use this version to determine compatibility or migrate data from one version to another

class ModelType(*args: Any, **kwargs: Any)¶

Bases: Base

to_domain() → ModelType¶

class NamedResource(*args, **kwargs)¶

Bases: Protocol[T_dom]

id: sqlalchemy.orm.Mapped.<class 'uuid.UUID'>¶

name: sqlalchemy.orm.Mapped.<class 'str'>¶

to_domain() → T_dom¶

class Options(*args: Any, **kwargs: Any)¶

Bases: Base

A table that should contain a single entry with a database’s options. It contains the database mode (see DatabaseMode) and the various STRICT_ options:

STRICT_DATASET_TYPES: when set, datasets may only be added to this database if their dataset_type exists. If unset, a dataset type will be created with format DatasetFormat.ENTITY_BASED if a dataset is about to be added with a non-exsiting type
STRICT_ENTITY_TYPES: when set, an entity type must exist before data for an entity group of that type can be added in either a dataset or an update. When unset, an entity type will be created when adding data for an entity group of a non-existing type
STRICT_ATTRIBUTE_TYPES: when set, an attribute type must exist before an entity group can contain data for that attribute. When unset, an attribute type will be created with an inferred data type when storing attribute data for that type.
STRICT_MODEL_TYPES: when set, a model type must exist before a scenario can be added that uses a model of that type. When unset, a model type will be created when adding a scenario that uses a model of that type. It will be created with a pass-all schema so that any config is allowed, but this also means that no references to datasets, entity groups or attributes can be made in the model config
STRICT_SCENARIO_DATASETS: when set, every dataset in a scenario config "dataset" section must exist before the scenario config may be added. When unset, any a stub for every dataset that does not exist will be added when adding a scenario config

Furthermore, the options singleton contains a reference to the default scenario and/or default workspace if they exist (determined by the database mode mode)

mode: Mapped[DatabaseMode]¶

class RawData(*args: Any, **kwargs: Any)¶

Bases: Base

compression: Mapped[str | None]¶

encoding: Mapped[str | None]¶

class RawDataChunk(*args: Any, **kwargs: Any)¶

Bases: Base

bytes: Mapped[bytes]¶

sequence: Mapped[int]¶

class RowptrArray(*args: Any, **kwargs: Any)¶

Bases: Base

data: Mapped[bytes]¶

to_numpy() → ndarray¶

Reconstruct numpy array from stored data.

Returns:: Reconstructed NumPy array

class Scenario(*args: Any, **kwargs: Any)¶

Bases: Base

display_name: Mapped[str]¶

epsg_code: Mapped[int | None]¶

name: Mapped[str]¶

status: Mapped[ScenarioStatus]¶

to_domain() → Scenario¶

class ScenarioDataset(*args: Any, **kwargs: Any)¶

Bases: Base

dataset¶: alias of Dataset

scenario¶: alias of Scenario

to_domain()¶

class ScenarioModel(*args: Any, **kwargs: Any)¶

Bases: Base

config: Mapped[dict]¶

sequence: Mapped[int]¶

to_domain()¶

class ScenarioModelReference(*args: Any, **kwargs: Any)¶

Bases: Base

path: Mapped[str]¶

to_domain()¶

class Update(*args: Any, **kwargs: Any)¶

Bases: Base

iteration: Mapped[int]¶

timestamp: Mapped[int]¶

to_domain() → Update¶

class UpdateAttribute(*args: Any, **kwargs: Any)¶: Bases: Base

class Workspace(*args: Any, **kwargs: Any)¶

Bases: Base

to_domain() → Workspace¶

to_domain_or_none(obj: NamedResource[T_dom] | None) → T_dom | None¶