Skip to content

Memory Provider

The Memory provider is an in-memory database adapter that stores data in Python dictionaries. It is the default provider in Protean and requires no external dependencies.

Overview

The Memory provider is designed for:

  • Development environments where simplicity and speed are key
  • Testing scenarios where deterministic behavior is required
  • Prototyping when you want to defer technology decisions
  • CI pipelines that should run without external services

The Memory provider stores data using Python dictionaries protected by threading locks:

  • Records are kept in defaultdict(dict) keyed by entity class name
  • Thread-safe access via Lock for concurrent operations
  • All data is lost when the process terminates

Because it is the default provider, no configuration is needed at all, a fresh domain uses the Memory provider automatically.

Configuration

[databases.default]
provider = "memory"

Configuration Options

Option Default Description
provider Required Must be "memory" for the Memory provider

No other options are needed. There is no connection URI, no pool size, no schema, just provider = "memory".

Capabilities

The Memory provider supports the following capabilities:

  • ✅ CRUD: Create, Read, Update, Delete single records
  • ✅ FILTER: Query/filter records with lookup criteria
  • ✅ BULK_OPERATIONS: update_all(), delete_all()
  • ✅ ORDERING: Server-side ordering of results
  • ✅ SIMULATED_TRANSACTIONS: Copy-on-write UoW semantics
  • ✅ OPTIMISTIC_LOCKING: Version-based concurrency control
  • ✅ RAW_QUERIES: JSON-string query criteria
  • ❌ TRANSACTIONS: No real database-level atomicity
  • ❌ SCHEMA_MANAGEMENT: No tables or indices to manage
  • ❌ CONNECTION_POOLING: No external connections
  • ❌ NATIVE_JSON: No native JSON column type
  • ❌ NATIVE_ARRAY: No native array column type

Note

The Memory provider uses simulated transactions, not real ones. Within a Unit of Work, changes are tracked and applied on commit, but a rollback only discards uncommitted changes. It cannot undo side effects that already happened in the Python process.

Indexes

The memory provider accepts Index declarations and validates them for shape at Domain.init(). Non-unique indexes are advisory (the in-memory store performs no index-backed lookups), but unique indexes (Index(..., unique=True)) are enforced: a duplicate insert or update that violates a single-column or composite unique index raises ValidationError (the same error Protean raises for a field-level unique=True; a SQL backend rejects the duplicate at the database instead). NULLs are treated as distinct (PostgreSQL/SQLite semantics). This lets you declare the indexes your aggregates need, develop against the memory provider, and switch to a SQL backend without changing the domain.

Raw Queries

The Memory provider supports raw queries through JSON-string criteria that are evaluated against in-memory records:

results = domain.providers["default"].raw(
    '{"age__gt": 21, "status": "active"}'
)

The query string is parsed as JSON and interpreted as filter criteria using the same lookup syntax as the QuerySet API.

Limitations

  • No Persistence: Data is lost on process restart. The Memory provider is purely ephemeral.
  • No Real Transactions: Simulated transactions track changes but cannot provide true rollback or ACID guarantees.
  • No Distribution: All data lives in a single Python process. Cannot scale across multiple processes or machines.
  • No Schema Management: There are no tables or indices to create or drop. _create_database_artifacts() and _drop_database_artifacts() are no-ops.
  • No Native JSON/Array: Complex fields are stored as serialized Python objects, not as native database types.

Migration Path

The Memory provider is designed to be easily replaced with production-ready providers:

# Development configuration (default)
[databases.default]
provider = "memory"

# Production configuration (same domain code works!)
[databases.default]
provider = "postgresql"
database_uri = "postgresql://user:pass@db.example.com:5432/myapp"

Your domain code remains unchanged when switching providers, because all providers implement the same BaseProvider interface. update the configuration and Protean handles the rest.