Skip to content

Memory Provider

The Memory provider is an in-memory database adapter that stores data in Python dictionaries. It is the default provider in Protean and requires no external dependencies.

Overview

The Memory provider is designed for:

  • Development environments where simplicity and speed are key
  • Testing scenarios where deterministic behavior is required
  • Prototyping when you want to defer technology decisions
  • CI pipelines that should run without external services

The Memory provider stores data using Python dictionaries protected by threading locks:

  • Records are kept in defaultdict(dict) keyed by entity class name
  • Thread-safe access via Lock for concurrent operations
  • All data is lost when the process terminates

Because it is the default provider, no configuration is needed at all -- a fresh domain uses the Memory provider automatically.

Configuration

[databases.default]
provider = "memory"

Configuration Options

Option Default Description
provider Required Must be "memory" for the Memory provider

No other options are needed. There is no connection URI, no pool size, no schema -- just provider = "memory".

Capabilities

The Memory provider supports the following capabilities:

  • ✅ CRUD -- Create, Read, Update, Delete single records
  • ✅ FILTER -- Query/filter records with lookup criteria
  • ✅ BULK_OPERATIONS -- update_all(), delete_all()
  • ✅ ORDERING -- Server-side ordering of results
  • ✅ SIMULATED_TRANSACTIONS -- Copy-on-write UoW semantics
  • ✅ OPTIMISTIC_LOCKING -- Version-based concurrency control
  • ✅ RAW_QUERIES -- JSON-string query criteria
  • ❌ TRANSACTIONS -- No real database-level atomicity
  • ❌ SCHEMA_MANAGEMENT -- No tables or indices to manage
  • ❌ CONNECTION_POOLING -- No external connections
  • ❌ NATIVE_JSON -- No native JSON column type
  • ❌ NATIVE_ARRAY -- No native array column type

Note

The Memory provider uses simulated transactions, not real ones. Within a Unit of Work, changes are tracked and applied on commit, but a rollback only discards uncommitted changes -- it cannot undo side effects that already happened in the Python process.

Raw Queries

The Memory provider supports raw queries through JSON-string criteria that are evaluated against in-memory records:

results = domain.providers["default"].raw(
    '{"age__gt": 21, "status": "active"}'
)

The query string is parsed as JSON and interpreted as filter criteria using the same lookup syntax as the QuerySet API.

Limitations

  • No Persistence -- Data is lost on process restart. The Memory provider is purely ephemeral.
  • No Real Transactions -- Simulated transactions track changes but cannot provide true rollback or ACID guarantees.
  • No Distribution -- All data lives in a single Python process. Cannot scale across multiple processes or machines.
  • No Schema Management -- There are no tables or indices to create or drop. _create_database_artifacts() and _drop_database_artifacts() are no-ops.
  • No Native JSON/Array -- Complex fields are stored as serialized Python objects, not as native database types.

Migration Path

The Memory provider is designed to be easily replaced with production-ready providers:

# Development configuration (default)
[databases.default]
provider = "memory"

# Production configuration (same domain code works!)
[databases.default]
provider = "postgresql"
database_uri = "postgresql://user:pass@db.example.com:5432/myapp"

Your domain code remains unchanged when switching providers, because all providers implement the same BaseProvider interface. Just update the configuration and Protean handles the rest.

Next Steps