1. Your First Aggregate: Create the foundation with a working Book aggregate.
2. Rich Fields and Value Objects: Add rich field types and create Money and Address value objects.
3. Entities and Associations: Build the Order aggregate with child entities.
4. Business Rules: Add invariants and encapsulate behavior in aggregate methods.

Part II: Making It Real

5. Commands and Handlers: Formalize state changes with commands and command handlers.
6. Events and Reactions: Define events, raise them from aggregates, and react with handlers.
7. Projections: Build read-optimized views from events.
8. Connecting a Real Database: Switch to PostgreSQL with a configuration change.
9. Structuring the Project: Organize domain code into a proper project layout with auto-discovery.
10. Exposing the Domain Through an API: Build a FastAPI web layer for commands and projections.
11. Testing Your Domain: Testing strategies for every layer of the application.

Part III: Growing the System

12. Going Async with the Server: Move side effects to background processing with the Protean server.
13. Domain Services: Cross-aggregate business logic with domain services.
14. Subscribers: Consume messages from external systems.
15. Fact Events: Auto-generated snapshots for reporting pipelines.

Part IV: Production Operations

16. Message Tracing: Track commands and events with correlation and causation IDs.
17. Dead Letter Queues: Inspect, replay, and purge failed messages.
18. Monitoring Health: Monitor subscription health with the Observatory dashboard.
19. Priority Lanes: Bulk imports without starving production traffic.

Part V: System Mastery

20. Process Managers: Orchestrate multi-step workflows with compensation.
21. Advanced Query Patterns: Filtering, sorting, pagination, and cross-aggregate projections.
22. The Full Picture: Complete architecture reference and where to go next.

Guides

Comprehensive reference organized by topic. Each guide goes deep on a specific area.

Guide Overview: How the guides are organized and where to start.
How Do I...?: Task-oriented index: find the right guide by what you're trying to do.

Choose a Path

Architectural Pathways: Three approaches that build on each other.
DDD Pathway: Aggregates, application services, and repositories.
CQRS Pathway: Separate reads from writes with commands and projections.
Event Sourcing Pathway: Derive state from event replay.

Set Up the Domain

The Domain Object: The composition root that manages elements, configuration, and adapters.
Register Elements: How elements register themselves with the domain.
Initialize the Domain: Call init() to wire everything together.
Activate the Domain: Bind the domain to a context for use.
When to Compose: Lifecycle and timing of domain composition.
Configure for Production: Environment overlays, env var substitution, adapter selection.
Choosing Adapters: Pick the right database, broker, event store, and cache for each workload.
Inspecting the IR: Generate and explore the domain's Intermediate Representation.
Schema Generation: Generate JSON Schema files for domain elements.
Domain Constructor: Domain() parameters: root_path, name, config, identity_function.
Element Decorators: Decorators that construct and register domain elements.
Object Model: Common structure and traits shared by all domain elements.

Define Domain Elements

Defining Concepts: Foundational domain concepts using DDD tactical patterns.
Aggregates: Model domain concepts with unique identity.
Entities: Objects with identity that compose aggregates.
Value Objects: Immutable descriptive objects identified by their attributes.
Fields: Declare attributes on domain elements: types, required flags, defaults, constraints, validators.
Declaring Indexes: Declare composite, unique, partial, and covering indexes on aggregates and entities.
Identity: Configure identity generation: UUIDs, integers, user-supplied keys, or a custom function.
Expressing Relationships: Model associations between domain elements.
Events: Model past changes as discrete, meaningful facts.
Choosing Element Types: Checklists and decision flows for choosing element types.

Field & Identity Reference

Fields Overview: Field types, attributes, options, and functionalities.
Defining Fields: Three styles for declaring fields: annotation, assignment, and raw Pydantic.
Simple Fields: String, Text, Integer, Float, Boolean, and other primitives.
Container Fields: Fields that hold and embed value objects.
Association Fields: HasOne, HasMany, and Reference fields for relationships.
Common Arguments: Shared field arguments like required, default, and description.
Identity Reference: Identity strategies, types, and per-field customization.
Indexes Reference: The Index API, options, dialect support matrix, and Index.from_sql escape hatch.

Add Rules and Behavior

Domain Behavior: Enforcing business rules through validations, invariants, and methods.
Validations: Field-level validation using types, options, and custom validators.
Invariants: Business rules that must always hold true within an aggregate.
Mutating Aggregates: Modify state through named methods reflecting actions and events.
Status Transitions: Enforce lifecycle state machines with the Status field.
Raising Events: Notify other parts of the system through domain events.
Message Tracing: Track the full causal chain of commands and events with correlation and causation IDs, and traverse causation chains programmatically.
Correlation and Causation IDs: Complete guide to correlation and causation ID propagation across HTTP headers, OTEL spans, Observatory traces, structured logging, and cross-service boundaries.
Explore Your Domain in the Observatory: Use the Observatory's Domain Visualizer (topology, event flows, process managers) and Timeline (event browser, correlation chains, traces) to navigate a running domain.
Domain Services: Complex domain logic that spans multiple aggregates.
Error Handling: Raise, propagate, and handle domain exceptions.

Change State

Changing State: Mechanisms for state changes: services, commands, and handlers.
Application Services: Bridge between the API layer and the domain model.
Commands: Data transfer objects expressing intention to change state.
Command Handlers: Process commands and execute domain logic.
Repositories: Define custom repositories, the DAO layer, and database-specific persistence.
Persist Aggregates: Save aggregates using a repository's add method.
Retrieve Aggregates: QuerySets, filtering, Q objects, bulk operations, and result navigation.
Temporal Queries: Reconstitute event-sourced aggregates at a specific version or point in time.
Unit of Work: Automatic transaction management for aggregate changes.
Custom Database Models: Override auto-generated storage schemas with adapter-specific tuning.
Event Store Setup: Choose, configure, and operate an event store.
Snapshots: Optimize event-sourced aggregate loading with periodic checkpoints.

React to Changes

Consuming State Changes: React to state changes through handlers, projections, and subscribers.
Event Handlers: Consume events to sync state or trigger side effects.
Process Managers: Coordinate multi-step processes across aggregates with stateful, event-sourced coordination.
Projections: Create read-optimized views built from events.
Projectors: Define event handlers that maintain projections.
Query Handlers: Process queries and return results from projections.
Subscribers: Consume messages from external brokers.
CloudEvents Interoperability: Serialize events to CloudEvents v1.0 for external systems, and consume CloudEvents from external sources.
Stream Categories: How messages are organized and routed.
Event Upcasting: Transforming old event schemas to match the current version during replay.
Evolving Events Over Time: The full event-evolution workflow: add fields, rename with renamed_from, deprecate/supersede, upcast, and check compatibility.

Run in Production

Configuration: Configure Protean through domain.toml and environment variables.
FastAPI Integration: Domain context middleware, exception handlers, and best practices for using Protean with FastAPI.
HTTP wide events: One wide event per HTTP request via DomainContextMiddleware, correlation with domain-layer wide events, [logging.http] configuration.

Observability

Logging: Configure structured logging, enrich wide events with business context, tail sampling, security events, disable auto-configuration.
Logging Reference: Every [logging] key, every framework logger, every structured event and its fields.
Logging Concepts: Wide events, query-oriented field design, the two-layer HTTP + domain split, tail sampling, high-cardinality backends, redaction as a pipeline stage.

Server

Server Overview: Asynchronous message processing engine for events, commands, and external messages.
Engine Architecture: Core async processing, managing subscriptions and lifecycle.
Subscriptions: Connect handlers to message sources.
Subscription Types: Stream and EventStore subscriptions for different use cases.
Subscription Configuration: Flexible configuration with priority hierarchy.
Outbox Pattern: Reliable message delivery via same-transaction storage.
Observability: Real-time tracing, the Observatory monitoring server, SSE streaming, and Prometheus metrics.
OpenTelemetry Integration: Distributed tracing, metrics, APM setup, and TraceParent propagation with OpenTelemetry.
Running the Server: Start, configure, and operate the Protean server.
Harden the Server: Raise pool limits, enable K8s health probes, run DLQ maintenance, pick subscription profiles, emit OTEL metrics, and shut down gracefully.
Tuning Subscriptions: Pick a profile, override one setting, cap stream growth, use the circuit breaker, process events for one entity in order, and check consumer lag.
Server Hardening Reference: Every option, default, metric, and profile shipped by the hardening epic.
Error Handling: Retry logic, dead letter queues, and recovery mechanisms for message processing failures.
Dead Letter Queues: Discover, inspect, replay, and purge messages that failed after retries were exhausted.
Using the Outbox: Reliably publish domain events via the outbox pattern; configuration, retries, cleanup.
Dispatching Published Events to External Brokers: Configure external brokers to deliver published events to other bounded contexts.

CLI

CLI Overview: The protean command-line interface for scaffolding and management.
protean check: Validate a domain and report architecture fitness diagnostics as rich, JSON, SARIF, or GitHub annotations.
protean upgrade-check: Read-only diagnostic that reports the 0.16 upgrade actions for a domain (elements, config, infra) and generates the outbox migration SQL.
Domain Discovery: Use --domain to load and initialize domains.
protean new: Initialize new projects.
protean shell: Interactive shell with the domain pre-loaded.
protean server: Run the async background message processing server.
protean docs: Live preview server for documentation.
protean test: Run Protean's framework test suite (development only).
protean ir show: Display the domain's IR as JSON or a human-readable summary.
protean schema: Generate and inspect JSON Schema files for data-carrying domain elements.
protean snapshot: Create snapshots for event-sourced aggregates.
protean projection: Rebuild projections by replaying events from the event store.
protean events: Inspect the event store: read streams, view stats, search events, trace aggregate history, and follow causal chains as a tree or flat table.
Type Checking: Static type checking with the Protean mypy plugin.
Architecture Fitness Functions: Enforce DDD architectural decisions on every commit with protean check.
Fitness Function Catalog: Every diagnostic rule, its rationale, and its fix.
Init & Runtime Diagnostics: The coded exceptions raised at init and by runtime accessors, each with its rationale and fix.

Test Your Application

Compatibility Checking: Detect breaking changes to your domain model with IR diffing, pre-commit hooks, and CI integration.
Testing Strategy: Layered testing approach with fast in-memory adapters.
Domain Model Tests: Unit tests for aggregates, entities, value objects, and invariants.
Application Tests: Validate commands, handlers, and services.
Event Sourcing Tests: Fluent test DSL for event-sourced aggregates using protean.testing.given.
Integration Tests: Verify behavior with real infrastructure.
Test Query Shape: Assert query count and shape to catch query-cost regressions.
Fixtures and Patterns: Reusable pytest fixtures and conftest recipes.
Pytest Plugin Reference: --protean-env, --update-snapshots, and the registered markers.

Python API Reference

Auto-generated documentation from source code docstrings.

API Reference Overview: Landing page for the Python API reference.
Domain: The central registry class.
BaseAggregate: Aggregate root entity base class.
BaseEntity: Entity base class.
BaseValueObject: Immutable value object base class.
Messages: BaseCommand and BaseEvent.
Handlers: BaseCommandHandler and BaseEventHandler.
Services: BaseApplicationService and BaseDomainService.
BaseRepository: Repository base class.
UnitOfWork: Transaction boundary.
QuerySet: Chainable query builder.
BaseDatabaseModel: Database model base class.
Read Models: BaseProjection and BaseProjector.
Event Processing: BaseSubscriber, BaseProcessManager, and BaseUpcaster.
Fields: All field types and FieldSpec.
Ports: Adapter interface contracts.
Testing DSL: Fluent test helpers for event-sourced aggregates.

Core Concepts

Architectural theory and the building blocks of domain-driven systems.

Foundations

Ubiquitous Language: The shared vocabulary between domain experts and developers.
Bounded Contexts: Boundaries within which a domain model is defined and applicable.
Analysis Model: Bridging the gap between domain understanding and software implementation.
Identity: The property that distinguishes one domain object from all others.
Invariants: Business rules that must always hold true within a domain concept.
Changing State: Principles governing how state transitions occur in DDD.
Streams: The primary unit of organization in evented systems.

Architecture Patterns

Domain-Driven Design: Tactical elements and their roles in the DDD pattern.
CQRS: Separating read and write responsibilities.
Event Sourcing: Deriving state from replaying event sequences.
Choosing an Architecture: When to use CQRS vs Event Sourcing.

Building Blocks

Domain Elements Overview: Tactical patterns organized into four layers.
Aggregates: Clusters of objects treated as a single unit for data changes.
Entities: Mutable objects with distinct identity.
Value Objects: Immutable elements distinguished by properties.
Domain Services: Domain logic that doesn't fit within aggregates.
Events: Immutable facts indicating state changes.
Commands: Intentions to change system state.
Command Handlers: Process commands and execute domain logic.
Event Handlers: React to events with side effects and state synchronization.
Application Services: Coordinate use cases at the boundary between external world and domain.
Repositories: Collection-oriented persistence abstraction for aggregates.
Subscribers: Consume messages from external brokers.
Projections: Read-optimized denormalized views.
Projectors: Specialized event handlers that maintain projections.
Queries: Immutable, validated read intents targeting a projection.
Query Handlers: Process queries and return read-side results from projections.
Process Managers: Stateful coordinators for multi-step processes across aggregates.

Adapters

Plug-in infrastructure that keeps your domain code free of technology dependencies.

Ports and Adapters: Hexagonal Architecture in Protean.
Adapters Catalog: Available database, broker, cache, and event store adapters.

Database

Database Providers: Overview of supported database adapters, capabilities, and provider registry.
Memory: Default in-memory provider for development and testing.
SQLite: File-based relational provider using SQLAlchemy.
PostgreSQL: SQLAlchemy-based adapter for PostgreSQL.
MSSQL: SQLAlchemy and pyodbc adapter for Microsoft SQL Server.
Elasticsearch: Document store adapter for search and analytics.
Custom Database Adapters: Build your own database adapter with entry-point registration.

Brokers

Broker Overview: Unified interface for message broker implementations.
Inline Broker: Synchronous in-memory broker for development and testing.
Redis Streams: Durable ordered messaging with consumer groups.
Broker Partitioning Contract: The port contract an adapter implements to support sequential_by: partition discovery, fenced ownership leases, and stream trimming.
Redis PubSub: Redis Lists-based queuing with consumer groups.
Custom Brokers: Build your own broker adapter.

Caches

Cache Overview: Cache port interface, available providers, and configuration.
Redis Cache: Persistent distributed cache with TTL support.

Event Stores

Event Store Overview: Event store port, core operations, temporal queries, and causation tracing.
MessageDB: MessageDB event store adapter.

Patterns & Recipes

In-depth guides for recurring patterns in domain-driven applications. These span multiple domain elements and represent good practices that Protean supports but does not enforce.

Aggregate Design

Design Small Aggregates: Draw boundaries around consistency requirements, not data relationships.
One Aggregate Per Transaction: Modify one aggregate per handler; use events for cross-aggregate side effects.
Optimistic Concurrency as a Design Tool: Classify version conflicts by business meaning: last-writer-wins, business rejection, or conditional merge.
Designing for Concurrent Event Processing: Prevent race conditions when multiple handlers process events for the same aggregate concurrently.
Encapsulate State Changes: Express every state change as a named method capturing business intent.
Replace Primitives with Value Objects: Extract strings and numbers into value objects with format rules and operations.
Factory Methods for Aggregate Creation: Encapsulate complex construction in factory classmethods on the aggregate or standalone factory classes.
Model Aggregate Lifecycle as a State Machine: Define explicit lifecycle states and guarded transition methods to enforce valid state changes.

Event-Driven Patterns

Design Events for Consumers: Events carry enough context for consumers to act independently.
Idempotent Event Handlers: Produce the same result whether an event is processed once or many times.
Event Versioning and Evolution: Evolve event schemas without breaking consumers or the event store.
Command Idempotency: Ensure processing the same command twice produces the same effect.
Coordinating Long-Running Processes: Use a process manager to coordinate multi-step workflows across aggregates, with idempotent handlers, compensation, and timeout strategies.
Message Tracing in Event-Driven Systems: Correlation and causation IDs for end-to-end traceability across commands and events.
Enrich Messages with Cross-Cutting Metadata: Inject tenant IDs, user context, and request trace IDs into events and commands via enrichment hooks, keeping the domain model clean.
Multi-Tenancy in Event-Driven Systems: Row-level tenant isolation using enrichers, metadata.extensions, and automatic context propagation through async processing.
CloudEvents as a Boundary Contract: Serialize Protean events to the CloudEvents v1.0 standard at system boundaries; use to_cloudevent() and from_cloudevent() as an anti-corruption layer for interoperability.

Architecture & Quality

Organize by Domain Concept: The folder tree owns domain concepts; the framework carries layer metadata. Organize by aggregate, colocate capabilities, separate projections.
Index Aggregates for Query Paths: Declare indexes for the query paths an aggregate actually has; match composite order to filter-then-sort; keep hot-set indexes small with partial indexes.
Validation Layering: Different validation belongs at different layers: fields, value objects, invariants, handlers.
Model Reference Data as Domain Concepts: Avoid the One True Lookup Table; model each kind as an enumeration value object or an aggregate keyed by its natural code.
Track Audit and Lifecycle Fields: Stamp created_at/updated_at and created_by/updated_by via an abstract base with auto_now/auto_now_add timestamps and a pre-persist enricher, instead of hand-writing them in every mutator.
Thin Handlers, Rich Domain: Handlers orchestrate; aggregates and domain services contain all logic.
Choose Between Application Services and Command Handlers: Decision tree for choosing synchronous application services vs async command handlers.
Design Projection Granularity Around Consumer Needs: Shape projections around UI views and API resources, not domain entities or endpoints.
Treat Projection Rebuilds as a Deployment Strategy: Rebuild projections from the event store instead of migrating database schemas.
Bridge the Eventual Consistency Gap in User Interfaces: Three strategies (optimistic UI, write-side result, version polling) to handle the CQRS read/write delay.
Testing Domain Logic in Isolation: Test aggregates and value objects directly, without infrastructure.

Identity & Communication

Creating Identities Early: Generate aggregate identities at creation, not at the database.
Connecting Concepts Across Bounded Contexts: Synchronize the same real-world concept across multiple contexts.
Consuming Events from Other Domains: Subscribers as anti-corruption layers for external events.
Sharing Event Classes Across Domains: Share schemas, not code; use contract tests for compatibility.
Use Fact Events as Cross-Context Integration Contracts: Publish full aggregate snapshots for external consumers instead of forcing them to reconstruct state from granular deltas.
Publishing Events to External Brokers: Deliver published events to external brokers via the outbox with independent retry, failure isolation, and stripped metadata envelopes.

Testing & Infrastructure

Test Event-Driven Flows End-to-End: Three testing levels for event chains: domain unit tests, sync flow tests, and async E2E tests with the Engine in test mode.
Setting Up and Tearing Down Databases for Tests: Manage database schema and test data lifecycles separately for fast, isolated integration tests.

Operations

Running Data Migrations with Priority Lanes: Route migration events to a separate backfill lane so they do not block production event processing.
Classify and Handle Async Processing Errors: Override handle_error() to classify failures as transient, data, or logic errors and route each to the right recovery path.
Temporal Queries for Audit, Debugging, and Compliance: Use at_version and as_of on event-sourced repositories for compliance audits, incident investigation, and customer support.

Internals

Design reasoning and internal architecture for contributors and advanced users.

Internals Overview: What this section covers.
Field System: How FieldSpec translates domain vocabulary to Pydantic, and why three definition styles are supported.
Shadow Fields: How ValueObject and Reference fields are flattened into database columns via shadow fields.
Query System: How the Repository → DAO → QuerySet → Provider chain works, Q object expression trees, lookup resolution, and lazy evaluation.
Event Sourcing: How raise_() invokes @apply handlers, aggregate reconstitution, version tracking, causation chain traversal algorithms, and projection rebuilding.
Event Upcasting: How old event payloads are transparently transformed to the current schema during deserialization.
IR Specification: The portable JSON schema capturing domain model topology.

Migration

Compatibility Reference: Breaking change rules, three-tier taxonomy, deprecation lifecycle, and config.toml reference.
Migrating to 0.18: Upgrade guide for the 0.18 release. Covers the right-sized dependency surface: the web/observatory stack, the shell, and the scaffolder move behind install extras.
Migrating to 0.17: Upgrade guide for the 0.17 API-consolidation release. Covers the event_sourced option rename, the internalized is_fact_event option, and the email subsystem deprecation.
Migrating to 0.16: Upgrade guide for the 0.16 stability release. Covers the outbox column-bounds structural change and its per-backend ALTER TABLE recipes.
Migrating to 0.15: Upgrade guide for the Pydantic v2 foundation release. Covers breaking changes, field style migration, and new features.

Reference

Glossary: Definitions of key terms.
Philosophy & Design Principles: The convictions that guide Protean's design.
The Always-Valid Domain: How four validation layers guarantee domain objects are never invalid.
Applicability Charter: What Protean is a good fit for, and the shapes of systems it is not, with the reason in each case.
Consistency & Delivery Guarantees: Per-port, per-adapter ordering, delivery, consistency, and isolation, the contract conformance and property tests cite.
Versioning Policy: What a version number promises: code that runs warning-free on 1.N runs unmodified on 1.N+1.
Stable Surface: Which imports, options, config keys, and commands the compatibility contract covers, tier by tier.
Why Protean?: Four capabilities that set Protean apart: domain compiler, always-valid domain, progressive architecture, infrastructure portability.

Testing

Testing Reference: Testing tools and plugins.
Pytest Plugin: Protean's pytest integration and fixtures.
Adapter Conformance Testing: Verify that custom adapters correctly implement their declared capabilities using the protean test test-adapter CLI and pytest plugin.

Community

Community: Get help and connect with other Protean users.
Development Setup: Set up Protean locally for contributing.
Testing Protean: Test strategy, fixtures, and running the suite.
Mutation Testing: Find under-asserted code paths with make mutation and close them with new tests.
Building Adapters: Overview of creating custom adapters for databases, brokers, event stores, and caches with links to per-port guides.

Contents

Getting Started

Tutorial: Building Bookshelf