Migrating to 0.18
From 0.17 to 0.18
Protean 0.18 right-sizes the runtime dependency surface. A plain
pip install protean no longer forces a web server, an ASGI stack, an
interactive REPL, and a project scaffolder onto every install. Those concerns
now live behind install extras, matching how the database and broker adapters
have always worked. See
ADR-0029
for the boundary and the rationale.
This is a Tier-1 breaking change, but a small one: nothing in your code changes, and the fix when you hit it is a one-word extra. If a feature you use now needs an extra, the CLI or the import tells you exactly which one.
On this page:
- What moved behind extras
- The one-line fix:
protean[all]restores the old install - What each feature needs now
- cffi and greenlet left the core install
- werkzeug left the core install
checkandverifymachine output changed shape and exit codesevents catalog,subscriptions status, andprojection statusJSON moved onto the envelope- The memory cache no longer raises on an absent key:
remove,remove_by_keyandset_ttlare silent get_ttlreturnsNoneormath.infinstead of a raw sentinel: the same three answers on every adapter- Cache key patterns are a glob on every adapter: the memory cache matches with
fnmatch, like Redis - Cache
get_allis now the private, unpaginated_get_all: a bounded utility, not a production read path - The IR schema is now v0.2.0: regenerate a committed
.protean/ir.jsonbaseline protean irrejects an unknown format value: a typo in a script now fails instead of printing the default formatDAO.update()andQuerySet.update()are deprecated: the patch-and-persist path warns now and is removed in 1.0.0- String and Text no longer sanitize by default: the
sanitizedefault flipped toFalse; sanitization is opt-in now - The InlineBroker no longer dead-letters a message its subscription owns:
{stream}:dlqis the one destination; nothing to change in your code
What moved behind extras
Five packages that used to be hard runtime dependencies are now install extras:
| Package(s) | Extra | Feature it gates |
|---|---|---|
fastapi, uvicorn, jinja2 |
protean[server] |
protean observatory, protean.integrations.fastapi |
ipython |
protean[shell] |
protean shell |
copier |
protean[scaffold] |
protean new |
Two convenience bundles are also available:
protean[cli]: The full interactive CLI experience (shell+scaffold).protean[all]: Everything that used to ship in the core install (server+cli).
import protean and defining a domain, persisting through the memory adapter,
and running protean server (the async engine) need none of these: they
were already lazy, and they stay in the lean core.
The one-line fix
If your project relied on the old fat install and you would rather not think
about which extras you need, install all:
pip install "protean[all]"
That restores every package that shipped in the 0.17 core. From there you can trim to the specific extras you actually use whenever you like.
What each feature needs now
You only need to act if you use one of these features and you install a
trimmed set of dependencies (for example in a production image that installs
protean without [all]):
- Running the Observatory (
protean observatory) or using the FastAPI integration (from protean.integrations.fastapi import ...):
pip install "protean[server]"
- Opening the domain shell (
protean shell):
pip install "protean[shell]"
- Scaffolding a project (
protean new):
pip install "protean[scaffold]"
If you run one of these without its extra, the command fails with a clear
message naming the extra to install, not a raw ModuleNotFoundError:
Error: 'protean observatory' requires the 'fastapi' package. Install it with 'pip install "protean[server]"'.
bleach stays in the core install. String and Text fields no longer sanitize
by default (see
String and Text no longer sanitize by default),
but bleach still backs the opt-in sanitize=True path.
cffi and greenlet left the core install
0.17 installed cffi and greenlet as hard dependencies, but nothing in
Protean imports either package. They only ever arrived as transitive
dependencies: SQLAlchemy (the postgresql, sqlite, and mssql extras)
requires greenlet on common platforms, and cryptography (pulled in by the
sendgrid extra) requires cffi.
In 0.18 both are out of the core install. Their version floors moved onto the
extras that pull them (greenlet>=3.2.3,<4 on the SQLAlchemy extras,
cffi>=2.0.0 on sendgrid), so installing one of those extras still resolves
a version with wheels for the newest supported Python.
You only need to act if your own code imports cffi or greenlet and relied
on Protean to provide it. Declare the package in your own project instead.
werkzeug left the core install
0.17 used Werkzeug's LocalProxy/LocalStack to back current_domain,
current_uow, and g. In 0.18 these context locals are backed by stdlib
contextvars, and werkzeug is no longer a core dependency.
For nearly all code this is transparent: the same push/pop nesting, the same
proxy behavior, and the same async support (in fact, contextvars follows
asyncio tasks and await boundaries natively). The public surface of
current_domain, current_uow, and g is unchanged.
You only need to act if your own code depends on Werkzeug-specific details of
the proxy or stack, such as inspecting the underlying LocalStack directly.
Internal helpers like _domain_context_stack and _uow_context_stack remain
private and have changed their implementation type.
check and verify machine output changed shape and exit codes
protean check --format json and protean verify --json now emit one shared,
versioned result envelope and adopt one exit-code
convention (that other commands can converge onto later). check's JSON output
has shipped before (0.15.0; the diagnostic schema and [lint].level gating
landed in 0.17.0) and verify is new in this cycle, but Protean is still
pre-1.0 and neither has a stability guarantee yet, so this is a direct
correction rather than a deprecation-managed break. You only need to act if you
already parse check --format json or verify --json output, or branch on
their exit codes.
The JSON shape. check --format json was the raw Domain.check() document
(domain, status, errors, diagnostics, counts at the top level). It is
now the envelope:
{
"version": "0.1.0",
"status": "fail",
"data": { "domain": "...", "status": "warn", "errors": [], "counts": {} },
"diagnostics": [ "..." ]
}
The check report moved under data; diagnostics is at the top level. A
consumer reading counts now reads data.counts; one reading diagnostics
reads the top-level diagnostics (unchanged path). verify --json gained the
same frame: its old {verdict, stages} is now under data.
The sarif and github-annotations formats are unchanged: they are external
standard schemas and are not wrapped.
--level no longer filters the JSON envelope. --level is now a display
filter for the human rich/--quiet views only; the JSON, SARIF, and
github-annotations machine formats always carry the full, unfiltered set of
findings (a machine consumer can filter for itself). diagnostics can still be
empty on a status="fail" envelope, because a fatal error (a malformed [lint]
config, an unresolved reference) fails the run without ever building
diagnostics; check data.errors for those. If you relied on
check --format json --level error to pre-filter diagnostics, filter the
output with jq instead.
The exit codes.
check:
| Case | Old | New |
|---|---|---|
| Clean | 0 |
0 |
| Validator errors | 1 |
1 |
| Gating warning/info at the floor | 2 |
1 |
Bad option, unloadable domain, malformed [lint] config |
1 |
2 |
The 1-error-vs-2-warning split collapsed into a single findings-failure code
(1); the severity now lives in the envelope. Load and config errors became
usage errors (2).
verify shifted its codes up so 2 is reserved for usage:
| Case | Old | New |
|---|---|---|
| All green | 0 |
0 |
Usage (bad/missing --domain, --path not a directory) |
1 |
2 |
| Init failed | 2 |
3 |
| Check failed | 3 |
4 |
| Tests failed | 4 |
5 |
Clean stdout. Under machine output, stdout now carries only the envelope,
exactly one JSON object. Logs and error lines go to stderr, so
protean check --format json | jq and protean verify --json | jq stay
parseable.
events catalog, subscriptions status, and projection status JSON moved onto the envelope
protean events catalog --json, protean subscriptions status --json, and
protean projection status --json each printed a bare JSON array. They now emit
the same result envelope that check and verify use,
so one shape parses every command. The array moves under data, keyed by the
command:
| Command | Old top level | New location |
|---|---|---|
events catalog --json |
array of events | data.events |
subscriptions status --json |
array of subscriptions | data.subscriptions |
projection status --json |
array of projections | data.projections |
A run that read the array at the top level now reads it under its data key:
{
"version": "0.1.0",
"status": "pass",
"data": { "events": [] },
"diagnostics": []
}
All three are pre-1.0 with no stability guarantee, so this is a direct
correction with no deprecation window. You only need to act if you already parse
one of these --json outputs: read data.events, data.subscriptions, or
data.projections in place of the top-level array.
All three also adopt the shared exit codes and clean stdout on their error paths
under --json. Under --json, any usage or load failure (an unloadable
--domain, a missing/unreadable --ir file, or, for events catalog, neither
--domain nor --ir or both at once) is now the error envelope
(status="error", the message under data.error) on stdout and exits 2, so
| jq stays parseable.
Two things change without --json as well. The events catalog usage errors
(neither --domain nor --ir, or both at once) now print to stderr and exit
2, where they exited 1 before. A domain or --ir load failure without
--json is unchanged: it prints a human message and exits 1.
The memory cache no longer raises on an absent key
Behavioural break, memory cache only. remove(projection),
remove_by_key(key) and set_ttl(key, ttl) raised KeyError when the key was
not in the cache. They now do nothing, which is what the Redis cache has always
done: Redis' DEL and PEXPIRE are no-ops on a missing key. The port
documents the silent behaviour as the contract, so the two adapters answer
alike.
An expired entry counts as absent. A key whose TTL has run out is treated the same as one that was never written, on both adapters.
Who is affected
Anyone who used the KeyError as a presence check on a memory-backed cache.
That code no longer takes its except branch, and nothing tells you: the call
succeeds quietly. It is the only shape that breaks.
# Before: worked on the memory cache, never fired on Redis
try:
cache.remove_by_key(key)
except KeyError:
log.info("nothing cached for %s", key)
# After: ask before removing, and it works on either adapter
if cache.get(key) is None:
log.info("nothing cached for %s", key)
cache.remove_by_key(key)
Nothing inside Protean relied on the KeyError, so this only affects your own
code.
What did not change
set_ttl still rejects an invalid TTL when the key is absent. A TTL that is
not a positive, finite number of seconds raises ConfigurationError whether or
not there is a key to apply it to, so a malformed TTL cannot hide behind a
missing key.
get_ttl now answers alike on both adapters too; see
get_ttl returns None or math.inf instead of a raw sentinel.
Why it was not put behind a flag
The behaviour was a divergence between two adapters of the same port, not an
intentional contract, so no program that runs on both could have depended on
it. A flag preserving the KeyError would have kept the divergence alive on
the adapter that is documented for development and testing, which is the one
place the inconsistency is least worth paying for.
get_ttl returns None or math.inf instead of a raw sentinel
Behavioural break, both cache adapters. get_ttl(key) used to answer
differently depending on the adapter, and neither answer was easy to use. The
memory cache raised KeyError for a missing key and could never report a
never-expiring key. Redis returned its raw PTTL sentinels: -2 for a missing
key, -1 for a key with no expiry. A caller that wanted to work on both had to
handle an exception and two magic negatives.
Both adapters now answer the same three cases:
Nonewhen there is no such key.math.infwhen the key exists and never expires.- the seconds remaining otherwise (unchanged).
An expired entry counts as absent, so get_ttl returns None for it, the same
way get does. math.inf only ever comes from Redis: the memory cache writes
every entry with a concrete TTL, so it has no never-expiring keys to report.
Who is affected
Anyone who read get_ttl's old return value directly.
import math
# Before: two shapes, one per adapter
try:
ttl = cache.get_ttl(key) # memory: raises KeyError if absent
except KeyError:
ttl = None
if ttl is not None and ttl < 0: # redis: -2 absent, -1 no expiry
ttl = None
# After: one shape on both adapters
ttl = cache.get_ttl(key)
if ttl is None:
... # no such key
elif ttl == math.inf:
... # never expires
else:
... # seconds remaining
A ttl < 0 check on the old Redis return no longer fires, because the negative
sentinels are gone. A ttl < math.inf check is the way to ask "does this key
expire".
Why it was not put behind a flag
Like the absent-key change above, this was a divergence between two adapters of
the same port, not an intentional contract. BaseCache is a Provisional API, so
its signatures can still change without a deprecation period. Nothing inside
Protean read get_ttl, so only your own callers are affected.
Cache key patterns are a glob on every adapter
Behavioural break, memory cache. _get_all, count, and
remove_by_key_pattern take a key_pattern. The memory cache used to compile
it as a Python regular expression, while Redis passed it to SCAN ... MATCH as
a glob. The same string selected different keys depending on which adapter you
ran, and a pattern that leaned on regex behaviour matched nothing once you
switched to Redis.
The memory cache now matches with fnmatch, the same glob language Redis uses.
* matches any run of characters, ? matches one, [...] is a character class,
and other characters, including ., are literal. Adapters agree on *, ?, and
literal characters; bracket negation and escaping can differ, so keep patterns to
the name:::* shape.
Who is affected
Anyone who passed a regex-flavoured key_pattern to the memory cache. Cache
keys are name:::identifier, so most patterns are already a name and a *,
which is a valid glob and needs no change. A pattern that used regex syntax
needs rewriting:
# Before, on the memory cache: a regex (and the method was public then)
cache.get_all("user_profile:::.*")
# After, on every adapter: a glob (the method is now the private `_get_all`)
cache._get_all("user_profile:::*")
A literal . in a pattern used to match any character on the memory cache and
now matches only a .. BaseCache is a Provisional API, so this lands in a
minor release with no deprecation period.
Cache get_all is now the private, unpaginated _get_all
Breaking, both cache adapters. The public cache.get_all(key_pattern,
last_position, size) is gone. It is now cache._get_all(key_pattern): private
(the leading underscore), and it returns every matching entry in key order with
no pagination.
A cache is for point reads by key. Enumerating a match set, and paging through it, is not what a cache is for, and offering offset pagination over it invited use it cannot support: on a store with no native key order (Redis) every call scans the whole keyspace, and paging a large set is quadratic. So the method is marked private and scoped to what it is actually good for, a convenience for tests and small, bounded stores.
It returns at most 1000 entries (GET_ALL_MAX). Past that it truncates to the
first 1000 in key order and logs a warning, so a store that outgrew the cache
surfaces as a warning rather than a silent partial read.
Who is affected
Anyone calling cache.get_all(...). The method no longer exists under that name,
and it no longer takes last_position or size.
# Before
for start in range(0, total, 50):
for entry in cache.get_all(pattern, last_position=start, size=50):
process(entry)
# After: a bounded read of the whole match set (small stores, tests)
for entry in cache._get_all(pattern):
process(entry)
If you need to page a large projection set, that belongs in the projection's
repository, not the cache. Query repository_for(Projection).query, which has
real indexes and native pagination. The cache stays for hot point lookups by key.
Why it was renamed rather than deprecated
BaseCache is a Provisional API, so it can change without a deprecation period,
like the other cache changes above. _get_all is private for the same reason it
is unpaginated: it is not a surface to build on. count and
remove_by_key_pattern stay public and take the same key_pattern glob; note
that count also scans the whole keyspace on Redis.
The IR schema is now v0.2.0
The materialized IR carries a new method_edges key on the elements that own
methods, so the schema version moved from 0.1.0 to 0.2.0. Documents and
schemas validate both ways: a 0.1.0 document still passes the 0.2.0 schema
(method_edges is optional), and a 0.2.0 document still passes the 0.1.0
schema. The v0.1.0 schema stays published, so a pinned $schema URL keeps
resolving.
What does change is the checksum. Adding a key changes the canonical bytes, so a
baseline built under 0.1.0 cannot be compared against a live 0.2.0 IR.
Who is affected
Anyone with a committed .protean/ir.json. protean ir check now reports a
version mismatch and exits 3 instead of comparing checksums. The
protean-check-staleness pre-commit hook prints the same message and fails (it
exits 1 for every failure), so a CI job that gates on the hook goes red until
the baseline is regenerated.
protean ir show --domain <module> --canonical > .protean/ir.json
The pre-commit hook rewrites it for you if you run it with --fix.
Regenerating is the whole migration: your domain code does not change. The rest
of the document is byte-identical apart from $schema, ir_version, the
checksum, the new method_edges entries, and the sanitize key on String and
Text fields. That key now appears only for a field that passes an explicit
sanitize=True, so it drops off every field that left sanitize unset (see
String and Text no longer sanitize by default).
What a cross-version diff looks like
protean ir diff against a 0.1.0 baseline reports method_edges as added on
every element that has one, and sanitize as removed on every String/Text field
declared without an explicit sanitize=. Compatibility classification only
reads field names and types, so protean ir compat and the
protean-check-compat hook ignore both and do not report a break. Regenerate
the baseline and diff two 0.2.0 documents if you want the noise gone.
protean ir rejects an unknown format value
protean ir show, protean ir diff, and protean ir check check --format
before they load anything. An unknown value prints the accepted ones and aborts:
$ protean ir diff --left baseline.json --right current.json --format=summary
Error: invalid --format: 'summary'. Choose from: text, json, event-model
Aborted.
Each command takes its own set:
| Command | Accepted --format values |
Default |
|---|---|---|
protean ir show |
json, summary |
json |
protean ir diff |
text, json, event-model |
text |
protean ir check |
text, json |
text |
Who is affected
Anyone whose script passes a value outside that command's set. It used to fall
through to the command's default: protean ir diff --format=summary printed the
text diff, and protean ir show --format=text printed the full JSON. That was
silent, so a typo in a CI job or a value borrowed from a sibling command read as
success for as long as nobody compared the output to the flag.
The failure exits 1, the same code every other argument error in these
commands uses. On protean ir diff that is also the breaking-changes code, so a
CI gate that branches on the exit code sees a usage error as a break until the
flag is fixed; the invalid --format line on stdout is what tells them apart.
The fix is to pass a value from the table. Nothing else about the commands changed, and a script that already passed a valid value is unaffected.
protean new --pretend is now --dry-run, and it writes nothing
Flag rename and behavioural break, the protean new dry run only. The flag
is --dry-run now, and --pretend (with its -p short form) is gone. Passing
--pretend is a usage error. --pretend was copier's name for the option; the
rest of the CLI already says --dry-run, and so does the create_project core
it calls (dry_run=).
The dry run also behaves differently. It used to render through copier's own
pretend flag, and everything around that render ran for real. So a dry run
reached the target directory before it printed anything: with --force it
cleared the target, and without --force on a non-empty target it raised
FileExistsError. Neither is something a preview should do.
--dry-run renders into a system temp directory and prints the
project-relative path of each file it would create, one per line, sorted. The
target directory is left alone whether or not it already has files in it, and
--force no longer clears anything under a dry run. The list it prints is the
same set of files an apply produces from the same inputs.
The preview is also complete now. .protean/project.json, AGENTS.md, the
CLAUDE.md bridge, and .protean/dx-state.json are written after the copier
render, so the old dry run never listed them even though an apply creates all
four.
Who is affected
Anyone passing --pretend or -p to protean new. Pass --dry-run instead.
Anyone parsing the output of the dry run. It used to be copier's coloured
create <path> log, in copier's render order; it is now a plain list of
relative paths, sorted.
# Before: copier's own log, no manifest, no AGENTS.md
$ protean new myproject --pretend
Copying from template version None
create .gitignore
create docker-compose.override.yml
...
# After: one project-relative path per line, sorted, and the full set
$ protean new myproject --dry-run
.copier-answers.yml
.dockerignore
.env.example
...
.protean/dx-state.json
.protean/project.json
AGENTS.md
CLAUDE.md
...
src/myproject/domain.py
Anyone who used --pretend --force to clear a directory is also affected. That
was a side effect of how the dry run was wired, not a documented feature, but it
did delete files. Use protean new <name> --force without --dry-run if
clearing the target is what you want.
Why it was not put behind a flag
A flag defaulting to the old behaviour would default to a preview that deletes your files. There is no reading of a dry run under which that is the contract a user asked for, so there is nothing to preserve.
protean new . and protean new .. are rejected
Input break. A project name has to be a single directory segment now: a
non-empty string, not . or .., and with no <>:"/\|?* or whitespace. The
first two used to pass validation. Passing one raises ValueError before any
file is touched.
Who is affected
Anyone calling protean new . expecting it to scaffold in place. It never did
that. project_name feeds the template, so . produced a project with no
package directory (src/domain.py instead of src/<name>/domain.py) and
name = "." in pyproject.toml, which no installer accepts. The generated tree
did not build.
# Before: rendered a tree that could not be installed
mkdir myproject && cd myproject && protean new .
# After: name the project, then work inside it
protean new myproject && cd myproject
The rule also closes a hole in --force. The clear removes the contents of
<output_folder>/<project_name>, so a name that escaped its segment (. or
..) pointed the clear at the output folder or its parent. Requiring a single
segment means the clear can only ever reach the project directory.
protean new refuses a target symlinked out of the output directory
Behavioural break, symlinked targets only. protean new <name> writes to
<output-dir>/<name>. If something already at that path was a symlink, every
step followed it: --force cleared the directory the link pointed at, and the
render wrote its files there. A link pointing outside the output directory meant
files outside the output directory were deleted and written.
The command now resolves the target first. If it does not sit directly inside
the resolved output directory, the command stops with a ValueError before it
clears or writes anything. protean.scaffold.create_project raises the same
error, on apply and under dry_run.
Who is affected
Anyone who symlinked a project directory to another disk or another checkout and
then re-ran protean new over it. Render into the real location and symlink to
it afterwards, or point --output-dir at the directory that actually holds the
project:
# Before: followed the link and wrote (or cleared) at /mnt/work/myproject
ln -s /mnt/work/myproject myproject
protean new myproject --force
# After: name the real location
protean new myproject --output-dir /mnt/work --force
A symlink is still fine when it resolves to a path directly inside the output
directory, say <output-dir>/other-name. One that resolves deeper, like
<output-dir>/nested/other-name, is refused too: the check is that the target's
resolved parent is the resolved output directory.
DAO.update() and QuerySet.update() are deprecated
A deprecation, not a break. Both methods still work in 0.18. They now emit a
RemovedInProtean10Warning (a DeprecationWarning subclass) and are removed in
1.0.0. Nothing else about their behaviour changes.
DAO.update() and QuerySet.update() take a field patch and write it straight
to the store. That skips the aggregate: the change does not go through a
behaviour method, so the rule that owns the change and the events it should raise
never run. The replacement is to load the aggregate, call a method on it that
applies the change, and persist it with repository.add().
Single aggregate
# Before: patch-and-persist through the DAO
repo = domain.repository_for(Order)
dao = repo._dao
order = dao.get(order_id)
dao.update(order, status="shipped")
# After: load, invoke a behaviour method, add
repo = domain.repository_for(Order)
order = repo.get(order_id)
order.ship() # the aggregate applies the change and raises OrderShipped
repo.add(order)
A child entity has no repository of its own. Load the aggregate root, change the
child through the root, and repo.add() the root.
Many aggregates
QuerySet.update() patched every matched row in one call, but it was never one
transaction: it looped and persisted each row on its own. Load the matched
aggregates, invoke the behaviour method on each, and repo.add() each one.
# Before: bulk patch through the query set
repo = domain.repository_for(Order)
repo._dao.query.filter(status="pending").update(status="cancelled")
# After: load each match and invoke the behaviour method on it
repo = domain.repository_for(Order)
for order in repo._dao.query.filter(status="pending").all():
order.cancel()
repo.add(order)
Each add() commits in its own transaction, which is the same granularity
QuerySet.update() gave you. Do not wrap the loop in one UnitOfWork to make
the batch atomic: every Order is its own consistency boundary, and
one transaction, one aggregate
is the rule. If the whole sweep has to land or fail together, it is a backfill,
not domain behaviour.
Large backfills
A one-off migration that rewrites a column across a whole table is an
infrastructure job, not domain behaviour, the same way a hard delete is. Do it at
the adapter or database level (a SQL UPDATE, an adapter-specific bulk write)
rather than reaching for the deprecated domain-level path.
String and Text no longer sanitize by default
Behavioural break, String and Text fields. An ordinary String or Text field
that left sanitize unset used to run bleach.clean() on every value, so the
stored value was HTML-escaped. The default is now False: an unset field stores
the raw input. sanitize=True and sanitize=False still mean exactly what they
did.
Two shapes were never sanitized and so do not change: a field with a real
choices= value, and a String or Text used as a container content spec such as
List(String(max_length=50)). upgrade-check skips both, as listed below.
This is a fail-open change: where a field silently escaped its input before, it now stores it verbatim. That matters because sanitization is a display-layer concern. Escaping on write bakes one encoding (HTML) into stored data, corrupts values rendered in any other context (a plain-text email, a JSON API, a CSV export), and is not a substitute for encoding output where it is actually rendered. The default now matches that: store the real value, encode at the point of display.
Who is affected
Anyone who relied on the old default to escape stored values, on a String or
Text field with no explicit sanitize=. After the upgrade that field stores the
raw input, and nothing tells you at runtime: the write succeeds.
protean upgrade-check --opportunities finds these sites for you once you are on
0.18.0. It reports each String/Text field that leaves sanitize unset, by
module:line, listing every one so the report works as a checklist. Unset means
either not passing sanitize at all or passing None, which the field layer
reads the same way.
What it skips, because none of these changed behaviour:
- A field that gives
sanitizea real value either way, by keyword (sanitize=True) or positionally (Text(True),String(255, None, True)). It declared its intent. - A field with a real
choices=value. Choice values were never sanitized. (choices=Noneis not a choices field, so it is still reported.) - A
String/Textused as a container content spec, as inList(String(max_length=50)). A container reads the inner spec's type and choices and never applies its sanitization, so the inner field was not sanitized before the flip either. - A
StringorTextthat is not Protean's. The scan resolves the name back to aprotean.fieldsimport, sofrom sqlalchemy import Stringis not reported andfrom protean.fields import String as StringFieldstill is. - Every field, if the domain sets
[field_defaults] sanitize = true. That opts the whole domain back into the old behaviour, so there is nothing to migrate.
A field whose arguments are hidden behind a splat (String(**opts)) is also
skipped, because sanitize could be inside and a static scan cannot see it.
Check those by hand.
The fix
Encode output where these values are rendered as HTML. That is the durable fix, and it protects the values a write-time escape never could (a template that renders the same field in an email or a CSV).
To restore the old behaviour instead, opt back in. For one field:
name = String(max_length=100, sanitize=True)
For a whole domain, set the domain-level default:
[field_defaults]
sanitize = true
Precedence is field kwarg > domain default > framework default, so a domain
default of true sanitizes every unset String/Text field, and a field can still
opt out with sanitize=False.
Why it was not put behind a flag defaulting to the old behaviour
This is a deliberate deviation from the standard Tier-2 transition in
ADR-0004,
which is opt-in flag → warning window → default flip, with the flag defaulting
to the old behaviour. Here the default flips in a single release, there is no
per-field runtime warning, and the [field_defaults] sanitize flag defaults to
the new behaviour (you set it to true to opt back in). It is silent at
runtime: the write succeeds and nothing warns.
That deviation is intentional. Storing an HTML-escaped value is the wrong
default for a domain model, and bleach on write is not the security control it
looks like, so the old default was not a contract worth a multi-release
transition to preserve. A per-field runtime warning was rejected as noise on a
path that is usually harmless. The migration net is static instead:
protean upgrade-check --opportunities enumerates the exact sites (see above),
so the flip is discoverable at upgrade time even though it is silent at runtime.
This is not one of the two narrow single-release exceptions ADR-0004 already
lists (operational defaults, silent correctness bugs); it is a security-relevant
default flip settled on its own merits.
Reloading data written under a different default
The sanitize decision is read from the active domain at validation time, and a
DB load, event-sourced replay, or serialization round-trip re-runs the
validator. So a value written while sanitization was off, then reloaded after
[field_defaults] sanitize is flipped to true, is cleaned on the way back in.
Because bleach can lengthen a string (& becomes &), a value that fit
max_length on write can grow past it on reload and fail to load with
... characters after sanitization, exceeding max_length. Flip the domain
default with this in mind: it changes how already-stored values validate, not
just new writes. An explicit sanitize=True field does not have this hazard,
because it cleaned on write too, so its stored value already satisfies the bound
(see ADR-0026).
The InlineBroker no longer dead-letters a message its subscription owns
On the InlineBroker, a stream consumed by a BrokerSubscription or a
StreamSubscription now has one dead-letter destination: the {stream}:dlq
stream the subscription publishes. The broker's own nack ceiling and its native
dead-letter queue no longer apply to that stream. A nacked message is held and
redelivered instead, which is what Redis Streams already did. See
ADR-0042.
Who is affected
Almost nobody, and there is nothing to change in your code. The old behaviour
needed a {stream}:dlq publish that kept failing inside the same process, on an
in-memory test double. If you had that, a message the subscription was holding
eventually landed in the broker's native DLQ and showed up under the
{stream}:dlq name in protean dlq list. It no longer does: it stays held and
is redelivered until the DLQ publish succeeds.
A test that asserted such a message reaches protean dlq list (or
broker.get_dlq_messages) has to assert the hold instead. Drive the DLQ outage
with retry_delay=0 so the held message comes back on the next read.
A stream consumed straight off the broker with no subscription is unchanged: it still hits the InlineBroker's ceiling and lands in the native DLQ. Production adapters are unchanged too. Redis Streams holds a nacked message pending and Redis PubSub does not support nack, so neither had a second dead-letter destination to reconcile.
Why it was not put behind a flag
Two layers counting the same failure was the bug, and a flag that kept the old path alive would keep the broker dead-lettering underneath the subscription. This takes the silent-correctness-bug exception in ADR-0004.
Replay hazards on an event-sourced aggregate are now breaking
protean ir diff reports four changes it previously passed clean. Each one stops
an existing event-sourced aggregate being rebuilt from its events, and none of
them is a field change, so nothing in the checker had anything to say about them:
| Change | Reported as |
|---|---|
is_event_sourced turned on or off |
event_sourcing_changed |
An aggregate's stream_category moved |
stream_category_changed |
| The aggregate's identity moved to another field | identity_field_changed |
An @apply handler dropped while its event survives |
apply_handler_removed |
Who is affected
A domain that makes one of these changes and runs protean ir diff in a check or
a pre-commit hook. The command passed before and now exits non-zero.
If your domains make none of these changes, nothing about your diff output moves.
What to do
Treat the report as correct and roll the change back, because each one strands data that already exists:
- An aggregate writes to
f"{stream_category}-{identifier}", so moving the stream category leaves the whole history under the old name and a load of an existing aggregate finds an empty stream. Keep the old category. If the rename is genuinely wanted, copy or replay the existing streams under the new name first, as a data migration, outside the compatibility checker. - Moving the identity to another field asks for a stream keyed by a value no writer ever used, with the same result. Keep the identity where it is.
- Turning event sourcing on leaves the aggregate's existing state as table rows that replay cannot rebuild; turning it off leaves it as a stream the table-backed loader never reads. Either direction needs a data migration first.
- Dropping an
@applyhandler leaves stored events of that type with nothing to apply them, andBaseAggregate._apply_handlerraises on one with no handler. Keep the handler for as long as events of that type are in the store, even after you stop raising the event.
Where the change is deliberate and the data has been migrated by hand, the
config-level exclude list is the escape hatch, as it is for any break the
checker cannot verify:
[compatibility]
exclude = ["ordering.order.order.Order"]
Why it was not put behind a flag
The checker's purpose is to report a change that strands stored data, and these four do. Passing them clean was the bug. A flag that kept the old behaviour alive would keep the checker silent about a domain whose aggregates can no longer load, which is the case it exists to catch. This takes the silent-correctness-bug exception in ADR-0004.