Performance
Identifiers recommends the struct form of an identifier, because of its memory layout. This page has the measurements behind that advice, including two places where the measurements first disagreed with it. The Tenancy section measures the database shapes behind the Tenancy supporting domain. Seven sections measure code beside the identifiers: value object equality, FluentValidation, a read that skips a named query filter, a save on a context that tracks many aggregates, a batch of outbox messages, access checks and several requirements on one request.
The benchmarks are in Benchmarks/DTS.DDD.Benchmarks and use BenchmarkDotNet.
Run them with:
dotnet build Benchmarks/DTS.DDD.Benchmarks/DTS.DDD.Benchmarks.csproj -c Release
Benchmarks/DTS.DDD.Benchmarks/bin/Release/net10.0/DTS.DDD.Benchmarks.exe --filter *
Both identifiers wrap a Guid and have the same prefix. Only the form differs:
[EntityId<Guid>("ORD")]
public readonly partial record struct StructOrderId;
[EntityId<Guid>("ORD")]
public partial record RecordOrderId
{
public static RecordOrderId From(Guid value) => new(value);
}
Test machine
Every number below comes from one run on one machine. Compare the ratios. The absolute times will differ on your machine.
BenchmarkDotNet v0.15.8, Windows 11 (10.0.26200.9445/25H2)
12th Gen Intel Core i9-12900K 3.20GHz, 1 CPU, 24 logical and 16 physical cores
.NET SDK 10.0.301
.NET 10.0.9 (10.0.9, 10.0.926.27113), X64 RyuJIT x86-64-v3
GC=Concurrent Workstation
The Entity Framework benchmarks run against SQLite in memory, the fastest database a benchmark can use. So the share of a round trip that Entity Framework and the identifier take is as large as it can be. On a real server across a network, the same difference is smaller.
Creating identifiers
The benchmark builds 10,000 identifiers from existing values, into an array. The Guid row (RawGuid in the
benchmark) is the floor: the same array without a wrapper.
| Mean | Allocated | |
|---|---|---|
Guid | 42.89 μs | 156.29 KB |
| Struct id | 42.67 μs | 156.29 KB |
| Record id | 58.54 μs | 625.02 KB |
- The struct id costs the same as the raw
Guidarray. It allocates the same bytes, and its time is within the measurement error. - The record id costs 4 times the memory, 64 bytes per identifier against 16. That is an 8-byte reference in
the array plus a 56-byte object. The object holds the object header, the
Guid, the prefix reference, and two validation fields that it inherits fromValueObject.
Identifiers counts the same 64 bytes. The two validation fields, _isValid and
_validationErrors, hold the validation state of a value object. An identifier never uses them.
The time difference here (1.37x) is mostly garbage collection. It is the least reliable number on this page. The record run had a standard deviation of 13.8 μs against a mean of 58.5 μs, because 625 KB per operation makes the garbage collector run during the measurement.
Comparing ids in an array
The benchmark compares 10,000 identifiers in an array with a target that matches the last element, so every comparison runs.
| Mean | Ratio | Allocated | |
|---|---|---|---|
| Struct id, compare each | 3.37 μs | 1.00 | none |
| Record id, compare each | 8.46 μs | 2.51 | none |
Struct id, read .Value | 7.23 μs | 2.15 | none |
Record id, read .Value | 8.63 μs | 2.57 | none |
Reading the value through a reference costs about 20% more. That is the dereference that Identifiers predicts. Comparing costs 2.5 times as much, for the same dereference twice plus the call.
The ratio of the record id used to be 69.6, with 1,440,000 bytes allocated. A record identifier inherited its
equality from ValueObject, which compares the sequences of GetEqualityComponents():
return Enumerable.SequenceEqual(GetEqualityComponents(), other.GetEqualityComponents());
Every comparison made two iterator objects, boxed two Guids and called LINQ. That fits a value object with
several components. For an identifier that wraps one Guid, it was far too expensive, and nobody had measured it.
The generators now write a direct comparison of Value for entity ids and single value objects. The answer is
the same, because both types have one component. The numbers above are from after that change.
Reading a read-only collection
order.Lines returns a read-only view over the generated backing field. List<T>.AsReadOnly() is
new ReadOnlyCollection<T>(this) and caches nothing. So a property written as => _lines.AsReadOnly() builds a
new wrapper on every read. The generator keeps the view in a second field instead.
The benchmark reads one property 16 times:
| Time | Allocated | |
|---|---|---|
| A wrapper per read | 68.2 ns | 384 B |
| The wrapper held in a field | 10.5 ns | 0 B |
| The bare list, no protection | 5.2 ns | 0 B |
384 bytes is 16 wrappers of 24 bytes. The number of elements changes nothing. At 4 elements and at 256 the numbers are the same, because the wrapper does not copy the list.
The third row is the floor, and no real option. Returning _lines allocates nothing and satisfies
IReadOnlyList<T>. But a caller can cast it back to List<T> and change the aggregate past every invariant it
has. The 5 nanoseconds between the second and the third row are the cost of that protection, now that the
wrapper is built once.
Dictionary and set lookup
The benchmark makes 1,000 lookups that hit, in a container of 10,000 entries.
| Mean | Ratio | Allocated | |
|---|---|---|---|
Dictionary keyed by struct id | 3.53 μs | 1.00 | none |
Dictionary keyed by record id | 5.35 μs | 1.52 | none |
HashSet of struct ids | 3.55 μs | 1.01 | none |
HashSet of record ids | 4.92 μs | 1.40 | none |
Every lookup computes a hash and makes at least one equality check. So this benchmark showed the same defect as the comparison above. The ratios of the record id used to be 17.5 and 21.1, with 272,000 bytes allocated. With the direct comparison, the record form costs about 1.5 times the struct form, for the dereference and the call, and allocates nothing.
Equality and hashing
One comparison and one hash, measured alone.
| Mean | Allocated | |
|---|---|---|
Struct id == | 0.20 ns | none |
Record id == | 0.23 ns | none |
Struct id GetHashCode | not measurable | none |
Record id GetHashCode | 0.19 ns | none |
Both forms now cost close to nothing, and these numbers are at the limit of what BenchmarkDotNet can measure. For the struct hash it reports "the method duration is indistinguishable from the empty method duration", so the table gives no figure for it.
Before the generators wrote a direct comparison, an equality check of the record form took 30.47 ns and 144 bytes, and a hash took 23.81 ns and 128 bytes. Those two costs caused the whole gap of 17x to 70x in the two sections above.
ToString and Parse
ToString and Parse, the operations at the edges of the system:
| Mean | Allocated | |
|---|---|---|
Struct id ToString() | 15.52 ns | 104 B |
Record id ToString() | 19.18 ns | 104 B |
Struct id Parse | 23.62 ns | 96 B |
Record id Parse | 28.11 ns | 152 B |
This measurement went against the recommendation. The ToString of the struct identifier used to allocate
2.2 times as much as that of the record identifier: 232 bytes against 104.
The cause was the generated code of the struct. The generator used to write:
public override string ToString()
=> IdPrefix.Length == 0 ? ValueToString() : IdPrefix + "_" + ValueToString();
private string ValueToString() => Convert.ToString(Value, CultureInfo.InvariantCulture) ?? string.Empty;
Convert.ToString(object, IFormatProvider) boxed the Guid and built the 36-character string. The concatenation
then built a second string of 40 characters. That made three allocations, against one for the record form, which
used an interpolated string. A struct identifier written into a log line on every request cost 232 bytes.
The generator now writes the interpolated form for the struct too, with the invariant culture:
public override string ToString()
=> IdPrefix.Length == 0
? string.Create(CultureInfo.InvariantCulture, $"{Value}")
: string.Create(CultureInfo.InvariantCulture, $"{IdPrefix}_{Value}");
Nothing is boxed, and one string is built. Both forms now allocate 104 bytes. The table above is from after that change.
Parse without a substring
The 96 bytes of the struct's Parse were one string. The generated TryParse called Substring to cut the value
out of the input after the prefix, and parsed that copy. It now parses a span of the input. A wrapped type without a
TryParse over ReadOnlySpan<char> still calls Substring.
| Before | After | |
|---|---|---|
Struct id Parse | 45.40 ns, 96 B | 27.60 ns, none |
Record id Parse | 61.47 ns, 152 B | 41.06 ns, 56 B |
The 56 bytes of the record form are the identifier object itself. This table and the ones below come from a later run
than the tables above, with --inProcess. In that run, the times of unchanged code differed by up to a factor of two
between the before and the after run. The allocations are exact, so compare the bytes first.
Value object equality
A [ValueObject] with two properties, Money(decimal Amount, string Currency): one comparison of two equal values,
one hash, and 1,000 lookups that hit in a dictionary keyed by the value object.
| Before | After | |
|---|---|---|
Equals, same value | 89.08 ns, 144 B | 6.66 ns, none |
GetHashCode | 59.92 ns, 128 B | 11.61 ns, none |
| 1,000 dictionary lookups | 153.25 μs, 272,001 B | 38.03 μs, none |
The generator wrote the equality of every value object over GetEqualityComponents(), as it did for record
identifiers before the fix above:
return Enumerable.SequenceEqual(GetEqualityComponents(), other.GetEqualityComponents());
Each comparison allocated two iterators and boxed the decimal twice. The hash allocated an iterator, a Select
iterator and a box. The hash had two more faults. Aggregate had no seed, so GetHashCode() threw for a value object
without compared properties. XOR gave (a, b) and (b, a) the same hash.
The generator now compares each property with EqualityComparer<T>.Default of its type, and adds each property to a
HashCode:
return EqualityComparer<decimal>.Default.Equals(Amount, other.Amount)
&& EqualityComparer<string>.Default.Equals(Currency, other.Currency);
Equals returns the same result as before, because it compares the same properties with the same equality. The hash
is a different number, and it depends on the order of the properties.
FluentValidation
The benchmark creates a single value object with two FluentValidation rules, NotEmpty().EmailAddress(), and reads
IsValid. Each call creates a new value, so the validation runs on every call. The benchmarks are in their own
project, Benchmarks/DTS.DDD.Benchmarks.FluentValidation. The FluentValidation generator adds a validator to every
value object and record id of a project. In DTS.DDD.Benchmarks that would change the identifiers measured
above.
| Before | After | |
|---|---|---|
| A valid value | 1,573 ns, 2.97 KB | 169 ns, 736 B |
| An invalid value | 3,447 ns, 4.76 KB | 1,015 ns, 2,592 B |
The generated Validate() used to build a new Validator on every run:
var validator = new Validator();
var result = validator.Validate(this);
The constructor of a validator runs every rule declaration and builds an expression tree and rule objects for each.
That was most of the cost. Every value also allocated an empty list for its failures, and every read of Errors
allocated a new wrapper. Now the first validation of a type builds the validator, and every value of the type uses
that instance. A valid value uses one shared empty Errors collection. The remaining bytes are the value itself and
FluentValidation's own allocations for one run.
The Entity Framework round trip
This section measures whether the form of the identifier matters to persistence. Identifiers says it does not, based on this measurement. Both aggregates have the same shape and the same payload column. Only the key differs:
[AggregateRoot<StructOrderId>] public partial class StructOrder { ... }
[AggregateRoot<RecordOrderId>] public partial class RecordOrder { ... }
Inserting
The benchmark adds and saves 1,000 aggregates, in a new database for each measured run.
| Mean | Allocated | |
|---|---|---|
| Struct id | 30.26 ms ± 3.68 | 7.38 MB |
| Record id | 31.44 ms ± 4.15 | 7.71 MB |
The times overlap, so neither form wins. The error bars are ±12%, the distribution has two peaks, and a database write cannot be measured to three significant figures. The allocation is deterministic. The record form allocates 4% more, for the 1,000 identifier objects.
Reading
The table has 2,000 rows. "Read all" loads every row through a new context. "Find by key" makes 100 lookups of one row each.
| Mean | Allocated | |
|---|---|---|
| Read all, struct id | 1.52 ms | 1090.03 KB |
| Read all, record id | 1.39 ms | 1293.20 KB |
| Find by key, struct id | 1.11 ms | 888.16 KB |
| Find by key, record id | 1.20 ms | 899.90 KB |
The claim holds, with one correction. Neither form costs anything in persistence.
- Both forms go through the same generated value converter to the same column type. The provider test suite checks that the two forms produce the same column type.
- The database work is far larger than anything either form adds.
- The record form allocates 19% more when it reads all rows. That is one object per row, small next to the 1 MB that loading 2,000 rows costs anyway.
So persistence is no reason to choose either form.
Summary: struct or record
Identifiers recommends the struct form. The table lists the reasons for that recommendation, and what the measurements say about each:
| Claim | Verdict |
|---|---|
| The struct is 16 bytes, and the record adds a reference and a header. | True, and the difference is larger: 64 bytes against 16, because the record also carries validation state. |
| A list of ten thousand is one block instead of ten thousand objects. | True. Reading through the reference costs about 20%. |
| The struct costs nothing in persistence. | True. The record costs nothing either, so this is no reason to choose one form. |
| (unstated) Equality and hashing | This was the strongest reason, at 17x to 70x with an allocation on every comparison. The measurements for this page found the cause, and it is fixed. The gap is now 1.4x to 2.5x, with nothing allocated. |
(unstated) ToString | This was the one place where the record form won, because of an inefficiency in the generated struct code. That is fixed too. Both forms allocate 104 bytes. |
Choose the struct form to avoid an object per identifier and a dereference on every read. Persistence is no reason for either form, and equality is no longer a reason. Two of the five rows describe defects that existed because nobody had measured. That is why the benchmarks stay in the repository.
Provider coverage
The benchmarks above run on SQLite only. A separate test suite checks the mapping on other providers.
Tests/DTS.DDD.EntityFramework.Providers.Tests runs the mapping, concurrency, outbox and inbox tests against
PostgreSQL 17 and SQL Server 2022 in Testcontainers.
- Without Docker, every test in the suite skips with a message that names the reason.
- With the environment variable
DDDTOOLKIT_REQUIRE_CONTAINERSset, a container that does not start fails the test instead.
The suite checks three things that SQLite cannot:
- The
dddschema works. SQLite has no schemas and ignores the schema argument without an error. So the default schema of the outbox and the inbox was never tested there. On both servers,EnsureCreatedcreates the schema, and creates both tables in it. - A struct id and a record id produce the same column. The column is
uuidon PostgreSQL anduniqueidentifieron SQL Server, for the key, for a plain property and for a nullable one. This is the other half of the persistence numbers above. - The outbox and the inbox keep their timestamps in the provider's own instant type. That is
timestamp with time zoneon PostgreSQL anddatetimeoffseton SQL Server. The toolkit converts to a UTCDateTimeon SQLite only, because SQLite cannot order by aDateTimeOffset. A database made withDomainEventTimestamps.UtcDateTimekeeps itsdatetime2columns on SQL Server, and both shapes read back the same instant (Outbox and inbox timestamps).
The suite also shows one difference between the providers. A primitive collection of converted ids becomes a
native integer[] on PostgreSQL and a JSON nvarchar on SQL Server. Both round trip, and a LINQ query over the
collection works on both providers. A SQL query that reads inside the collection needs a different form for each
provider.
Skipping a named query filter
Tenancy's tenant filter is a named query filter. A read across tenants skips it by name, and the seat lookup does that
on every authenticated request. The benchmark reads one row by key and skips one named filter. It passes the filter
key in two ways. IgnoreQueryFilters([name]) creates a new collection on every call. IgnoreTenantFilter() passes
the same static array on every call. The benchmark ran on Entity Framework 10.0.0, the lowest version the packages
allow, and on 10.0.12, the version the repository builds with.
| Entity Framework 10.0.0 | Entity Framework 10.0.12 | |
|---|---|---|
| One array of keys | 13.37 μs, 9.77 KB | 46.29 μs, 9.90 KB |
| A new collection per call | 2,440.48 μs, 162.35 KB | 47.82 μs, 9.95 KB |
The two columns are separate runs, so compare the rows within a column.
- On 10.0.0, a new collection compiles the query on every call. Entity Framework stores the keys as a constant in the query. Its query cache compares that constant by reference unless it is an array. A new collection is a new reference, so the lookup misses the cache, compiles the query again and adds a cache entry. The miss takes 182 times the time and 17 times the memory of a hit.
- From 10.0.3, Entity Framework copies the keys into an array. The cache then hits for both forms. The new collection costs only the copy.
IgnoreTenantFilter()passes the same array on every call. The cache hits from 10.0.0. The seat directory and the tenant listing use it, and an application can use it too. The benchmark isQueryFilterKeyBenchmarks. To run it on 10.0.0, build with-p:DDDDependencyVersions=Floorand run with--inProcess.
A save on a context that tracks many aggregates
The benchmark changes one aggregate and calls SaveChanges on a context that tracks 10 or 1,000 aggregates. It runs
the save with the toolkit's interceptors for domain events, invariants and versions, and as a baseline with Entity
Framework alone. The benchmark is SaveChangesBenchmarks.
| Tracked | Before | After | |
|---|---|---|---|
| Entity Framework alone | 10 | 11.21 μs, 11.68 KB | 11.16 μs, 11.68 KB |
| With the toolkit | 10 | 28.52 μs, 35.95 KB | 16.69 μs, 23.78 KB |
| Entity Framework alone | 1,000 | 208.66 μs, 328.79 KB | 118.60 μs, 328.79 KB |
| With the toolkit | 1,000 | 1,442.94 μs, 2,402.67 KB | 442.42 μs, 1,292.21 KB |
The baseline code did not change, so the difference in its time is noise between the two runs. Compare the ratios instead. At 1,000 tracked aggregates, the save with the toolkit took 6.9 times the baseline before and 3.7 times after. It allocated 7.3 times the baseline before and 3.9 times after.
Each interceptor used to run this code:
if (context.ChangeTracker.AutoDetectChangesEnabled)
{
context.ChangeTracker.DetectChanges();
}
foreach (var entry in context.ChangeTracker.Entries().ToList())
Entries() calls DetectChanges again and creates an EntityEntry for every tracked entity, unchanged ones
included. With the event, invariant and version interceptors, one save called DetectChanges 7 times, including the
call Entity Framework makes itself.
- One
DetectChangesper interceptor. The event interceptor calls it once per round. A save without events now calls it 4 times, including Entity Framework's own call. - Only the changed entries. The interceptors read the added, modified and deleted entries from Entity Framework's state manager, which keeps a separate set of entries for each state. The invariant interceptor uses one list for the roots and for the entries.
- An
EntityEntryonly for an aggregate with events. The event interceptor still checks every tracked aggregate for events, because an aggregate can raise an event without a change to a column.AggregateRoot.DomainEventsreturns the same read-only view on every read. - One walk from a child to its root. The invariant and the version interceptor share it. It reuses its collections for all children of a save, caches the owner keys per entity type, and gets a tracked principal's entry from the state manager.
Tenancy's save interceptor and Membership's hold interceptor read the changed entries in the same way. Tenancy's
projection writer creates an EntityEntry only for Tenancy's own classes.
Most of the remaining cost above the baseline is the three DetectChanges calls of the toolkit's interceptors. Each
call compares every tracked entity with its snapshot, and costs about as much as Entity Framework's own call. An
interceptor cannot skip the call, because another interceptor may change an entity between two of them.
Delivering a batch of outbox messages
The benchmark puts 10, 100 or 1,000 pending rows in an outbox, and delivers them with one ProcessPendingAsync call
to a sink that does nothing. The sink is not registered in the container, so the processor creates it. Each operation
first makes the rows pending again with one ExecuteUpdate, which takes 0.07 to 1.1 ms and allocates 62 KB. The
benchmark is OutboxProcessorBenchmarks.
| Messages | Before | After |
|---|---|---|
| 10 | 713 μs, 570 KB | 569 μs, 303 KB |
| 100 | 34.8 ms, 30,797 KB | 5.45 ms, 1,678 KB |
| 1,000 | 2,811 ms, 2,950,340 KB | 45.8 ms, 15,421 KB |
Before, the time and the memory grew with the square of the batch size. Now they grow with the batch size. The times differed by up to a factor of two between runs on this machine. The allocations were the same in every run.
The processor saves the mark of each message on its own. It used to read the batch tracked, so each save compared
every row of the batch with its snapshot. A save calls DetectChanges 4 times, including Entity Framework's own call.
A batch of 1,000 rows therefore made 4 million row comparisons.
- The batch is read untracked. The processor attaches a row while it delivers the row, and detaches it after the save. A row that the context already tracks, because the caller saved its event through the same context, keeps its entry.
- Each sink is resolved once per batch. The processor resolved every sink for every message, so a sink that is not registered in the container was created for every message.
- The check for a handler's changes reads only the changed entries. Where the host requires an explicit current user, the processor checks before and after each in-process dispatch which entities a handler changed. It reads the added, modified and deleted entries from the state manager, as the save interceptors do.
The cost of the claim
The processor now claims its batch with a lease before it delivers it (Several processors on one table). The same benchmark, run again with the claim:
| Messages | After | With the claim |
|---|---|---|
| 10 | 569 μs, 303 KB | 824 μs, 373 KB |
| 100 | 5.45 ms, 1,678 KB | 4.02 ms, 1,997 KB |
| 1,000 | 45.8 ms, 15,421 KB | 33.2 ms, 18,244 KB |
The claim adds two statements per batch: the UPDATE that writes the claim, and the query that reads the claimed
rows. Each mark also sets LockedBy and LockedUntil back to null, and carries the claim in its WHERE clause.
The allocations grow by 70 KB for 10 messages and by about 2.8 KB per message for 1,000. The times are within the
factor of two by which runs differ on this machine, so the run shows no change in time.
Access checks
Each benchmark makes one check of a request, and the check passes. The scope and its context exist already, so their
cost is not part of the numbers. Each check reads one statement. OwnerCheckBenchmarks is in this repository.
MemberCheckBenchmarks and TenancyCheckBenchmarks are in DTS Pro's Benchmarks/DTS.Pro.Benchmarks, beside
Membership and Tenancy, and use the test hosts of the two supporting domains. The benchmarks run on SQLite in memory,
and on Postgres 17 in Docker 29.6 on the test machine, reached over localhost. They start the container themselves, so
Docker must run.
On SQLite
| Check | Before | After |
|---|---|---|
[Own] on a parcel's id, by the parcel's courier | 17.0 μs, 13.6 KB | 6.0 μs, 3.9 KB |
[OnResource(permission)] on a document, by a member whose role gives the permission | 399 μs, 203 KB | 95 μs, 14.5 KB |
[RequiresPermission] of Tenancy, for the whole tenant, by its administrator | 61.7 μs, 31.9 KB | 20.1 μs, 6.6 KB |
[AtUnit(permission)], by a supervisor at its own unit | 68.9 μs, 36.5 KB | 24.1 μs, 7.2 KB |
The times differed by up to a factor of 1.6 between runs on this machine, because other processes ran beside the benchmarks. The allocations differed by less than 3%. Before and after were measured one after the other.
On Postgres
The round trip varied a lot on this machine. Within one run, the standard deviation was 0.4 to 1 ms per check. Two runs before the ones in the table had means of up to 43 ms. So the table has the median of each run instead of the mean. Before and after were measured in turn, twice: before, after, before, after. Each cell has the two medians and the allocations.
| Check | Before | After |
|---|---|---|
[Own] on a parcel's id, by the parcel's courier | 2.01 and 1.70 ms, 15.1 KB | 1.74 and 1.24 ms, 5.4 KB |
[OnResource(permission)] on a document, by a member whose role gives the permission | 4.08 and 3.90 ms, 115.7 KB | 3.39 and 3.18 ms, 17.5 KB |
[RequiresPermission] of Tenancy, for the whole tenant, by its administrator | 2.63 and 5.11 ms, 34.2 KB | 2.04 and 2.25 ms, 9.0 KB |
[AtUnit(permission)], by a supervisor at its own unit | 2.53 and 2.54 ms, 38.7 KB | 2.42 and 2.17 ms, 9.4 KB |
- The median after is lower in each pair. In seven of the eight pairs it is 0.11 to 0.73 ms lower. On SQLite the checks save 10 to 300 μs of work in the process. The spread of the round trip is larger than both, so these runs do not tell how much of the difference on Postgres the compiled queries make.
- The allocations drop as on SQLite. They differed by less than 3% between runs.
- Before, the Membership check allocated 116 KB on Postgres and 203 KB on SQLite. This fits the list parameter of Entity Framework 10, which cost about 100 KB for each execution on SQLite. On Postgres a list parameter costs a few KB (see Several requirements on one request).
- The round trip was slower in these runs than in the next section's. There a Tenancy check at one unit took 0.85 ms. Here it took 2.2 to 2.4 ms. Compare times on Postgres only within one section.
What changed
Before, each check built its query as a LINQ expression and handed it to Entity Framework. Entity Framework walked the
expression to take out its parameters, and computed a hash of it to find the compiled query in its cache. It did not
translate the query again: a count of Entity Framework's query compilations showed one for each shape of statement.
Building and walking the expression made most of the cost. Now each check runs a query compiled with
EF.CompileAsyncQuery, and sends its values as parameters.
[Own]. The package finds the owner's column of an aggregate the first time a request asks about the aggregate. It compiles the query that reads the column at the same moment. A check sends the aggregate's id as the one parameter.[OrOwn]on an aggregate's id reads the owner in the same way.- Tenancy.
ITenancyReadSourcehas three members for the checks:HoldsAtTenantAsync,HoldsAtAsyncandHasUnitAsync. A source answers each with a query it keeps, or returnsnull. The Entity Framework source answers with queries that it compiles once per model. For a source that returnsnull, the questions are worked out over its rows, as before. - Membership. The statement of a check depends on the rules and the permission, never on the caller. It depends on
how the caller can hold the permission, how it sees the resource, whether it is the application's own work, and which
roles give the permission. The package compiles the statement once for each of these shapes. The caller and the
moment are parameters. The roles are constants of the statement.
- Entity Framework 10 sends a list as one parameter per element by default. With the roles as a list parameter, the check allocated 114 KB on SQLite. With the roles as constants, it allocates 14.5 KB.
- A statement with roles kept as rows, or with a permission held from above, contains a query that the application answers for the caller. Such a statement is still composed for each check.
The queries that a module composes into its own query, such as WhereIHold and db.Tenancy().UnitsWhereIHold, are
still built for each call.
The query filters and row level security apply as before. A compiled query applies the query filters of the context it runs on, and evaluates a filter that reads the context again for every execution. Its command goes through the same interceptors. The tests check this with a filter that reads a member of the context, and with Tenancy's tenant filter.
Several requirements on one request
A request with a permission marker on each of two members requires both, AccessRequirement.All of the two. The toolkit
checks the cases one after the other, and each case that reads sends a statement of its own. A marker on a list of ids
reads every resource of the list in one statement. The benchmarks compare the two on SQLite in memory, where a statement
costs no round trip, and on Postgres, where every statement is one. The benchmarks are MemberRoundTripBenchmarks and
TenancyRoundTripBenchmarks, in DTS Pro's Benchmarks/DTS.Pro.Benchmarks. Like the access checks above, they check in
a scope that already has its context, and every check passes.
Postgres 17 ran in Docker 29.6 on the test machine, reached over localhost. The benchmarks start the container themselves, so Docker must run.
The Membership benchmark requires the permission that shares on documents. The caller manages every document.
| Request | SQLite | Postgres |
|---|---|---|
One document, ShareDocument | 88.8 μs, 14.4 KB | 1,586 μs, 17.5 KB |
Two documents, a marker on each, ShareOnBoth | 183.5 μs, 29.2 KB | 3,103 μs, 35.2 KB |
The same two documents, one marker on the list, ShareOnEach | 198.6 μs, 116.4 KB | 1,588 μs, 20.2 KB |
The Tenancy benchmark requires the permission that manages seats. A supervisor holds it at a region, and so at the two units below it.
| Request | SQLite | Postgres |
|---|---|---|
One unit, ManageSeatsAt | 20.1 μs, 7.2 KB | 851 μs, 9.4 KB |
Two units, a marker on each, MoveSeat | 40.7 μs, 14.5 KB | 1,866 μs, 19.0 KB |
- Each case that reads costs a statement. On Postgres, the second marker doubles the time of the check: about 1 ms for Tenancy and 1.5 ms for Membership. On SQLite in memory a check takes 20 to 95 μs (see Access checks). So on Postgres, the round trip and the work in the database make almost all of the time.
- One statement for both documents costs as much as one for one document. On Postgres, the marker on the list checks two documents in 1,588 μs, and the marker on one document checks it in 1,586 μs.
- On SQLite, the list costs 100 KB more than one document. The statement compares the ids with a list parameter. Entity Framework 10 sends a list as one parameter per element by default, and on SQLite that cost about 100 KB for each execution. On Postgres, the list costs 2.7 KB more than one document.
The cases of a combination are not put into one statement. A case that refuses ends an All before the cases after it
are read, and a case that lets the caller through ends an Any. The checks of the cases belong to different packages,
and may read different contexts.
Tenancy: the shape of the access data
The Tenancy supporting domain makes three choices:
- it stores facts, such as which seat holds which role at which unit;
- it starts every query from the person who asks;
- it keeps the organization tree as a closure table.
The measurements below support these choices. They compare Tenancy's shape with a shape that permission models often grow into:
- a table of what every person may do on every project, kept up to date by triggers;
- a copy of the organization path on every project;
- row level security (RLS) policies that call a function for every row.
The data
Both shapes hold the same data, and both give every person the same result:
- one large tenant: 3,111 units in five levels (the root, 10 regions, 100 areas, 1,000 units below the areas and 2,000 below those), 49,776 projects, 1,000 seats, and a crew of three on every project: a crew lead, a surveyor and an observer;
- 20 area managers who hold
projects.editat an area, and one who holds it at the root; - 49 small tenants of 111 units and 999 projects each, 98,727 projects in all.
Storage and the project list
Measured on PostgreSQL 17, for two people:
- an area manager, who holds the permission at one area, is also on some crews, and sees 643 projects;
- a surveyor, who holds no permission in the organization, and is on the crew of 150 projects.
| Per-person table, per-row policies | Per-person table, set-shaped policy | Facts, set-shaped | |
|---|---|---|---|
| Storage | 411 MB | 411 MB | 60 MB |
| "My projects", area manager | 10,900 ms | 185 ms | 5.2 ms |
| "My projects", surveyor | 10,364 ms | 155 ms | 4.6 ms |
| Buffers touched (shared hits and reads), area manager's list | 1,243,555 | 450,277 | 1,019 |
| 50 projects opened one by one, total, area manager | 27 ms | 9,126 ms | 24 ms |
| 50 projects opened one by one, total, surveyor | 26 ms | 7,356 ms | 26 ms |
| Growth of the connection's memory | +1.0 MB | +0.8 MB | +0.6 MB |
| Cached plans in the connection | 293 KB | 49 KB | none |
Three causes explain the difference:
- Storing outcomes costs the space. The table of what every person may do on every project has one row per person, permission and project. It is 256 MB on its own. The facts it is computed from, the crews and the permissions of each role, take 39 MB.
- A policy that runs per row reads the whole tenant. For a list, the policy calls its function once for each project of the tenant, whatever the person may see. A set-shaped policy computes the set of permitted projects once per statement. That makes the list fast, and makes the other case slow. A single project now pays for the whole set, 9 seconds for 50 projects. Starting from the person is fast in both cases, because the set is only as large as what that person may do.
- Memory per connection is multiplied by the number of connections. PostgreSQL runs every connection in its own process, and each process keeps its cached plans and catalogue entries for its lifetime. The per-row policies call PL/pgSQL functions, whose plans each connection caches. The set functions of the facts shape are SQL functions that run once per statement, and PostgreSQL 17 caches no plan for them. A connection grew by 0.6 MB instead of 1.0 to 1.1 MB. That is small per connection, and it adds up over hundreds of connections.
The organization tree
The tree has its own data set: the same tenants, units and projects, without seats or crews. Three people hold
projects.edit:
- an area manager at two areas, who sees 992 projects;
- an area manager at the root;
- a crew lead at one unit below an area.
The query asks which projects are below the units where the person holds the permission. The table shows the median execution time of seven runs on PostgreSQL, for each way to store the tree:
| Who, and how many projects | ltree | Text path, a range | Closure table | Path copied on every project, per row |
|---|---|---|---|---|
| Area manager at two areas, 992 | 0.71 ms | 0.40 ms | 0.33 ms | 21.49 ms |
| Area manager at the root, 49,776 | 10.41 ms | 10.23 ms | 10.31 ms | 16.15 ms |
| Crew lead at one unit, 48 | 0.22 ms | 0.16 ms | 0.14 ms | 15.22 ms |
- The three ways to store the tree are equally fast.
- The path copied onto every project is slower for everyone, the manager at the root included, because the query checks every project of the tenant.
- Moving an area with its 30 units below another region takes 0.4 ms with a text path, 0.7 ms with
ltree, and 3.3 ms with the closure table. The closure table replaces the 62 rows that link the moved units to the units above them that they leave and gain. None of the three changes a project row.
The closure table works on every database. It uses plain joins on ids. It needs no extension and no collation for a range query, and it has no limit on the length of an index key. The same query on SQL Server 2022, against a text path with a binary collation, and on SQLite in memory:
| Who | SQL Server, text path | SQL Server, closure table | SQLite, text path | SQLite, closure table |
|---|---|---|---|---|
| Area manager at two areas | 1.28 ms | 1.27 ms | 0.05 ms | 0.04 ms |
| Area manager at the root | 4.00 ms | 4.01 ms | 2.52 ms | 1.88 ms |
| Crew lead at one unit | 1.24 ms | 1.26 ms | 0.01 ms | 0.01 ms |
Compare the columns within one database, and do not compare the databases. SQL Server's times are taken inside the server, over 200 executions at a time, because its clock moves in steps of about 4 ms. SQLite runs in the process, in memory.
How it was measured
On the test machine above, with PostgreSQL 17 and SQL Server 2022 in Docker, one session at a time, and a warm cache. This is no load test. The behavior with many people at once is not measured.
The tree comparison is in Benchmarks/Tenancy. Its README.md says how to run it and how each number is taken.
The script of the storage and list comparison is not in the repository. Its numbers are given as measured.
Not measured
- Other providers. Everything before the Tenancy section runs on SQLite in memory, except the access checks and several requirements on one request, which run on Postgres as well. The suite above checks the mapping on PostgreSQL and SQL Server, but does not time it.
- The generators. Nothing measures the build-time cost of the source generators.
- The outbox. The throughput of
OutboxProcessorunder load, and the effect of theProcessedAtindex as the table grows, are open questions. - JSON and GraphQL. The generated converters have no benchmarks.
- Larger value objects. Nothing measures a value object with more than two properties, or with a collection.