There is something interesting about a database automatically deleting its own data.
You insert a document, attach an expiration timestamp, create a TTL index, and MongoDB takes responsibility for eventually removing that document when its time is up.
From an application developer's perspective, this looks almost trivial:
db.sessions.createIndex(
{ createdAt: 1 },
{ expireAfterSeconds: 86400 }
);One index. One expiration policy. No cron job. No scheduled deletion script. No application-level cleanup logic.
But what actually happens internally?
When millions of documents expire, how does MongoDB identify them? Does it scan the entire collection? Does it iterate over every expired record? Does it parse timestamps repeatedly? How does it prevent deletion from overwhelming the database? Why can't a document simply destroy itself at the exact moment it expires?
And perhaps the most important question: if a database can automatically expire data, can we trust that mechanism to execute mission-critical financial business logic at precisely the scheduled time?
These questions lead to some interesting insights into database internals, indexing, storage engines, scheduling, distributed systems and financial correctness.
Let's work through them.
1. First, understand what a TTL index actually does
TTL stands for Time To Live.
MongoDB provides TTL indexes to automatically remove documents after a specified period or at a specified expiration date.
Consider a session collection:
{
_id: ObjectId("..."),
sessionId: "SESSION_1001",
userId: "USER_101",
createdAt: ISODate("2026-11-01T10:00:00Z")
}Suppose we want every session to expire after 24 hours.
We create:
db.sessions.createIndex(
{ createdAt: 1 },
{ expireAfterSeconds: 86400 }
);MongoDB now has an expiration policy associated with the indexed date field.
The important thing to understand is that creating a TTL index does not create an individual timer for every document.
MongoDB does not attach a background process, timer or execution thread to each record.
Instead, it maintains an index and uses a background mechanism to identify and remove expired documents.
That distinction becomes important when the collection grows from a few thousand documents to millions or hundreds of millions.
2. How does MongoDB find expired documents efficiently?
Let's say a collection contains 10 million documents.
A naive implementation would be:
Read every document.
Extract its expiration timestamp.
Compare it against the current time.
Decide whether it has expired.
Delete it if necessary.
This would require examining the entire collection, even if only 100 documents had expired.
That would be an extremely inefficient way to implement expiration.
MongoDB takes a different approach.
It uses the TTL index.
For example, imagine an indexed expiration field containing:
2026-10-01 08:00:00
2026-10-01 09:00:00
2026-10-01 10:00:00
2026-10-02 08:00:00
2026-10-03 08:00:00
2026-10-04 08:00:00
2026-10-05 08:00:00The index is ordered by the indexed date value.
MongoDB calculates the expiration threshold and can seek to the relevant position in the index rather than examining every document in the collection.
For an index on createdAt with a retention period of 24 hours:
Expiration threshold = Current time - 24 hoursFor an absolute expiration date, the TTL index can use:
db.sessions.createIndex(
{ expiresAt: 1 },
{ expireAfterSeconds: 0 }
);Here, the date itself determines when the document becomes eligible for deletion.
The index is doing something important: it makes locating the beginning of the expired range efficient.
But this leads to another question.
3. Is TTL deletion actually O(1)?
No. And this is where the distinction between identifying records and deleting records becomes important.
Suppose:
Total documents: 10 million.
Expired documents: 5 million.
Non-expired documents: 5 million.
The index helps MongoDB locate the expired range efficiently.
Conceptually, we can describe the work as:
[
O(\log N + K)
]
Where:
(N) is the total number of indexed entries.
(K) is the number of expired entries that must be processed.
The first component represents locating the relevant position in an ordered index.
The second component represents traversing and processing the expired entries.
This is an important distinction.
MongoDB does not need to perform a full collection scan to identify expired documents, but it cannot magically remove five million individual records without doing the work associated with those records.
Each deletion has consequences:
The corresponding document must be removed.
Relevant index entries must be maintained.
Storage-engine metadata must be updated.
Replication must account for the deletion.
Storage pages and internal structures must be managed.
MongoDB may optimise and batch parts of this work, but the amount of expired data still matters.
An efficient lookup algorithm does not make the deletion operation itself constant-time.
This is the first major insight: indexing reduces the cost of identifying expired data, not the fundamental cost of removing a large volume of individual records.
4. What actually runs inside MongoDB?
MongoDB has an internal background process known as the TTL Monitor.
It runs within the mongod process.
It is responsible for checking TTL indexes and removing documents that have passed their expiration threshold.
This is not an application-level cron job. You do not need to schedule a separate Node.js process to periodically inspect the collection.
MongoDB handles this internally.
The process is periodic rather than continuous.
By default, the TTL Monitor runs approximately every 60 seconds.
This means that if a document expires at:
12:00:00It does not follow that the document will disappear at exactly 12:00:00.
It becomes eligible for deletion.
The next monitoring cycle may identify it and begin removing it. If the database has a large deletion backlog, the actual removal may take longer.
Now let's examine how MongoDB prevents this background process from monopolising database resources.
5. How does MongoDB handle millions of expired documents?
MongoDB does not attempt to delete an unlimited number of documents from one TTL index in a single uninterrupted operation.
Its deletion process uses stopping conditions.
For each TTL index, MongoDB processes expired documents until one of these conditions is met:
It has deleted 50,000 documents from the current index.
It has spent one second deleting documents from the current index.
There are no more expired documents to delete from that index.
It then moves to the next TTL index.
After checking the TTL indexes, MongoDB performs another sub-pass to look for remaining expired documents.
This continues until it completes a pass without finding more eligible documents to delete.
There is also a 60-second limit on an individual deletion loop to prevent a single large deletion workload from monopolising the monitor.
Starting with MongoDB 6.1, MongoDB can batch multiple document deletions for efficiency.
There is another important implementation detail: the TTL deletion process is single-threaded.
It does not independently run multiple TTL deletion workers concurrently across all TTL indexes.
Consider a simplified example.
Suppose five million documents have expired and the monitor reaches its 50,000-document stopping threshold on each iteration.
The number of iterations at that threshold would be:
[
\frac{5,000,000}{50,000}=100
]
That is 100 deletion iterations.
But it would be incorrect to conclude that the entire process takes 100 seconds.
Each iteration can also stop after one second, and actual throughput depends on document size, storage performance, index maintenance, replication and concurrent database activity.
This explains why a large TTL backlog can take considerable time to clear.
It also explains why TTL should not be treated as an immediate deletion mechanism.
6. An interesting architectural question: why should we delete millions of records individually at all?
This is where I started thinking about a different approach.
Imagine a collection containing 10 million temporary application logs.
All records have the same retention policy: 30 days.
MongoDB's TTL mechanism identifies expired documents and deletes them individually.
It does this efficiently compared with a full collection scan, but it still has to process the expired records.
Now consider another possibility.
Instead of storing all records in one collection, what if we organised them into time-based storage segments?
For example:
application_logs_2026_10_01
application_logs_2026_10_02
application_logs_2026_10_03
application_logs_2026_10_04
application_logs_2026_10_05Each collection contains records belonging to a particular day.
When the retention period expires for an entire segment, instead of deleting millions of individual documents, we drop the collection:
db.application_logs_2026_10_01.drop();The unit of deletion has changed.
Instead of:
Find millions of expired documents
|
v
Delete each document
|
v
Update associated indexesWe now have:
Identify expired storage segment
|
v
Drop the collectionThis is a fundamentally different approach to data lifecycle management.
The objective is no longer to make individual deletion faster.
It is to make individual deletion unnecessary.
Why is this attractive?
Consider the differences:
DimensionIndividual TTL deletionTime-based collection deletionExpiration identificationIndexed range lookupSegment-level decisionDeletion unitIndividual documentEntire collectionIndex maintenancePer-document deletion workCollection structures removedLarge deletion backlogCan require substantial processingAvoids individual record deletionRetention granularityIndividual expiration datesSegment-level expirationOperational complexityRelatively lowHigher partition and lifecycle management
There is an important caveat.
Dropping a collection is not guaranteed to be literally O(1) in every storage-engine, filesystem or replication operation. Physical disk space reclamation may also be deferred.
However, it can avoid the enormous amount of per-document deletion work associated with deleting millions of records individually.
This principle is used in large-scale data systems: align data retention with storage boundaries so that entire segments can be retired efficiently.
7. But what if every document has a different expiration time?
This is where the time-segment approach becomes more complicated.
Suppose:
Document A: expires in 2 hours
Document B: expires in 7 days
Document C: expires in 15 daysIf all three are placed in the same daily collection, dropping that collection after two hours would prematurely destroy documents B and C.
We need to group data according to its lifecycle.
There are a few possible approaches:
Approach A: Smaller time buckets
Use hourly or even smaller partitions.
This increases granularity but also increases the number of collections and the operational overhead of managing them.
Approach B: Group by expiration date
Store documents in segments according to when they expire rather than when they were created.
This can work well when expiration periods are predictable.
Approach C: Hybrid lifecycle management
Use time-based segments for predictable, high-volume retention workloads and TTL indexes for irregular expiration requirements.
This is often the more practical architecture.
For example:
Data typePotential lifecycle strategyOTP recordsTTL indexTemporary sessionsTTL indexHigh-volume application logsTime-based segmentationHistorical telemetryObject storage archival and segment deletionFinancial transaction recordsExplicit retention and archival policies
The important lesson is that the most efficient lifecycle mechanism depends on the shape of the data and its retention requirements.
8. Doesn't Redis already solve expiration differently?
Redis provides another interesting perspective.
When a Redis key is assigned an expiration time, Redis does not create an independent operating-system timer for every key.
It uses two complementary mechanisms.
Passive expiration
When an application accesses a key, Redis checks whether the key has expired.
If it has expired, Redis removes it rather than returning it as a valid key.
This means an expired key can be removed when it is accessed, without waiting for a background cleanup cycle.
Active expiration
Redis also periodically samples keys with expiration times and removes expired keys in the background.
This prevents expired keys that are never accessed from remaining in memory indefinitely.
The combination is important.
Passive expiration makes sure expired data is not treated as valid when accessed.
Active expiration gradually cleans up expired data that nobody accesses.
Redis therefore uses a different strategy from MongoDB, but the underlying principle is similar: expiration eligibility and physical cleanup are separate concerns.
Even Redis does not attach an independent self-destruction thread to every key.
9. The self-destructing document idea
This brings us to a more interesting thought experiment.
Why can't every document carry its own destruction schedule?
Imagine a record that behaves like a self-destructing object:
Document created
|
v
Expiration timestamp assigned
|
v
Document waits
|
v
Expiration time reached
|
v
Document destroys itselfAt first glance, this sounds like a better model.
There would be no need for a background process to periodically inspect records. There would be no repeated timestamp comparisons. Every record would somehow know when its lifetime ends.
But there is a fundamental problem.
A MongoDB document is not an independently executing program.
It is data stored and managed by a storage engine.
It does not own an execution thread. It cannot independently wake up, execute code, remove its index entries, update storage metadata and coordinate replication.
Even if we implemented one timer per document outside the database, we would create a new problem.
Imagine five million records, each with an independent timer.
We would need to manage:
Five million scheduled events.
Timer memory and scheduling overhead.
Process restarts.
Timer persistence.
Clock changes and synchronisation.
Duplicate execution.
Missed events.
Recovery after failures.
Coordination across multiple application instances.
We would have moved the complexity rather than eliminated it.
There is another fundamental distinction: even if a document could trigger a deletion operation at the exact expiration time, the storage engine would still need to perform the actual work of removing that document and maintaining its internal structures.
The scheduling of deletion and the physical execution of deletion are two different problems.
This is why the time-segment approach is interesting: instead of creating millions of independent destruction events, it reduces the number of lifecycle operations by changing the storage architecture.
10. The most important question: can TTL execute financial business logic?
Now let's move from database housekeeping to something much more consequential.
Suppose we are operating a financial marketplace.
A user reserves an investment opportunity for 15 minutes.
We create a MongoDB document:
{
reservationId: "RES_1001",
investorId: "INV_101",
amount: 50000,
status: "PENDING",
expiresAt: ISODate("2026-11-02T12:15:00.000Z")
}The business rule is:
If the investor does not complete the transaction within 15 minutes, expire the reservation and release the reserved amount.
A tempting implementation would be to create a TTL index on expiresAt and assume that MongoDB will delete the document when the deadline arrives.
Then, perhaps, some application process observes the deletion and releases the funds.
This is dangerous.
There are several reasons.
First: deletion is not immediate.
The document becomes eligible for deletion at its expiration time. The TTL Monitor may not process it until a later cycle.
Second: TTL does not execute business logic.
Deleting a reservation document does not automatically release funds, update a financial ledger, cancel an external payment or notify the investor.
Third: the process can be delayed or interrupted.
Database load, replication activity, process restarts and other operational conditions can affect when cleanup occurs.
Fourth: deleting a document removes information that may be important for reconciliation and auditing.
Financial state should not disappear simply because a timer has expired.
The conclusion is straightforward:
TTL is a data lifecycle mechanism, not a financial workflow engine.
11. Timestamp precision is not execution precision
There is another subtle but important distinction.
A database can store a timestamp representing a particular instant.
That does not mean it can guarantee that an operation executes at that exact instant.
Consider:
Expiration timestamp:
12:15:00.000000000Even if a system represents a timestamp with nanosecond precision, it does not follow that its scheduler, operating system, database or network can execute a business operation at nanosecond precision.
There are multiple independent sources of uncertainty:
ComponentWhy exact execution is not guaranteedClock sourceClock drift and synchronisation errorSchedulerVariable scheduling and queueing delaysOperating systemThread scheduling is not inherently real-timeDatabaseLocking, write contention and storage activityReplicationChanges must propagate to other membersNetworkVariable transmission and retransmission delaysExternal dependenciesIndependent processing and availability
MongoDB TTL itself is periodic, making exact-time execution particularly unsuitable.
There is also a precision detail worth remembering: BSON Date values represent time in milliseconds since the Unix epoch. A timestamp's representation is not a guarantee of when an operation will execute.
A nanosecond timestamp does not create a nanosecond scheduler.
12. What should a financially correct expiration architecture look like?
We need to separate three responsibilities:
Enforcing the business deadline.
Executing the expiration workflow.
Cleaning up historical data.
These should not be coupled to one background deletion mechanism.
Consider the following architecture.
Step 1: Persist the business state
Create the reservation with an explicit expiration timestamp.
{
reservationId: "RES_1001",
status: "PENDING",
expiresAt: ISODate("2026-11-02T12:15:00Z")
}The expiration timestamp is part of the business state.
It is not merely a TTL configuration.
Step 2: Schedule the expiration event
Use a durable scheduler or job queue to schedule an expiration task.
For example:
Reservation created
|
v
Persist reservation
|
v
Schedule expiration job
|
v
Wait until deadline
|
v
Execute expiration workerThe scheduler's job is to trigger the workflow near the intended deadline.
It should not be treated as an exact-time execution guarantee.
Step 3: Verify the business condition
When the worker runs, it must check the actual transaction state.
A reservation should expire only if:
It is still pending.
Its deadline has passed.
It has not already been completed or cancelled.
This matters because the payment might have completed just before the expiration worker started.
The worker must not blindly expire a reservation based only on the existence of a scheduled job.
Step 4: Perform an atomic state transition
For example:
const now = new Date();
const result = await db.collection("reservations").updateOne(
{
reservationId: "RES_1001",
status: "PENDING",
expiresAt: { $lte: now }
},
{
$set: {
status: "EXPIRED",
expiredAt: now
}
}
);
If modifiedCount is zero, the worker did not successfully transition the reservation.
Perhaps it was already completed. Perhaps another worker processed it. Perhaps the deadline has not actually passed.
This conditional update is important because multiple workers or duplicate job deliveries must not cause multiple successful financial transitions.
For high-integrity systems, use an authoritative time source and carefully handle the clock-skew implications of application-generated timestamps. The deadline check and state transition should be designed as one atomic decision.
Step 5: Record the financial consequences
If expiration requires releasing reserved funds, that operation must be represented in the financial ledger.
For example:
Reservation expired
|
v
Release reserved balance
|
v
Record ledger entry
|
v
Publish expiration event
|
v
Notify investorThe ledger operation should be idempotent.
Running the same expiration job twice must not release the same funds twice.
Where the state transition and ledger entry share a database, a transaction may be appropriate. Where they cross database or service boundaries, use an appropriate distributed consistency pattern, such as a transactional outbox, idempotent consumers and reconciliation.
The key is that the system must be able to recover from a crash between any two steps.
Step 6: Reconcile missed events
Even a durable scheduler can experience failures.
Jobs can be delayed, workers can crash, queues can become unavailable, and events can be delivered more than once.
Therefore, implement a reconciliation process that periodically finds records satisfying:
status = PENDING
AND expiresAt <= current timeIt can then schedule or execute any missing expiration transitions.
This creates a recovery mechanism independent of whether the original scheduled event was successfully delivered.
Step 7: Use TTL for eventual cleanup
Once a record is no longer needed for operational or audit purposes, TTL can remove it according to the retention policy.
This is where TTL belongs.
It should clean up data after the business workflow has completed, not determine whether the business workflow happens.
13. What does "exact deadline" actually mean in financial systems?
There are two requirements that are often confused.
Requirement A: A transaction must not be accepted after its deadline.
This is a business correctness requirement.
It should be enforced synchronously when processing the transaction.
For example, if a reservation expires at 12:15, a payment request received at 12:15:01 should not be accepted as a valid payment against that reservation.
The application must check the authoritative deadline and perform the appropriate atomic state transition.
Requirement B: The system must release reserved funds at exactly 12:15.
This is a scheduling and execution requirement.
A durable scheduler can target 12:15, but it cannot guarantee that every worker will execute at precisely that instant under arbitrary system conditions.
Instead, define an acceptable execution-latency objective.
For example:
Deadline enforcement: no transaction accepted after the authoritative deadline.
Expiration processing: target execution within a defined operational window.
Recovery: all missed expiration events eventually processed.
Financial correctness: no duplicate release, no lost ledger entry and no invalid state transition.
This is a much stronger engineering contract than promising nanosecond-level execution.
The important insight is that financial correctness should not depend on the scheduler waking up at precisely the right moment.
It should depend on the business rules being enforced correctly whenever the operation is attempted.
14. How should we monitor MongoDB TTL internally?
One useful aspect of MongoDB is that it exposes TTL-specific metrics through serverStatus.
Run:
db.serverStatus().metrics.ttlUseful metrics include:
MetricWhat it tells usdeletedDocumentsTotal documents deleted by TTLpassesCompleted TTL monitoring passessubPassesCompleted TTL monitoring sub-passes
Newer MongoDB versions expose additional metrics, including examined documents, examined keys and cumulative deletion duration.
For example, in versions that support them:
db.serverStatus().metrics.ttlYou can use these counters to understand whether the TTL monitor is processing data and how much work it is doing.
A useful production monitoring strategy is to track:
Growth in expired-but-not-yet-deleted documents.
TTL deletion throughput.
TTL deletion duration.
Database write latency during large cleanup periods.
Replication lag.
Storage utilisation.
TTL pass and sub-pass progression.
One important point: cumulative counters should be monitored as rates and differences over time, not interpreted as instantaneous values.
If a collection continuously generates expired records faster than the TTL monitor can remove them, the backlog will grow.
That is a capacity and lifecycle-design problem, not simply an indexing problem.
15. The larger engineering lesson
We started with a seemingly simple question:
How does MongoDB delete millions of expired documents?
But the discussion reveals several broader engineering principles.
First: efficient identification is not the same as efficient removal.
Indexes reduce the cost of finding expired records, but each individual deletion still has a cost.
Second: the best optimisation may be changing the unit of work.
Instead of deleting millions of documents individually, organise data into lifecycle-aligned segments and retire entire segments when possible.
Third: expiration and execution are different concepts.
A timestamp tells us when data becomes eligible for expiration. It does not guarantee that an operation will execute at that exact instant.
Fourth: background cleanup should never be the foundation of financial correctness.
Business rules must be enforced independently of cleanup schedules.
Fifth: distributed systems must be designed for delayed, duplicated and missed events.
Schedulers, queues and workers can fail. Idempotency, atomic state transitions, durable event handling and reconciliation are what make the overall system reliable.
Final thoughts
What I find most interesting about TTL is not the index itself. It is the separation of responsibilities behind it.
MongoDB does not attempt to make every document independently responsible for its destruction. It maintains an ordered index, runs a background monitor, processes expired records in controlled batches and accepts that physical deletion may happen after the expiration timestamp.
That is a deliberate engineering trade-off.
It prioritises manageable background processing over exact-time deletion.
And that leads to a principle I believe is worth carrying into much larger system designs:
Never confuse when something becomes eligible to happen with when the system guarantees that it will happen.
A database can tell us that a record has expired. A scheduler can tell us that an event is due. A queue can deliver that event. But none of these mechanisms, on its own, guarantees that a financial operation has been executed correctly.
That guarantee must come from the architecture: explicit business invariants, atomic state transitions, idempotent execution, durable event handling and reconciliation.
TTL is excellent at what it was designed to do.
The engineering mistake is asking it to do something it was never designed to guarantee.
Gunjan Sharma
Staff Platform Engineer | Security & GenAI
Exploring distributed systems, database internals, platform reliability, and the engineering principles behind financially correct systems.
https://gunjan.tech | Linkedin