Data Modeling hard
In an Event Sourced system, replaying 1 million events to calculate 'Current Balance' is too slow. How is this solved without abandoning the pattern?
- A Snapshotting, which periodically saves the materialized state so that the system only needs to replay the small number of events that occurred after the latest snapshot.
- B Event Truncation, where the system identifies and permanently deletes every 'stale' historical event that has already been factored into the current state calculation.
- C Hardware Acceleration, which leverages specialized high-frequency multicore CPUs to perform millions of event-replay operations in a single, ultra-low latency cycle.
- D Distributed Caching, where the final calculated result is stored in an external Redis cluster with a permanent TTL to ensure that the event log is never accessed again.
Answer: A. Snapshotting, which periodically saves the materialized state so that the system only needs to replay the small number of events that occurred after the latest snapshot.
As the event log grows, the time required to calculate the current state ('folding' the events) increases linearly. Snapshots act as a performance shortcut: by saving the result (e.g., 'Account Balance = $500 at Event #1,000,000'), the system can skip the first million events during the next load. This maintains the 'Source of Truth' status of the log while providing the O(1) read performance expected of modern database systems.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- When modeling a Distributed Transaction using the Saga Pattern (Choreography or Orchestration), what type of consistency do you achieve?
- Which practice prevents SQL Injection attacks?
- What is 'Bi-Temporal' data modeling (e.g., in SQL:2011 standard)?
- Adding an Index to a database column generally improves Read performance. What is the negative trade-off?
- Log Systems Merge (LSM) Trees (used in RocksDB, Cassandra) optimize for fast writes. What is their primary downside regarding Disk I/O?
- What is the primary objective of normalizing a relational database schema (e.g., to 3NF)?