Managing Long Running Tasks hard
How do you atomically update your local database AND publish a message to a queue without losing sync if the process crashes in between?
- A Outbox Pattern. Insert the message into an 'Outbox' table in the SAME database transaction as the business data. A separate background process reads the Outbox and pushes to the Queue.
- B Publish first, then commit DB.
- C Commit DB first, then publish.
- D Use Redis.
Answer: A. Outbox Pattern. Insert the message into an 'Outbox' table in the SAME database transaction as the business data. A separate background process reads the Outbox and pushes to the Queue.
Transactional publishing guarantees either both happen or neither happens.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- What is the primary purpose of a Dead Letter Queue (DLQ)?
- The architectural pattern where one service creates tasks and another processes them is called:
- In a Priority Queue system (High vs Low), what is 'Starvation' and how do you fix it?
- What does 'Fire and Forget' mean?
- Why might you 'Throttle' (limit the speed of) your background workers?
- When implementing retries for failed tasks, why use 'Exponential Backoff' with 'Jitter'?