Managing Long Running Tasks hard
If a worker crashes *after* charging a credit card but *before* deleting the message from the queue, the message will redeliver. How do you prevent double charging?
- A Delete message first.
- B Check a side-table 'processed_payments' with a unique transaction_id before charging. If present, skip charging and just delete the message.
- C Refund the user.
- D Use a faster server.
Answer: B. Check a side-table 'processed_payments' with a unique transaction_id before charging. If present, skip charging and just delete the message.
The 'Check-Then-Act' pattern (with DB transactions) creates the Idempotency.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- What is the primary purpose of a Dead Letter Queue (DLQ)?
- The architectural pattern where one service creates tasks and another processes them is called:
- In a Priority Queue system (High vs Low), what is 'Starvation' and how do you fix it?
- How do you atomically update your local database AND publish a message to a queue without losing sync if the process crashes in between?
- What does 'Fire and Forget' mean?
- Why might you 'Throttle' (limit the speed of) your background workers?