Multi Step Processes medium
What is a 'Poison Pill' in the context of stream processing?
- A A virus in the software.
- B A command to shut down the server.
- C A specific message payload that crashes the consumer, causing the message to be returned to the queue and redelivered infinitely.
- D A security token.
Answer: C. A specific message payload that crashes the consumer, causing the message to be returned to the queue and redelivered infinitely.
Poison Pills (or 'messages of death') block partition progress because the consumer crashes, restarts, picks up the same message, and crashes again.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- A user registration process involves two steps: inserting a row into the 'Users' database table and inserting a row into the 'Profiles' table. If the first insert succeeds but the second fails, the database is left in an inconsistent state. What mechanism prevents this?
- In a multi-step workflow across different services (Order Service -> Payment Service), you cannot use a single database transaction. If the Payment fails after the Order was created, how do you fix the Order state?
- You are writing a script to process 10,000 images. The script runs on a server that might restart unexpectedly. To ensure you don't have to start over from image #1 after a crash, what should you implement?
- A client app sends a 'Pay $50' request to your API. The server processes the payment but the network response is lost. The client automatically retries the request. What prevents the user from being charged $100?
- A user uploads a large video file that needs to be transcoded (converted), which takes 5 minutes. Why is it bad practice to keep the HTTP connection open while the server processes the video?
- How does CDC (reading the database transaction log) solve the 'Dual Write' problem better than the Application-level Outbox pattern?