Data Modeling hard
How do Time Series databases (like Gorilla/Prometheus) achieve 10x compression on timestamp data?
- A Delta-of-Delta Encoding, which stores only the difference between a timestamp and its predicted value based on a fixed, previously observed interval.
- B Generic GZIP Streaming, which utilizes a massive, high-latency dictionary to compress blocks of thousand timestamps into a single, opaque binary blob.
- C Precision Truncation, where the database engine automatically drops millisecond or nanosecond data until the resulting integers are small enough to pack.
- D Binary Deduplication, which identifies and removes every duplicate timestamp entry from the influx stream to ensure only one unique point exists per second.
Answer: A. Delta-of-Delta Encoding, which stores only the difference between a timestamp and its predicted value based on a fixed, previously observed interval.
Because time-series data is usually highly regular (e.g., one point every 10 seconds), the 'delta' between timestamps remains constant. 'Delta-of-Delta' encoding exploits this: if the interval doesn't change, the delta-of-delta is zero, requiring only a single bit to store. This domain-specific optimization allows TSDBs to store months of high-resolution data in a fraction of the space required by a generic database.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- When modeling a Distributed Transaction using the Saga Pattern (Choreography or Orchestration), what type of consistency do you achieve?
- Which practice prevents SQL Injection attacks?
- What is 'Bi-Temporal' data modeling (e.g., in SQL:2011 standard)?
- Adding an Index to a database column generally improves Read performance. What is the negative trade-off?
- Log Systems Merge (LSM) Trees (used in RocksDB, Cassandra) optimize for fast writes. What is their primary downside regarding Disk I/O?
- What is the primary objective of normalizing a relational database schema (e.g., to 3NF)?