Back To Schedule
Friday, November 16 • 2:10pm - 2:50pm
FiloDB: Real-time, In-Memory Time Series at Massive SMACK Scale

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Time series and event data is becoming huge for every business, and ingesting millions of series reliably while answering many concurrent queries from users is a huge challenge. In this talk I share the story of developing and productionizing FiloDB, an open source, in-memory time series solution built with the Scala, Akka, Kafka, Cassandra, Mesos (SMACK) stack. FiloDB is able to reliably ingest monitoring/time series data and answer tons of low latency queries at massive scale. * Why we developed our own solution after looking at Prometheus, OpenTSDB, Cassandra, etc. * Time series data model and low-latency distributed querying at scale * The benefits and challenges of off-heap, in-memory data processing at scale * Building a database for modern container environments * Challenges with scaling the Prometheus data model while remaining compatible * Persistent, recoverable data at scale with Kafka and Cassandra * Key lessons in building massively scalable, real-time, low-latency data systems

avatar for Evan Chan

Evan Chan

Senior Data Engineer, UrbanLogiq
Evan is currently Senior Data Engineer at UrbanLogiq, where he is using Rust, among other tools, in building robust data platforms to help public servants build better communities. Evan has been a distributed systems / data / software engineer for twenty years. He led a team developing... Read More →

Friday November 16, 2018 2:10pm - 2:50pm PST