Scale By the Bay 2018

Learn about the streaming data pipeline that supports the Mux Video service to process CDN access logs to support performance monitoring, dynamic CDN selection, log enrichment, and utility billing. The talk will cover the architecture & technologies (Go, Java, Kafka, Flink) and some of the challenges we’ve faced, including variable log delivery schedules that are addressed with Flink’s powerful windowing features, and dynamic CDN selection based on CDN performance in a geographic locale. This talk will appeal to developers faced with processing large volumes of data with minimal latency (a requirement that has become increasingly common) and supporting a growing number of business applications.