Scale By the Bay 2018

I present the Distributed Chemical Machine (DCM) - a purely functional, fully declarative framework for parallel, concurrent, and distributed computing in Scala. The DCM can automatically run multi-core concurrent code on any number of machines connected to (one or more) Zookeeper instances. Zookeeper provides data coordination, persistence, and fault tolerance, allowing the programmer to focus on the distributed application logic in a peer-to-peer architecture. The DCM builds upon the Chemical Machine (http://chemist.io), a data-driven, message-passing concurrency paradigm that significantly improves upon the Actor Model, achieving automatic parallelism and a higher level of declarative expressiveness for concurrency. Previously I implemented the (single-JVM, multi-core) Chemical Machine as an embedded DSL in Scala. With very few code changes and little configuration, a Chemical Machine-based application can now be ported to the DCM and run on a cluster, achieving automatic distribution.