FoundationDB Summit 2018

FoundationDB is an integral part of the architecture of SnowflakeDB, a cloud SQL analytics database. FDB is effectively their OLTP engine and has allowed them to build some truly amazing and differentiating features.

Snowflake have been operating it for 4+ years, and as they have grown, FDB has continued to scale from tens of servers to thousands of servers at present.

In this talk Ashish will present why FDB was their preferred choice, what purposes it serves, and the reliability and scale of their FDB clusters.

Ryan will demonstrate how FoundationDB can be applied to solve real business problems today and how to map common infrastructure components like logs, tables, and indexes into a cohesive system within FoundationDB. His example applies these techniques to a problem ClickFunnels was facing in mid-2018, which required scanning millions of end-user data points for each of their tens of thousands of customers multiple times per hour. Through custom bitmap indexes built on top of FoundationDB, queries which simply wouldn't finish now take milliseconds, which enables new use cases never thought possible.

Document Layer is a stateless micro server on top of FoundationDB that allows management of JSON documents at large scale. It exposes the traits of FoundationDB through document data model, such as fully ordered documents, consistent indexes and serializable transactions. It does all this while maintaining wire compatibility with MongoDB® API. As the compatibility is done at wire level, existing MongoDB® tools and drivers work seem less. In this talk, we explore how FoundationDB core strengths play well together with Document data model to make it easier to use and reliable database.

JanusGraph is a popular open source property graph database that serves a variety of transactional and analytical use cases. It was originally designed to run on top of a number of different distributed storage engines including Apache Cassandra and Apache HBase. This talk will discuss the new JanusGraph FoundationDB storage adapter which adds distributed ACID support to JanusGraph. Topics will include an intro to JanusGraph and FoundationDB layer development followed by a deep dive into how the property graph model and read/write access patterns have been mapped on to FoundationDB. We will also discuss how FoundationDB's unique features  address a number of JanusGraph pain points and discuss further improvements that it will enable going forward.

This presentation will explain the role of FDB's storage engine component, how the storage engine's requirements are influenced by FDB's distributed architecture, and it will give a brief overview of the upcoming first storage engine to be designed with FDB's specific read and commit paths in mind.  In addition to higher write throughput, this new design will have a smaller space footprint via key prefix compression and a configurable ability to enable long running read transactions by trading off a reasonable amount of disk space to persist old data versions on disk.