Loading…
FoundationDB Summit 2018 has ended

Sign up or log in to bookmark your favorites and sync them to your phone or calendar.

Monday, December 10
 

8:00am

Registration + Breakfast
FoundationDB Summit registration will be available directly outside of 3AB at the Washington State Convention Center.


Monday December 10, 2018 8:00am - 9:00am
3AB, Washington State Convention Center

9:00am

Welcome + Introduction - Dave Lester, Apple
Speakers
DL

Dave Lester

Open Source Program Manager, Apple


Monday December 10, 2018 9:00am - 9:10am
3AB, Washington State Convention Center

9:10am

State of the FoundationDB Community - Ben Collins, Apple
This talk will provide a birds-eye view of the current state of the FoundationDB project and its community. This will cover some details of interesting deployments and collaborators as well as community projects of note. To make sure that attendees, both new to the project and more seasoned, have good context on the workings of the community, this will cover current and emerging community process.

Speakers
BC

Ben Collins

Apple
Ben Collins is Project Lead for FoundationDB. Ben was an early committer on the project and has worked on core features, testing, and bindings. He currently manages the FoundationDB team at Apple, which both develops on the FoundationDB KV Store and operates it as a service for internal... Read More →


Monday December 10, 2018 9:10am - 9:30am
3AB, Washington State Convention Center

9:30am

Technical Overview of FoundationDB - Evan Tschannen, Apple
This presentation will walk through what is happening in the internals of the FoundationDB in response to user operations. The talk will focus on explaining how FoundationDB provides external consistency, and how FoundationDB scales as more processes join a cluster.

Speakers
ET

Evan Tschannen

Software Engineer, Apple
Evan Tschannen has been working on FoundationDB for over 8 years. He has implemented many of the critical components of the system. He now manages the development team for FoundationDB at Apple.


Monday December 10, 2018 9:30am - 10:10am
3AB, Washington State Convention Center

10:10am

How FoundationDB powers SnowflakeDB's metadata! - Ashish Motivala, Snowflake Computing
FoundationDB is an integral part of the architecture of SnowflakeDB, a cloud SQL analytics database. FDB is effectively their OLTP engine and has allowed them to build some truly amazing and differentiating features.

Snowflake have been operating it for 4+ years, and as they have grown, FDB has continued to scale from tens of servers to thousands of servers at present.

In this talk Ashish will present why FDB was their preferred choice, what purposes it serves, and the reliability and scale of their FDB clusters.

Speakers
AM

Ashish Motivala

Engineering Manager, Snowflake Computing
Ashish has worked on databases for over 12 years. For the last 5+ years, he's worked at Snowflake Computing, a cloud-based SQL analytics database, where he built various parts of the database and the service. He currently manages the FoundationDB development team at Snowflake, which... Read More →


Monday December 10, 2018 10:10am - 10:30am
3AB, Washington State Convention Center

10:30am

AM Break
Monday December 10, 2018 10:30am - 10:50am
3AB, Washington State Convention Center

10:50am

Solving Everyday Data Problems with FoundationDB - Ryan Worl, Ryan Worl
Ryan will demonstrate how FoundationDB can be applied to solve real business problems today and how to map common infrastructure components like logs, tables, and indexes into a cohesive system within FoundationDB. His example applies these techniques to a problem ClickFunnels was facing in mid-2018, which required scanning millions of end-user data points for each of their tens of thousands of customers multiple times per hour. Through custom bitmap indexes built on top of FoundationDB, queries which simply wouldn't finish now take milliseconds, which enables new use cases never thought possible.

Speakers
avatar for Ryan Worl

Ryan Worl

Consultant, Ryan Worl
Ryan is software engineer primarily focused on improving the performance of online services. His primary client is ClickFunnels, a sales funnel automation platform, where he is developing systems to process millions of customer interactions at low-latency using FoundationDB.



Monday December 10, 2018 10:50am - 11:20am
3AB, Washington State Convention Center

11:20am

Running FDB at scale at Wavefront - Matthew Zeier + Mike McMahon, VMware
Wavefront has been running FoundationDB in production for years. Our fleet consists of 500 instances of FoundationDB spanning 35 Wavefront clusters. About 400 of those are running FoundationDB as both memory & ssd systems.

Wavefront Ops will show how we manage FoundationDB at scale, including automatically provisioning new clusters, expanding/contracting existing clusters and how we replace an entire FoundationDB fleet.

We’ll include a live demo showing how we use Terraform to quickly add capacity and how we use Ansible to manage exclusions (or replace an entire running fleet).

The discussion will also cover the architecture design of a single Fdb instance, including disk layouts and instance store caching that enable us to achieve memory write rates of 800KHz and ssd write rates of 70KHz.

Speakers
avatar for Mike McMahon

Mike McMahon

Staff Engineer, Wavefront Operations, Wavefront by VMware, Wavefront
Wavefront's most senior staff engineer, Mike has mastered FoundationDB and uses this knowledge alongside Wavefront to operate and monitor hundreds of FoundationDB instances at scale.
avatar for Matthew Zeier

Matthew Zeier

Sr. Manager, Wavefront Operations, Wavefront by VMware
As Senior Manager, Technical Operations, Matthew is focused on delivering the highest level of reliability and velocity for the services his teams support.An industry veteran with over 20 years' experience in Network Engineering & Operations, Matthew understands the challenges and... Read More →



Monday December 10, 2018 11:20am - 11:50am
3AB, Washington State Convention Center

11:50am

12:20pm

Lunch
Monday December 10, 2018 12:20pm - 1:20pm
3AB, Washington State Convention Center

1:20pm

FoundationDB Document Layer - Bhaskar Muppana, Apple
Document Layer is a stateless micro server on top of FoundationDB that allows management of JSON documents at large scale. It exposes the traits of FoundationDB through document data model, such as fully ordered documents, consistent indexes and serializable transactions. It does all this while maintaining wire compatibility with MongoDB® API. As the compatibility is done at wire level, existing MongoDB® tools and drivers work seem less. In this talk, we explore how FoundationDB core strengths play well together with Document data model to make it easier to use and reliable database.

Speakers
avatar for Bhaskar Muppana

Bhaskar Muppana

Software Engineer, Apple Inc
Bhaskar is a Software Engineer at Apple. His time before Apple, he worked as a computer architect at Intel and worked on an in-kernel Cassandra storage backend. In Apple, he worked some time on Cassandra. Now working on FoundationDB layers and he is the lead developer for Document... Read More →


Monday December 10, 2018 1:20pm - 1:40pm
3AB, Washington State Convention Center

1:40pm

The JanusGraph FoundationDB Storage Adapter - Ted Wilmes, Expero Inc.
JanusGraph is a popular open source property graph database that serves a variety of transactional and analytical use cases. It was originally designed to run on top of a number of different distributed storage engines including Apache Cassandra and Apache HBase. This talk will discuss the new JanusGraph FoundationDB storage adapter which adds distributed ACID support to JanusGraph. Topics will include an intro to JanusGraph and FoundationDB layer development followed by a deep dive into how the property graph model and read/write access patterns have been mapped on to FoundationDB. We will also discuss how FoundationDB's unique features  address a number of JanusGraph pain points and discuss further improvements that it will enable going forward.

Speakers
avatar for Ted Wilmes

Ted Wilmes

Data Architect, Expero Inc.
Ted Wilmes, Data Architect at Expero, is a graduate of Trinity University where he studied computer science and art history. At Expero, Ted enjoys putting his deep knowledge of graph computing to work as he helps customers of all types navigate the burgeoning property graph database... Read More →



Monday December 10, 2018 1:40pm - 2:00pm
3AB, Washington State Convention Center

2:00pm

Avoiding Conflicts in Highly-Concurrent Applications - Alec Grieser, Apple
This is intended to be a somewhat quick how-to on how to build useful primitives using FoundationDB features to help avoid conflicts in highly-concurrent systems. In particular, thinking about minimizing contention and not running into correctness issues related to multiple workers going at once. This will be demonstrated by explaining the design of a few simple patterns that are often useful within larger applications. This could include implementing (or avoiding) an auto-incrementing primary key replacement, including the naïve approach, an approach using a max_index, and an approach using versionstamps. Another application might be a high-contention queue (like our high-contention queue layer), where applications must be able to both append to the end of a growing thing as well as allow for multiple readers to pop from the beginning. Another might be something like a semaphore where you might have multiple requests at once attempting to take or release locks and doing that in an efficient way might be important. (In principle, this could be shortened to one example to meet the 20 minute time limit.)

Speakers
avatar for Alec Grieser

Alec Grieser

Software Engineer, Apple
Alec is a software engineer at Apple working on CloudKit, a managed cloud database. He is one of the contributors to the FoundationDB Record Layer, an open source library which provides roughly relational semantics on top of the FoundationDB key-value store. He has spoken about FoundationDB... Read More →


Monday December 10, 2018 2:00pm - 2:20pm
3AB, Washington State Convention Center

2:20pm

Snowcannon: Synchronous Data Replication in FDB - Markus Pilman + David Youngworth, Snowflake Computing
The current FDB disaster recovery solution uses asynchronous replication, therefore failover might result in data loss.  

For Snowflake, this is an unacceptable business risk.

Snowcannon is a synchronous, distributed queueing system built within FDB.  An FDB cluster can synchronously replicate transactions to Snowcannon, and Snowcannon asychronously pushes these transactions to a secondary cluster.  The queueing system has three benefits: (1) It can replay rollbacks and redos (2) It allows us to bring down the secondary cluster for maintenance while the primary remains online. (3) In the event that both primary and secondary cluster fails, it allows recovery from backups with zero data loss.

This talk will go over the design of Snowcannon, including the replication protocol, failover switch, queueing architecture, and the quorum protocol.

Speakers
MP

Markus Pilman

Software Engineer, Snowflake Computing
Markus Pilman is the engineering lead of the FoundationDB team at Snowflake Computing. He joined Snowflake two years ago after graduating as a Doctor of Sciences from ETH Zurich.He was the author or co-author of and presenter of several papers in the academic database community.
avatar for David Youngworth

David Youngworth

Software Engineer, Snowflake Computing
David Youngworth joined the FoundationDB team at Snowflake Computing nearly 2 years ago.  He is a core developer on the Snowcannon project, Snowflake's synchronous replication engine and HA failover solution, among other FoundationDB related projects. He has a variety of systems... Read More →


Monday December 10, 2018 2:20pm - 2:50pm
3AB, Washington State Convention Center

2:50pm

3:20pm

Future of FoundationDB Storage Engines - Steve Atherton, Apple
This presentation will explain the role of FDB's storage engine component, how the storage engine's requirements are influenced by FDB's distributed architecture, and it will give a brief overview of the upcoming first storage engine to be designed with FDB's specific read and commit paths in mind.  In addition to higher write throughput, this new design will have a smaller space footprint via key prefix compression and a configurable ability to enable long running read transactions by trading off a reasonable amount of disk space to persist old data versions on disk.

Speakers
SA

Steve Atherton

Software Engineer, Apple
Steve Atherton is a core developer on FoundationDB at Apple.


Monday December 10, 2018 3:20pm - 3:40pm
3AB, Washington State Convention Center

3:40pm

Cross-Region Replication Options with FoundationDB - Evan Tschannen, Apple
FoundationDB has a number of different configurations for replicating data between regions. This presentation will go through these different options and discuss the tradeoffs between them.

Speakers
ET

Evan Tschannen

Software Engineer, Apple
Evan Tschannen has been working on FoundationDB for over 8 years. He has implemented many of the critical components of the system. He now manages the development team for FoundationDB at Apple.


Monday December 10, 2018 3:40pm - 4:00pm
3AB, Washington State Convention Center

4:00pm

Growing A Successful and Sustainable Project - Nithya Ruff, Comcast
The basis for many open source projects is to solve a problem and deliver value.  But the growth and adoption comes from the way the project is setup for positive governance, community growth and sustainability.  Nithya will talk about the elements needed to set up a community for growth and opening doors to new contributors.

Speakers
avatar for Nithya Ruff

Nithya Ruff

Open Source Program Office Chair, LF Board Member, Comcast
Nithya Ruff is a well-known and well-regarded leader in Open Source. As the Head of Comcast’s Open Source Practice, she is responsible for growing Open Source culture inside of Comcast and engagement with external communities. She is a passionate advocate for opening doors to new... Read More →


Monday December 10, 2018 4:00pm - 4:30pm
3AB, Washington State Convention Center

4:30pm

Lightning Talk: Highlights of the Snowflake Elastic Data Warehouse - Torsten Grabs, Snowflake
Snowflake is an analytics database provided as a cloud service. The Snowflake data warehouse uses a new SQL database engine that built from scratch and was designed for the cloud. In this talk Sameet will describe the design of Snowflake's novel multi-cluster, shared-data cloud architecture. The talk will highlight some of the key features of Snowflakelike extreme and instant elasticity, support for semi-structured and schema-less data, and data sharing. The talk will also present upcoming features.

Speakers
TG

Torsten Grabs

Snowflake
Torsten currently serves as Director of Product Management at Snowflake. Torsten’s focus are Snowflake on Microsoft Azure, Snowflake's developer platform and integration with developer ecosystems such as Spark. Before joining Snowflake, Torsten spent more than a decade at Microsoft... Read More →


Monday December 10, 2018 4:30pm - 4:37pm
3AB, Washington State Convention Center

4:40pm

Lightning Talk: Instant Backups and Restores - Senthil Kumar Ramamoorthy, Snowflake
Current backups and restore mechanism available for FDB can be quite resource and time intensive operation. Senthil, will present a mechanism to do instant backup and restores across FDB clusters that are equivalent. The mechanism relies on a snapshottable file-system made available to TLogs and Storage processes.

Speakers
SK

SENTHIL Kumar RAMAMOORTHY

Software Engineer, Snowflake
- Currently Software Engineer at Snowflake and working on Foundation DB related projects- Prior to SnowFlake, was a Distinguished Technologist at HPE and was responsible for the storage stack of Nimble Storage.- Have presented at team, group and SE level talks.


Monday December 10, 2018 4:40pm - 4:47pm
3AB, Washington State Convention Center

4:50pm

Lightning Talk: Hotspot detection and handling in FoundationDB - Ashish Motivala, Snowflake Computing
Hotspot, are a source of problems for all databases. On FDB they can result in a system-wide performance degradation. This talk will demonstrate how SnowflakeDB, which uses FDB as its OLTP database, keeps track of FDB transactions and maps data to partitions on FDB storage nodes. The enables detection of hotspots, and discusses ways to handle them.

Speakers
AM

Ashish Motivala

Engineering Manager, Snowflake Computing
Ashish has worked on databases for over 12 years. For the last 5+ years, he's worked at Snowflake Computing, a cloud-based SQL analytics database, where he built various parts of the database and the service. He currently manages the FoundationDB development team at Snowflake, which... Read More →


Monday December 10, 2018 4:50pm - 4:57pm
3AB, Washington State Convention Center

5:00pm

Lightning Talk: Performance Testing of FoundationDB using TPC-C - Trevor Clinkenbeard, Snowflake Computing
This talk discusses the performance testing of FoundationDB done at Snowflake Computing. This includes discussion of a TPC-C workload that has been used to discover performance regressions and to test the performance effects of new features, rolling upgrades, and various cluster configurations.

Speakers
TC

Trevor Clinkenbeard

Software Developer, Snowflake Computing
I graduated from the University of Waterloo earlier this year, and had been interning with Snowflake Computing since 2016. I've been with the FoundationDB team within Snowflake since 2017 and started full-time in May of this year. I'm currently working on improving client-side throttling... Read More →



Monday December 10, 2018 5:00pm - 5:07pm
3AB, Washington State Convention Center

5:10pm

Lightning Talk: Towards supporting backward compatibility - Andrew Noyes , Snowflake Computing
FoundationDB's serialization library provides an abstraction that allows it to be replaced with a new serialization format without changing (much) client code. In this talk, Andrew will describe a flatbuffers implementation that is nearly a drop-in replace for the current serialization, and how it allows rolling upgrades and backward compatibility. Andrew will also describe how this might be tested in the simulator.

Speakers
AN

Andrew Noyes

Software Engineer, Snowflake Computing
Andrew is a software engineer working at Snowflake Computing on FoundationDB development.



Monday December 10, 2018 5:10pm - 5:17pm
3AB, Washington State Convention Center

5:20pm

Lightning Talk: Entity Store: A FoundationDB Layer for Versioned Entities with Fine Grained Authorization and Lineage - Stephen Pimentel, Apple
Entity Store is a FoundationDB layer that implements a data model for versioned entities with fine-grained authorization and lineage. It is implemented as a Python library exposing its own API above the FoundationDB Python bindings. Versioning of entities is automatic, with modification to immutable fields resulting in a new version rather than mutation. Versions form a parentage tree and can be explicitly selected for use. Read and write authorizations are separately recorded at the level of individual fields (a.k.a “cell-based security.”) Lineage is recorded via labeled, directed multigraphs and used to represent version parentage and other forms of provenance.

Each entity is modeled as a collection of objects that represent its distinct versions, along with a “core” object for non-versioned fields. Objects are schemaless and consist of any number of fields with values. Fields can be single-valued or set-valued, and either case can optionally be indexed. Fields can also have large blob values.

FoundationDB was chosen because it allows rich data models to be flexibly mapped to its Key-Value Store. Required features such as versioning and lineage were straightforward to incorporate into the layer. We rely on FoundationDB’s distributed transactions to allow multiple clients concurrent access to entities without danger of erroneous results or data corruption.

Our use case for Entity Store is as a backend for a metadata manager in a service for data science and machine learning pipelines. These pipelines interact with multiple types of entities, including data sets, notebooks, machine learning models, and experiments. Each of these entities has associated metadata that must be versioned, protected by authorization, and tracked for lineage. The metadata manager uses GraphQL to define schemas for entity types, queries, and mutations.

Speakers
SP

Stephen Pimentel

Software Engineer, Apple
Stephen is a software engineer at Apple working on a machine learning platform. Before its acquisition by Apple, he worked at FoundationDB.


Monday December 10, 2018 5:20pm - 5:27pm
3AB, Washington State Convention Center

5:30pm