Bigtable Optimizations
How did Bigtable make itself faster, cheaper and more reliable?
This will be my final post on Bigtable. In case you missed my earlier posts in this series I recommend you check those out before reading this post.
This post is going to focus on a hodgepodge of optimizations within Bigtable. This post differs from my other posts in that each section will not build on the previous sections, it really will just be five independent optimizations.
The following diagram was used in a previous post to show how reads ands writes are satisfied. It is included here simply to help readers reload mental context on Bigtable before diving into the optimizations. For more context on this diagram check out this post.
Now, let’s dive into those optimizations…
1) Locality Groups
Bigtable enables grouping multiple column families together into a locality group. Each locality group is intended to store logically similar column families. Typically what this means is that those column families are accessed together. Bigtable offers config knobs that apply per locality group and dedicated separate SSTables per locality group.
There are two advantages offered by locality groups.
- Faster Reads: By virtue of dedicating SSTables to a single locality group it can speed up reads. This is because reads which access columns in the locality group only need to fan out to the SSTables which are dedicated to that locality group. Without locality groups reads need to fan out to all SSTables.
- Config Tuning: Bigtable exposes configuration per locality group. This makes sense only when we consider that column families within a locality group should be logically related. Given a set of related column families, it makes sense to be able to apply configuration over the whole set. An example of such configuration is Bigtable’s
lazy_in_memory_loadoption. If this is specified for a locality group Bigtable will lazily store the data in SSTables in tablet server memory. This is a useful option for small pieces of data that are read often. Bigtable uses locality groups and this option to support its internal metadata table.
2) Multi-Layer Caching
So far we have talked about Bigtable storing data in two places — the memtable and the SSTables. The memtable stores recently written data in memory and the SSTables store older data on disk.
In addition to these two extremes Bigtable also has two levels of caching.
Scan Cache: Used to store key/value pairs that have been recently read from SSTables. This cache is useful when clients read the same row multiple times in short succession.
Block Cache: Used to store SSTable blocks that recently had to be loaded in order to satisfy a read. Note that not all data within a block will have been used to satisfy a read but everything within that block will be lexicographically close to something that has recently been read. This cache is useful for access patterns that frequently access pieces of data that are close together.
3) Bloom Filters
Bigtable offers a per locality group config option to specify a bloom filter. The bloom filter will be used to determine if a given row/column could possibly exist within an SSTable.
For certain types of locality groups this bloom filter can be effective at filtering out disk accesses which could not possibly return relevant SSTable blocks.
The bloom filter takes up some memory in the tablet server and is therefore not enabled by default for all locality groups. The user needs to specify the bloom filter as a config on the desired locality group.
4) Improving Tablet Recovery
When the leader indicates to a tablet server that it should unload a given tablet, the tablet server will first perform a minor compaction. Remember that a minor compaction involves serializing the memtable into an SSTable.
This process helps out the tablet server which will have to load that tablet because there will be a smaller amount of data from the commit log to read through. In more general terms, the smaller the data size which has not yet been saved to SSTables, the faster the recovery.
5) Using Immutability
The SSTables are immutable in Bigtable. This makes concurrency dead simple — there is nothing that needs to be done to make immutable objects concurrency safe.
Well folks, that is it for Bigtable. Thanks for following along. Next up I will be writing about another classic NoSQL technology: DynamoDB.
