Surfstore is a distributed key-value store that allows users to store and synchronize their files in the cloud. The key-value store consists of Meta Stores, which store file metadata, and Block Stores, which store the actual block data of the file. The RAFT Consensus Algorithm is implemented with the Meta Store to make the service fault-tolerant. For the Block Stores, a technique called Consistent Hashing is adopted to efficiently scale the service. A binary search algorithm is used in the Consistent Hashing Ring to accelerate the server lookup.
Stack
Go, gRPC, Protobuf, SQLite, RAFT Consensus Algorithm, Consistent HashingLink
https://github.com/ucsd-cse224-wi23/proj5-chrisliang12Associated with
CSE 224 - Networked SystemsHighlights
- A scalable key-value store that provides cloud file syncing.
- Maintained a local index file implemented with SQLite.
- Scaled the cloud storage with Consistent Hashing.
- Implemented binary search in Consistent Hashing Ring for fast server lookup.
- Increased fault-tolerance with a subset of RAFT Consensus Algorithm.
- Developed conflict resolution and persistence using object versioning
Demo
(Note: VPN is needed for the video if you are accessing from China)