Surfstore is a distributed key-value store that allows users to store and synchronize their files in the cloud. The key-value store consists of Meta Stores, which store file metadata, and Block Stores, which store the actual block data of the file. The RAFT Consensus Algorithm is implemented with the Meta Store to make the service fault-tolerant. For the Block Stores, a technique called Consistent Hashing is adopted to efficiently scale the service. A binary search algorithm is used in the Consistent Hashing Ring to accelerate the server lookup.

Stack

Go, gRPC, Protobuf, SQLite, RAFT Consensus Algorithm, Consistent Hashing

Link

https://github.com/ucsd-cse224-wi23/proj5-chrisliang12

Associated with

CSE 224 - Networked Systems

Highlights

  • A scalable key-value store that provides cloud file syncing.
  • Maintained a local index file implemented with SQLite.
  • Scaled the cloud storage with Consistent Hashing.
  • Implemented binary search in Consistent Hashing Ring for fast server lookup.
  • Increased fault-tolerance with a subset of RAFT Consensus Algorithm.
  • Developed conflict resolution and persistence using object versioning

Demo

(Note: VPN is needed for the video if you are accessing from China)

© 2023 Yong Liang | Powered by