Borg: Large-scale Cluster Management at Google with Borg

February 19, 2024 · 557 words · 3 min · Borg Kubernetes Cluster Management Paper Reading

Borg is a cluster management system, similar to the closed-source version of Kubernetes (k8s).

• It achieves high utilization through admission control, efficient task packing, overcommitment, machine sharing, and process-level performance isolation.
• It provides runtime features to reduce failure recovery time for high-availability applications and scheduling policies that reduce the probability of correlated failures.
• It offers a declarative job description language, DNS integration, real-time job monitoring, and tools for analyzing and simulating system behavior, simplifying usage for end-users.

The paper aims to introduce the system design and share the experiences Google has gained behind it. This blog mainly focuses on system design, specifically the services Borg offers in terms of SLA, its abstraction of workloads, resources, and scheduling.

Percolator: Large-scale Incremental Processing Using Distributed Transactions and Notifications

September 28, 2023 · 1135 words · 3 min · Distributed System Transaction Paper Reading

It has been a while since I last studied, and I wanted to learn something interesting. This time, I’ll be covering Percolator, a distributed transaction system. I won’t translate the paper or delve into detailed algorithms; I’ll just document my understanding.

MIT6.824-Raft

February 21, 2022 · 953 words · 5 min · Paper Reading Consensus Distributed System MIT6.824

Finally, I managed to complete Lab 02 during this winter break, which had been on hold for quite some time. I was stuck on one of the cases in Test 2B for a while. During the winter break, I revisited the implementations from experts, and finally completed all the tasks, so I decided to document them briefly.

DFS-Haystack

October 6, 2021 · 1284 words · 7 min · DFS Paper Reading Distributed System

The primary project in my group is a distributed file system (DFS) that provides POSIX file system semantics. The approach to handle “lots of small files” (LOSF) is inspired by Haystack, which is specifically designed for small files. I decided to read through the Haystack paper and take some notes as a learning exercise.

MIT6.824 Bigtable

September 16, 2021 · 1908 words · 9 min · Paper Reading MIT6.824 DFS Distributed System

I recently found a translated version of the Bigtable paper online and saved it, but hadn’t gotten around to reading it. Lately, I’ve noticed that Bigtable shares many design similarities with a current project in our group, so I took some time over the weekend to read through it.