kraken
P2P Docker registry capable of distributing TBs of data in seconds
Tech stack & purpose
Kraken is a peer-to-peer Docker registry built in Go and designed for scalable distribution of container images in hybrid cloud environments. It uses a BitTorrent-inspired protocol coordinated by a tracker to distribute Docker blobs across a network of agents running on each host, achieving over 50% of maximum download speeds even in clusters with thousands of hosts. The system consists of five main components—Agent, Origin, Tracker, Proxy, and Build-Index—that use self-healing hash rings for distribution and high availability, and it supports pluggable storage backends including S3, GCS, ECR, and HDFS. Kraken has been in production at Uber since 2018, where it distributes over 1 million blobs per day across clusters of 15,000+ hosts.
Community & reference links
Languages
Contributors (top 30 of 54)
- Cody Gibb424 commits
- Yiran Wang159 commits
- evelynl73 commits
- Evelyn Liu68 commits
- Igor Morozov57 commits
- akalpakchiev40 commits
- hweawer32 commits
- Sambhav Jain30 commits
- Thijmen21 commits
- dependabot[bot]19 commits · 0 followers
- Egor Grishechko18 commits
- Gaurav Kumar17 commits
- Antoine Pourchet17 commits
- Ed Oakes13 commits
- xinlongz17 commits
- zerosnake05 commits
- Zach Cheung4 commits
- rmalpani-uber4 commits
- Anton Kalpakchiev4 commits
- Arnav Sankaran3 commits
- Ori Rawlings3 commits
- Edward Oakes3 commits
- Thao Tran3 commits
- mmpei3 commits
- Elton Hu3 commits
- Antonio Alaniz3 commits
- Xin Ding2 commits
- Lionel Nicolas2 commits
- Mike Deats2 commits
- Magnus Vinther2 commits
