leofs

Tech stack & purpose

LeoFS is a distributed object/blob storage system designed for data lakes and web applications, built on Erlang and supporting multiple access protocols including RESTful interfaces, Amazon S3-API compatibility, and NFS v3. The system consists of three core components—LeoGateway (handling HTTP requests with built-in object caching), LeoStorage (managing object and metadata storage with replication and recovery mechanisms), and LeoManager (monitoring and coordinating the cluster via consistent-hash-based routing)—all written in Erlang and built with the Rebar build system. LeoFS was originally sponsored by Rakuten, Inc. from 2012 to 2018 and is currently sponsored by Lions Data, Ltd.

Community & reference links

• Amazon S3 compatible REST API
• docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html
• Server Side Object Encryption
• AWS Signature v4
• leo-project.net/leofs/
• here
• leo-project.net/leofs/download.html
• docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPUTacl.html
• the versioning policy
• en.wikipedia.org/wiki/Data_lake
• GDPR
• en.wikipedia.org/wiki/Network_File_System
• github.com/basho/basho_bench
• eleveldb
• Cowboy
• kerl
• the cache mecanism
• github.com/leo-project/leo_center
• github.com/leo-project/leofs
• LeoFS Bench's repository
• large_object.chunked_obj_len
• large_object.reading_chunked_obj_len
• github.com/leo-project/leofs/blob/master/apps/leo_gateway/priv/leo_gateway.conf
• github.com/leo-project/leofs/blob/master/apps/leo_manager/priv/leo_manager_0.conf
• github.com/leo-project/leofs/blob/master/apps/leo_manager/priv/leo_manager_1.conf
• github.com/leo-project/leofs/blob/master/apps/leo_storage/priv/leo_storage.conf
• leofs-adm
• LEOFS_Issue#1017
• LEOFS_Issue#1021
• LeoFS Issue#531
• v1.0
• v1.1
• v1.2
• github.com/leo-project/leofs/releases/tag/1.4.4
• leofs-mon
• LeoFS Gateway
• LeoFS Storage
• Cowboy
• rebar
• gitter.im/leo-project/leofs
• gitter.im/leo-project/leofs?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
• Kubernetes Persistent Volumes
• leo-project.net/leofs/docs/
• REST-API
• leo-project.net/leofs/docs/admin/settings/cluster/
• LeoGateway
• LeoManager
• LeoStorage
• leofs-architecture-1

Languages

Contributors

• 
  Yosuke Hara1,052 commits · 60 followers
• 
  yoshiyuki.kanno205 commits · 19 followers
• 
  Yosuke Hara150 commits · 60 followers
• 
  Heinz N. Gies78 commits · 289 followers
• 
  Windkit Li45 commits · 7 followers
• 
  yoshiyuki.kanno42 commits · 19 followers
• 
  Vladimir Mosgalin35 commits · 1 followers
• 
  Satoshi Egi24 commits · 326 followers
• Mikolaj Golub8 commits
• Glass_saga6 commits
• 
  Vladimir Mosgalin4 commits · 1 followers
• 
  Kevin Meziere3 commits · 7 followers
• 
  yoshiyuki.kanno3 commits · 19 followers
• 
  gerald2 commits · 69 followers
• 
  Loïc Hoguin1 commits · 861 followers
• Shuichiro MAKIGAKI1 commits
• Maksim Zhylinski1 commits
• Max Kochubey1 commits
• Kunal Tyagi1 commits
• The Gitter Badger1 commits
• hiroaki-iwase1 commits