systemd/casync

casync is a content-addressable data synchronization tool that combines the rsync algorithm with content-addressable storage. It splits large data streams into variable-sized chunks based on content, stores these chunks in compressed files keyed by a strong hash, and uses a chunk index for reassembly. The tool introduces a well-defined, reproducible, random-access serialization format for directory trees, enabling efficient storage and retrieval of multiple related versions of large file systems or directory trees. It is designed for delivering and updating OS, VM, IoT, and container images over the Internet in an HTTP and CDN friendly way, and also serves as an efficient backup system. Similar contents result in mostly the same chunks, minimizing disk usage and network traffic.