A proof-of-idea for a read-finest filesystem with random contents generated on
search recordsdata from.
Licensed below the Blue Oak Model License 1.0.0.
- Rust and Cargo (tested with Rust 1.47.0)
- fuse (including the headers, that is
libfuse-devon Debian) (tested with
Easy how one can bustle
Within the muse of the repo:
mkdir mnt cargo bustle --commence mnt
This is able to per chance mount PlentyFS into
mnt listing and block. You may well per chance per chance also work with the
listing in a special terminal.
Even as you’re executed, cd to the muse of the repo and make contact with:
fusermount -u mnt
This is able to per chance unmount PlentyFS and commence the foremost terminal.
The filesystem may well well per chance well also simply aloof:
- be read-finest
- have recordsdata with random contents
- retailer (nearly) nothing in memory or on disk
The structure below test
For this PoC, the FS contains a single listing (root) with a mounted series of
recordsdata (1,000) of mounted dimension (1 megabyte each and every). The finest factor that is random is
the contents of the file.
Upon mounting, we purchase the PID of the fuse program. That is our “root seed”, and
or now not it’s the finest price that PlentyFS retail outlets in memory. All the pieces else is computed
from it and the meta-recordsdata.
Upon each and every
read(), we generate the blocks that beget requested recordsdata. To retain out
generate a “file seed”: mix root seed with the file’s inode number, and
hash the result;
generate the block: mix the file seed with the block number, and hash the
This structure is “embarrassingly parallel”, for the reason that contents of the file
blocks depend finest on the “root seed”, file’s inode, and the block offset. This
may well well per chance well also simply aloof allow it to scale linearly to many cores.
Little print of this implementation:
- file inodes are numbered from 2 upwards;
- the hash we use is SHA-1.
On Intel i7-7700HQ,
tar -cvf /dev/shm/plentyfs.tar /mnt achieves 115 MB/s.
/dev/shm/ is a
tempfs. The tempo became once little by this program,
which maxed out a single core for your whole length of the benchmark.
Why the title
Horn of plenty (conrnucopia) is a tiny object overflowing with meals and riches.
In an identical vogue, PlentyFS is a minute filesystem containing as vital recordsdata as that you may well even