Added inline_max, to optionally limit the size of inlined files

Inlined files live in metadata and decrease storage requirements, but
may be limited to improve metadata-related performance. This is
especially important given the current plague of metadata performance.

Though decreasing inline_max may make metadata more dense and increase
block usage, so it's important to benchmark if optimizing for speed.

The underlying limits of inlined files haven't changed:
1. Inlined files need to fit in RAM, so <= cache_size
2. Inlined files need to fit in a single attr, so <= attr_max
3. Inlined files need to fit in 1/8 of a block to avoid metadata
   overflow issues, this is after limiting by metadata_max,
   so <= min(metadata_max, block_size)/8

By default, the largest possible inline_max is used. This preserves
backwards compatibility and is probably a good default for most use
cases.

This does have the awkward effect of requiring inline_max=-1 to
indicate disabled inlined files, but I don't think there's a good
way around this.
This commit is contained in:
Christopher Haster
2023-12-20 22:56:26 -06:00
parent 09972a1710
commit 8b8fd14187
7 changed files with 79 additions and 25 deletions

10
lfs.h
View File

@@ -272,6 +272,15 @@ struct lfs_config {
// Defaults to block_size when zero.
lfs_size_t metadata_max;
// Optional upper limit on inlined files in bytes. Inlined files live in
// metadata and decrease storage requirements, but may be limited to
// improve metadata-related performance. Must be <= cache_size, <=
// attr_max, and <= block_size/8. Defaults to the largest possible
// inline_max when zero.
//
// Set to -1 to disable inlined files.
lfs_size_t inline_max;
#ifdef LFS_MULTIVERSION
// On-disk version to use when writing in the form of 16-bit major version
// + 16-bit minor version. This limiting metadata to what is supported by
@@ -451,6 +460,7 @@ typedef struct lfs {
lfs_size_t name_max;
lfs_size_t file_max;
lfs_size_t attr_max;
lfs_size_t inline_max;
#ifdef LFS_MIGRATE
struct lfs1 *lfs1;