Compare commits

...

9 Commits

Author SHA1 Message Date
Christopher Haster
68d28b5114 Merge pull request #966 from BrianPugh/fix-divide-by-zero-full-filesystem
Fix DivideByZero exception when filesystem is completely full.
2024-04-17 12:38:22 -05:00
Christopher Haster
1bc14933b7 Tweaked on-disk config comments for consistency
- Prefer "defaults to blablabla when zero" to hint that this is the
  default state when both explicitly set to zero and implicitly set to
  zero thanks to C's initializers.

- Prefer "disk" when referencing something stored "on disk". Other terms
  can quickly get ambiguous. Except maybe "block device"...
2024-04-17 00:16:20 -05:00
Christopher Haster
01b6a47ea8 Extended test_alloc to test inferred block_count
The block allocator is an area where inferred block counts (when
cfg.block_count=0) are more likely to cause problems.

As is shown by the recent divide-by-zero-exhaustion issue.
2024-04-17 00:04:56 -05:00
Brian Pugh
749a45650f Fix DivideByZero exception when filesystem is completely full. 2024-04-16 20:32:12 -07:00
Christopher Haster
4dd30c1b8f Merge pull request #948 from littlefs-project/fix-sync-ordering
Fix sync issue where data writes could appear before metadata writes
2024-03-08 16:49:59 -06:00
Christopher Haster
5c0d332ecd Merge pull request #939 from Graveflo/master
Add nim-littlefs to readme
2024-03-08 16:49:11 -06:00
Christopher Haster
cf68333a55 Merge pull request #937 from littlefs-project/fix-pending-rm-get-underflow
Fix synthetic move underflows in lfs_dir_get
2024-03-08 16:48:50 -06:00
Ryan McConnell
2752d8c486 add nim-littlefs to readme 2024-02-07 02:53:16 -05:00
Christopher Haster
ddbfcaa722 Fixed synthetic move underflows in lfs_dir_get
By "luck" the previous code somehow managed to not be broken, though it
was possible to traverse the same file twice in lfs_fs_traverse/size
(which is not an error).

The problem was an underlying assumption in lfs_dir_get that it would
never be called when the requested id is pending removal because of a
powerloss. The assumption was either:

1. lfs_dir_find would need to be called first to find the id, and it
   would correctly toss out pending-rms with LFS_ERR_NOENT.

2. lfs_fs_mkconsistent would be implicitly called before any filesystem
   traversals, cleaning up any pending-rms. This is at least true for
   allocator scans.

But, as noted by andriyndev, both lfs_fs_traverse and lfs_fs_size can
call lfs_fs_get with a pending-rm id if called in a readonly context.

---

By "luck" this somehow manages to not break anything:

1. If the pending-rm id is >0, the id is decremented by 1 in lfs_fs_get,
   returning the previous file entry during traversal. Worst case, this
   reports any blocks owned by the previous file entry twice.

   Note this is not an error, lfs_fs_traverse/size may return the same
   block multiple times due to underlying copy-on-write structures.

2. More concerning, if the pending-rm id is 0, the id is decremented by
   1 in lfs_fs_get and underflows. This underflow propagates into the
   type field of the tag we are searching for, decrementing it from
   0x200 (LFS_TYPE_STRUCT) to 0x1ff (LFS_TYPE_INTERNAL(UNUSED)).

   Fortunately, since this happens to underflow to the INTERNAL tag
   type, the type intended to never exist on disk, we should never find
   a matching tag during our lfs_fs_get search. The result? lfs_dir_get
   returns LFS_ERR_NOENT, which is actually what we want.

Also note that LFS_ERR_NOENT does not terminate the mdir traversal
early. If it did we would have missed files instead of duplicating
files, which is a slightly worse situation.

---

The fix is to add an explicit check for pending-rms in lfs_dir_get, just
like in lfs_dir_find. This avoids relying on unintended underflow
propagation, and should make the internal API behavior more consistent.

This is especially important for potential future gc extensions.

Found by andriyndev
2024-02-04 15:12:31 -06:00
4 changed files with 73 additions and 29 deletions

View File

@@ -258,6 +258,9 @@ License Identifiers that are here available: http://spdx.org/licenses/
use with the MirageOS library operating system project. It is interoperable
with the reference implementation, with some caveats.
- [nim-littlefs] - A Nim wrapper and API for littlefs. Includes a fuse
implementation based on [littlefs-fuse]
[BSD-3-Clause]: https://spdx.org/licenses/BSD-3-Clause.html
[littlefs-disk-img-viewer]: https://github.com/tniessen/littlefs-disk-img-viewer
[littlefs-fuse]: https://github.com/geky/littlefs-fuse
@@ -274,3 +277,4 @@ License Identifiers that are here available: http://spdx.org/licenses/
[littlefs-python]: https://pypi.org/project/littlefs-python/
[littlefs2-rust]: https://crates.io/crates/littlefs2
[chamelon]: https://github.com/yomimono/chamelon
[nim-littlefs]: https://github.com/Graveflo/nim-littlefs

13
lfs.c
View File

@@ -688,7 +688,7 @@ static int lfs_alloc(lfs_t *lfs, lfs_block_t *block) {
if (lfs->lookahead.ckpoint <= 0) {
LFS_ERROR("No more free space 0x%"PRIx32,
(lfs->lookahead.start + lfs->lookahead.next)
% lfs->cfg->block_count);
% lfs->block_count);
return LFS_ERR_NOSPC;
}
@@ -710,11 +710,14 @@ static lfs_stag_t lfs_dir_getslice(lfs_t *lfs, const lfs_mdir_t *dir,
lfs_tag_t ntag = dir->etag;
lfs_stag_t gdiff = 0;
// synthetic moves
if (lfs_gstate_hasmovehere(&lfs->gdisk, dir->pair) &&
lfs_tag_id(gmask) != 0 &&
lfs_tag_id(lfs->gdisk.tag) <= lfs_tag_id(gtag)) {
// synthetic moves
gdiff -= LFS_MKTAG(0, 1, 0);
lfs_tag_id(gmask) != 0) {
if (lfs_tag_id(lfs->gdisk.tag) == lfs_tag_id(gtag)) {
return LFS_ERR_NOENT;
} else if (lfs_tag_id(lfs->gdisk.tag) < lfs_tag_id(gtag)) {
gdiff -= LFS_MKTAG(0, 1, 0);
}
}
// iterate over dir block backwards (for faster lookups)

16
lfs.h
View File

@@ -59,7 +59,8 @@ typedef uint32_t lfs_block_t;
#endif
// Maximum size of custom attributes in bytes, may be redefined, but there is
// no real benefit to using a smaller LFS_ATTR_MAX. Limited to <= 1022.
// no real benefit to using a smaller LFS_ATTR_MAX. Limited to <= 1022. Stored
// in superblock and must be respected by other littlefs drivers.
#ifndef LFS_ATTR_MAX
#define LFS_ATTR_MAX 1022
#endif
@@ -203,7 +204,8 @@ struct lfs_config {
// program sizes.
lfs_size_t block_size;
// Number of erasable blocks on the device.
// Number of erasable blocks on the device. Defaults to block_count stored
// on disk when zero.
lfs_size_t block_count;
// Number of erase cycles before littlefs evicts metadata logs and moves
@@ -252,18 +254,18 @@ struct lfs_config {
// Optional upper limit on length of file names in bytes. No downside for
// larger names except the size of the info struct which is controlled by
// the LFS_NAME_MAX define. Defaults to LFS_NAME_MAX when zero. Stored in
// superblock and must be respected by other littlefs drivers.
// the LFS_NAME_MAX define. Defaults to LFS_NAME_MAX or name_max stored on
// disk when zero.
lfs_size_t name_max;
// Optional upper limit on files in bytes. No downside for larger files
// but must be <= LFS_FILE_MAX. Defaults to LFS_FILE_MAX when zero. Stored
// in superblock and must be respected by other littlefs drivers.
// but must be <= LFS_FILE_MAX. Defaults to LFS_FILE_MAX or file_max stored
// on disk when zero.
lfs_size_t file_max;
// Optional upper limit on custom attributes in bytes. No downside for
// larger attributes size but must be <= LFS_ATTR_MAX. Defaults to
// LFS_ATTR_MAX when zero.
// LFS_ATTR_MAX or attr_max stored on disk when zero.
lfs_size_t attr_max;
// Optional upper limit on total space given to metadata pairs in bytes. On

View File

@@ -8,17 +8,22 @@ defines.FILES = 3
defines.SIZE = '(((BLOCK_SIZE-8)*(BLOCK_COUNT-6)) / FILES)'
defines.GC = [false, true]
defines.COMPACT_THRESH = ['-1', '0', 'BLOCK_SIZE/2']
defines.INFER_BC = [false, true]
code = '''
const char *names[] = {"bacon", "eggs", "pancakes"};
lfs_file_t files[FILES];
lfs_t lfs;
lfs_format(&lfs, cfg) => 0;
lfs_mount(&lfs, cfg) => 0;
struct lfs_config cfg_ = *cfg;
if (INFER_BC) {
cfg_.block_count = 0;
}
lfs_mount(&lfs, &cfg_) => 0;
lfs_mkdir(&lfs, "breakfast") => 0;
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
for (int n = 0; n < FILES; n++) {
char path[1024];
sprintf(path, "breakfast/%s", names[n]);
@@ -39,7 +44,7 @@ code = '''
}
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
for (int n = 0; n < FILES; n++) {
char path[1024];
sprintf(path, "breakfast/%s", names[n]);
@@ -62,17 +67,22 @@ defines.FILES = 3
defines.SIZE = '(((BLOCK_SIZE-8)*(BLOCK_COUNT-6)) / FILES)'
defines.GC = [false, true]
defines.COMPACT_THRESH = ['-1', '0', 'BLOCK_SIZE/2']
defines.INFER_BC = [false, true]
code = '''
const char *names[] = {"bacon", "eggs", "pancakes"};
lfs_t lfs;
lfs_format(&lfs, cfg) => 0;
lfs_mount(&lfs, cfg) => 0;
struct lfs_config cfg_ = *cfg;
if (INFER_BC) {
cfg_.block_count = 0;
}
lfs_mount(&lfs, &cfg_) => 0;
lfs_mkdir(&lfs, "breakfast") => 0;
lfs_unmount(&lfs) => 0;
for (int n = 0; n < FILES; n++) {
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
char path[1024];
sprintf(path, "breakfast/%s", names[n]);
lfs_file_t file;
@@ -91,7 +101,7 @@ code = '''
lfs_unmount(&lfs) => 0;
}
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
for (int n = 0; n < FILES; n++) {
char path[1024];
sprintf(path, "breakfast/%s", names[n]);
@@ -113,19 +123,24 @@ code = '''
defines.FILES = 3
defines.SIZE = '(((BLOCK_SIZE-8)*(BLOCK_COUNT-6)) / FILES)'
defines.CYCLES = [1, 10]
defines.INFER_BC = [false, true]
code = '''
const char *names[] = {"bacon", "eggs", "pancakes"};
lfs_file_t files[FILES];
lfs_t lfs;
lfs_format(&lfs, cfg) => 0;
struct lfs_config cfg_ = *cfg;
if (INFER_BC) {
cfg_.block_count = 0;
}
for (int c = 0; c < CYCLES; c++) {
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
lfs_mkdir(&lfs, "breakfast") => 0;
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
for (int n = 0; n < FILES; n++) {
char path[1024];
sprintf(path, "breakfast/%s", names[n]);
@@ -143,7 +158,7 @@ code = '''
}
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
for (int n = 0; n < FILES; n++) {
char path[1024];
sprintf(path, "breakfast/%s", names[n]);
@@ -159,7 +174,7 @@ code = '''
}
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
for (int n = 0; n < FILES; n++) {
char path[1024];
sprintf(path, "breakfast/%s", names[n]);
@@ -175,19 +190,24 @@ code = '''
defines.FILES = 3
defines.SIZE = '(((BLOCK_SIZE-8)*(BLOCK_COUNT-6)) / FILES)'
defines.CYCLES = [1, 10]
defines.INFER_BC = [false, true]
code = '''
const char *names[] = {"bacon", "eggs", "pancakes"};
lfs_t lfs;
lfs_format(&lfs, cfg) => 0;
struct lfs_config cfg_ = *cfg;
if (INFER_BC) {
cfg_.block_count = 0;
}
for (int c = 0; c < CYCLES; c++) {
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
lfs_mkdir(&lfs, "breakfast") => 0;
lfs_unmount(&lfs) => 0;
for (int n = 0; n < FILES; n++) {
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
char path[1024];
sprintf(path, "breakfast/%s", names[n]);
lfs_file_t file;
@@ -232,10 +252,15 @@ code = '''
# exhaustion test
[cases.test_alloc_exhaustion]
defines.INFER_BC = [false, true]
code = '''
lfs_t lfs;
lfs_format(&lfs, cfg) => 0;
lfs_mount(&lfs, cfg) => 0;
struct lfs_config cfg_ = *cfg;
if (INFER_BC) {
cfg_.block_count = 0;
}
lfs_mount(&lfs, &cfg_) => 0;
lfs_file_t file;
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
size_t size = strlen("exhaustion");
@@ -263,7 +288,7 @@ code = '''
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_RDONLY);
size = strlen("exhaustion");
lfs_file_size(&lfs, &file) => size;
@@ -276,10 +301,15 @@ code = '''
# exhaustion wraparound test
[cases.test_alloc_exhaustion_wraparound]
defines.SIZE = '(((BLOCK_SIZE-8)*(BLOCK_COUNT-4)) / 3)'
defines.INFER_BC = [false, true]
code = '''
lfs_t lfs;
lfs_format(&lfs, cfg) => 0;
lfs_mount(&lfs, cfg) => 0;
struct lfs_config cfg_ = *cfg;
if (INFER_BC) {
cfg_.block_count = 0;
}
lfs_mount(&lfs, &cfg_) => 0;
lfs_file_t file;
lfs_file_open(&lfs, &file, "padding", LFS_O_WRONLY | LFS_O_CREAT);
@@ -317,7 +347,7 @@ code = '''
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, cfg) => 0;
lfs_mount(&lfs, &cfg_) => 0;
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_RDONLY);
size = strlen("exhaustion");
lfs_file_size(&lfs, &file) => size;
@@ -330,10 +360,15 @@ code = '''
# dir exhaustion test
[cases.test_alloc_dir_exhaustion]
defines.INFER_BC = [false, true]
code = '''
lfs_t lfs;
lfs_format(&lfs, cfg) => 0;
lfs_mount(&lfs, cfg) => 0;
struct lfs_config cfg_ = *cfg;
if (INFER_BC) {
cfg_.block_count = 0;
}
lfs_mount(&lfs, &cfg_) => 0;
// find out max file size
lfs_mkdir(&lfs, "exhaustiondir") => 0;