LFS_WCOMPAT_RDONLY seems generally useful for tools that just want to
mark a filesystem is read-only. This is a common flag that exists in
other filesystems (RO_COMPAT_READONLY in ext4 for example).
LFS_RCOMPAT_WRONLY, on the other hand, is a bit more of a joke, but
there could be some niche use cases for it (preventing double mounts?).
Fortunately, these flags require no extra code, and fall out naturally
from our wcompat/rcompat handling.
---
Originally, the idea was to also add LFS_F_RDONLY, to match LFS_M_RDONLY
and set the LFS_WCOMPAT_RDONLY flag during format.
But this doesn't really work with the current API, since lfsr_format
would just give you an empty filesystem you can't write to. Which is a
bit silly.
Maybe we should add something like lfsr_fs_mkrdonly in the future? This
is probably low-priority.
Mainly to add LFS_RCOMPAT_MSPROUT. It makes sense that a littlefs driver
may not want to support mroot-inlined mdirs, and this flag would be the
only way to indicate that. (Currently inlined mdir -> mtree is one way,
but this may not always be the case.)
This also makes space for a couple planned features:
LFS_RCOMPAT_NONSTANDARD 0x00000001 Non-standard filesystem format
LFS_RCOMPAT_WRONLY* 0x00000002 Reading is disallowed
LFS_RCOMPAT_GRM 0x00000004 May use a global-remove
LFS_RCOMPAT_MSPROUT 0x00000010 May use an inlined mdir
LFS_RCOMPAT_MLEAF 0x00000020 May use a single mdir pointer
LFS_RCOMPAT_MSHRUB 0x00000040 May use an inlined mtree
LFS_RCOMPAT_MTREE 0x00000080 May use an mdir btree
LFS_RCOMPAT_BSPROUT 0x00000100 Files may use inlined data
LFS_RCOMPAT_BLEAF 0x00000200 Files may use single block pointers
LFS_RCOMPAT_BSHRUB 0x00000400 Files may use inlined btrees
LFS_RCOMPAT_BTREE 0x00000800 Files may use btrees
*Planned
I've gone ahead and included rcompat flags we reserve but don't
currently use (LFS_RCOMPAT_MSHRUB). It seems like a good idea to make
these reservations explicit. Though we should still prohibit their use
until there is a good reason, in case we want to repurpose these flags
in the future.
Code changes minimal (larger literal? compiler noise?):
code stack ctx
before: 37788 2608 620
after: 37792 (+0.0%) 2608 (+0.0%) 620 (+0.0%)
This is mainly to free up space for flags, we're pretty close to running
out of 32-bits with future planned features:
1. Reduced file type info from 8 -> 4 bits
We don't really need more than this, but it does mean type info is
no longer a simple byte load.
2. Moved most internal file-state flags into the next 4 bits
These are mostly file-type specific (except LFS_o_ZOMBIE), so we
don't need to worry too much about overlap.
3. Compacted ck-flags into 5 bits:
LFS_M_CKPROGS 0x00000800
LFS_M_CKFETCHES 0x00001000
LFS_M_CKPARITY 0x00002000
LFS_M_CKMETAREDUND* 0x00004000
LFS_M_CKDATACKSUMS 0x00008000
*Planned
Now that ck-flags are a bit more mature, it's pretty clear we'll
probably never have CKMETACKSUMS (ckcksums + small tag reads is
crazy expensive) or CKDATAREDUND (non-trivial parity fanout makes
this crazy expensives. So reserving bits for these just wastes bits.
This also moves things around so ck-flags no longer overlap with open
flags.
It's a tight fit, and I still think file-specific ck-flags are out-of-
scope, but this at least decreases flag ambiguity.
New jenga:
8 8 8 8
.----++----++----++----.
.-..-..-.-------.------.
o_flags: |t||f||t| | o |
|-||-||-|-------:--.---'
|-||-||-'--.----.------.
t_flags: |t||f|| t | | tstt |
'-''-'|----|----'------'
.----.|----|.--.:--:.--.
m_flags: | f || t ||c ||o ||m |
|----||-.--'|--|'--''--'
|----||-|---|--|.------.
f_flags: | f ||t| |c || f |
'----''-'---'--''------'
Fortunately no major code costs:
code stack ctx
before: 37792 2608 620
after: 37788 (-0.0%) 2608 (+0.0%) 620 (+0.0%)
dbgerr.py and dbgtag.py have proven to be incredibly useful for quick
debugging/introspection, so I figured why not have more of that.
My favorite part is being able to quickly see all flags set on an open
file handle:
(gdb) p file.o.o.flags
$2 = 24117517
(gdb) !./scripts/dbgflags.py o 24117517
LFS_O_WRONLY 0x00000001 Open a file as write only
LFS_O_CREAT 0x00000004 Create a file if it does not exist
LFS_O_EXCL 0x00000008 Fail if a file already exists
LFS_O_DESYNC 0x00000100 Do not sync or recieve file updates
LFS_o_REG 0x01000000 Type = regular-file
LFS_o_UNFLUSH 0x00100000 File's data does not match disk
LFS_o_UNSYNC 0x00200000 File's metadata does not match disk
LFS_o_UNCREAT 0x00400000 File does not exist yet
The only concern is if dbgflags.py falls out-of-sync often, I suspect
flag encoding will have quite a bit more churn than flags/tags. But we
can always drop this script in the future if this turns into a problem.
---
While poking around this also ended up with a bunch of other small
changes:
- Added LFS_*_MODE masks for consistency with other "type<->flag
embeddings"
- Added compat flag comments
- Adopted lowercase prefix for internal flags (LFS_o_ZOMBIE), though
not sure if I'll keep this yet...
- Tweaked dbgerr.py to also match ERR_ prefixes and to ignore case
Since we dropped lfsr_gc_setflags/setsteps, it was no longer possible to
set gc_flags to zero (perfectly valid and useful for system bringup/
testing things). Supporting gc_flags=0 means it's not possible to
provide a default, but this is probably ok as users need to opt-in to
LFS_GC anyways.
Note that at least gc_steps=0 doesn't make sense, so the default there
is reasonable.
Fixing this also highlighted that gc_flags/steps are no longer mutable,
making the comment in lfs_init out-of-date. Dropping these saves a bit
of lfs_t size, so that's nice.
And then testing also revealed that LFS_GC_CKDATA implying LFS_GC_CKDATA
means it should probably clear the LFS_I_CKMETA flag as well.
---
And here I thought this was going to be just a simple test-writing
exercise!
Code changes:
code stack ctx
default before: 37792 2608 620
default after: 37792 (-0.0%) 2608 (+0.0%) 620 (+0.0%)
gc before: 37896 2608 768
gc after: 37848 (-0.1%) 2608 (+0.0%) 760 (-1.0%)
This just makes lfsr_fs_stat and lfsr_fs_gc_ that much simpler, at the
risk of the duplicate state falling out-of-sync.
Some minor code savings:
code stack ctx
before: 37804 2608 620
after: 37792 (-0.0%) 2608 (+0.0%) 620 (+0.0%)
The argument for this flag is pretty brittle. Yes it's _technically_
possible to end up with a compactable filesystem during lfsr_format, but
it's pretty unlikely. And keeping LFS_F_COMPACT around means we'd always
need the lfsr_mtree_gc circuitry in lfsr_format, for such a niche
situation, that can be easily cleaned up in lfsr_mount.
So dropping for now.
No code changes, but this does mean one less feature to support:
code stack ctx
before: 37804 2608 620
after: 37804 (+0.0%) 2608 (+0.0%) 620 (+0.0%)
Looking at future planned features, we're running into some real issues
fitting all these flags into 32 bits.
I think the only real use case for LFS_T_MTREEONLY is in
lfsr_traversal_t, where the depth of traversal can't be infered. So no
reason to keep this flag around in the other APIs.
No code changes:
code stack ctx
default before: 37804 2608 620
default after: 37804 (+0.0%) 2608 (+0.0%) 620 (+0.0%)
gc before: 37940 2608 768
gc after: 37940 (+0.0%) 2608 (+0.0%) 768 (+0.0%)
- LFS_I_INCONSISTENT -> LFS_I_MKCONSISTENT
- LFS_I_CANLOOKAHEAD -> LFS_I_LOOKAHEAD
- LFS_I_UNCOMPACTED -> LFS_I_COMPACT
- LFS_I_CANCKMETA -> LFS_I_CKMETA
- LFS_I_CANCKDATA -> LFS_I_CKDATA
This just makes everything easier to read/pattern match, even if it's
a bit inaccurate english-wise. The imperative transformations were also
wildly inconsistent...
Looks like this was never updated after changing the -Y/--summary +
-c/--compare hack to its own -Q/--small-table flag. Fortunately a single
character fix.
Unrelated, but I was considering dropping the make *-diff rules, until
the different compile-time targets proved they are _very_ useful when
jumping around various commits/builds.
- lfsr_gc -> lfsr_fs_gc
- lfsr_gc_unck -> lfsr_fs_unck
lfsr_fs_unck is surprisingly still useful in non-gc builds, since we
still have ckmeta/ckdata state. These flags can still be queried with
lfsr_fs_stat and cleared with lfsr_fs_ckmeta/ckdata/lfsr_traversal_t, so
it seems useful to keep this function around.
It's also a relatively cheap function.
Though this does mean it deserves a rename. Dropping the gc prefix
hopefully makes it clearer this function is not entirely gc-specific.
And since we no longer have lfsr_gc_setflags/setsteps, it makes sense to
rename lfsr_gc back to lfsr_fs_gc, to be consistent with the other
filesystem-wide utilities.
Code changes, apparently lfsr_fs_unck costs 12 bytes:
code stack ctx
default before: 37792 2608 620
default after: 37804 (+0.0%) 2608 (+0.0%) 620 (+0.0%)
gc before: 37938 2608 768
gc after: 37940 (+0.0%) 2608 (+0.0%) 768 (+0.0%)
LFS_GC_CKMETA and LFS_GC_CKDATA are a bit unique in that their work is
never really done.
Where LFS_GC_MKCONSISTENT/COMPACT can prove things about the system,
LFS_GC_CKMETA/CKDATA can't, because it's always possible for new
bit-errors to develop. Even _during_ an LFS_GC_CKMETA/CKDATA traversal.
But while this is technically true, it's not a very useful state of
things for our lfsr_gc API...
---
What we really want is some way to know if ckmeta/ckdata has completed
"recently" (for some definition of recently), and to let users indicate
when they need another ckmeta/ckdata scan.
To try to solve this:
1. Added LFS_I_CANCKMETA and LFS_I_CANCKDATA to indicate when lfsr_gc
has not checked metadata/data.
These are set during mount (unless mounting with
LFS_M_CKMETA/CKDATA), and cleared when either lfsr_gc completes or
lfsr_fs_ckmeta/data is called. Once cleared, littlefs will not reset
them on its own.
2. Added lfsr_gc_unck to allow users to explicitly reset LFS_I_CKMETA
and/or LFS_I_CKDATA, which will tell lfsr_gc to check metadata/data
again on the next call.
There is some subtlety around clobbering ongoing traversals, but a
mask and some tests should prevent this from being a problem.
Currently, lfsr_gc_unck also allows clearing of other gc flags, but
I'm not sure there's any real use-case for this...
Note that you can still get the previous behavior if you just call
lfsr_gc_unck after every lfsr_gc call.
This also changes info flag behavior slightly in default mode, with
LFS_I_CANCKMETA/CANCKDATA telling you if metadata/data has been checked
since mount. Which does seem useful? Maybe these flags deserve a better
name?
Code changes:
code stack ctx
default before: 37796 (+0.0%) 2608 (+0.0%) 620 (+0.0%)
default after: 37792 (+0.0%) 2608 (+0.0%) 620 (+0.0%)
gc before: 37896 2608 768
gc after: 37938 (+0.1%) 2608 (+0.0%) 768 (+0.0.%)
Now that you can provide gc_flags/gc_steps in lfs_config, I think it's a
bit more clear that _mutating_ the flags/steps is a niche feature, and
not worth implementing/testing.
It raises the question why not have a similar lfsr_setflags or
lfsr_file_setflags, and the answer there is it would be a pain-in-the-
ass to make sure all possible corner cases are covered.
It actually already was a pain-in-the-ass to test lfsr_gcsetflags/
setsteps... but just because we already did the work is not a good
reason for keeping complexity around.
---
Note that most of the use cases for lfsr_gc_setflags/setsteps can be
covered by either remounting the filesystem or through the
lfsr_traversal_t APIs directly.
The end result is a bit of code savings when incremental gc is enabled:
code stack ctx
default before: 37796 2608 620
default after: 37796 (+0.0%) 2608 (+0.0%) 620 (+0.0%)
gc before: 37944 2608 768
gc after 37896 (-0.1%) 2608 (+0.0%) 768 (+0.0%)
Incremental gc, being stateful and not gc-able (ironic), was always
going to need to be conditionally compilable.
This moves incremental gc behind the LFS_GC define, so that we can focus
on the "default" costs. This cuts lfs_t in nearly half!
lfs_t with LFS_GC: 308
lfs_t without LFS_C: 168 (-45.5%)
This does save less code than one might expect though. We still need
most of the internal traversal/gc logic for things like block allocation
and orphan cleanup, so most of the savings is limited to the RAM storing
the incremental state:
code stack ctx
before: 37916 2608 768
after with LFS_CFG: 37944 (+0.1%) 2608 (+0.0%) 768 (+0.0%)
after without LFS_CFG: 37796 (-0.3%) 2608 (+0.0%) 620 (-19.3%)
On the flip side, this does mean most of the incremental gc
functionality is still availables in the lfsr_traversal_t APIs.
Applications with more advanced gc use-cases may actually benefit from
_not_ enabling the incremental gc APIs, and instead use the
lfsr_traversal_t APIs directly.
Before:
int lfsr_fs_gc(lfs_t *lfs, lfs_soff_t steps, uint32_t flags);
After:
int lfsr_gc(lfs_t *lfs);
int lfsr_gc_setflags(lfs_t *lfs, uint32_t flags);
int lfsr_gc_setsteps(lfs_t *lfs, lfs_soff_t steps);
---
The interesting thing about the lfsr_gc API is that the caller will
often be very different from whoever configures the system. One example
being an OS calling lfsr_gc in a background loop, while leaving
configuration up to the user.
The idea here, is instead of forcing the OS to come up with its own
stateful system to pass flags to lfsr_gc, we just embed this state in
littlefs directly. The whole point of lfsr_gc is that it's a stateful
system anyways.
Unfortunately this state does require a bit more logic to maintain,
which adds code/ctx cost:
code stack ctx
before: 37812 2608 752
after: 37916 (+0.3%) 2608 (+0.0%) 768 (+2.1%)
Found a bug in our toml parser that's difficult to work around:
defines.GC_FLAGS = """ => {
LFS_GC_MKCONSISTENT "GC_FLAGS": "blablabla",
| LFS_GC_LOOKAHEAD } // where did defines go?
"""
This appears to be this bug:
https://github.com/uiri/toml/issues/286
But since it was opened 4 years ago, I think it's safe to say this toml
library is now defunct...
---
Apparently tomllib/tomli is the new hotness, which started as tomli
before being adopt in Python 3.11 as tomllib. Fortunately tomli is still
maintained so we don't have to worry about Python versions too much.
Adopting tomli was relatively straightforward, the only hiccup being
that it doesn't support text files? Curious, but fortunately Python
exposes the underlying binary file handle in f.buffer.
I think lfsr_traversal_rewind_ originally reset everything manually
because it was cheaper than creating a compound-literal, but this is no
longer relevant now that lfsr_traversal_init exists.
This saves a bit of code:
code stack ctx
before: 37844 2608 752
after: 37812 (-0.1%) 2608 (+0.0%) 752 (+0.0%)
While they are a bit more annoying to call, init functions give the
compiler a chance to deduplicate common struct initialization logic. So
we should probably prefer init functions for any structs larger than a
couple words.
The cost of each init is small, but it really adds up!
code stack ctx
before: 38036 2608 752
after: 37844 (-0.5%) 2608 (+0.0%) 752 (+0.0%)
Unfortunately this is undefined behavior.
As far as I can tell, there's no well-defined way in C to express that
we don't need a full struct allocation.
Curiously this ended up saving code? I guess because of better compiler
assumptions when using the correct types. This hack was supposed to save
stack, but it's possible the single saved word was lost due to alignment/
measurement noise:
code stack ctx
before: 38060 2608 752
after: 38036 (-0.1%) 2608 (+0.0%) 752 (+0.0%)
These should be the last implicit buffers in LFSR_DATA_* macros, leaving
only LFSR_RAT_* macros with implicit stack-allocations (which are wayyy
too useful to give up).
There's an argument to keep these macros implicit, since they represent
relatively small things, but a stack allocation is a stack allocation.
It's safer to make stack allocations explicit, though it does risk
buffer overflow if these fall out-of-sync...
I guess we're forced to choose our poison...
In the end consistency with other LFSR_DATA_* macros wins.
---
And, again, compound-literals are so poorly optimized this minor cleanup
somehow saves code:
code stack ctx
before: 38060 2608 752
after: 38000 (-0.2%) 2608 (+0.0%) 752 (+0.0%)
See previous commit for more details on why this doesn't work:
1. Losing the simple/compiler friendly lfsr_data_t costs more code/stack
than we save inlined small pieces of data (dids, leb128s, flags,
etc).
2. Inlined lfsr_data_t is fundamentally incompatible with the new
lightweight lfsr_rat_t representation for simple data.
Though I did add a comment, and marked lfsr_data_fromslice as inline.
The compiler was apparently already inlining lfsr_data_fromslice (and
it's a valuable optimization!), but making this explicit helps document/
influence future changes.
Code changes:
code stack ctx
before: 38128 2672 752
after: 38060 (-0.2%) 2608 (-2.4%) 752 (+0.0%)
The idea, which has floated up a few times, is to add a third
representation of lfsr_data_t where the data is inlined in the struct
directly. In theory saving RAM for small pieces of data such as dids,
leb128s, flags, etc:
inlined: in-RAM buffer: on-disk:
.---+---+---+---. .---+---+---+---. .---+---+---+---.
|01| size | |00| size | |1| size |
+---+---+---+---+ +---+---+---+---+ +---+---+---+---+
| inlined data | | ptr -------. | block |
+ + +---+---+---+---+ | +---+---+---+---+
| | | (unused) | | | off |
'---+---+---+---' '---+---+---+---' | '---+---+---+---'
.---+---+---+---. |
| data |<'
: : :
Unfortunately in practice this just doesn't work out.
It turns out we benefit a lot from the _simplicity_ of lfsr_data_t. When
lfsr_data_t is built out of simple words, the compiler can make some
pretty strong assumptions and basically break it down into simple
register operations.
When you stick a byte array in the middle of the struct, this sort of
breaks down.
---
We can see this in our code measurements. After adding inlined data, but
before implementing slicing (in lfsr_data_fromslice), we can see decent
stack savings. But as soon as we add the memmove to lfsr_data_fromslice,
any benefit is lost:
code stack ctx
before: 38060 2608 752
without slicing: 38056 (-0.0%) 2568 (-1.5%) 752 (+0.0%)
after: 38128 (+0.2%) 2672 (+2.5%) 752 (+0.0%)
One reason for this is the extra logic does cause lfsr_data_fromslice to
be no longer inlined, but adding __attribute__((always_inline)) only
claws back some of the code/stack savings (though it's interesting to
note the compiler heuristic failure here):
code stack ctx
before: 38060 2608 752
after+inline: 38128 (+0.2%) 2672 (+2.5%) 752 (+0.0%)
after+always_inline: 38684 (+1.6%) 2656 (+1.8%) 752 (+0.0%)
---
Oh, and inlined lfsr_data_t is no longer compatible with LFSR_RAT's
simple data conversion, since lfsr_rat_t's can only point to existing
buffers. This causes tests to fail rather quickly.
This should be reverted, but I think the hidden cost of inlined
lfsr_data_t is surprising and interesting to note.
This was a disappointing failure of compount-literals.
These macros protect against mismatched buffer sizes, which is great for
preventing bugs caused by simple typos, but the overhead of compound-
literals requiring initialization make them simply unusable.
This commit leaves only a couple macros with implicit buffers:
LFSR_DATA_LEB128, and the LFSR_RAT_CAT/LFSR_RATS macros.
Even the tiny cleanup of the one remaining implicit-buffer macro still
in use, LFSR_DATA_GEOMETRY, saved some code:
code stack ctx
before: 38084 2608 752
after: 38060 (-0.1%) 2608 (+0.0%) 752 (+0.0%)
Well, renamed lfsr_rcompat_* really. But this avoids making rcompat
special and treats all rcompat/wcompat/ocompat logic as specializations
of the shared lfsr_compat_* logic.
No code changes:
code stack ctx
before: 38084 2608 752
after: 38084 (+0.0%) 2608 (+0.0%) 752 (+0.0%)
- Fixed issue where some overflowed compat flags could end up ignored.
A simple typo: incrementing by the unrelated d variable, meant we
were skipping overflowed compat flags whenever the previous logic sets
d > 1.
- Fixed issue where any zero padding was treated as overflowed compat
flags.
Note this hid the previous issue from our tests.
Added more tests to prevent a regression here. Letting bad compat flag
parsing through would be _very_ annoying in the future.
Code changes:
code stack ctx
before: 38148 2608 752
after: 38084 (-0.2%) 2608 (+0.0%) 752 (+0.0%)
This is to be consistent with other LFSR_DATA_* constructors. The code
is also a bit more readable when all LFSR_DATA_* constructors are
capitalized.
Not really sure why, but this saved a bit of stack? Probably just
compiler noise:
code stack ctx
before: 38144 2616 752
after: 38148 (+0.0%) 2608 (-0.3%) 752 (+0.0%)
I also explored adding in-place slice/truncate/fruncate functions as
well, but the impact on stack usage was REALLY BAD:
code stack ctx
by-value: 38148 2608 752
in-place: 38144 (-0.0%) 2688 (+3.1%) 752 (+0.0%)
I think maybe because the in-place functions end up with too many
pointers for the compiler to make safe assumptions about compound-
literal lifetimes?
Or what's left of lfsr_cat_t anyways.
lfsr_cat_t ended up being a pretty lfsr_rat_t specific optimization, so
it makes sense to drop the special type and simplify the code base a
little bit.
Curiously this trades some code for stack, I guess because inlining
static functions is a bit of a difficult heuristic mess:
code stack ctx
before: 38128 2624 752
after: 38144 (+0.0%) 2616 (-0.3%) 752 (+0.0%)
This reduces lfsr_ck_t to just the cksize/cksum fields, and moves all of
the compile-time ifdef LFS_CKDATACKSUMS logic up into the relevant
lfsr_data_* functions.
This doesn't solve the lfsr_data_t/lfsr_bptr_t duplication problem,
unfortunately, but does simplify the code base a bit.
No significant code changes:
code stack ctx
default before: 38128 2624 752
default after: 38128 (+0.0%) 2624 (+0.0%) 752 (+0.0%)
ckdatacksums before: 39240 3008 752
ckdatacksums after: 39232 (-0.0%) 3008 (+0.0%) 752 (+0.0%)
To clarify this only checks data reads, and to makes space for future
theoretical ck-operations:
- ckmetaredund - likely
- ckdataredund - unlikely, expensive
- ckmetacksums - unlikely, expensive
- ckdatacksums - implemented
This also tweaks the relevant mount/format/info flags a bit:
LFS_M_CKPROGS 0x00100000 Check progs by reading back progged data
LFS_M_CKFETCHES 0x00200000 Check block checksums before first use
LFS_M_CKPARITY 0x00400000 Check metadata tag parity bits
LFS_M_CKMETAREDUND+ 0x01000000 Check metadata redund blocks on reads
LFS_M_CKDATAREDUND* 0x02000000 Check data redund blocks on reads
LFS_M_CKMETACKSUMS* 0x04000000 Check metadata checksums on reads
LFS_M_CKDATACKSUMS 0x08000000 Check data checksums on reads
+Planned
*Hypothetical
No code changes.
Unfortunately ckparity has proven itself to be much less useful than
originally thought.
The use of leb128 encoding in our tags means that ckparity can't even
detect single bit-errors reliably. Which raises the question: is
ckparity really worth all of the extra baggage necessary to track parity
in our codebase?
Fortunately we don't have to toss out ckparity entirely!
If we only check parity bits in lfsr_bd_readtag_, instead of on every
read, we still have a reasonable chance of noticing parity errors during
metadata lookups.
This does weaken ckparity, but allows us to drop a lot of lfsr_data_t's
ckparity baggage, at the cost of no longer, uh, unreliably detecting
parity errors during reads?
The limited error detection of ckparity means we can't reliably detect
errors during reads anyways, so we might as well keep the code/RAM/
maintenance implications at a minimum to make ckparity remotely worth
it.
---
Note the significant savings for both LFS_CKPARITY and LFS_CKCKSUMS.
Tracking parity info in lfsr_data_t had a heavy cost:
code stack ctx
default before: 38128 2624 752
default after: 38128 (+0.0%) 2624 (+0.0%) 752 (+0.0%)
ckparity before: 39700 3048 760
ckparity after: 38476 (-3.1%) 2696 (-11.5%) 760 (+0.0%)
ckcksums before: 39396 3096 760
ckcksums after: 39240 (-0.4%) 3008 (-2.8%) 752 (-1.1%)
This also means lfsr_ck_ckprefix/cksuffix calls always have a ckoff of 0
(bptrs only), which means even more code savings, yay!
So... Long store short, checking metadata cksums is just intractably
slow.
But data cksums?
Yes checking data cksums is still O(b^2), but unlike metadata lookups,
which involve many small backwards reads, data reads are very easy to
cache. So instead of O(b^2), it's more like O(b^2/c), where c is your
rcache size.
Still O(b^2) when c << b, but I'm not sure that's avoidable without
adding more cksums.
At the very least, if you have enough RAM, c == b reduces this to O(b),
which is nice for "large" systems that want hardened reads without a
performance loss.
---
But why bother checking data cksums if we still have a read-hole with
metadata cksums?
Well, while considering the problem in the context of future features, I
noticed something _really interesting_:
- ckredund + metadata - reasonable ✓
- ckredund + data - impractical ✗, parity fanout + O(f+r) is bad
- ckcksums + metadata - impractical ✗, small reads + O(b^2) is bad
- ckcksums + data - reasonable ✓, assuming enough rcache
The current planned design for data redundancy makes it also intractably
slow to check every read, since it would require xoring all blocks that
contribute to the relevant parity block, but this isn't a problem for
metadata redundancy.
So while neither ckredund nor ckcksums can tractably close the read-hole
on their own, it looks like together they will be able to cover
everything without completely sacrificing performance. Neat!
Of course this isn't possible if ckcksums/ckredund imply checking both
metadata and data, so they need to be split apart.
And I don't really see a point in keeping the intractable variants
around in the codebase.
---
Dropping metadata ckcksums also means we can get rid of the ugly
lfsr_bd_ckrbydprefix and lfsr_bd_ckrbydsuffix functions, which were
basically duplicating all of lfsr_rbyd_fetch. That was quite a wart!
This saves a nice chunk of code when ckcksums is enabled:
code stack ctx
default before: 38128 2624 752
default after: 38128 (+0.0%) 2624 (+0.0%) 752 (+0.0%)
ckparity before: 39724 3048 764
ckparity after: 39700 (-0.1%) 3048 (+0.0%) 760 (-0.5%)
ckcksums before: 40612 3184 772
ckcksums after: 39396 (-3.0%) 3096 (-2.8%) 760 (-1.6%)
In theory this gives callers more flexibility around what file states
they actually care about.
This didn't actually end up saving any code (I guess due to const
propagation?), but is kind of neat:
code stack ctx
before: 38128 2624 752
after: 38128 (+0.0%) 2624 (+0.0%) 752 (+0.0%)
One of the unexpected side-effects of lazy file creation is that
suddenly LFS_O_EXCL doesn't make sense.
The standard definition: "Fail if the file exists", is easy enough to
implement, but doesn't really match what the user expects.
The user expects one of these calls to fail:
lfsr_file_open(&lfs, &file_a, "file.txt",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfsr_file_open(&lfs, &file_b, "file.txt",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
But because we create files lazily (to prevent zero-length files after
powerloss), these both succeed.
---
I considered deferring the "file exists" check until we actually would
create the file, but while this _technically_ satisfies the
exclusitivity requirement, I decided against it as I think it just makes
the API way too confusing:
lfsr_file_open(&lfs, &file_a, "file.txt",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfsr_file_open(&lfs, &file_b, "file.txt",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfsr_file_close(&lfs, &file_a) => 0;
lfsr_file_close(&lfs, &file_b) => LFS_ERR_EXIST;
---
Instead, a simpler, more pragmatic approach: Fail if the file exists
_or_ if the file is open in a mode that will create the file:
lfsr_file_open(&lfs, &file_a, "file.txt",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfsr_file_open(&lfs, &file_b, "file.txt",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => LFS_ERR_EXIST;
This explicitly does _not_ error on zombie/desync files:
lfsr_file_open(&lfs, &file_a, "file.txt",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfsr_file_desync(&lfs, &file_a) => 0;
lfsr_file_open(&lfs, &file_b, "file.txt",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
And it does mean we aren't necessarily guaranteeing the file will be
created, but I think this does more-or-less what the user expects:
- open(a) -> desync(a) -> open(b) -> resync(a) is roughly equivalent to
opening a after creating b, which is perfectly fine with LFS_O_EXCL.
- open(a) -> open(b) (errors) -> desync(a) is one way to not actually
create the file, but is somewhat similar to removing the file after
creation.
If you're using desync files you should probably have a good
understanding of littlefs's sync model anyways.
And of course the user can always sync immediately after open to
guarantee file creation, while opting into the possibility of
zero-length files after powerloss.
Code changes:
code stack ctx
before: 38084 2624 752
after: 38128 (+0.1%) 2624 (+0.0%) 752 (+0.0%)
Making lfsr_tag_isunknown its own function suggests this is a cheap
operation, but it's not really since we need to check against all known
types.
Dropping lfsr_tag_isunknown and changing the relevant logic to fallback
to LFS_ERR_NOTSUP saves a surprising amount of code (ok, not that much,
but still surprising):
code stack ctx
before: 38140 2624 752
after: 38084 (-0.1%) 2624 (+0.0%) 752 (+0.0%)
This is the tradeoff of not erroring on unknown filetypes during mount.
- lfsr_file_open and lfsr_mtree_pathlookup now returns LFS_ERR_NOTSUP
instead of LFS_ERR_NOTDIR/LFS_ERR_ISDIR if it encounters an unkown
filetype.
This gets a bit subtle. You might think LFS_ERR_NOTDIR is reasonable,
but it's possible for our unknown filetype to be something dir-like.
Symlinks are an excellent example.
- lfsr_remove/lfsr_rename now bail with LFS_ERR_NOTSUP if encountering
an unknown filetype.
This conflicts with the POSIX philosophy of remove always being
allowed, but I'm not sure what other option there is. Maybe allowing
removes when mounted with LFS_M_FORCE?
We can't just allow removes by default because of the risk of leaking
resources. Directories being the main example of this (need to clean
up bookmarks).
Maybe leaky filetypes should also set WCOMPAT flags?
Not doing something is cheaper than doing something, so unfortunately
this costs us more than what we saved from dropping the orphan/unknown
scan during mount:
code stack ctx
bail: 38120 2624 725
no-error-no-bail (before): 38020 (-0.3%) 2624 (+0.0%) 752 (+0.0%)
error-no-bail (after): 38140 (+0.1%) 2624 (+0.0%) 752 (+0.0%)
But this is probably worth it for the extra flexibility.
The motivation here is to simplify lfsr_mount, but there's a number of
knock-on effects.
For one, lfsr_mount should now be faster on filesystems with large
blocks:
O(nb(log b)(log_b n)) -> O(nb(log_b n))
But we now no longer check if our filesystem contains orphaned
stickynotes or unknown filetypes:
- Orphaned stickynotes turned out to not be a big deal. If we find
orphans we'd need to do a second traversal to remove them anyways (no
mutation allowed in lfsr_mount), so this actually ends up a net
improvement in the found-orphan case.
If anything, doing a traversal on first write sets user expectations
correctly, and can be offloaded with lfsr_fs_mkconsistent or
lfsr_fs_gc.
- Unknown filetypes are a bit more annoying (I actually forgot about
this check), but unknown filetypes that require special care should
probably set WCOMPAT/RCOMPAT flags.
Allowing unknown filetypes is a bit more flexible in cases where a
filesystem image is being shared between drivers with different
features (bootloader + app for example).
Though we should probably add more checks/tests that we're handling
these correctly now that we no longer just bail during mount...
Also renamed LFS_I_HASORPHANS -> LFS_I_UNTIDY.
Not doing something is cheaper than doing something, so this saves a bit
of code:
code stack ctx
before: 38120 2624 752
after: 38020 (-0.3%) 2624 (+0.0%) 752 (+0.0%)
- test_attrs_fattr_zombie_no_receive
- test_attrs_fattr_mvrm_fuzz_fuzz
And renamed a number of broadcast tests to try to make it clear exactly
what we're testing:
- test_attrs_fattr_wronly_broadcast -> *_wronly_no_receive
- test_attrs_fattr_rdonly_broadcast -> *_rdonly_no_broadcast
- test_attrs_fattr_desync_broadcast -> *_desync_no_receive
- test_attrs_fattr_resync_broadcast -> *_resync_receive
- test_attrs_fattr_zombie_broadcast -> *_zombie_no_broadcast
As a part of the effort to undo the overuse of the term "orphan".
I can't really think of a better name, and uncreat gets the point
across. At least it matches LFS_O_UNSYNC/LFS_O_UNFLUSH.
Apparently the Uncreated are a race of aliens in the Marvel universe?
Unfortunately the import sys in the argparse block was hiding missing
sys imports.
The mistake was assuming the import sys in Python would limit the scope
to that if block, but Python's late binding strikes again...
Apparently __builtins__ is a CPython implementation detail, and behaves
differently when executed vs imported???
import builtins is the correct way to go about this.
I've been unhappy with LFSR_TAG_ORPHAN for a while now. While it's true
these represent orphaned files, they also represent zombied files. And
as long as a reference to the file exists in-RAM, I find it hard to say
these files are truely "orphaned".
We're also just using the term "orphan" for too many things.
Really this tag just represents an mid reservation. The term stickynote
works well enough for this, and fits in with the other internal tag,
LFSR_TAG_BOOKMARK.
We already have lfsr_cat_t so...
lfsr_rattr_t is a pretty fundamental type for littlefs, unfortunately
the name "rattr" is a mouthful. Shortening this to just "rat" hopefully
makes things easier to read at the cost of it being a bit less clear
what lfsr_rat_t actually is.
Though it's possible I've been staring at the dwarf spec (DW_AT_*) for
too long...
Moved local import hack behind if __name__ == "__main__"
These scripts aren't really intended to be used as python libraries.
Still, it's useful to import them for debugging and to get access to
their juicy internals.
Previously, we were using the checksum of the full path to generate
dids. This mostly works, but means that different paths can end up with
different dids for the same directory:
- crc32c("a/b") => 0xfb0b40a3
- crc32c("/a/c/../b") => 0x223404f6
Not the end of the world, but this is the stuff heisenbugs are made of.
Now we instead checksum only the file name, which doesn't have this
problem, and xor with our parent's did to prevent collisions between
same-named files in different directories:
did = parent_did xor crc32c(name)
As an extra plus, this should also trivially work for the theoretical
lfsr_mkdirat function.
Code size unchanged, which is humorous but not surprising:
code stack ctx
before: 38120 2624 752
after: 38120 (+0.0%) 2624 (+0.0%) 752 (+0.0%)
