Commit Graph

3 Commits

Author SHA1 Message Date
Christopher Haster
1e4d4cfdcf Tried to write errors to stderr consistently in scripts 2023-11-05 15:55:07 -06:00
Christopher Haster
d0a6ef0c89 Changed scripts to not infer field purposes from CSV values
Note there's a bit of subtlety here, field _types_ are still infered,
but the intention of the fields, i.e. if the field contains data vs
row name/other properties, must be unambiguous in the scripts.

There is still a _tiny_ bit of inference. For most scripts only one
of --by or --fields is strictly needed, since this makes the purpose of
the other fields unambiguous.

The reason for this change is so the scripts are a bit more reliable,
but also because this simplifies the data parsing/inference a bit.

Oh, and this also changes field inference to use the csv.DictReader's
fieldnames field instead of only inspecting the returned dicts. This
should also save a bit of O(n) overhead when parsing CSV files.
2023-11-04 15:24:18 -05:00
Christopher Haster
2be3ff57c5 Moved post-bench amor/avg analysis out into amor.py and avg.py
1. Being able to inspect results before benchmarks complete was useful
   to track their status. It also allows some analysis even if a
   benchmark fails.

2. Moving these scripts out of bench.py allows them to be a bit more
   flexible, at the cost of CSV parsing/structuring overhead.

3. Writing benchmark measurements immediately avoids RAM buildup as we
   store intermediate measurements for each bench permutation. This may
   increase the IO bottleneck, but we end up writing the same number of
   lines, so not sure...

I realize avg.py has quite a bit of overlap with summary.py, but I don't
want to entangle them further. summary.py is already trying to do too
much as is...
2023-11-04 13:16:50 -05:00