According to the zlib FAQ (*1), zlib does not support compressed data
larger than 4 GiB when the compiler's long type is 32 bits. Therefore,
we need to report an error if a zlib-compressed debug section exceeds
4 GiB on LLP64 machines.
(*1) https://zlib.net/zlib_faq.html#faq32
Signed-off-by: Rui Ueyama <rui314@gmail.com>
z_stream's avail_in and avail_out are defined as "unsigned int", so it
cannot decode an entire compressed stream in one pass if the stream is
larger than 4 GiB. The simplest solution to this problem is to use zlib's
convenient uncompress2() function, which handles the details for us.
Signed-off-by: Rui Ueyama <rui314@gmail.com>
This modifies _bfd_elf_free_cached_info to unmap/free section
contents. To do that we need to *not* free sections where contents
are bfd_alloc'd or point to constant strings or somesuch. I've chosen
to implement this be adding another flag to struct bfd_section,
"alloced" to say the section contents can't be freed. Most of the
patch is about setting that flag in many places.
If a BFD user is making use of a function like
bfd_get_section_contents to read a section into a pre-allocated
buffer, then that BFD user might also want to make use of
_bfd_section_size_insane prior to allocating the buffer they intend to
use in order to validate that the buffer size that plan to allocate is
sane.
This commit makes _bfd_section_size_insane public, by renaming it to
bfd_section_size_insane.
I've updated the existing uses within bfd/, I don't believe this
function is used outside of bfd/ currently.
One place that I plan to make use of this function is in
gdb/gdb_bfd.c, in the function gdb_bfd_get_full_section_contents.
This change isn't included in this commit, but will come later if/when
this has been merged into bfd.
There should be no change in behaviour after this commit.
bfd/
* bfd-in2.h (bfd_section_size_insane): Add declaration.
* compress.c (bfd_get_full_section_contents): Update for new name
of _bfd_section_size_insane.
(bfd_init_section_compress_status): Likewise.
* dwarf2.c (read_section): Likewise.
(_bfd_dwarf2_slurp_debug_info): Likewise.
* libbfd.h (_bfd_section_size_insane): Remove declaration.
* section.c (_bfd_section_size_insane): Rename to ...
(bfd_section_size_insane): ... this.
binutils/
* readelf.c (uncompress_section_contents): Update comment to
account for new name of _bfd_section_size_insane.
Adds two new external authors to etc/update-copyright.py to cover
bfd/ax_tls.m4, and adds gprofng to dirs handled automatically, then
updates copyright messages as follows:
1) Update cgen/utils.scm emitted copyrights.
2) Run "etc/update-copyright.py --this-year" with an extra external
author I haven't committed, 'Kalray SA.', to cover gas testsuite
files (which should have their copyright message removed).
3) Build with --enable-maintainer-mode --enable-cgen-maint=yes.
4) Check out */po/*.pot which we don't update frequently.
With specially crafted compressed sections, it's possible to tickle a
problem when decompressing: If the compression headers says the
uncompressed size is zero, this will be seen as an error return from
bfd_compress_section_contents. On errors the caller should free any
malloc'd input buffers, but this isn't really an error and the section
contents have been updated to a bfd_alloc'd buffer which can't be
freed.
* compress.c (bfd_compress_section_contents): Return -1 as error
rather than 0.
(bfd_init_section_compress_status, bfd_compress_section): Adjust.
bfd/doc/chew.c extracts documentation from source code comments
annotated with keywords, and generates much of bfd.h and libbfd.h from
those same comments. The docs have suffered from people (me too)
adding things like CODE_FRAGMENT to the source to put code into bfd.h
without realising that CODE_FRAGMENT also puts @example around said
code into the docs. So we have random senseless things in the docs.
This patch fixes that problem (well, the senseless things from
CODE_FRAGMENT), moves most of the code out of bfd-in.h, and improves a
few chew.c features. libbfd.h now automatically gets ATTRIBUTE_HIDDEN
prototypes, and indentation in bfd.h and libbfd.h is better.
This function doesn't just initialise for compression, it actually
compresses. This patch sanity checks section size before allocating
buffers for the uncompressed contents.
* compress.c (bfd_init_section_compress_status): Sanity check
section size.
When reading the BFD manual, I noticed text like this:
-- Function: bool bfd_close (bfd *abfd);
Close a BFD. If the BFD was open for writing, then pending
operations are completed and the file written out and closed. If
...
*Returns*
'TRUE' is returned if all is ok, otherwise 'FALSE'.
The *Returns*, like the *Synopsis* in the earlier patch, is
un-info-like. It's also used inconsistently.
This patch removes all the uses of the RETURNS word and removes it
entirely from the chew scripts. Now this example reads:
-- Function: bool bfd_close (bfd *abfd);
Close a BFD. If the BFD was open for writing, then pending
operations are completed and the file written out and closed. If
...
'TRUE' is returned if all is ok, otherwise 'FALSE'.
In a few cases I had to slightly reword the comment. There were also
a couple of cases where there was redundant text. In these cases I
just dropped the RETURNS copy.
2023-02-07 Tom Tromey <tom@tromey.com>
* bfd.c, cache.c, compress.c, opncls.c: Remove RETURNS from
documentation comments.
* doc/doc.str, doc/proto.str (RETURNS): Remove.
The newer update-copyright.py fixes file encoding too, removing cr/lf
on binutils/bfdtest2.c and ld/testsuite/ld-cygwin/exe-export.exp, and
embedded cr in binutils/testsuite/binutils-all/ar.exp string match.
This patch removes the bfd_malloc in default_indirect_link_order and
bfd_simple_get_relocated_section_contents, pushing the allocation down
to bfd_get_relocated_section_contents. The idea is to make use of the
allocation done with sanity checking in bfd_get_full_section_contents,
which is called by bfd_generic_get_relocated_section_contents.
Doing this exposed a bug in bfd_get_full_section_contents. With
relaxation it is possible that an input section rawsize is different
to the section size. In that case we want to use the larger of
rawsize (the on-disk size for input sections) and size.
* reloc.c (bfd_generic_get_relocated_section_contents),
* reloc16.c (bfd_coff_reloc16_get_relocated_section_contents),
* coff-alpha.c (alpha_ecoff_get_relocated_section_contents),
* coff-sh.c (sh_coff_get_relocated_section_contents),
* elf-m10200.c (mn10200_elf_get_relocated_section_contents),
* elf-m10300.c (mn10300_elf_get_relocated_section_contents),
* elf32-avr.c (elf32_avr_get_relocated_section_contents),
* elf32-cr16.c (elf32_cr16_get_relocated_section_contents),
* elf32-crx.c (elf32_crx_get_relocated_section_contents),
* elf32-h8300.c (elf32_h8_get_relocated_section_contents),
* elf32-nds32.c (nds32_elf_get_relocated_section_contents),
* elf32-sh.c (sh_elf_get_relocated_section_contents),
* elfxx-mips.c (_bfd_elf_mips_get_relocated_section_contents):
Handle NULL data buffer.
* bfd.c (bfd_get_section_alloc_size): New function.
* bfd-in2.h: Regenerate.
* compress.c (bfd_get_full_section_contents): Correct section
malloc size.
* linker.c (default_indirect_link_order): Don't malloc memory
here before calling bfd_get_relocated_section_contents.
* simple.c (bfd_simple_get_relocated_section_contents): Likewise.
Tidies:
- Move stuff from bfd-in.h and libbfd.c to compress.c
- Delete COMPRESS_DEBUG from enum compressed_debug_section_type
- Move compress_debug field out of link_info to ld_config.
Fixes:
- Correct test in bfd_convert_section_setup to use obfd flags,
not ibfd.
- Apply bfd_applicable_file_flags to compression bfd flags added
by gas and ld to the output bfd.
bfd/
* bfd-in.h (enum compressed_debug_section_type),
(struct compressed_type_tuple),
(bfd_get_compression_algorithm),
(bfd_get_compression_algorithm_name),
* libbfd.c (compressed_debug_section_names),
(bfd_get_compression_algorithm),
(bfd_get_compression_algorithm_name): Move..
* compress.c: ..to here, deleting COMPRESS_DEBUG from
enum compressed_debug_section_type.
(bfd_convert_section_setup): Test obfd flags not ibfd for
compression flags.
* elf.c (elf_fake_sections): Replace link_info->compress_debug
test with abfd->flags test.
* bfd-in2.h: Regenerate.
binutils/
* objcopy.c (copy_file): Tidy setting of bfd compress flags.
Expand comment.
gas/
* write.c (compress_debug): Test bfd compress flags rather than
flag_compress_debug.
(write_object_file): Apply bfd_applicable_file_flags to compress
debug flags added to output bfd.
include/
* bfdlink.h (struct bfd_link_info): Delete compress_debug.
ld/
* ld.h (ld_config_type): Add compress_debug.
* emultempl/elf.em: Replace references to link_info.compress_debug
with config.compress_debug.
* lexsup.c (elf_static_list_options): Likewise.
* ldmain.c (main): Likewise. Apply bfd_applicable_file_flags
to compress debug flags added to output bfd.
When bfd_size_type is unsigned 64-bit integer and sizeof is unsigned
32-bit integer, subtraction in
*new_size += sizeof (Elf32_External_Chdr) - sizeof (Elf64_External_Chdr);
will overflow. Use
*new_size -= sizeof (Elf64_External_Chdr) - sizeof (Elf32_External_Chdr);
to avoid overflow.
PR binutils/29860
* compress.c (bfd_convert_section_setup): Avoid signed overflow
for new_size adjustment.
SEC_ELF_RENAME is a flag used to effect section name changes when
compressing/decompressing zlib-gnu debug sections. This can be
accomplished more directly in one of the objcopy specific bfd
functions. Renaming for ld input is simplified too. Ld input object
files always have BFD_DECOMPRESS set.
bfd/
* compress.c (bfd_convert_section_size): Rename to..
(bfd_convert_section_setup): ..this. Handle objcopy renaming
of compressed/decompressed debug sections.
* elf.c (_bfd_elf_make_section_from_shdr): Only rename zdebug
input for linker.
(elf_fake_sections): Don't handle renaming of debug sections for
objcopy here.
* section.c (SEC_ELF_RENAME): Delete.
* bfd-in2.h: Regenerate.
binutils/
* objcopy.c (setup_section): Call bfd_convert_section_setup.
Don't call bfd_convert_section_size.
Define an enum instead of using ELFCOMPRESS_ZLIB and ELFCOMPRESS_ZSTD
in bfd and binutils, and move some functions from bfd.c to compress.c.
When looking at the COFF/PE debug compression support, I wondered
about extending it to support zstd. I likely won't do that, but
the compression header ch_type field isn't just ELF specific if these
headers are to be used in COFF/PE too.
bfd/
* bfd.c (bfd_update_compression_header),
(bfd_check_compression_header, bfd_get_compression_header_size),
(bfd_convert_section_size, bfd_convert_section_contents): Move to..
* compress.c: ..here.
(enum compression_type): New. Use it throughout file.
* elf.c (_bfd_elf_make_section_from_shdr): Replace uses of
ELFCOMPRESS_ZLIB and ELFCOMPRESS_ZSTD with ch_compress_zlib and
ch_compress_zstd.
* bfd-in2.h: Regenerate.
binutils/
* readelf.c (process_section_headers, dump_section_as_strings),
(dump_section_as_bytes, load_specific_debug_section): Replace
uses of ELFCOMPRESS_ZLIB and ELFCOMPRESS_ZSTD with
ch_compress_zlib and ch_compress_zstd.
Move a couple of elf.c functions to compress.c.
* compress.c (bfd_debug_name_to_zdebug): New inline function.
(bfd_zdebug_name_to_debug): Likewise.
* elf.c (convert_debug_to_zdebug, convert_zdebug_to_debug): Delete.
(_bfd_elf_make_section_from_shdr, elf_fake_sections),
(_bfd_elf_assign_file_positions_for_non_load): Adjust to suit.
* coffgen.c (make_a_section_from_file): Use new inlines here.
This patch provides a new function to sanity check section sizes.
It's mostly extracted from what we had in bfd_get_full_section_contents
but also handles compressed debug sections.
Improvements are:
- section file offset is taken into account,
- added checks that a compressed section can be read from file.
The function is then used when handling multiple .debug_* sections
that need to be read into a single buffer, to sanity check sizes
before allocating the buffer.
PR 26946, PR 28834
* Makefile.am (LIBBFD_H_FILES): Add section.c.
* compress.c (bfd_get_full_section_contents): Move section size
sanity checks..
* section.c (_bfd_section_size_insane): ..to here. New function.
* dwarf2.c (read_section): Use _bfd_section_size_insane.
(_bfd_dwarf2_slurp_debug_info): Likewise.
* Makefile.in: Regenerate.
* libbfd.h: Regenerate.
So we had a zlib-gabi .debug_info section that increased in size with
zstd, so much so that it was better to leave the section
uncompressed. Things went horribly wrong when the section was read
again later. The section was read again off disk using the
uncompressed size. So you get the zlib section again with some
garbage at the end. Fix that particular problem by setting the
section flag SEC_IN_MEMORY. Any future read will get sec->contents.
Also, if the section is to be left uncompressed, the input
SHF_COMPRESSED flag needs to be reset otherwise objcopy will copy it
to output.
Finally, bfd_convert_section_contents needed a small update to handle
zstd compressed sections, and I've deleted bfd_cache_section_contents.
* bfd.c (bfd_convert_section_contents): Handle zstd.
* compress.c (bfd_compress_section_contents): When section
contents are uncompressed set SEC_IN_MEMORY flag,
compress_status to COMRESS_SECTION_NONE, and clear
SHF_COMPRESSED. Set SEC_IN_MEMORY for compressed contents.
(bfd_get_full_section_contents): Don't check section size
against file size when SEC_IN_MEMORY.
(bfd_cache_section_contents): Delete function.
* elf32-arm.c (elf32_arm_get_synthetic_symtab): Expand
bfd_cache_section_contents here.
* bfd-in2.h: Regenerate.
Commit 2cac01e3ff lacked support for objcopy changing compression
style. Add that support, which meant a rewrite of
bfd_compress_section_contents. In the process I've fixed some memory
leaks.
* compress.c (bfd_is_section_compressed_info): Rename from
bfd_is_section_compressed_with_header and add ch_type param
to return compression header ch_type field.
Update all callers.
(decompress_section_contents): Remove buffer and size params.
Rewrite. Update callers.
(bfd_init_section_compress_status): Free contents on failure.
(bfd_compress_section): Likewise.
* elf.c (_bfd_elf_make_section_from_shdr): Support objcopy
changing between any of the three compression schemes. Report
"unable to compress/decompress" rather than "unable to
initialize compress/decompress status" on compress/decompress
failures.
* bfd-in2.h: Regenerate.
PR29397 PR29563: Add new configure option --with-zstd which defaults to
auto. If pkgconfig/libzstd.pc is found, define HAVE_ZSTD and support
zstd compressed debug sections for most tools.
* bfd: for addr2line, objdump --dwarf, gdb, etc
* gas: support --compress-debug-sections=zstd
* ld: support ELFCOMPRESS_ZSTD input and --compress-debug-sections=zstd
* objcopy: support ELFCOMPRESS_ZSTD input for
--decompress-debug-sections and --compress-debug-sections=zstd
* gdb: support ELFCOMPRESS_ZSTD input. The bfd change references zstd
symbols, so gdb has to link against -lzstd in this patch.
If zstd is not supported, ELFCOMPRESS_ZSTD input triggers an error. We
can avoid HAVE_ZSTD if binutils-gdb imports zstd/ like zlib/, but this
is too heavyweight, so don't do it for now.
```
% ld/ld-new a.o
ld/ld-new: a.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support
...
% ld/ld-new a.o --compress-debug-sections=zstd
ld/ld-new: --compress-debug-sections=zstd: ld is not built with zstd support
% binutils/objcopy --compress-debug-sections=zstd a.o b.o
binutils/objcopy: --compress-debug-sections=zstd: binutils is not built with zstd support
% binutils/objcopy b.o --decompress-debug-sections
binutils/objcopy: zstd.o: section .debug_abbrev is compressed with zstd, but BFD is not built with zstd support
...
```
The result of running etc/update-copyright.py --this-year, fixing all
the files whose mode is changed by the script, plus a build with
--enable-maintainer-mode --enable-cgen-maint=yes, then checking
out */po/*.pot which we don't update frequently.
The copy of cgen was with commit d1dd5fcc38ead reverted as that commit
breaks building of bfp opcodes files.
Investigating the PR28530 testcase, which has a fuzzed compression
header with an enormous size, I noticed that decompress_contents is
broken when the size doesn't fit in strm.avail_out. It wouldn't be
too hard to support larger sizes (patches welcome!) but for now just
stop decompress_contents from returning rubbish.
PR 28530
* compress.c (decompress_contents): Fail when uncompressed_size
is too big.
(bfd_init_section_decompress_status): Likewise.
Always check sections with the corrupt size for non-MMO files. Skip MMO
files for compress_status == COMPRESS_SECTION_NONE since MMO has special
handling for COMPRESS_SECTION_NONE.
PR binutils/28530
* compress.c (bfd_get_full_section_contents): Always check
sections with the corrupt size.
Rather than reporting "memory exhausted", report "file truncated".
You can hit this error on small fuzzed object files, or on files that
are actually truncated. In either case sizes can be such that an out
of memory error is a little confusing.
* compress.c (bfd_get_full_section_contents): Set
bfd_error_file_truncated rather than bfd_error_no_memory when
section size exceeds file size.
So that no one need worry about the value of Z_OK.
bfd/
* compress.c (decompress_contents): Tidy inflateEnd result test.
binutils/
* readelf.c (uncompress_section_contents): Tidy inflateEnd result test.
The check in bfd_get_full_section_contents is trying to check that we don't
allocate more space for a section than the size of the section is on disk.
Previously we excluded linker created sections since they didn't have a size on
disk. However we also need to exclude sections with no content as well such as
the BSS section. Space for these would not have been allocated by the assembler
and so the check would incorrectly fail.
bfd/ChangeLog:
PR binutils/24753
* compress.c (bfd_get_full_section_contents): Exclude sections with no
content.
gas/ChangeLog:
PR binutils/24753
* testsuite/gas/arm/pr24753.d: New test.
* testsuite/gas/arm/pr24753.s: New test.
PR 24753
bfd * compress.c (bfd_get_full_section_contents): Do not complain
about linker created sections that are larger than the file size.
ld * emultempl/aarch64elf.em (_aarch64_add_stub_section): Include the
LINKER_CREATED section flag when creating the stub section.
PR 24708
* elf.c (_bfd_elf_slurp_version_tables): Check for an excessively
large version reference section.
* compress.c (bfd_get_full_section_contents): Check for an
uncompressed section whose size is larger than the file size.
Initialize *uncompressed_align_pow_p to 0 since *uncompressed_align_pow_p
is passed to bfd_is_section_compressed_with_header as uninitialized,
PR binutils/23919
* compress.c (bfd_is_section_compressed_with_header): Initialize
*uncompressed_align_pow_p to 0.
PR binutils/23919
binutils* readelf.c (dump_sections_as_strings): Remove bogus addralign check.
(dump_sections_as_bytes): Likewise.
(load_specific_debug_sections): Likewise.
* testsuite/binutils-all/dw2-3.rS: Adjust alignment.
* testsuite/binutils-all/dw2-3.rt: Likewise.
bfd * bfd.c (bfd_update_compression_header): Explicitly set alignment.
(bfd_check_compression_header): Add uncompressed_alignment_power
argument. Check ch_addralign is a power of 2.
* bfd-in2.h: Regenerated.
* compress.c (bfd_compress_section_contents): Get and set
orig_uncompressed_alignment_pow if section is decompressed.
(bfd_is_section_compressed_with_header): Add and get
uncompressed_align_pow_p argument.
(bfd_is_section_compressed): Add uncompressed_align_power argument
to bfd_is_section_compressed_with_header call.
(bfd_init_section_decompress_status): Get and set
uncompressed_alignment_power.
* elf.c (_bfd_elf_make_section_from_shdr): Add
uncompressed_align_power argument to
bfd_is_section_compressed_with_header call.
