forked from Imagelibrary/binutils-gdb
ea21a1b2ae4987d81b876e47ba784c347a84e21f
90 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Nick Alcock
|
ea21a1b2ae |
libctf: create, types: variables and datasecs (REVIEW NEEDED)
This is an area of significant difference from CTFv3. The API changes significantly, with quite a few additions to allow creation and querying of these new datasec entities: -typedef int ctf_variable_f (const char *name, ctf_id_t type, void *arg); +typedef int ctf_variable_f (ctf_dict_t *, const char *name, ctf_id_t type, + void *arg); +typedef int ctf_datasec_var_f (ctf_dict_t *fp, ctf_id_t type, size_t offset, + size_t datasec_size, void *arg); +/* Search a datasec for a variable covering a given offset. + + Errors with ECTF_NODATASEC if not found. */ + +ctf_id_t ctf_datasec_var_offset (ctf_dict_t *fp, ctf_id_t datasec, + uint32_t offset); + +/* Return the datasec that a given variable appears in, or ECTF_NODATASEC if + none. */ + +ctf_id_t ctf_variable_datasec (ctf_dict_t *fp, ctf_id_t var); +int ctf_datasec_var_iter (ctf_dict_t *, ctf_id_t, ctf_datasec_var_f *, + void *); +ctf_id_t ctf_datasec_var_next (ctf_dict_t *, ctf_id_t, ctf_next_t **, + size_t *size, size_t *offset); -int ctf_add_variable (ctf_dict_t *, const char *, ctf_id_t); +/* ctf_add_variable adds variables to no datasec at all; + ctf_add_section_variable adds them to the given datasec, or to no datasec at + all if the datasec is NULL. */ + +ctf_id_t ctf_add_variable (ctf_dict_t *, const char *, int linkage, ctf_id_t); +ctf_id_t ctf_add_section_variable (ctf_dict_t *, uint32_t, + const char *datasec, const char *name, + int linkage, ctf_id_t type, + size_t size, size_t offset); We tie datasecs quite closely to variables at addition (and, as should become clear later, dedup) time: you never create datasecs, you only create variables *in* datasecs, and the datasec springs into existence when you do so: datasecs are always found in the same dict as the variables they contain (the variables are never in the parent if the datasec is in a child or anything). We keep track of the variable->datasec mapping in ctf_var_datasecs (populating it at addition and open time), to allow ctf_variable_datasec to work at reasonable speed. (But, as yet, there are no tests of this function at all.) The datasecs are created unsorted (to avoid variable addition becoming O(n^2)) and sorted at serialization time, and when ctf_datasec_var_offset is invoked. We reuse the natural-alignment code from struct addition to get a plausible offset in datasecs if an alignment of -1 is specified: maybe this is unnecessary now (it was originally added when ctf_add_variable added variables to a "default datasec", while now it just leaves them out of all datasecs, like externs are). One constraint of this is that we currently prohibit the addition of nonrepresentable-typed variables, because we can't tell what their natural alignment is: if we dropped the whole "align" and just required everyone adding a variable to a datasec to specify an offset, we could drop that restriction. WDYT? One additional caveat: right now, ctf_lookup_variable() looks up the type of a variable (because when it was invented, variables were not entities in themselves that you could look up). This name is confusing as hell as a result. It might be less confusing to make it return the CTF_K_VAR, but that would be awful to adapt callers to, since both are represented with ctf_id_t's, so the compiler wouldn't warn about the needed change at all... I've vacillated on this three or four times now. |
||
|
Nick Alcock
|
2c1a0a70d1 |
libctf, create, types: encoding, BTF floats
This adds support for the nearly useless BTF_KIND_FLOAT, under the name
CTF_K_BTF_FLOAT. At the same time we fix up the ctf_add_encoding and
ctf_type_encoding machinery for the new API changes.
I expect this to change a bit: Ali Bahrami reckons I've oversimplified the
CTFv4 encoding representation and need to reintroduce at least a width.
New API:
ctf_id_t ctf_add_btf_float (ctf_dict_t *, uint32_t,
const char *, const ctf_encoding_t *);
|
||
|
Nick Alcock
|
03609073b0 |
libctf: create, types: enums and enum64s; type encoding
This commit adapts most aspects of enum handling: querying and iteration, enumerator querying and iteration, ctf_type_add, etc. We have to adapt to enum64s and to signed versus unsigned enums, to our vlen and DTD changes and other internal API changes to handle prefix types etc, and fix the types of things to allow for 64-bit enumerators. We can also (finally!) get useful info on enum size rather than being restricted to a value hardwired into libctf. We also adjust all the type-encoding functions for the internal API changes, since enums are the first encodable entities we have covered. API changes: -typedef int ctf_enum_f (const char *name, int val, void *arg); +typedef int ctf_enum_f (const char *name, int64_t val, void *arg); +typedef int ctf_unsigned_enum_f (const char *name, uint64_t val, void *arg); -extern const char *ctf_enum_name (ctf_dict_t *, ctf_id_t, int); -extern int ctf_enum_value (ctf_dict_t *, ctf_id_t, const char *, int *); +extern const char *ctf_enum_name (ctf_dict_t *, ctf_id_t, int64_t); +extern int ctf_enum_value (ctf_dict_t *, ctf_id_t, const char *, int64_t *); +extern int ctf_enum_unsigned_value (ctf_dict_t *, ctf_id_t, const char *, uint64_t *); + +/* Return 1 if this enum's contents are unsigned, so you can tell which of the + above functions to use. */ + +extern int ctf_enum_unsigned (ctf_dict_t *, ctf_id_t); -/* Return all enumeration constants in a given enum type. */ -extern int ctf_enum_iter (ctf_dict_t *, ctf_id_t, ctf_enum_f *, void *); +/* Return all enumeration constants in a given enum type. The return value, and + VAL argument, may need to be cast to uint64_t: see ctf_enum_unsigned(). */ +extern int64_t ctf_enum_iter (ctf_dict_t *, ctf_id_t, ctf_enum_f *, void *); extern const char *ctf_enum_next (ctf_dict_t *, ctf_id_t, ctf_next_t **, - int *); + int64_t *); + +/* enums are created signed by default. If you want an unsigned enum, + use ctf_add_enum_encoded() with an encoding of 0 (CTF_INT_SIGNED and + everything else off). This will not create a slice, unlike all other + uses of ctf_add_enum_encoded(), and the result is still representable + as BTF. */ + +extern ctf_id_t ctf_add_enum64_encoded (ctf_dict_t *, uint32_t, const char *, + const ctf_encoding_t *); +extern ctf_id_t ctf_add_enum64 (ctf_dict_t *, uint32_t, const char *); -extern int ctf_add_enumerator (ctf_dict_t *, ctf_id_t, const char *, int); +extern int ctf_add_enumerator (ctf_dict_t *, ctf_id_t, const char *, int64_t); The only aspects of enums that are not now handled are forwards to enums, dumping of enums, and deduplication of enums. |
||
|
Nick Alcock
|
0a3ee49dd0 |
libctf: types: add ctf_struct_bitfield (NEEDS REVIEW)
This new public API function allows you to find out if a struct has the bitfield flag set or not. (There are no other properties specific to a struct, so we needed a new function for it. I am open to a ctf_struct_info() function handing back a struct if people prefer.) New API: int ctf_struct_bitfield (ctf_dict_t *, ctf_id_t); |
||
|
Nick Alcock
|
20e6f72dc7 |
libctf: create: structure and union member addition
There is one API addition here:
int ctf_add_member_bitfield (ctf_dict_t *, ctf_id_t souid,
const char *, ctf_id_t type,
unsigned long bit_offset,
int bit_width);
SoU addition handles the representational changes for bitfields and for
CTF_K_BIG structs (i.e. all structs you can add members to), errors out if
you add bitfields to structs that aren't created with the
CTF_ADD_STRUCT_BITFIELDS flag, and arranges to add padding as needed if
there is too much of a gap for the offsets to encode in one hop (that
part is still untested).
|
||
|
Nick Alcock
|
cd8ea31666 |
libctf: create: struct/union addition
There's one API addition here: the existing CTF_ADD_ROOT / CTF_ADD_NONROOT flags can have a new flag ORed with them, CTF_ADD_STRUCT_BITFIELDS, indicating that the newly-added struct/union is capable of having bitfields added to it via the new ctf_add_member_bitfield function (see a later commit). Without this, you can only add bitfields via the deprecated slice or base type encoding representations (the former will force CTF output). Implementation notes: structs and unions are always added with a CTF_K_BIG prefix: if promoting from a forward, one is added. These are elided at serialization time if they are not needed to encode this size of struct / this number of members. (This means you don't have to figure out in advance if your struct will be too big for BTF: you can just add members to it, and libctf will figure it out and upgrade the dict as needed, or tell you it can't if you've forbidden such things.) We take advantage of this to merge a couple of very similar functions, saving a bit of code. |
||
|
Nick Alcock
|
05a2970ad1 |
libctf: create, lookup: delete DVDs; ctf_lookup_by_kind
Variable handling in BTF and CTFv4 works quite differently from in CTFv3. Rather than a separate section containing sorted, bsearchable variables, they are simply named entities like types, stored in CTF_K_VARs. As a first stage towards migrating to this, delete most references to the ctf_varent_t and ctf_dvdef_t, including the DVD lookup code, all the linking code, and quite a lot of the serialization code. Note: CTF_LINK_OMIT_VARIABLES_SECTION, and the whole "delete variables that already exist in the symtypetabs section" stuff, has yet to be reimplemented. We can implement CTF_LINK_OMIT_VARIABLES_SECTION by simply excising all CTF_K_VARs at deduplication time if requested. (Note: symtypetabs should still point directly at the type, not at the CTF_K_VAR.) (Symtypetabs in general need a bit more thought -- perhaps we can now store them in a separate .ctf.symtypetab section with its own little four-entry header for the symtypetabs and their indexes, making .ctf even more like .BTF; the only difference would then be that .ctf could include prefix types, CTF_K_FLOAT, and external string refs. For later discussion.) We also add ctf_lookup_by_kind() at this stage (because it is hopelessly diff-entangled with ctf_lookup_variable): this looks up a type of a particular kind, without needing a per-kind lookup function for it, nor needing to hack around adding string prefixes (so you can do ctf_lookup_by_kind (fp, CTF_K_STRUCT, "foo") rather than having to do ctf_lookup_by_name (fp, "struct foo"): often this is more convenient, and anything that reduces string buffer manipulation in C is good.) |
||
|
Nick Alcock
|
64b65a0a34 |
libctf: types: struct/union member querying and iteration
This commit revises ctf_member_next, ctf_member_iter, ctf_member_count, and
ctf_member_info for the new CTFv4 world. This also pulls in a bunch of
infrastructure used by most of the type querying functions, and fundamental
changes to the way DTD records are represented in libctf (ctf-create not yet
adjusted). Other type querying functions affected by changes in struct
representation are also changed.
There are some API changes here: new bit-width fields in ctf_member_f,
ctf_membinfo_t and ctf_member_next, and a fix to the type of the offset in
ctf_member_f, ctf_membinfo_t and and ctf_member_count. (ctf_member_next got
the offset type right already.)
ctf_member_f also gets a new ctf_dict_t arg so that you can actually use
the member type it passes in without having to package up and pass in the
dict type yourself (a frequent need). This change is later echoed in most
of the rest of the *_f typedefs.
typedef struct ctf_membinfo
{
ctf_id_t ctm_type; /* Type of struct or union member. */
- unsigned long ctm_offset; /* Offset of member in bits. */
+ size_t ctm_offset; /* Offset of member in bits. */
+ int ctm_bit_width; /* Width of member in bits: -1: not bitfield */
} ctf_membinfo_t;
-typedef int ctf_member_f (const char *name, ctf_id_t membtype,
- unsigned long offset, void *arg);
+typedef int ctf_member_f (ctf_dict_t *, const char *name, ctf_id_t membtype,
+ size_t offset, int bit_width, void *arg);
extern ssize_t ctf_member_next (ctf_dict_t *, ctf_id_t, ctf_next_t **,
const char **name, ctf_id_t *membtype,
- int flags);
+ int *bit_width, int flags);
-int ctf_member_count (ctf_dict_t *, ctf_id_t);
+ssize_t ctf_member_count (ctf_dict_t *, ctf_id_t);
The DTD changes are that where before the ctf_dtdef_t had a dtd_data which
was the ctf_type_t type node for a type, and a separate dtd_vlen which was
the vlen buffer which (in the final serialized representation) would
directly follow that type, now it has one single buffer, dtd_buf, which
consists of a stream of one or more ctf_type_t nodes, followed by a vlen,
as it will appear in the final serialized form. This buffer has internal
pointers into it: dtd_data is a pointer to the last ctf_type_t in the stream
(the true type node, after all prefixes), and dtd_vlen is a pointer to the
vlen (precisely one ctf_type_t after the dtd_data). This representation is
nice because it means there is even less distinction between a dynamic type
added by ctf_add_*() and a static one read directly out of a dict: you can
traverse the entire type without caring where it came from, simplifying
most of the type querying functions.
(There are a few more things in there which will be useful mostly when
adding new types: their uses will be seen later.)
Two new nontrivial functions exist (one of which is annoyingly tangled up in
the diff, sorry about that): ctf_find_prefix, which hunts down a given
prefix (if it exists) among the possibly many that may exist on a type (so
you can ask it to find the CTF_K_BIG prefix for a type if it exists, and
it'll return you a pointer to its ctf_type_t record), and ctf_vlen, which
you hand a type ID and its ctf_type_t *, and it gives you back a pointer to
its vlen and tells you how long it is. (This is one of only two places left
in ctf-types.c which cares whether a type is dynamic or not. The other has
yet to be added). Almost every function in ctf-types.c will end up calling
ctf_lookup_by_id and ctf_vlen in turn.
ctf_next_t has changed significantly: the ctn_type member is split in two so
that we can tell whether a given iterator works using types or indexes, and
we gain the ability to iterate over enum64s, DTDs themselves, and datasecs
(most of this will only be used in later commits).
The old internal function ctf_struct_member, which handled the distinction
between ctf_member_t and ctf_lmember_t, is gone. Instead we have new code
that handles the different representation of bitfield versus non-bitfield
structs and unions, and more code to handle the different representation of
CTF_K_BIG structs and unions (their offsets are the distance from the last
offset, rather than the distance from the start of the structure).
|
||
|
Nick Alcock
|
ad13b7d44f |
libctf: CTFv4: type opening
The majority of this commit rejigs the core type table opening code for CTFv4: there are a few ancillary bits it drags in, indicated below. The internal definition of a child dict (that may not have type or string lookups performed in it until ctf_open time) used to be 'has a cth_parent_name', but since BTF doesn't have one of those at all, we add an additional check: a dict the first byte of whose strtab is not 0 must be a child. (If *either* is true, this is a child dict, which allows for the possibility of CTF dicts with non-deduplicated strtabs -- thus with leading \0's -- to exist in future.) The initial sweep through the type table in init_static_types (to size the name-table lookup hashes) also now checks for various types which indicate that this must be a CTF dict, in addition to being adjusted to cater for new CTFv4 representations of things like forwards. (At this early stage, we cannot rely on the functions in ctf-type.c to abstract over this for us.) We make some new hashtables for new namespace-like things: datasecs and type and decl tags. The main name-population loop in init_static_types_names_internal takes prefixes into account, looking for the name on the suffix type (where the name is always found). LSTRUCT handling is removed (they no longer exist); ENUM64s, enum forwards, VARs, datasecs, and type and decl tags get their names suitably populated. Some buggy code which tried to populate the name tables for cvr-quals (which are nameless) was dropped. We add an extra pass which traverses all datasecs and keeps track of which datasec each var is instantiated in (if any) in a new ctf_var_datasecs hash table. (This uses a number of type-querying functions which don't yet exist: they'll be added in the upcoming commits.) We handle the type 0 == void case by pointing the first element of ctf_txlate at a type read in named "void" (making type 0 an alias to it), or, if one doesn't exist, creating a new one (outside the type table and dtd arrays), and pointing type 0 at that. Since it is numbered 0 and not in the type table or dtd arrays, it will never be written out at serialization time, but since it is *present*, libctf consumers who expect the void type to have an integral definition rather than being a magic number will get what they expect. |
||
|
Nick Alcock
|
7bcd444b9c |
libctf, include: debuggability improvements
When --enable-libctf-hash-debugging is on, make ctf_set_errno and ctf_set_typed_errno into real functions, not inlines, so you can drop breakpoints on them. Since we are breaking API, also move ECTF_NEXT_END to the start of the _CTF_ERRORS array, so you can check for real (non-ECTF_NEXT_END) errors in breakpooints on those functions by checking for err > 1000. |
||
|
Nick Alcock
|
e0490fbc73 |
include, libctf, binutils: drop labels
These have never been implemented properly and don't work with the linker or deduplicator: BTF has nothing like them, so the default assumption should be that we drop them. If we need something like them in future, we can add them back (which we do not expect). Quite a bit of label detritus is left in libctf after this: it's tied up with later changes so will be removed as part of later commits. (Because the entire thing is disabled, the non-compilability of this intermediate state is not a concern.) |
||
|
Nick Alcock
|
de5a31a8ca |
include, libctf: header and soname changes for CTFv4
These changes bump the current file format version to CTF_VERSION_4, and introduce a new VERSION_5 identical with it to get the version integer and the name identical again. A great many changes are made to account for the changes to handle CTFv4 (which is a BTF superset). libctf will not compile after these changes, which is why it's been diked out of the build system and forced-off until the series is complete. Because all the CTF_K constants have changed values, this is necessarily an ABI break: add a #define to make picking up this break at compile time obvious. Note that the ABI has broken by bumping the soname (deriving it now from libctf/libtool-version) and folding all newer symbols in the symbol version file into a new LIBCTF_2.0 version, which is now the only exported version. |
||
|
Nick Alcock
|
06f77d49f6 |
libctf: document parameters to ctf_import
One is a parent dict, one is a child: you shouldn't have to look at the source to tell which is which. include/ * ctf-api.h (ctf_import): Name parameters. |
||
|
Nick Alcock
|
b875301e74 |
libctf: drop LCTF_TYPE_ISPARENT/LCTF_TYPE_ISCHILD
Parent/child determination is about to become rather more complex, making a macro impractical. Use the ctf_type_isparent/ischild function calls everywhere and remove the macro. Make them more const-correct too, to make them more widely usable. While we're about it, change several places that hand-implemented ctf_get_dict() to call it instead, and armour several functions against the null returns that were always possible in this case (but previously unprotected-against). |
||
|
Nick Alcock
|
3bec4f1f3c |
include, libctf: string lookup and writeout of a parent-shared strtab
The next stage of strtab sharing is actual lookup of strings in such
strtabs, interning of strings in such strtabs and writing out of
such strtabs (but not actually figuring out which strings should
be shared: that's next).
We introduce several new internal ctf_str_* API functions to augment the
existing rather large set: ctf_str_add_copy, which adds a string and always
makes a copy of it (used when deduplicating to stop dedupped strings holding
permanent references on the input dicts), and ctf_str_no_dedup_ref (which
adds a ref to a string while preventing it from ever being deduplicated,
used for header fields like the parent name, which is the same for almost
all child dicts but had still better not be stored in the parent!).
ctf_strraw_explicit, the ultimate underlying "look up a string" function
that backs ctf_strptr et al, gains the ability to automatically find strings
in the parent if the offset is < cth_parent_strlen, and generally make all
offsets parent-relative (so something at offset 1 in the child strlen will
need to be looked up at offset 257 if cth_parent_strlen is 256). This
suffices to paste together the parent and child from the perspective
of lookup.
We do quite a lot of new checks in here, simply because it's called all over
the place and it's preferable to emit a nice error into the ctf_err_warning
stream if things go wrong. Among other things this traps cases where you
accidentally added a string to the parent, throwing off all the offsets.
Completely invalid offsets also now add a message to the err_warning
stream.
Insertion of new atoms (the deduplicated entities underlying strings in a
given dict), already a flag-heavy operation, gains more flags, corresponding
to the new ctf_str_add_copy and ctf_str_no_dedup_ref functions: atom
addition also checks the ctf_max_children set by ctf_import and prevents
addition of new atoms to any dicts with ctf_imported children and an
already-serialized strtab.
strtab writeout gains more checks as well: you can't write out a strtab for
a child dict whose parent hasn't been serialized yet (and thus doesn't have
a serialized strtab itself); you can't write it out if the child already
depended on a shared parent strtab and that strtab has changed length. The
null atom at offset 0 is only written to the parent strtab; and ref updating
changes to look up offsets in the parent's atoms table iff a new
CTF_STR_ATOM_IN_PARENT flag is set on the atom (this will be set by
deduplication to ensure that serializing a dict will update all its refs
properly even though a bunch of them have moved to the parent dict).
None of this actually has any *effect* yet because no string deduplication
is being carried out, and the cth_parent_strlen is still locked at 0.
include/
* ctf-api.h (_CTF_ERRORS) [ECTF_NOTSERIALIZED]: New.
(ECTF_NERR): Updated.
libctf/
* ctf-impl.h (CTF_STR_ATOM_IN_PARENT): New.
(CTF_STR_ATOM_NO_DEDUP): Likewise.
(ctf_str_add_no_dedup_ref): New.
(ctf_str_add_copy): New.
* ctf-string.c (ctf_strraw_explicit): Look in parents if necessary:
use parent-relative offsets.
(ctf_strptr_validate): Avoid duplicating errors.
(ctf_str_create_atoms): Update comment.
(CTF_STR_COPY): New.
(CTF_STR_NO_DEDUP): Likewise.
(ctf_str_add_ref_internal): Use them, setting the corresponding
csa_flags, prohibiting addition to serialized parents, and copying
strings if so requested.
(ctf_str_add): Turn into a wrapper around...
(ctf_str_add_flagged): ... this new function. The offset is now
parent-relative.
(ctf_str_add_ref): Likewise.
(ctf_str_add_movable_ref): Likewise.
(ctf_str_add_copy): New.
(ctf_str_add_no_dedup_ref): New.
(ctf_str_write_strtab): Prohibit writes when the parent has
changed length or is not serialized. Only write the null atom
to parent strtabs. Chase refs to the parent if necessary.
|
||
|
Nick Alcock
|
70d05ab0b2 |
libctf: add mechanism to prohibit most operations without a strtab
We are about to add machinery that deduplicates a child dict's strtab
against its parent. Obviously if you open such a dict but do not import its
parent, all strtab lookups must fail: so add an LCTF_NO_STR flag that is set
in that window and make most operations fail if it's not set. (Two more
that will be set in future commits are serialization and string lookup
itself.)
Notably, not all symbol lookup is impossible in this window: you can still
look up by symbol index, as long as this dict is not using an indexed
strtypetab (which obviously requires string lookups to get the symbol name).
include/
* ctf-api.h (_CTF_ERRORS) [ECTF_HASPARENT]: New.
[ECTF_WRONGPARENT]: Likewise.
(ECTF_NERR): Update.
Update comments to note the new limitations on ctf_import et al.
libctf/
* ctf-impl.h (LCTF_NO_STR): New.
* ctf-create.c (ctf_rollback): Error out when LCTF_NO_STR.
(ctf_add_generic): Likewise.
(ctf_add_struct_sized): Likewise.
(ctf_add_union_sized): Likewise.
(ctf_add_enum): Likewise.
(ctf_add_forward): Likewise.
(ctf_add_unknown): Likewise.
(ctf_add_enumerator): Likewise.
(ctf_add_member_offset): Likewise.
(ctf_add_variable): Likewise.
(ctf_add_funcobjt_sym_forced): Likewise.
(ctf_add_type): Likewise (on either dict).
* ctf-dump.c (ctf_dump): Likewise.
* ctf-lookup.c (ctf_lookup_by_name): Likewise.
(ctf_lookup_variable): Likewise. Likewise.
(ctf_lookup_enumerator): Likewise.
(ctf_lookup_enumerator_next): Likewise.
(ctf_symbol_next): Likewise.
(ctf_lookup_by_sym_or_name): Likewise, if doing indexed lookups.
* ctf-types.c (ctf_member_next): Likewise.
(ctf_enum_next): Likewise.
(ctf_type_aname): Likewise.
(ctf_type_name_raw): Likewise.
(ctf_type_compat): Likewise, for either dict.
(ctf_member_info): Likewise.
(ctf_enum_name): Likewise.
(ctf_enum_value): Likewise.
(ctf_type_rvisit): Likewise.
(ctf_variable_next): Note that we don't need to test LCTF_NO_STR.
|
||
|
Nick Alcock
|
9a74ab12c8 |
include, libctf: start work on libctf v4
This format is a superset of BTF, but for now we just do the minimum to
declare a new file format version, without actually introducing any format
changes.
From now on, we refuse to reserialize CTFv1 dicts: these have a distinct
parent/child boundary which obviously cannot change upon reserialization
(that would change the type IDs): instead, we encoded this by stuffing in
a unique CTF version for such dicts. We can't do that now we have one
version for all CTFv4 dicts, and testing such old dicts is very hard these
days anyway, and is not automated: so just drop support for writing them out
entirely. (You still *can* write them out, but you have to do a full-blown
ctf_link, which generates an all-new fresh dict and recomputes type IDs as
part of deduplication.)
To prevent this extremely-not-ready format escaping into the wild, add a
new mechanism whereby any format version higher than the new #define
CTF_STABLE_VERSION cannot be serialized unless I_KNOW_LIBCTF_IS_UNSTABLE is
set in the environment.
include/
* ctf-api.h (_CTF_ERRORS) [ECTF_CTFVERS_NO_SERIALIZE]: New.
[ECTF_UNSTABLE]: New.
(ECTF_NERR): Update.
* ctf.h: Small comment improvements..
(ctf_header_v3): New, copy of ctf_header.
(CTF_VERSION_4): New.
(CTF_VERSION): Now CTF_VERSION_4.
(CTF_STABLE_VERSION): Still 4, CTF_VERSION_3.
ld/
* testsuite/ld-ctf/*.d: Update to CTF_VERSION_4.
libctf/
* ctf-impl.h (LCTF_NO_SERIALIZE): New.
* ctf-dump.c (ctf_dump_header): Add CTF_VERSION_4.
* ctf-open.c (ctf_dictops): Likewise.
(upgrade_header): Rename to...
(upgrade_header_v2): ... this.
(upgrade_header_v3): New.
(upgrade_types): Support upgrading from CTF_VERSION_3.
Turn on LCTF_NO_SERIALIZE for CTFv1.
(init_static_types_internal): Upgrade all types tables older than
* CTF_VERSION_4.
(ctf_bufopen): Support CTF_VERSION_4: error out if we forget to
update this switch in future. Add header upgrading from v3 and
below. Improve comments slightly.
* ctf-serialize.c (ctf_serialize): Block serialization of unstable
file formats, and of file formats for which LCTF_NO_SERIALIZE is
turned on (v1).
|
||
|
Alan Modra
|
e8e7cf2abe | Update year range in copyright notice of binutils files | ||
|
Nick Alcock
|
6da9267482 |
libctf, include: add ctf_dict_set_flag: less enum dup checking by default
The recent change to detect duplicate enum values and return ECTF_DUPLICATE when found turns out to perturb a great many callers. In particular, the pahole-created kernel BTF has the same problem we historically did, and gleefully emits duplicated enum constants in profusion. Handling the resulting duplicate errors from BTF -> CTF converters reasonably is unreasonably difficult (it amounts to forcing them to skip some types or reimplement the deduplicator). So let's step back a bit. What we care about mostly is that the deduplicator treat enums with conflicting enumeration constants as conflicting types: programs that want to look up enumeration constant -> value mappings using the new APIs to do so might well want the same checks to apply to any ctf_add_* operations they carry out (and since they're *using* the new APIs, added at the same time as this restriction was imposed, there is likely to be no negative consequence of this). So we want some way to allow processes that know about duplicate detection to opt into it, while allowing everyone else to stay clear of it: but we want ctf_link to get this behaviour even if its caller has opted out. So add a new concept to the API: dict-wide CTF flags, set via ctf_dict_set_flag, obtained via ctf_dict_get_flag. They are not bitflags but simple arbitrary integers and an on/off value, stored in an unspecified manner (the one current flag, we translate into an LCTF_* flag value in the internal ctf_dict ctf_flags word). If you pass in an invalid flag or value you get a new ECTF_BADFLAG error, so the caller can easily tell whether flags added in future are valid with a particular libctf or not. We check this flag in ctf_add_enumerator, and set it around the link (including on child per-CU dicts). The newish enumerator-iteration test is souped up to check the semantics of the flag as well. The fact that the flag can be set and unset at any time has curious consequences. You can unset the flag, insert a pile of duplicates, then set it and expect the new duplicates to be detected, not only by ctf_add_enumerator but also by ctf_lookup_enumerator. This means we now have to maintain the ctf_names and conflicting_enums enum-duplication tracking as new enums are added, not purely as the dict is opened. Move that code out of init_static_types_internal and into a new ctf_track_enumerator function that addition can also call. (None of this affects the file format or serialization machinery, which has to be able to handle duplicate enumeration constants no matter what.) include/ * ctf-api.h (CTF_ERRORS) [ECTF_BADFLAG]: New. (ECTF_NERR): Update. (CTF_STRICT_NO_DUP_ENUMERATORS): New flag. (ctf_dict_set_flag): New function. (ctf_dict_get_flag): Likewise. libctf/ * ctf-impl.h (LCTF_STRICT_NO_DUP_ENUMERATORS): New flag. (ctf_track_enumerator): Declare. * ctf-dedup.c (ctf_dedup_emit_type): Set it. * ctf-link.c (ctf_create_per_cu): Likewise. (ctf_link_deduplicating_per_cu): Likewise. (ctf_link): Likewise. (ctf_link_write): Likewise. * ctf-subr.c (ctf_dict_set_flag): New function. (ctf_dict_get_flag): New function. * ctf-open.c (init_static_types_internal): Move enum tracking to... * ctf-create.c (ctf_track_enumerator): ... this new function. (ctf_add_enumerator): Call it. * libctf.ver: Add the new functions. * testsuite/libctf-lookup/enumerator-iteration.c: Test them. |
||
|
Nick Alcock
|
adc74ae98f |
include, libctf: improve ECTF_DUPLICATE error message
It applies to enums now, so it should mention them. include/ * ctf-api.h (_CTF_ERRORS) ECTF_DUPLICATE]: Mention enums. |
||
|
Nick Alcock
|
5f1077e69e |
libctf: improve ECTF_NOPARENT error message
This erorr doesn't just indicate that there is no parent dictionary (that's routine, and true of all dicts that are parents themselves) but that a parent is *needed* but wasn't found. include/ * ctf-api.h (_CTF_ERRORS) [ECTF_NOPARENT]: Improve error message. ld/ * testsuite/ld-ctf/diag-parname.d: Adjust. |
||
|
Nick Alcock
|
2fa4b6e6df |
libctf, include: new functions for looking up enumerators
Three new functions for looking up the enum type containing a given
enumeration constant, and optionally that constant's value.
The simplest, ctf_lookup_enumerator, looks up a root-visible enumerator by
name in one dict: if the dict contains multiple such constants (which is
possible for dicts created by older versions of the libctf deduplicator),
ECTF_DUPLICATE is returned.
The next simplest, ctf_lookup_enumerator_next, is an iterator which returns
all enumerators with a given name in a given dict, whether root-visible or
not.
The most elaborate, ctf_arc_lookup_enumerator_next, finds all
enumerators with a given name across all dicts in an entire CTF archive,
whether root-visible or not, starting looking in the shared parent dict;
opened dicts are cached (as with all other ctf_arc_*lookup functions) so
that repeated use does not incur repeated opening costs.
All three of these return enumerator values as int64_t: unfortunately, API
compatibility concerns prevent us from doing the same with the other older
enum-related functions, which all return enumerator constant values as ints.
We may be forced to add symbol-versioning compatibility aliases that fix the
other functions in due course, bumping the soname for platforms that do not
support such things.
ctf_arc_lookup_enumerator_next is implemented as a nested ctf_archive_next
iterator, and inside that, a nested ctf_lookup_enumerator_next iterator
within each dict. To aid in this, add support to ctf_next_t iterators for
iterators that are implemented in terms of two simultaneous nested iterators
at once. (It has always been possible for callers to use as many nested or
semi-overlapping ctf_next_t iterators as they need, which is one of the
advantages of this style over the _iter style that calls a function for each
thing iterated over: the iterator change here permits *ctf_next_t iterators
themselves* to be implemented by iterating using multiple other iterators as
part of their internal operation, transparently to the caller.)
Also add a testcase that tests all these functions (which is fairly easy
because ctf_arc_lookup_enumerator_next is implemented in terms of
ctf_lookup_enumerator_next) in addition to enumeration addition in
ctf_open()ed dicts, ctf_add_enumerator duplicate enumerator addition, and
conflicting enumerator constant deduplication.
include/
* ctf-api.h (ctf_lookup_enumerator): New.
(ctf_lookup_enumerator_next): Likewise.
(ctf_arc_lookup_enumerator_next): Likewise.
libctf/
* libctf.ver: Add them.
* ctf-impl.h (ctf_next_t) <ctn_next_inner>: New.
* ctf-util.c (ctf_next_copy): Copy it.
(ctf_next_destroy): Destroy it.
* ctf-lookup.c (ctf_lookup_enumerator): New.
(ctf_lookup_enumerator_next): New.
* ctf-archive.c (ctf_arc_lookup_enumerator_next): New.
* testsuite/libctf-lookup/enumerator-iteration.*: New test.
* testsuite/libctf-lookup/enum-ctf-2.c: New test CTF, used by the
above.
|
||
|
Nick Alcock
|
1f62f2a9b5 |
include: libctf: comment improvements
Describe a bit more clearly what effects a type being non-root-
visible has. More consistently use the term non-root-visible
rather than hidden. Document ctf_enum_iter.
include/
* ctf-api.h (ctf_enum_iter): Document.
(ctf_type_iter): Hidden, not non-root. Mention that
parent dictionaries are not traversed.
|
||
|
Nick Alcock
|
6e09d4a6e6 |
libctf: prohibit addition of enums with overlapping enumerator constants
libctf has long prohibited addition of enums with overlapping constants in a
single enum, but now that we are properly considering enums with overlapping
constants to be conflciting types, we can go further and prohibit addition
of enumeration constants to a dict if they already exist in any enum in that
dict: the same rules as C itself.
We do this in a fashion vaguely similar to what we just did in the
deduplicator, by considering enumeration constants as identifiers and adding
them to the core type/identifier namespace, ctf_dict_t.ctf_names. This is a
little fiddly, because we do not want to prohibit opening of existing dicts
into which the deduplicator has stuffed enums with overlapping constants!
We just want to prohibit the addition of *new* enumerators that violate that
rule. Even then, it's fine to add overlapping enumerator constants as long
as at least one of them is in a non-root type. (This is essential for
proper deduplicator operation in cu-mapped mode, where multiple compilation
units can be smashed into one dict, with conflicting types marked as
hidden: these types may well contain overlapping enumerators.)
So, at open time, keep track of all enums observed, then do a third pass
through the enums alone, adding each enumerator either to the ctf_names
table as a mapping from the enumerator name to the enum it is part of (if
not already present), or to a new ctf_conflicting_enums hashtable that
tracks observed duplicates. (The latter is not used yet, but will be soon.)
(We need to do a third pass because it's quite possible to have an enum
containing an enumerator FOO followed by a type FOO: since they're processed
in order, the enumerator would be processed before the type, and at that
stage it seems nonconflicting. The easiest fix is to run through the
enumerators after all type names are interned.)
At ctf_add_enumerator time, if the enumerator to which we are adding a type
is root-visible, check for an already-present name and error out if found,
then intern the new name in the ctf_names table as is done at open time.
(We retain the existing code which scans the enum itself for duplicates
because it is still an error to add an enumerator twice to a
non-root-visible enum type; but we only need to do this if the enum is
non-root-visible, so the cost of enum addition is reduced.)
Tested in an upcoming commit.
libctf/
* ctf-impl.h (ctf_dict_t) <ctf_names>: Augment comment.
<ctf_conflicting_enums>: New.
(ctf_dynset_elements): New.
* ctf-hash.c (ctf_dynset_elements): Implement it.
* ctf-open.c (init_static_types): Split body into...
(init_static_types_internal): ... here. Count enumerators;
keep track of observed enums in pass 2; populate ctf_names and
ctf_conflicting_enums with enumerators in a third pass.
(ctf_dict_close): Free ctf_conflicting_enums.
* ctf-create.c (ctf_add_enumerator): Prohibit addition of duplicate
enumerators in root-visible enum types.
include/
* ctf-api.h (CTF_ADD_NONROOT): Describe what non-rootness
means for enumeration constants.
(ctf_add_enumerator): The name is not a misnomer.
We now require that enumerators have unique names.
Document the non-rootness of enumerators.
|
||
|
Nick Alcock
|
f7b02dc3e1 |
include: fix libctf ECTF_NOENUMNAM error message
ECTF_NOENUMNAM is emitted when enumerator constant names don't exist. Call them that, not 'enum elements'. include/ * ctf-api.h (ECTF_NOENUMNAM): fix error message. |
||
|
Nick Alcock
|
6f0fe858b8 |
include, libctf: improve documentation
Some review comments came in after I pushed the last lot of ctf-api.h comment improvements. They were good, so I've incorporated them. Mostly: better _next iterator usage info, better info on ctf_*open functions, and better info on ctf_type_aname and ctf_type_name_raw. include/ * ctf-api.h: improve documentation. |
||
|
Nick Alcock
|
9ea82bfdd6 |
include, libctf: add a bunch of documentation to ctf-api.h
Hopefully this library is no longer quite so much a "you have to look in the source to understand anything" library. No semantic changes, though some functions have been moved around for clarity. include/ ctf-api.h: Add comments. |
||
|
Alan Modra
|
fd67aa1129 |
Update year range in copyright notice of binutils files
Adds two new external authors to etc/update-copyright.py to cover bfd/ax_tls.m4, and adds gprofng to dirs handled automatically, then updates copyright messages as follows: 1) Update cgen/utils.scm emitted copyrights. 2) Run "etc/update-copyright.py --this-year" with an extra external author I haven't committed, 'Kalray SA.', to cover gas testsuite files (which should have their copyright message removed). 3) Build with --enable-maintainer-mode --enable-cgen-maint=yes. 4) Check out */po/*.pot which we don't update frequently. |
||
|
Alan Modra
|
d87bef3a7b |
Update year range in copyright notice of binutils files
The newer update-copyright.py fixes file encoding too, removing cr/lf on binutils/bfdtest2.c and ld/testsuite/ld-cygwin/exe-export.exp, and embedded cr in binutils/testsuite/binutils-all/ar.exp string match. |
||
|
Nick Alcock
|
44c70fb01f |
libctf: add a comment explaining how to use ctf_*open
Specifically, tell users what to pass to those functions that accept raw section content, since it's fairly involved and easy to get wrong. (.dynsym / .dynstr when CTF_F_DYNSTR is set, otherwise .symtab / .strtab). include/ChangeLog: * ctf-api.h (ctf_*open): Improve comment. |
||
|
Alan Modra
|
a2c5833233 |
Update year range in copyright notice of binutils files
The result of running etc/update-copyright.py --this-year, fixing all the files whose mode is changed by the script, plus a build with --enable-maintainer-mode --enable-cgen-maint=yes, then checking out */po/*.pot which we don't update frequently. The copy of cgen was with commit d1dd5fcc38ead reverted as that commit breaks building of bfp opcodes files. |
||
|
Nick Alcock
|
49da556c65 |
libctf, include: support an alternative encoding for nonrepresentable types
Before now, types that could not be encoded in CTF were represented as references to type ID 0, which does not itself appear in the dictionary. This choice is annoying in several ways, principally that it forces generators and consumers of CTF to grow special cases for types that are referenced in valid dicts but don't appear. Allow an alternative representation (which will become the only representation in format v4) whereby nonrepresentable types are encoded as actual types with kind CTF_K_UNKNOWN (an already-existing kind theoretically but not in practice used for padding, with value 0). This is backward-compatible, because CTF_K_UNKNOWN was not used anywhere before now: it was used in old-format function symtypetabs, but these were never emitted by any compiler and the code to handle them in libctf likely never worked and was removed last year, in favour of new-format symtypetabs that contain only type IDs, not type kinds. In order to link this type, we need an API addition to let us add types of unknown kind to the dict: we let them optionally have names so that GCC can emit many different unknown types and those types with identical names will be deduplicated together. There are also small tweaks to the deduplicator to actually dedup such types, to let opening of dicts with unknown types with names work, to return the ECTF_NONREPRESENTABLE error on resolution of such types (like ID 0), and to print their names as something useful but not a valid C identifier, mostly for the sake of the dumper. Tests added in the next commit. include/ChangeLog 2021-05-06 Nick Alcock <nick.alcock@oracle.com> * ctf.h (CTF_K_UNKNOWN): Document that it can be used for nonrepresentable types, not just padding. * ctf-api.h (ctf_add_unknown): New. libctf/ChangeLog 2021-05-06 Nick Alcock <nick.alcock@oracle.com> * ctf-open.c (init_types): Unknown types may have names. * ctf-types.c (ctf_type_resolve): CTF_K_UNKNOWN is as non-representable as type ID 0. (ctf_type_aname): Print unknown types. * ctf-dedup.c (ctf_dedup_hash_type): Do not early-exit for CTF_K_UNKNOWN types: they have real hash values now. (ctf_dedup_rwalk_one_output_mapping): Treat CTF_K_UNKNOWN types like other types with no referents: call the callback and do not skip them. (ctf_dedup_emit_type): Emit via... * ctf-create.c (ctf_add_unknown): ... this new function. * libctf.ver (LIBCTF_1.2): Add it. |
||
|
Nick Alcock
|
8915c559d4 |
libctf, include: remove the nondeduplicating CTF linker
The nondeduplicating CTF linker was kept around when the deduplicating one was added so that people had something to fall back to in case the deduplicating linker turned out to be buggy. It's now much more stable than the nondeduplicating linker, in addition to much faster, using much less memory and producing much better output. In addition, while libctf has a linker flag to invoke the nondeduplicating linker, ld does not expose it: the only way to turn it on within ld is an intentionally- undocumented environment variable. So we can remove it without any ABI or user-visibility concerns (the only thing we leave around is the CTF_LINK_NONDEDUP flag, which can easily be interpreted as "deduplicate less", though right now it does nothing). This lets us remove a lot of complexity associated with tracking filenames and CU names separately (something the deduplcating linker never bothered with, since the cunames are always reliable and ld never hands us useful filenames anyway) The biggest lacuna left behind is the ctf_type_mapping machinery, which slows down deduplicating links quite a lot. We can't just ditch it because ctf_add_type uses it: removing the slowdown from the deduplicating linker is a job for another commit. include/ChangeLog 2021-03-02 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (CTF_LINK_SHARE_DUPLICATED): Note that this might merely change how much deduplication is done. libctf/ChangeLog 2021-03-02 Nick Alcock <nick.alcock@oracle.com> * ctf-link.c (ctf_create_per_cu): Drop FILENAME now that it is always identical to CUNAME. (ctf_link_deduplicating_one_symtypetab): Adjust. (ctf_link_one_type): Remove. (ctf_link_one_input_archive_member): Likewise. (ctf_link_close_one_input_archive): Likewise. (ctf_link_one_input_archive): Likewise. (ctf_link): No longer call it. Drop CTF_LINK_NONDEDUP path. Improve header comment a bit (dicts, not files). Adjust ctf_create_per_cu call. (ctf_link_deduplicating_variables): Simplify. (ctf_link_in_member_cb_arg_t) <cu_name>: Remove. <in_input_cu_file>: Likewise. <in_fp_parent>: Likewise. <done_parent>: Likewise. (ctf_link_one_variable): Turn uses of in_file_name to in_cuname. |
||
|
Nick Alcock
|
f4f60336da |
libctf, include: find types of symbols by name
The existing ctf_lookup_by_symbol and ctf_arc_lookup_symbol functions suffice to look up the types of symbols if the caller already has a symbol number. But the caller often doesn't have one of those and only knows the name of the symbol: also, in object files, the caller might not have a useful symbol number in any sense (and neither does libctf: the 'symbol number' we use in that case literally starts at 0 for the lexicographically first-sorted symbol in the symtypetab and counts those symbols, so it corresponds to nothing useful). This means that even though object files have a symtypetab (generated by the compiler or by ld -r), the only way we can look up anything in it is to iterate over all symbols in turn with ctf_symbol_next until we find the one we want. This is unhelpful and pointlessly inefficient. So add a pair of functions to look up symbols by name in a dict and in a whole archive: ctf_lookup_by_symbol_name and ctf_arc_lookup_symbol_name. These are identical to the existing functions except that they take symbol names rather than symbol numbers. To avoid insane repetition, we do some refactoring in the process, so that both ctf_lookup_by_symbol and ctf_arc_lookup_symbol turn into thin wrappers around internal functions that do both lookup by symbol index and lookup by name. This massively reduces code duplication because even the existing lookup-by-index stuff wants to use a name sometimes (when looking up in indexed sections), and the new lookup-by-name stuff has to turn it into an index sometimes (when looking up in non-indexed sections): doing it this way lets us share most of that. The actual name->index lookup is done by ctf_lookup_symbol_idx. We do not anticipate this lookup to be as heavily used as ld.so symbol lookup by many orders of magnitude, so using the ELF symbol hashes would probably take more time to read them than is saved by using the hashes, and it adds a lot of complexity. Instead, do a linear search for the symbol name, caching all the name -> index mappings as we go, so that future searches are likely to hit in the cache. To avoid having to repeat this search over and over in a CTF archive when ctf_arc_lookup_symbol_name is used, have cached archive lookups (the sort done by ctf_arc_lookup_symbol* and the ctf_archive_next iterator) pick out the first dict they cache in a given archive and store it in a new ctf_archive field, ctfi_crossdict_cache. This can be used to store cross-dictionary cached state that depends on things like the ELF symbol table rather than the contents of any one dict. ctf_lookup_symbol_idx then caches its name->index mappings in the dictionary named in the crossdict cache, if any, so that ctf_lookup_symbol_idx in other dicts in the same archive benefit from the previous linear search, and the symtab only needs to be scanned at most once. (Note that if you call ctf_lookup_by_symbol_name in one specific dict, and then follow it with a ctf_arc_lookup_symbol_name, the former will not use the crossdict cache because it's only populated by the dict opens in ctf_arc_lookup_symbol_name. This is harmless except for a small one-off waste of memory and time: it's only a cache, after all. We can fix this later by using the archive caching machinery more aggressively.) In ctf-archive, we do similar things, turning ctf_arc_lookup_symbol into a wrapper around a new function that does both index -> ID and name -> ID lookups across all dicts in an archive. We add a new ctfi_symnamedicts cache that maps symbol names to the ctf_dict_t * that it was found in (so that linear searches for symbols don't need to be repeated): but we also *remove* a cache, the ctfi_syms cache that was memoizing the actual ctf_id_t returned from every call to ctf_arc_lookup_symbol. This is pointless: all it saves is one call to ctf_lookup_by_symbol, and that's basically an array lookup and nothing more so isn't worth caching. (Equally, given that symbol -> index mappings are cached by ctf_lookup_by_symbol_name, those calls are nearly free after the first call, so there's no point caching the ctf_id_t in that case either.) We fix up one test that was doing manual symbol lookup to use ctf_arc_lookup_symbol instead, and enhance it to check that the caching layer is not totally broken: we also add a new test to do lookups in a .o file, and another to do lookups in an archive with conflicted types and make sure that sort of multi-dict lookup is actually working. include/ChangeLog 2021-02-17 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_arc_lookup_symbol_name): New. (ctf_lookup_by_symbol_name): Likewise. libctf/ChangeLog 2021-02-17 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (ctf_dict_t) <ctf_symhash>: New. <ctf_symhash_latest>: Likewise. (struct ctf_archive_internal) <ctfi_crossdict_cache>: New. <ctfi_symnamedicts>: New. <ctfi_syms>: Remove. (ctf_lookup_symbol_name): Remove. * ctf-lookup.c (ctf_lookup_symbol_name): Propagate errors from parent properly. Make static. (ctf_lookup_symbol_idx): New, linear search for the symbol name, cached in the crossdict cache's ctf_symhash (if available), or this dict's (otherwise). (ctf_try_lookup_indexed): Allow the symname to be passed in. (ctf_lookup_by_symbol): Turn into a wrapper around... (ctf_lookup_by_sym_or_name): ... this, supporting name lookup too, using ctf_lookup_symbol_idx in non-writable dicts. Special-case name lookup in dynamic dicts without reported symbols, which have no symtab or dynsymidx but where name lookup should still work. (ctf_lookup_by_symbol_name): New, another wrapper. * ctf-archive.c (enosym): Note that this is present in ctfi_symnamedicts too. (ctf_arc_close): Adjust for removal of ctfi_syms. Free the ctfi_symnamedicts. (ctf_arc_flush_caches): Likewise. (ctf_dict_open_cached): Memoize the first cached dict in the crossdict cache. (ctf_arc_lookup_symbol): Turn into a wrapper around... (ctf_arc_lookup_sym_or_name): ... this. No longer cache ctf_id_t lookups: just call ctf_lookup_by_symbol as needed (but still cache the dicts those lookups succeed in). Add lookup-by-name support, with dicts of successful lookups cached in ctfi_symnamedicts. Refactor the caching code a bit. (ctf_arc_lookup_symbol_name): New, another wrapper. * ctf-open.c (ctf_dict_close): Free the ctf_symhash. * libctf.ver (LIBCTF_1.2): New version. Add ctf_lookup_by_symbol_name, ctf_arc_lookup_symbol_name. * testsuite/libctf-lookup/enum-symbol.c (main): Use ctf_arc_lookup_symbol rather than looking up the name ourselves. Fish it out repeatedly, to make sure that symbol caching isn't broken. (symidx_64): Remove. (symidx_32): Remove. * testsuite/libctf-lookup/enum-symbol-obj.lk: Test symbol lookup in an unlinked object file (indexed symtypetab sections only). * testsuite/libctf-writable/symtypetab-nonlinker-writeout.c (try_maybe_reporting): Check symbol types via ctf_lookup_by_symbol_name as well as ctf_symbol_next. * testsuite/libctf-lookup/conflicting-type-syms.*: New test of lookups in a multi-dict archive. |
||
|
Nick Alcock
|
caa170493e |
libctf: prohibit nameless ints, floats, typedefs and forwards
Now that "anonymous typedef nodes" have been extirpated, we can mandate that things that have names in C must have names in CTF too. (Unlike the no-forwards embarrassment, the deduplicator does nothing special with names: types that have names in C will have the same name in CTF. So we can assume that the CTF rules and the C rules are the same.) include/ChangeLog 2021-01-27 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ECTF_NONAME): New. (ECTF_NERR): Adjust. libctf/ChangeLog 2021-01-27 Nick Alcock <nick.alcock@oracle.com> * ctf-create.c (ctf_add_encoded): Add check for non-empty name. (ctf_add_forward): Likewise. (ctf_add_typedef): Likewise. |
||
|
Nick Alcock
|
35a01a0454 |
libctf, ld: fix symtypetab and var section population under ld -r
The variable section in a CTF dict is meant to contain the types of variables that do not appear in the symbol table (mostly file-scope static declarations). We implement this by having the compiler emit all potential data symbols into both sections, then delete those symbols from the variable section that correspond to data symbols the linker has reported. Unfortunately, the check for this in ctf_serialize is wrong: rather than checking the set of linker-reported symbols, we check the set of names in the data object symtypetab section: if the linker has reported no symbols at all (usually if ld -r has been run, or if a non-linker program that does not use symbol tables is calling ctf_link) this will include every single symbol, emptying the variable section completely. Worse, when ld -r is in use, we want to force writeout of every symtypetab entry on the inputs, in an indexed section, whether or not the linker has reported them, since this isn't a final link yet and the symbol table is not finalized (and may grow more symbols than the linker has yet reported). But the check for this is flawed too: we were relying on ctf_link_shuffle_syms not having been called if no symbols exist, but that function is *always* called by ld even when ld -r is in use: ctf_link_add_linker_symbol is the one that's not called when there are no symbols. We clearly need to rethink this. Using the emptiness of the set of reported symbols as a test for ld -r is just ugly: the linker already knows if ld -r is underway and can just tell us. So add a new linker flag CTF_LINK_NO_FILTER_REPORTED_SYMS that is set to stop the linker filtering the symbols in the symtypetab sections using the set that the linker has reported: use the presence or absence of this flag to determine whether to emit unindexed symtabs: we only remove entries from the variable section when filtering symbols, and we only remove them if they are in the reported symbol set, fixing the case where no symbols are reported by the linker at all. (The negative sense of the new CTF_LINK flag is intentional: the common case, both for ld and for simple tools that want to do a ctf_link with no ELF symbol table in sight, is probably to filter out symbols that no linker has reported: i.e., for the simple tools, all of them.) There's another wrinkle, though. It is quite possible for a non-linker to add symbols to a dict via ctf_add_*_sym and then write it out via the ctf_write APIs: perhaps it's preparing a dict for a later linker invocation. Right now this would not lead to anything terribly meaningful happening: ctf_serialize just assumes it was called via ctf_link if symbols are present. So add an (internal-to-libctf) flag that indicates that a writeout is happening via ctf_link_write, and set it there (propagating it to child dicts as needed). ctf_serialize can then spot when it is not being called by a linker, and arrange to always write out an indexed, sorted symtypetab for fastest possible future symbol lookup by name in that case. (The writeouts done by ld -r are unsorted, because the only thing likely to use those symtabs is the linker, which doesn't benefit from symtypetab sorting.) Tests added for all three linking cases (ld -r, ld -shared, ld), with a bit of testsuite framework enhancement to stop it unconditionally linking the CTF to be checked by the lookup program with -shared, so tests can now examine CTF linked with -r or indeed with no flags at all, though the output filename is still foo.so even in this case. Another test added for the non-linker case that endeavours to determine whether the symtypetab is sorted by examining the order of entries returned from ctf_symbol_next: nobody outside libctf should rely on this ordering, but this test is not outside libctf :) include/ChangeLog 2021-01-26 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (CTF_LINK_NO_FILTER_REPORTED_SYMS): New. ld/ChangeLog 2021-01-26 Nick Alcock <nick.alcock@oracle.com> * ldlang.c (lang_merge_ctf): Set CTF_LINK_NO_FILTER_REPORTED_SYMS when appropriate. libctf/ChangeLog 2021-01-27 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.c (_libctf_nonnull_): Add parameters. (LCTF_LINKING): New flag. (ctf_dict_t) <ctf_link_flags>: Mention it. * ctf-link.c (ctf_link): Keep LCTF_LINKING set across call. (ctf_write): Likewise, including in child dictionaries. (ctf_link_shuffle_syms): Make sure ctf_dynsyms is NULL if there are no reported symbols. * ctf-create.c (symtypetab_delete_nonstatic_vars): Make sure the variable has been reported as a symbol by the linker. (symtypetab_skippable): Mention relationship between SYMFP and the flags. (symtypetab_density): Adjust nonnullity. Exit early if no symbols were reported and force-indexing is off (i.e., we are doing a final link). (ctf_serialize): Handle the !LCTF_LINKING case by writing out an indexed, sorted symtypetab (and allow SYMFP to be NULL in this case). Turn sorting off if this is a non-final link. Only delete nonstatic vars if we are filtering symbols and the linker has reported some. * testsuite/libctf-regression/nonstatic-var-section-ld-r*: New test of variable and symtypetab section population when ld -r is used. * testsuite/libctf-regression/nonstatic-var-section-ld-executable.lk: Likewise, when ld of an executable is used. * testsuite/libctf-regression/nonstatic-var-section-ld.lk: Likewise, when ld -shared alone is used. * testsuite/libctf-regression/nonstatic-var-section-ld*.c: Lookup programs for the above. * testsuite/libctf-writable/symtypetab-nonlinker-writeout.*: New test, testing survival of symbols across ctf_write paths. * testsuite/lib/ctf-lib.exp (run_lookup_test): New option, nonshared, suppressing linking of the SOURCE with -shared. |
||
|
Nick Alcock
|
6c3a38777b |
libctf, include: support unnamed structure members better
libctf has no intrinsic support for the GCC unnamed structure member extension. This principally means that you can't look up named members inside unnamed struct or union members via ctf_member_info: you have to tiresomely find out the type ID of the unnamed members via iteration, then look in each of these. This is ridiculous. Fix it by extending ctf_member_info so that it recurses into unnamed members for you: this is still unambiguous because GCC won't let you create ambiguously-named members even in the presence of this extension. For consistency, and because the release hasn't happened and we can still do this, break the ctf_member_next API and add flags: we specify one flag, CTF_MN_RECURSE, which if set causes ctf_member_next to automatically recurse into unnamed members for you, returning not only the members themselves but all their contained members, so that you can use ctf_member_next to identify every member that it would be valid to call ctf_member_info with. New lookup tests are added for all of this. include/ChangeLog 2021-01-05 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (CTF_MN_RECURSE): New. (ctf_member_next): Add flags argument. libctf/ChangeLog 2021-01-05 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (struct ctf_next) <u.ctn_next>: Move to... <ctn_next>: ... here. * ctf-util.c (ctf_next_destroy): Unconditionally destroy it. * ctf-lookup.c (ctf_symbol_next): Adjust accordingly. * ctf-types.c (ctf_member_iter): Reimplement in terms of... (ctf_member_next): ... this. Support recursive unnamed member iteration (off by default). (ctf_member_info): Look up members in unnamed sub-structs. * ctf-dedup.c (ctf_dedup_rhash_type): Adjust ctf_member_next call. (ctf_dedup_emit_struct_members): Likewise. * testsuite/libctf-lookup/struct-iteration-ctf.c: Test empty unnamed members, and a normal member after the end. * testsuite/libctf-lookup/struct-iteration.c: Verify that ctf_member_count is consistent with the number of successful returns from a non-recursive ctf_member_next. * testsuite/libctf-lookup/struct-iteration-*: New, test iteration over struct members. * testsuite/libctf-lookup/struct-lookup.c: New test. * testsuite/libctf-lookup/struct-lookup.lk: New test. |
||
|
Nick Alcock
|
ffeece6ac2 |
libctf, ld: prohibit getting the size or alignment of forwards
C allows you to do only a very few things with entities of incomplete type (as opposed to pointers to them): make pointers to them and give them cv-quals, roughly. In particular you can't sizeof them and you can't get their alignment. We cannot impose all the requirements the standard imposes on CTF users, because the deduplicator can transform any structure type into a forward for the purposes of breaking cycles: so CTF type graphs can easily contain things like arrays of forward type (if you want to figure out their size or alignment, you need to chase down the types this forward might be a forward to in child TU dicts: we will soon add API functions to make doing this much easier). Nonetheless, it is still meaningless to ask for the size or alignment of forwards: but libctf didn't prohibit this and returned nonsense from internal implementation details when you asked (it returned the kind of the pointed-to type as both the size and alignment, because forwards reuse ctt_type as a type kind, and ctt_type and ctt_size overlap). So introduce a new error, ECTF_INCOMPLETE, which is returned when you try to get the size or alignment of forwards: we also return it when you try to do things that require libctf itself to get the size or alignment of a forward, notably using a forward as an array index type (which C should never do in any case) or adding forwards to structures without specifying their offset explicitly. The dumper will not emit size or alignment info for forwards any more. (This should not be an API break since ctf_type_size and ctf_type_align could both return errors before now: any code that isn't expecting error returns is already potentially broken.) include/ChangeLog 2021-01-05 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ECTF_INCOMPLETE): New. (ECTF_NERR): Adjust. ld/ChangeLog 2021-01-05 Nick Alcock <nick.alcock@oracle.com> * testsuite/ld-ctf/conflicting-cycle-1.parent.d: Adjust for dumper changes. * testsuite/ld-ctf/cross-tu-cyclic-conflicting.d: Likewise. * testsuite/ld-ctf/forward.c: New test... * testsuite/ld-ctf/forward.d: ... and results. libctf/ChangeLog 2021-01-05 Nick Alcock <nick.alcock@oracle.com> * ctf-types.c (ctf_type_resolve): Improve comment. (ctf_type_size): Yield ECTF_INCOMPLETE when applied to forwards. Emit errors into the right dict. (ctf_type_align): Likewise. * ctf-create.c (ctf_add_member_offset): Yield ECTF_INCOMPLETE when adding a member without explicit offset when this member, or the previous member, is incomplete. * ctf-dump.c (ctf_dump_format_type): Do not try to print the size of forwards. (ctf_dump_member): Do not try to print their alignment. |
||
|
Alan Modra
|
250d07de5c | Update year range in copyright notice of binutils files | ||
|
Nick Alcock
|
53651de80f |
libctf, include: support foreign-endianness symtabs with CTF
The CTF symbol lookup machinery added recently has one deficit: it assumes the symtab is in the machine's native endianness. This is always true when the linker is writing out symtabs (because cross linkers byteswap symbols only after libctf has been called on them), but may be untrue in the cross case when the linker or another tool (objdump, etc) is reading them. Unfortunately the easy way to model this to the caller, as an endianness field in the ctf_sect_t, is precluded because doing so would change the size of the ctf_sect_t, which would be an ABI break. So, instead, allow the endianness of the symtab to be set after open time, by calling one of the two new API functions ctf_symsect_endianness (for ctf_dict_t's) or ctf_arc_symsect_endianness (for entire ctf_archive_t's). libctf calls these functions automatically for objects opened via any of the BFD-aware mechanisms (ctf_bfdopen, ctf_bfdopen_ctfsect, ctf_fdopen, ctf_open, or ctf_arc_open), but the various mechanisms that just take raw ctf_sect_t's will assume the symtab is in native endianness and need a later call to ctf_*symsect_endianness to adjust it if needed. (This call is basically free if the endianness is actually native: it only costs anything if the symtab endianness was previously guessed wrong, and there is a symtab, and we are using it directly rather than using symtab indexing.) Obviously, calling ctf_lookup_by_symbol or ctf_symbol_next before the symtab endianness is correctly set will probably give wrong answers -- but you can set it at any time as long as it is before then. include/ChangeLog 2020-11-23 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h: Style nit: remove () on function names in comments. (ctf_sect_t): Mention endianness concerns. (ctf_symsect_endianness): New declaration. (ctf_arc_symsect_endianness): Likewise. libctf/ChangeLog 2020-11-23 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (ctf_dict_t) <ctf_symtab_little_endian>: New. (struct ctf_archive_internal) <ctfi_symsect_little_endian>: Likewise. * ctf-create.c (ctf_serialize): Adjust for new field. * ctf-open.c (init_symtab): Note the semantics of repeated calls. (ctf_symsect_endianness): New. (ctf_bufopen_internal): Set ctf_symtab_little_endian suitably for the native endianness. (_Static_assert): Moved... (swap_thing): ... with this... * swap.h: ... to here. * ctf-util.c (ctf_elf32_to_link_sym): Use it, byteswapping the Elf32_Sym if the ctf_symtab_little_endian demands it. (ctf_elf64_to_link_sym): Likewise swap the Elf64_Sym if needed. * ctf-archive.c (ctf_arc_symsect_endianness): New, set the endianness of the symtab used by the dicts in an archive. (ctf_archive_iter_internal): Initialize to unknown (assumed native, do not call ctf_symsect_endianness). (ctf_dict_open_by_offset): Call ctf_symsect_endianness if need be. (ctf_dict_open_internal): Propagate the endianness down. (ctf_dict_open_sections): Likewise. * ctf-open-bfd.c (ctf_bfdopen_ctfsect): Get the endianness from the struct bfd and pass it down to the archive. * libctf.ver: Add ctf_symsect_endianness and ctf_arc_symsect_endianness. |
||
|
Nick Alcock
|
97a2a623d0 |
libctf, include: add ctf_getsymsect and ctf_getstrsect
libctf has long provided ctf_getdatasect, which hands back a pointer to the CTF section a (read-only) dict came from. But it has no such functions to return pointers to the ELF symbol table or string table it's working from, which is unfortunate because several libctf functions (ctf_open, ctf_fdopen, and ctf_bfdopen) figure out which string and symbol table to use themselves, and don't tell the user what they decided, so the caller can't agree on which symtab to use with libctf even if it wanted to. Add a pair of functions to return the symtab and strtab in use. Like ctf_getdatasect, these return ctf_sect_t structures by value, filled with all-NULL/0 content if a symtab or strtab is not being used. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_getsymsect): New. (ctf_getstrsect): Likewise. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-open.c (ctf_getsymsect): New. (ctf_getstrsect): Likewise. * libctf.ver: Add them. |
||
|
Nick Alcock
|
2c78e92523 |
libctf, include: CTF-archive-wide symbol lookup
CTF archives may contain multiple dicts, each of which contain many types and possibly a bunch of symtypetab entries relating to those types: each symtypetab entry is going to appear in exactly one dict, with the corresponding entries in the other dicts empty (either pads, or indexed symtypetabs that do not mention that symbol). But users of libctf usually want to get back the type associated with a symbol without having to dig around to find out which dict that type might be in. This adds machinery to do that -- and since you probably want to do it repeatedly, it adds internal caching to the ctf-archive machinery so that iteration over archives via ctf_archive_next and repeated symbol lookups do not have to repeatedly reopen the archive. (Iteration using ctf_archive_iter will gain caching soon.) Two new API functions: ctf_dict_t * ctf_arc_lookup_symbol (ctf_archive_t *arc, unsigned long symidx, ctf_id_t *typep, int *errp); This looks up the symbol with index SYMIDX in the archive ARC, returning the dictionary in which it resides and optionally the type index as well. Errors are returned in ERRP. The dict should be ctf_dict_close()d when done, but is also cached inside the ctf_archive so that the open cost is only paid once. The result of the symbol lookup is also cached internally, so repeated lookups of the same symbol are nearly free. void ctf_arc_flush_caches (ctf_archive_t *arc); Flush all the caches. Done at close time, but also available as an API function if users want to do it by hand. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_arc_lookup_symbol): New. (ctf_arc_flush_caches): Likewise. * ctf.h: Document new auto-ctf_import behaviour. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h (struct ctf_archive_internal) <ctfi_dicts>: New, dicts the archive machinery has opened and cached. <ctfi_symdicts>: New, cache of dicts containing symbols looked up. <ctfi_syms>: New, cache of types of symbols looked up. * ctf-archive.c (ctf_arc_close): Free them on close. (enosym): New, flag entry for 'symbol not present'. (ctf_arc_import_parent): New, automatically import the parent from ".ctf" if this is a child in an archive and ".ctf" is present. (ctf_dict_open_sections): Use it. (ctf_archive_iter_internal): Likewise. (ctf_cached_dict_close): New, thunk around ctf_dict_close. (ctf_dict_open_cached): New, open and cache a dict. (ctf_arc_flush_caches): New, flush the caches. (ctf_arc_lookup_symbol): New, look up a symbol in (all members of) an archive, and cache the lookup. (ctf_archive_iter): Note the new caching behaviour. (ctf_archive_next): Use ctf_dict_open_cached. * libctf.ver: Add ctf_arc_lookup_symbol and ctf_arc_flush_caches. |
||
|
Nick Alcock
|
1136c37971 |
libctf: symbol type linking support
This adds facilities to write out the function info and data object
sections, which efficiently map from entries in the symbol table to
types. The write-side code is entirely new: the read-side code was
merely significantly changed and support for indexed tables added
(pointed to by the no-longer-unused cth_objtidxoff and cth_funcidxoff
header fields).
With this in place, you can use ctf_lookup_by_symbol to look up the
types of symbols of function and object type (and, as before, you can
use ctf_lookup_variable to look up types of file-scope variables not
present in the symbol table, as long as you know their name: but
variables that are also data objects are now found in the data object
section instead.)
(Compatible) file format change:
The CTF spec has always said that the function info section looks much
like the CTF_K_FUNCTIONs in the type section: an info word (including an
argument count) followed by a return type and N argument types. This
format is suboptimal: it means function symbols cannot be deduplicated
and it causes a lot of ugly code duplication in libctf. But
conveniently the compiler has never emitted this! Because it has always
emitted a rather different format that libctf has never accepted, we can
be sure that there are no instances of this function info section in the
wild, and can freely change its format without compatibility concerns or
a file format version bump. (And since it has never been emitted in any
code that generated any older file format version, either, we need keep
no code to read the format as specified at all!)
So the function info section is now specified as an array of uint32_t,
exactly like the object data section: each entry is a type ID in the
type section which must be of kind CTF_K_FUNCTION, the prototype of
this function.
This allows function types to be deduplicated and also correctly encodes
the fact that all functions declared in C really are types available to
the program: so they should be stored in the type section like all other
types. (In format v4, we will be able to represent the types of static
functions as well, but that really does require a file format change.)
We introduce a new header flag, CTF_F_NEWFUNCINFO, which is set if the
new function info format is in use. A sufficiently new compiler will
always set this flag. New libctf will always set this flag: old libctf
will refuse to open any CTF dicts that have this flag set. If the flag
is not set on a dict being read in, new libctf will disregard the
function info section. Format v4 will remove this flag (or, rather, the
flag has no meaning there and the bit position may be recycled for some
other purpose).
New API:
Symbol addition:
ctf_add_func_sym: Add a symbol with a given name and type. The
type must be of kind CTF_K_FUNCTION (a function
pointer). Internally this adds a name -> type
mapping to the ctf_funchash in the ctf_dict.
ctf_add_objt_sym: Add a symbol with a given name and type. The type
kind can be anything, including function pointers.
This adds to ctf_objthash.
These both treat symbols as name -> type mappings: the linker associates
symbol names with symbol indexes via the ctf_link_shuffle_syms callback,
which sets up the ctf_dynsyms/ctf_dynsymidx/ctf_dynsymmax fields in the
ctf_dict. Repeated relinks can add more symbols.
Variables that are also exposed as symbols are removed from the variable
section at serialization time.
CTF symbol type sections which have enough pads, defined by
CTF_INDEX_PAD_THRESHOLD (whether because they are in dicts with symbols
where most types are unknown, or in archive where most types are defined
in some child or parent dict, not in this specific dict) are sorted by
name rather than symidx and accompanied by an index which associates
each symbol type entry with a name: the existing ctf_lookup_by_symbol
will map symbol indexes to symbol names and look the names up in the
index automatically. (This is currently ELF-symbol-table-dependent, but
there is almost nothing specific to ELF in here and we can add support
for other symbol table formats easily).
The compiler also uses index sections to communicate the contents of
object file symbol tables without relying on any specific ordering of
symbols: it doesn't need to sort them, and libctf will detect an
unsorted index section via the absence of the new CTF_F_IDXSORTED header
flag, and sort it if needed.
Iteration:
ctf_symbol_next: Iterator which returns the types and names of symbols
one by one, either for function or data symbols.
This does not require any sorting: the ctf_link machinery uses it to
pull in all the compiler-provided symbols cheaply, but it is not
restricted to that use.
(Compatible) changes in API:
ctf_lookup_by_symbol: can now be called for object and function
symbols: never returns ECTF_NOTDATA (which is
now not thrown by anything, but is kept for
compatibility and because it is a plausible
error that we might start throwing again at some
later date).
Internally we also have changes to the ctf-string functionality so that
"external" strings (those where we track a string -> offset mapping, but
only write out an offset) can be consulted via the usual means
(ctf_strptr) before the strtab is written out. This is important
because ctf_link_add_linker_symbol can now be handed symbols named via
strtab offsets, and ctf_link_shuffle_syms must figure out their actual
names by looking in the external symtab we have just been fed by the
ctf_link_add_strtab callback, long before that strtab is written out.
include/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-api.h (ctf_symbol_next): New.
(ctf_add_objt_sym): Likewise.
(ctf_add_func_sym): Likewise.
* ctf.h: Document new function info section format.
(CTF_F_NEWFUNCINFO): New.
(CTF_F_IDXSORTED): New.
(CTF_F_MAX): Adjust accordingly.
libctf/ChangeLog
2020-11-20 Nick Alcock <nick.alcock@oracle.com>
* ctf-impl.h (CTF_INDEX_PAD_THRESHOLD): New.
(_libctf_nonnull_): Likewise.
(ctf_in_flight_dynsym_t): New.
(ctf_dict_t) <ctf_funcidx_names>: Likewise.
<ctf_objtidx_names>: Likewise.
<ctf_nfuncidx>: Likewise.
<ctf_nobjtidx>: Likewise.
<ctf_funcidx_sxlate>: Likewise.
<ctf_objtidx_sxlate>: Likewise.
<ctf_objthash>: Likewise.
<ctf_funchash>: Likewise.
<ctf_dynsyms>: Likewise.
<ctf_dynsymidx>: Likewise.
<ctf_dynsymmax>: Likewise.
<ctf_in_flight_dynsym>: Likewise.
(struct ctf_next) <u.ctn_next>: Likewise.
(ctf_symtab_skippable): New prototype.
(ctf_add_funcobjt_sym): Likewise.
(ctf_dynhash_sort_by_name): Likewise.
(ctf_sym_to_elf64): Rename to...
(ctf_elf32_to_link_sym): ... this, and...
(ctf_elf64_to_link_sym): ... this.
* ctf-open.c (init_symtab): Check for lack of CTF_F_NEWFUNCINFO
flag, and presence of index sections. Refactor out
ctf_symtab_skippable and ctf_elf*_to_link_sym, and use them. Use
ctf_link_sym_t, not Elf64_Sym. Skip initializing objt or func
sxlate sections if corresponding index section is present. Adjust
for new func info section format.
(ctf_bufopen_internal): Add ctf_err_warn to corrupt-file error
handling. Report incorrect-length index sections. Always do an
init_symtab, even if there is no symtab section (there may be index
sections still).
(flip_objts): Adjust comment: func and objt sections are actually
identical in structure now, no need to caveat.
(ctf_dict_close): Free newly-added data structures.
* ctf-create.c (ctf_create): Initialize them.
(ctf_symtab_skippable): New, refactored out of
init_symtab, with st_nameidx_set check added.
(ctf_add_funcobjt_sym): New, add a function or object symbol to the
ctf_objthash or ctf_funchash, by name.
(ctf_add_objt_sym): Call it.
(ctf_add_func_sym): Likewise.
(symtypetab_delete_nonstatic_vars): New, delete vars also present as
data objects.
(CTF_SYMTYPETAB_EMIT_FUNCTION): New flag to symtypetab emitters:
this is a function emission, not a data object emission.
(CTF_SYMTYPETAB_EMIT_PAD): New flag to symtypetab emitters: emit
pads for symbols with no type (only set for unindexed sections).
(CTF_SYMTYPETAB_FORCE_INDEXED): New flag to symtypetab emitters:
always emit indexed.
(symtypetab_density): New, figure out section sizes.
(emit_symtypetab): New, emit a symtypetab.
(emit_symtypetab_index): New, emit a symtypetab index.
(ctf_serialize): Call them, emitting suitably sorted symtypetab
sections and indexes. Set suitable header flags. Copy over new
fields.
* ctf-hash.c (ctf_dynhash_sort_by_name): New, used to impose an
order on symtypetab index sections.
* ctf-link.c (ctf_add_type_mapping): Delete erroneous comment
relating to code that was never committed.
(ctf_link_one_variable): Improve variable name.
(check_sym): New, symtypetab analogue of check_variable.
(ctf_link_deduplicating_one_symtypetab): New.
(ctf_link_deduplicating_syms): Likewise.
(ctf_link_deduplicating): Call them.
(ctf_link_deduplicating_per_cu): Note that we don't call them in
this case (yet).
(ctf_link_add_strtab): Set the error on the fp correctly.
(ctf_link_add_linker_symbol): New (no longer a do-nothing stub), add
a linker symbol to the in-flight list.
(ctf_link_shuffle_syms): New (no longer a do-nothing stub), turn the
in-flight list into a mapping we can use, now its names are
resolvable in the external strtab.
* ctf-string.c (ctf_str_rollback_atom): Don't roll back atoms with
external strtab offsets.
(ctf_str_rollback): Adjust comment.
(ctf_str_write_strtab): Migrate ctf_syn_ext_strtab population from
writeout time...
(ctf_str_add_external): ... to string addition time.
* ctf-lookup.c (ctf_lookup_var_key_t): Rename to...
(ctf_lookup_idx_key_t): ... this, now we use it for syms too.
<clik_names>: New member, a name table.
(ctf_lookup_var): Adjust accordingly.
(ctf_lookup_variable): Likewise.
(ctf_lookup_by_id): Shuffle further up in the file.
(ctf_symidx_sort_arg_cb): New, callback for...
(sort_symidx_by_name): ... this new function to sort a symidx
found to be unsorted (likely originating from the compiler).
(ctf_symidx_sort): New, sort a symidx.
(ctf_lookup_symbol_name): Support dynamic symbols with indexes
provided by the linker. Use ctf_link_sym_t, not Elf64_Sym.
Check the parent if a child lookup fails.
(ctf_lookup_by_symbol): Likewise. Work for function symbols too.
(ctf_symbol_next): New, iterate over symbols with types (without
sorting).
(ctf_lookup_idx_name): New, bsearch for symbol names in indexes.
(ctf_try_lookup_indexed): New, attempt an indexed lookup.
(ctf_func_info): Reimplement in terms of ctf_lookup_by_symbol.
(ctf_func_args): Likewise.
(ctf_get_dict): Move...
* ctf-types.c (ctf_get_dict): ... here.
* ctf-util.c (ctf_sym_to_elf64): Re-express as...
(ctf_elf64_to_link_sym): ... this. Add new st_symidx field, and
st_nameidx_set (always 0, so st_nameidx can be ignored). Look in
the ELF strtab for names.
(ctf_elf32_to_link_sym): Likewise, for Elf32_Sym.
(ctf_next_destroy): Destroy ctf_next_t.u.ctn_next if need be.
* libctf.ver: Add ctf_symbol_next, ctf_add_objt_sym and
ctf_add_func_sym.
|
||
|
Nick Alcock
|
3d16b64e28 |
bfd, include, ld, binutils, libctf: CTF should use the dynstr/sym
This is embarrassing. The whole point of CTF is that it remains intact even after a binary is stripped, providing a compact mapping from symbols to types for everything in the externally-visible interface of an ELF object: it has connections to the symbol table for that purpose, and to the string table to avoid duplicating symbol names. So it's a shame that the hooks I implemented last year served to hook it up to the .symtab and .strtab, which obviously disappear on strip, leaving any accompanying the CTF dict containing references to strings (and, soon, symbols) which don't exist any more because their containing strtab has been vaporized. The original Solaris design used .dynsym and .dynstr (well, actually, .ldynsym, which has more symbols) which do not disappear. So should we. Thankfully the work we did before serves as guide rails, and adjusting things to use the .dynstr and .dynsym was fast and easy. The only annoyance is that the dynsym is assembled inside elflink.c in a fairly piecemeal fashion, so that the easiest way to get the symbols out was to hook in before every call to swap_symbol_out (we also leave in a hook in front of symbol additions to the .symtab because it seems plausible that we might want to hook them in future too: for now that hook is unused). We adjust things so that rather than being offered a whole hash table of symbols at once, libctf is now given symbols one at a time, with st_name indexes already resolved and pointing at their final .dynstr offsets: it's now up to libctf to resolve these to names as needed using the strtab info we pass it separately. Some bits might be contentious. The ctf_new_dynstr callback takes an elf_internal_sym, and this remains an elf_internal_sym right down through the generic emulation layers into ldelfgen. This is no worse than the elf_sym_strtab we used to pass down, but in the future when we gain non-ELF CTF symtab support we might want to lower the elf_internal_sym to some other representation (perhaps a ctf_link_symbol) in bfd or in ldlang_ctf_new_dynsym. We rename the 'apply_strsym' hooks to 'acquire_strings' instead, becuse they no longer have anything to do with symbols. There are some API changes to pieces of API which are technically public but actually totally unused by anything and/or unused by anything but ld so they can change freely: the ctf_link_symbol gains new fields to allow symbol names to be given as strtab offsets as well as strings, and a symidx so that the symbol index can be passed in. ctf_link_shuffle_syms loses its callback parameter: the idea now is that linkers call the new ctf_link_add_linker_symbol for every symbol in .dynsym, feed in all the strtab entries with ctf_link_add_strtab, and then a call to ctf_link_shuffle_syms will apply both and arrange to use them to reorder the CTF symtab at CTF serialization time (which is coming in the next commit). Inside libctf we have a new preamble flag CTF_F_DYNSTR which is always set in v3-format CTF dicts from this commit forwards: CTF dicts without this flag are associated with .strtab like they used to be, so that old dicts' external strings don't turn to garbage when loaded by new libctf. Dicts with this flag are associated with .dynstr and .dynsym instead. (The flag is not the next in sequence because this commit was written quite late: the missing flags will be filled in by the next commit.) Tests forthcoming in a later commit in this series. bfd/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * elflink.c (elf_finalize_dynstr): Call examine_strtab after dynstr finalization. (elf_link_swap_symbols_out): Don't call it here. Call ctf_new_symbol before swap_symbol_out. (elf_link_output_extsym): Call ctf_new_dynsym before swap_symbol_out. (bfd_elf_final_link): Likewise. * elf.c (swap_out_syms): Pass in bfd_link_info. Call ctf_new_symbol before swap_symbol_out. (_bfd_elf_compute_section_file_positions): Adjust. binutils/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * readelf.c (dump_section_as_ctf): Use .dynsym and .dynstr, not .symtab and .strtab. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * bfdlink.h (struct elf_sym_strtab): Replace with... (struct elf_internal_sym): ... this. (struct bfd_link_callbacks) <examine_strtab>: Take only a symstrtab argument. <ctf_new_symbol>: New. <ctf_new_dynsym>: Likewise. * ctf-api.h (struct ctf_link_sym) <st_symidx>: New. <st_nameidx>: Likewise. <st_nameidx_set>: Likewise. (ctf_link_iter_symbol_f): Removed. (ctf_link_shuffle_syms): Remove most parameters, just takes a ctf_dict_t now. (ctf_link_add_linker_symbol): New, split from ctf_link_shuffle_syms. * ctf.h (CTF_F_DYNSTR): New. (CTF_F_MAX): Adjust. ld/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ldelfgen.c (struct ctf_strsym_iter_cb_arg): Rename to... (struct ctf_strtab_iter_cb_arg): ... this, changing fields: <syms>: Remove. <symcount>: Remove. <symstrtab>: Rename to... <strtab>: ... this. (ldelf_ctf_strtab_iter_cb): Adjust. (ldelf_ctf_symbols_iter_cb): Remove. (ldelf_new_dynsym_for_ctf): New, tell libctf about a single symbol. (ldelf_examine_strtab_for_ctf): Rename to... (ldelf_acquire_strings_for_ctf): ... this, only doing the strtab portion and not symbols. * ldelfgen.h: Adjust declarations accordingly. * ldemul.c (ldemul_examine_strtab_for_ctf): Rename to... (ldemul_acquire_strings_for_ctf): ... this. (ldemul_new_dynsym_for_ctf): New. * ldemul.h: Adjust declarations accordingly. * ldlang.c (ldlang_ctf_apply_strsym): Rename to... (ldlang_ctf_acquire_strings): ... this. (ldlang_ctf_new_dynsym): New. (lang_write_ctf): Call ldemul_new_dynsym_for_ctf with NULL to do the actual symbol shuffle. * ldlang.h (struct elf_strtab_hash): Adjust accordingly. * ldmain.c (bfd_link_callbacks): Wire up new/renamed callbacks. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-link.c (ctf_link_shuffle_syms): Adjust. (ctf_link_add_linker_symbol): New, unimplemented stub. * libctf.ver: Add it. * ctf-create.c (ctf_serialize): Set CTF_F_DYNSTR on newly-serialized dicts. * ctf-open-bfd.c (ctf_bfdopen_ctfsect): Check for the flag: open the symtab/strtab if not present, dynsym/dynstr otherwise. * ctf-archive.c (ctf_arc_bufpreamble): New, get the preamble from some arbitrary member of a CTF archive. * ctf-impl.h (ctf_arc_bufpreamble): Declare it. |
||
|
Nick Alcock
|
ae41200ba8 |
libctf, include, binutils, gdb: rename CTF-opening functions
The functions that return ctf_dict_t's given a ctf_archive_t and a name are very clumsily named. It sounds like they return *archives*, not dictionaries, and the names are very long and clunky. Why do we have a ctf_arc_open_by_name when it opens a dictionary, not an archive, and when there is no way to open a dictionary in any other way? The answer is purely internal: the function is located in ctf-archive.c, and everything in there was called ctf_arc_*, and there is another way to open a dict (by offset in the archive), that is internal to ctf-archive.c and that nothing else can call. This is clearly bad naming. The internal organization of the source tree should not dictate public API names! So rename things (keeping the old, bad names for compatibility), and adjust all users. You now open a dict using ctf_dict_open, and open it giving ELF sections via ctf_dict_open_sections. binutils/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * objdump.c (dump_ctf): Use ctf_dict_open, not ctf_arc_open_by_name. * readelf.c (dump_section_as_ctf): Likewise. gdb/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctfread.c (elfctf_build_psymtabs): Use ctf_dict_open, not ctf_arc_open_by_name. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_arc_open_by_name): Rename to... (ctf_dict_open): ... this, keeping compatibility function. (ctf_arc_open_by_name_sections): Rename to... (ctf_dict_open_sections): ... this, keeping compatibility function. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-archive.c (ctf_arc_open_by_offset): Rename to... (ctf_dict_open_by_offset): ... this. Adjust callers. (ctf_arc_open_by_name_internal): Rename to... (ctf_dict_open_internal): ... this. Adjust callers. (ctf_arc_open_by_name_sections): Rename to... (ctf_dict_open_sections): ... this, keeping compatibility function. (ctf_arc_open_by_name): Rename to... (ctf_dict_open): ... this, keeping compatibility function. * libctf.ver: New functions added. * ctf-link.c (ctf_link_one_input_archive): Adjusted accordingly. (ctf_link_deduplicating_open_inputs): Likewise. |
||
|
Nick Alcock
|
139633c307 |
libctf, include, binutils, gdb, ld: rename ctf_file_t to ctf_dict_t
The naming of the ctf_file_t type in libctf is a historical curiosity. Back in the Solaris days, CTF dictionaries were originally generated as a separate file and then (sometimes) merged into objects: hence the datatype was named ctf_file_t, and known as a "CTF file". Nowadays, raw CTF is essentially never written to a file on its own, and the datatype changed name to a "CTF dictionary" years ago. So the term "CTF file" refers to something that is never a file! This is at best confusing. The type has also historically been known as a 'CTF container", which is even more confusing now that we have CTF archives which are *also* a sort of container (they contain CTF dictionaries), but which are never referred to as containers in the source code. So fix this by completing the renaming, renaming ctf_file_t to ctf_dict_t throughout, and renaming those few functions that refer to CTF files by name (keeping compatibility aliases) to refer to dicts instead. Old users who still refer to ctf_file_t will see (harmless) pointer-compatibility warnings at compile time, but the ABI is unchanged (since C doesn't mangle names, and ctf_file_t was always an opaque type) and things will still compile fine as long as -Werror is not specified. All references to CTF containers and CTF files in the source code are fixed to refer to CTF dicts instead. Further (smaller) renamings of annoyingly-named functions to come, as part of the process of souping up queries across whole archives at once (needed for the function info and data object sections). binutils/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * objdump.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t. (dump_ctf_archive_member): Likewise. (dump_ctf): Likewise. Use ctf_dict_close, not ctf_file_close. * readelf.c (dump_ctf_errs): Rename ctf_file_t to ctf_dict_t. (dump_ctf_archive_member): Likewise. (dump_section_as_ctf): Likewise. Use ctf_dict_close, not ctf_file_close. gdb/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctfread.c: Change uses of ctf_file_t to ctf_dict_t. (ctf_fp_info::~ctf_fp_info): Call ctf_dict_close, not ctf_file_close. include/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_file_t): Rename to... (ctf_dict_t): ... this. Keep ctf_file_t around for compatibility. (struct ctf_file): Likewise rename to... (struct ctf_dict): ... this. (ctf_file_close): Rename to... (ctf_dict_close): ... this, keeping compatibility function. (ctf_parent_file): Rename to... (ctf_parent_dict): ... this, keeping compatibility function. All callers adjusted. * ctf.h: Rename references to ctf_file_t to ctf_dict_t. (struct ctf_archive) <ctfa_nfiles>: Rename to... <ctfa_ndicts>: ... this. ld/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ldlang.c (ctf_output): This is a ctf_dict_t now. (lang_ctf_errs_warnings): Rename ctf_file_t to ctf_dict_t. (ldlang_open_ctf): Adjust comment. (lang_merge_ctf): Use ctf_dict_close, not ctf_file_close. * ldelfgen.h (ldelf_examine_strtab_for_ctf): Rename ctf_file_t to ctf_dict_t. Change opaque declaration accordingly. * ldelfgen.c (ldelf_examine_strtab_for_ctf): Adjust. * ldemul.h (examine_strtab_for_ctf): Likewise. (ldemul_examine_strtab_for_ctf): Likewise. * ldeuml.c (ldemul_examine_strtab_for_ctf): Likewise. libctf/ChangeLog 2020-11-20 Nick Alcock <nick.alcock@oracle.com> * ctf-impl.h: Rename ctf_file_t to ctf_dict_t: all declarations adjusted. (ctf_fileops): Rename to... (ctf_dictops): ... this. (ctf_dedup_t) <cd_id_to_file_t>: Rename to... <cd_id_to_dict_t>: ... this. (ctf_file_t): Fix outdated comment. <ctf_fileops>: Rename to... <ctf_dictops>: ... this. (struct ctf_archive_internal) <ctfi_file>: Rename to... <ctfi_dict>: ... this. * ctf-archive.c: Rename ctf_file_t to ctf_dict_t. Rename ctf_archive.ctfa_nfiles to ctfa_ndicts. Rename ctf_file_close to ctf_dict_close. All users adjusted. * ctf-create.c: Likewise. Refer to CTF dicts, not CTF containers. (ctf_bundle_t) <ctb_file>: Rename to... <ctb_dict): ... this. * ctf-decl.c: Rename ctf_file_t to ctf_dict_t. * ctf-dedup.c: Likewise. Rename ctf_file_close to ctf_dict_close. Refer to CTF dicts, not CTF containers. * ctf-dump.c: Likewise. * ctf-error.c: Likewise. * ctf-hash.c: Likewise. * ctf-inlines.h: Likewise. * ctf-labels.c: Likewise. * ctf-link.c: Likewise. * ctf-lookup.c: Likewise. * ctf-open-bfd.c: Likewise. * ctf-string.c: Likewise. * ctf-subr.c: Likewise. * ctf-types.c: Likewise. * ctf-util.c: Likewise. * ctf-open.c: Likewise. (ctf_file_close): Rename to... (ctf_dict_close): ...this. (ctf_file_close): New trivial wrapper around ctf_dict_close, for compatibility. (ctf_parent_file): Rename to... (ctf_parent_dict): ... this. (ctf_parent_file): New trivial wrapper around ctf_parent_dict, for compatibility. * libctf.ver: Add ctf_dict_close and ctf_parent_dict. |
||
|
Tom Tromey
|
0d01fbe64f |
Remove libctf/mkerrors.sed
This patch removes libctf/mkerrors.sed, replacing it with a macro in ctf-api.h. This simplifies the build and avoids possible unportable code in the sed script. 2020-10-21 Tom Tromey <tromey@adacore.com> * ctf-api.h (_CTF_ERRORS): New macro. libctf/ChangeLog 2020-10-21 Tom Tromey <tromey@adacore.com> * mkerrors.sed: Remove. * ctf-error.c (_CTF_FIRST): New define. (_CTF_ITEM): Define this, not _CTF_STR. (_ctf_errlist, _ctf_erridx): Use _CTF_ERRORS. (ERRSTRFIELD): Rewrite. (ERRSTRFIELD1): Remove. * Makefile.in: Rebuild. * Makefile.am (BUILT_SOURCES): Remove. (ctf-error.h): Remove. |
||
|
Nick Alcock
|
926c9e7665 |
libctf, binutils, include, ld: gettextize and improve error handling
This commit follows on from the earlier commit "libctf, ld, binutils: add textual error/warning reporting for libctf" and converts every error in libctf that was reported using ctf_dprintf to use ctf_err_warn instead, gettextizing them in the process, using N_() where necessary to avoid doing gettext calls unless an error message is actually generated, and rephrasing some error messages for ease of translation. This requires a slight change in the ctf_errwarning_next API: this API is public but has not been in a release yet, so can still change freely. The problem is that many errors are emitted at open time (whether opening of a CTF dict, or opening of a CTF archive): the former of these throws away its incompletely-initialized ctf_file_t rather than return it, and the latter has no ctf_file_t at all. So errors and warnings emitted at open time cannot be stored in the ctf_file_t, and have to go elsewhere. We put them in a static local in ctf-subr.c (which is not very thread-safe: a later commit will improve things here): ctf_err_warn with a NULL fp adds to this list, and the public interface ctf_errwarning_next with a NULL fp retrieves from it. We need a slight exception from the usual iterator rules in this case: with a NULL fp, there is nowhere to store the ECTF_NEXT_END "error" which signifies the end of iteration, so we add a new err parameter to ctf_errwarning_next which is used to report such iteration-related errors. (If an fp is provided -- i.e., if not reporting open errors -- this is optional, but even if it's optional it's still an API change. This is actually useful from a usability POV as well, since ctf_errwarning_next is usually called when there's been an error, so overwriting the error code with ECTF_NEXT_END is not very helpful! So, unusually, ctf_errwarning_next now uses the passed fp for its error code *only* if no errp pointer is passed in, and leaves it untouched otherwise.) ld, objdump and readelf are adapted to call ctf_errwarning_next with a NULL fp to report open errors where appropriate. The ctf_err_warn API also has to change, gaining a new error-number parameter which is used to add the error message corresponding to that error number into the debug stream when LIBCTF_DEBUG is enabled: changing this API is easy at this point since we are already touching all existing calls to gettextize them. We need this because the debug stream should contain the errno's message, but the error reported in the error/warning stream should *not*, because the caller will probably report it themselves at failure time regardless, and reporting it in every error message that leads up to it leads to a ridiculous chattering on failure, which is likely to end up as ridiculous chattering on stderr (trimmed a bit): CTF error: `ld/testsuite/ld-ctf/A.c (0): lookup failure for type 3: flags 1: The parent CTF dictionary is unavailable' CTF error: `ld/testsuite/ld-ctf/A.c (0): struct/union member type hashing error during type hashing for type 80000001, kind 6: The parent CTF dictionary is unavailable' CTF error: `deduplicating link variable emission failed for ld/testsuite/ld-ctf/A.c: The parent CTF dictionary is unavailable' ld/.libs/lt-ld-new: warning: CTF linking failed; output will have no CTF section: `The parent CTF dictionary is unavailable' We only need to be told that the parent CTF dictionary is unavailable *once*, not over and over again! errmsgs are still emitted on warning generation, because warnings do not usually lead to a failure propagated up to the caller and reported there. Debug-stream messages are not translated. If translation is turned on, there will be a mixture of English and translated messages in the debug stream, but rather that than burden the translators with debug-only output. binutils/ChangeLog 2020-08-27 Nick Alcock <nick.alcock@oracle.com> * objdump.c (dump_ctf_archive_member): Move error- reporting... (dump_ctf_errs): ... into this separate function. (dump_ctf): Call it on open errors. * readelf.c (dump_ctf_archive_member): Move error- reporting... (dump_ctf_errs): ... into this separate function. Support calls with NULL fp. Adjust for new err parameter to ctf_errwarning_next. (dump_section_as_ctf): Call it on open errors. include/ChangeLog 2020-08-27 Nick Alcock <nick.alcock@oracle.com> * ctf-api.h (ctf_errwarning_next): New err parameter. ld/ChangeLog 2020-08-27 Nick Alcock <nick.alcock@oracle.com> * ldlang.c (lang_ctf_errs_warnings): Support calls with NULL fp. Adjust for new err parameter to ctf_errwarning_next. Only check for assertion failures when fp is non-NULL. (ldlang_open_ctf): Call it on open errors. * testsuite/ld-ctf/ctf.exp: Always use the C locale to avoid breaking the diags tests. libctf/ChangeLog 2020-08-27 Nick Alcock <nick.alcock@oracle.com> * ctf-subr.c (open_errors): New list. (ctf_err_warn): Calls with NULL fp append to open_errors. Add err parameter, and use it to decorate the debug stream with errmsgs. (ctf_err_warn_to_open): Splice errors from a CTF dict into the open_errors. (ctf_errwarning_next): Calls with NULL fp report from open_errors. New err param to report iteration errors (including end-of-iteration) when fp is NULL. (ctf_assert_fail_internal): Adjust ctf_err_warn call for new err parameter: gettextize. * ctf-impl.h (ctfo_get_vbytes): Add ctf_file_t parameter. (LCTF_VBYTES): Adjust. (ctf_err_warn_to_open): New. (ctf_err_warn): Adjust. (ctf_bundle): Used in only one place: move... * ctf-create.c: ... here. (enumcmp): Use ctf_err_warn, not ctf_dprintf, passing the err number down as needed. Don't emit the errmsg. Gettextize. (membcmp): Likewise. (ctf_add_type_internal): Likewise. (ctf_write_mem): Likewise. (ctf_compress_write): Likewise. Report errors writing the header or body. (ctf_write): Likewise. * ctf-archive.c (ctf_arc_write_fd): Use ctf_err_warn, not ctf_dprintf, and gettextize, as above. (ctf_arc_write): Likewise. (ctf_arc_bufopen): Likewise. (ctf_arc_open_internal): Likewise. * ctf-labels.c (ctf_label_iter): Likewise. * ctf-open-bfd.c (ctf_bfdclose): Likewise. (ctf_bfdopen): Likewise. (ctf_bfdopen_ctfsect): Likewise. (ctf_fdopen): Likewise. * ctf-string.c (ctf_str_write_strtab): Likewise. * ctf-types.c (ctf_type_resolve): Likewise. * ctf-open.c (get_vbytes_common): Likewise. Pass down the ctf dict. (get_vbytes_v1): Pass down the ctf dict. (get_vbytes_v2): Likewise. (flip_ctf): Likewise. (flip_types): Likewise. Use ctf_err_warn, not ctf_dprintf, and gettextize, as above. (upgrade_types_v1): Adjust calls. (init_types): Use ctf_err_warn, not ctf_dprintf, as above. (ctf_bufopen_internal): Likewise. Adjust calls. Transplant errors emitted into individual dicts into the open errors if this turns out to be a failed open in the end. * ctf-dump.c (ctf_dump_format_type): Adjust ctf_err_warn for new err argument. Gettextize. Don't emit the errmsg. (ctf_dump_funcs): Likewise. Collapse err label into its only case. (ctf_dump_type): Likewise. * ctf-link.c (ctf_create_per_cu): Adjust ctf_err_warn for new err argument. Gettextize. Don't emit the errmsg. (ctf_link_one_type): Likewise. (ctf_link_lazy_open): Likewise. (ctf_link_one_input_archive): Likewise. (ctf_link_deduplicating_count_inputs): Likewise. (ctf_link_deduplicating_open_inputs): Likewise. (ctf_link_deduplicating_close_inputs): Likewise. (ctf_link_deduplicating): Likewise. (ctf_link): Likewise. (ctf_link_deduplicating_per_cu): Likewise. Add some missed ctf_set_errnos to obscure error cases. * ctf-dedup.c (ctf_dedup_rhash_type): Adjust ctf_err_warn for new err argument. Gettextize. Don't emit the errmsg. (ctf_dedup_populate_mappings): Likewise. (ctf_dedup_detect_name_ambiguity): Likewise. (ctf_dedup_init): Likewise. (ctf_dedup_multiple_input_dicts): Likewise. (ctf_dedup_conflictify_unshared): Likewise. (ctf_dedup): Likewise. (ctf_dedup_rwalk_one_output_mapping): Likewise. (ctf_dedup_id_to_target): Likewise. (ctf_dedup_emit_type): Likewise. (ctf_dedup_emit_struct_members): Likewise. (ctf_dedup_populate_type_mapping): Likewise. (ctf_dedup_populate_type_mappings): Likewise. (ctf_dedup_emit): Likewise. (ctf_dedup_hash_type): Likewise. Fix a bit of messed-up error status setting. (ctf_dedup_rwalk_one_output_mapping): Likewise. Don't hide unknown-type-kind messages (which signify file corruption). |
||
|
Nick Alcock
|
662df3c3f1 |
libctf, link: tie in the deduplicating linker
This fairly intricate commit connects up the CTF linker machinery (which operates in terms of ctf_archive_t's on ctf_link_inputs -> ctf_link_outputs) to the deduplicator (which operates in terms of arrays of ctf_file_t's, all the archives exploded). The nondeduplicating linker is retained, but is not called unless the CTF_LINK_NONDEDUP flag is passed in (which ld never does), or the environment variable LD_NO_CTF_DEDUP is set. Eventually, once we have confidence in the much-more-complex deduplicating linker, I hope the nondeduplicating linker can be removed. In brief, what this does is traverses each input archive in ctf_link_inputs, opening every member (if not already open) and tying child dicts to their parents, shoving them into an array and constructing a corresponding parents array that tells the deduplicator which dict is the parent of which child. We then call ctf_dedup and ctf_dedup_emit with that array of inputs, taking the outputs that result and putting them into ctf_link_outputs where the rest of the CTF linker expects to find them, then linking in the variables just as is done by the nondeduplicating linker. It also implements much of the CU-mapping side of things. The problem CU-mapping introduces is that if you map many input CUs into one output, this is saying that you want many translation units to produce at most one child dict if conflicting types are found in any of them. This means you can suddenly have multiple distinct types with the same name in the same dict, which libctf cannot really represent because it's not something you can do with C translation units. The deduplicator machinery already committed does as best it can with these, hiding types with conflicting names rather than making child dicts out of them: but we still need to call it. This is done similarly to the main link, taking the inputs (one CU output at a time), deduplicating them, taking the output and making it an input to the final link. Two (significant) optimizations are done: we share atoms tables between all these links and the final link (so e.g. all type hash values are shared, all decorated type names, etc); and any CU-mapped links with only one input (and no child dicts) doesn't need to do anything other than renaming the CU: the CU-mapped link phase can be skipped for it. Put together, large CU-mapped links can save 50% of their memory usage and about as much time (and the memory usage for CU-mapped links is significant, because all those output CUs have to have all their types stored in memory all at once). include/ * ctf-api.h (CTF_LINK_NONDEDUP): New, turn off the deduplicator. libctf/ * ctf-impl.h (ctf_list_splice): New. * ctf-util.h (ctf_list_splice): Likewise. * ctf-link.c (link_sort_inputs_cb_arg_t): Likewise. (ctf_link_sort_inputs): Likewise. (ctf_link_deduplicating_count_inputs): Likewise. (ctf_link_deduplicating_open_inputs): Likewise. (ctf_link_deduplicating_close_inputs): Likewise. (ctf_link_deduplicating_variables): Likewise. (ctf_link_deduplicating_per_cu): Likewise. (ctf_link_deduplicating): Likewise. (ctf_link): Call it. |
||
|
Nick Alcock
|
e3e8411bec |
libctf, link: add CTF_LINK_OMIT_VARIABLES_SECTION
This flag (not used anywhere yet) causes the variables section to be omitted from the output CTF dict. include/ * ctf-api.h (CTF_LINK_OMIT_VARIABLES_SECTION): New. libctf/ * ctf-link.c (ctf_link_one_input_archive_member): Check CTF_LINK_OMIT_VARIABLES_SECTION. |