Project

General

Profile

Feature #13280

CTF: provide option to truncate and continue

Added by Andy Fiddaman 5 months ago. Updated 4 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Andy Fiddaman
Category:
tools - gate/build tools
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:
1037

Description

See also: https://smartos.org/bugview/OS-6510

When converting some ELF objects, ctfconvert aborts because the CTF format is not able to accommodate structs, unions or enums with a large number of members since the format stores the object size in 10 bits.

% ctfconvert -o /dev/null ~/ctf/libperl.so
ctfconvert: CTF conversion failed: Limit on number of dynamic type members reached

In the case of libperl.so, there is an enumeration in charclass_invlists.h with ~1200 members.

typedef enum {
        PERL_BIN_PLACEHOLDER = 0, /* So no real value is zero */
        UNI_ADLM,
        UNI_AEGEANNUMBERS,
        UNI_AGE__10,
        UNI_AGE__11,
...
        UNI__PERL_NCHAR,
        UNI__PERL_PATWS,
        UNI__PERL_PROBLEMATIC_LOCALE_FOLDEDS_START,
        UNI__PERL_PROBLEMATIC_LOCALE_FOLDS,
        UNI__PERL_QUOTEMETA,
        UNI__PERL_SURROGATE
} binary_invlist_enum;

It would be nice if there was an option to accept truncation here and continue converting what we can, rather than ending up with an object that contains no CTF data at all.

Related issues

Precedes illumos gate - Feature #13366: Add CTF tests for new featuresIn ProgressAndy Fiddaman

Actions
#1

Updated by Andy Fiddaman 5 months ago

  • Description updated (diff)
#2

Updated by Electric Monk 5 months ago

  • Gerrit CR set to 1037
#3

Updated by Andy Fiddaman 5 months ago

I tested the attached with the problematic libperl.so and with a number of regression test cases.:

% ctfconvert -o /dev/null ~/ctf/libperl.so
ctfconvert: failed to add enumerator UNI_SC__HANO (1023) to <anon> (396)
ctfconvert: CTF conversion failed: Limit on number of dynamic type members reached

% ctfconvert -so test ~/ctf/libperl.so
ctfconvert: WARNING: truncating enumeration UNI_SC__HANO at <anon>

% ctfdump -S test
ctfdump: failed to dump labels: File does not contain any labels

- CTF Statistics ----------------------------------------------------------------

  total number of data objects        = 1782
...

Since the new warning callback introduced with this change is now used for each CU that is missing debug data, here's that output too:

% ctfconvert -o /dev/null ~/ctf/test
ctfconvert: WARNING: file test2.c is missing debug information
ctfconvert: WARNING: file test3.c is missing debug information
ctfconvert: CTF conversion failed due to missing debug data; use -m to override

% ctfconvert -mo test ~/ctf/test
ctfconvert: WARNING: file test2.c is missing debug information
ctfconvert: WARNING: file test3.c is missing debug information

% ctfdump -S test
ctfdump: failed to dump labels: File does not contain any labels

- CTF Statistics ----------------------------------------------------------------

  total number of data objects        = 22
#4

Updated by Andy Fiddaman 4 months ago

#5

Updated by Andy Fiddaman 4 months ago

In addition to the testing noted above, I've also done a full wsdiff between gate and this branch and the only differences to CTF data were in the files touched here.

#6

Updated by Electric Monk 4 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit dd4422524768709a579a2a93a10c78a88a6b0ecb

commit  dd4422524768709a579a2a93a10c78a88a6b0ecb
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Date:   2020-12-17T21:17:56.000Z

    13280 CTF: provide option to truncate and continue
    Reviewed by: Robert Mustacchi <rm@fingolfin.org>
    Reviewed by: Igor Kozhukhov <igor@dilos.org>
    Approved by: Rich Lowe <richlowe@richlowe.net>

Also available in: Atom PDF